In a paper published in Nature Machine Intelligence, Collaborations Pharmaceuticals, Inc. described how they used their MegaSyn machine learning model to create potentially deadly molecules with ease.
The model is normally used to help create potential therapeutic drugs by weeding out toxic molecules before scientists synthesize them. Collaboration Pharmaceuticals simply flipped that on its head by adding publicly available toxicity databases and instructing the AI to reward for toxicity, rather than weed it out.
Among the 40,000 compounds, false positives are likely present. But the AI independently created several already-known deadly compounds without them being in the original dataset. This suggests that most of the compounds are likely toxic.
In an interview with The Verge, Fabio Urbina, the paper’s lead author and a senior scientist at Collaborations Pharmaceuticals, stated that his company has no desire or plans to synthesize the chemicals to fully test the experiment’s efficiency.
The purpose of the experiment was to see how difficult it would be for someone to misuse AI learning techniques in this way. Urbina explains that a bad actor would only need a toxicity database, and many are publicly available. It would just take an open source machine learning algorithm, a little bit of Python coding and machine learning knowledge to recreate what they did.
That, fortunately, wouldn’t be all they need to create a deadly chemical weapon. The AI only creates virtual molecules, someone would still have to synthesize them. But as Urbina also points out, that may not be as hard as one might think.
“A lot of the chemical building blocks of these chemical warfare agents are well known and are watched. They’re regulated. But there’s so many synthesis companies. As long as it doesn’t look like a chemical warfare agent, they’re most likely going to just synthesize it and send it right back because who knows what the molecule is being used for, right?”
Despite the potential threat, Urbina does not argue for ending open-source knowledge sharing or a heavily regulated AI market. Instead, he argues that keeping track of who is using your data when it is potentially dangerous could be a good first step. He also argues that machine learning scientists should be taught about the potential dangers of misuse, much like chemistry students are.
