The researchers injected malware into an AI-powered model called AlexNet without significantly altering the functionality of the model itself. She continued to identify the images with almost the same accuracy as before. This is because the number of layers and the total number of neurons in the convolutional neural network is fixed prior to training, which means that, just like in the human brain, many neurons in the trained model are either turned on or completely dormant.
At the same time, the malware placed in the model was broken into parts, which prevented it from being detected by standard anti-virus systems. VirusTotal, a service that “scans objects with over 70 antivirus scanners and blocking services,” found no trace of the virus.
In the researchers’ method, you first need to choose the best layer to work in the already trained model, and then inject malware into that layer. In the existing trained model – for example, the widespread image classifier – the effect on the number of neurons was not recorded.
However, the researchers noted that malware can only be hidden by not running it. In order to run malware, it must be extracted from the model by another malicious program, and then brought into a working form. That being said, the bad news is that neural network models are usually huge, so attackers can hide a huge number of programs in them.
Cybersecurity researcher Lukasz Oleinik noted that the new technology cannot yet be used in full force. “It won’t be easy for antivirus software to detect a virus, but that’s only because no one is looking. However, this technique can evolve and then we can have big problems.”