This article is part of our reviews of AI research papers, a series of posts that explore the latest findings in artificial intelligence.

Most artificial intelligence researchers agree that one of the key concerns of machine learning is adversarial attacks, data manipulation techniques that cause trained models to behave in undesired ways. But dealing with adversarial attacks has become a sort of cat-and-mouse chase, where AI researchers develop new defense techniques and then find ways to circumvent them.

Among the hottest areas of research in adversarial attacks is computer vision, AI systems that process visual data. By adding an imperceptible layer of noise to images, attackers can fool machine learning algorithms to misclassify them. A proven defense method against adversarial attacks on computer vision systems is “randomized smoothing,” a series of training techniques that focus on making machine learning systems resilient against imperceptible perturbations. Randomized smoothing has become popular because it is applicable to deep learning models, which are especially efficient in performing computer vision tasks.

But randomized smoothing is not perfect. And a new paper accepted at this year’s Conference on Computer Vision and Pattern Recognition (CVPR), AI researchers at Tulane University, Lawrence Livermore National Laboratory, and IBM Research shows that machine learning systems can fail against adversarial examples even if they have been trained with randomized smoothing techniques. Titled “How Robust are Randomized Smoothing based Defenses to Data Poisoning?” the paper sheds light on previously overlooked aspects of adversarial machine learning.

Data poisoning and randomized smoothing

One of the known techniques to compromise machine learning systems is to target the data used to train the models. Called data poisoning, this technique involves an attacker inserting corrupt data in the training dataset to compromise a target machine learning model during training. Some data poisoning techniques aim to trigger a specific behavior in a computer vision system when it faces a specific pattern of pixels at inference time. For instance, in the following image, the machine learning model will tune its parameters to label any image with the purple logo as “dog.”

Other data poisoning techniques aim to reduce the accuracy of a machine learning model on one or more output classes. In this case, the attacker would insert carefully crafted adversarial examples into the dataset used to train the model. These manipulated examples are virtually impossible to detect because their modifications are not visible to the human eye.

Research shows that computer vision systems trained on these examples would be vulnerable to adversarial attacks on manipulated images of the target class. But the AI community has come up with training methods that can make machine learning models robust against data poisoning.

“All previous data poisoning methods assume that the victim will use the standard training procedure of minimizing the empirical error on the training data,” Akshay Mehra, Ph.D. student at Tulane University and lead author of the paper, told TechTalks. “However, the adversarial robustness community has highlighted that minimizing the empirical error is not suitable for model training since models trained with it are vulnerable to adversarial attacks. Several works have been published that try to improve the adversarial robustness of the models. Of these works, training procedures that can produce certifiably robust models are of the most interest due to the adversarial robustness guarantees of the models, trained using these methods.”

Random smoothing is a technique that cancels out the effects of data poisoning by establishing an average certified radius (ACR) during the training of a machine learning model. If a trained computer vision model classifies an image correctly, then adversarial perturbations within the certified radius will not affect its accuracy. The larger the ACR, the harder it becomes to stage an adversarial attack against the machine learning model without making the adversarial noise visible to the human eye.

Experiments show that deep learning models trained with random smoothing techniques maintain their accuracy even if their training dataset contains poisoned examples.

In their research, Mehra and his co-authors assumed that a victim has used random smoothing to make the target robust against adversarial attacks. “In our work, we explored three popular training procedures (Gaussian data augmentation, smooth adversarial training, and MACER) which have been shown to increase certified adversarial robustness of the models as measured by the state-of-the-art certification method based on randomized smoothing,” Mehra says.

Their findings show that even when trained with certified adversarial robustness techniques, machine learning models can be compromised through data poisoning.

Poisoning Against Certified Defenses and bilevel optimization

In their paper, the researchers introduce a new data poisoning method called “Poisoning Against Certified Defenses” (PACD). PACD uses a technique known as “bilevel optimization,” which achieves two goals: create poisoned data for models that have undergone robustness training and pass the certification procedure. PACD produces clean adversarial examples, which means the perturbations are not visible to the human eye.

“A few previous works have shown the effectiveness of solving the bilevel optimization problem to achieve better poisoning data,” Mehra says. “The difference in the formulation of the attack in this work is that instead of using the poison data to reduce the model accuracy we are targeting certified adversarial robustness guarantees obtained from state-of-the-art certification procedure based on randomized smoothing.”

The bilevel optimization process takes a set of clean training examples and gradually adds noise to them until they reach a level that can circumvent the target training technique. The ingenuity behind this data poisoning technique is that researchers were able to create a machine learning algorithm that optimizes the adversarial noise for the specific type of robustness training method used in the target model. The algorithm that creates the adversarial example is called ApproxGrad, and it can be adjusted for different robustness training methods.

Once the target model is trained on the tainted dataset, its ACR will be reduced considerably, and it will be highly vulnerable to adversarial attacks.

“In our approach, we explicitly generated poison data that when used for training, will lead to models with low certified adversarial robustness,” Mehra says. “To do this we used the training procedures that produce models with high certified adversarial robustness as our lower-level problem. The attacker’s objective (upper-level problem) is to lower the guarantees produced by the certification procedure. By approximately solving this bilevel optimization problem we were able to generate poison data that could significantly hurt the certified adversarial robustness guarantees of the models. The lowered guarantees lead to a loss of trust in the model’s prediction at test-time.”

The researchers applied PACD to the MNIST and CIFAR datasets and tested it on neural networks trained with all three popular adversarial robustness techniques. In all cases, PACD data poisoning resulted in a considerable decrease in the average certified radius of the trained model, making it vulnerable to adversarial attacks.

The AI researchers also tested to see whether a poisoned dataset targeted at one adversarial training technique would prove to be effective against others. Interestingly, their findings show that PACD transfers across different training techniques. For instance, even if a poisoned dataset has been optimized for gaussian data augmentation, it will still be effective on machine learning models that will go through the MACER and smooth adversarial training processes.

“We demonstrate, through transfer learning experiments, that the generated poison data works to reduce the certified adversarial robustness guarantees of models trained with different methods and also models with different architectures,” Mehra says.

But while PACD has proven to be effective, it comes with a few caveats. Adversarial attacks that assume full knowledge of the target model, including its architecture and weights, are called “white box attacks.” Adversarial attacks that only need access to the output of a machine learning model are “black box attacks.” PACD stands somewhere in between the two ends of the spectrum. The attacker needs to have some general knowledge of the target machine learning model before formulating the poisoned data.

“Our attack is a grey box attack since we are assuming knowledge of victim’s model architecture and training method,” Mehra says. “But we don’t assume knowledge of the particular weights of the network.”

Another problem with PACD is the cost of producing the poisoned dataset. ApproxGrad, the algorithm that generates the adversarial examples, becomes computationally expensive when applied to large machine learning models and complicated problems. In their experiments, the AI researchers focused on small convolutional neural networks trained to classify the MNIST and CIFAR-10 datasets, which contain no more than 60,000 training examples. In their paper, the researchers note, “For datasets like ImageNet where the optimization must be performed over a very large number of batches, obtaining the solution to bilevel problems becomes computationally hard. Due to this bottleneck we leave the problem of poisoning ImageNet for future work.”

ImageNet contains more than 14 million examples. A machine learning model that can perform well on the ImageNet dataset requires a convolutional neural network with dozens of layers and millions of parameters. Accordingly, creating PACD data would require large resources.

“Solving bilevel optimization problems can be computationally expensive, especially when using very large datasets and deep models,” Mehra says. “However, in our paper, we show that attacks generated against moderately deep models transfer well to much deeper models. It would be interesting to see if attacks generated against a portion of the large training data also work well on the entire training data.”

Today, machine learning applications have created new and complex attack vectors in the millions of parameters of trained models and the numerical values of image pixels, audio samples, and text documents. Adversarial attacks are presenting new challenges for the cybersecurity community, whose tools and methods are centered on finding and fixing bugs in source code.

The PACD technique shows that poisoned data can render proven adversarial defense methods ineffective. Mehra and his co-authors warn that data quality is an underrated factor in assessing adversarial vulnerabilities and developing defenses.

For instance, a malicious actor can develop a tainted dataset and deploy it online for others to use in training their machine learning models. Alternatively, the attacker can insert poisoned examples into crowdsourced machine learning datasets. The adversarial perturbations are imperceptible to the human eye, which makes it extremely difficult to detect them. And automated tools that vet software security can’t detect them.

PACD has important implications for the machine learning community. Machine learning engineers should be more careful about the datasets they use to train their models and make sure the source is trustworthy. Organizations that curate datasets for machine learning training should be more careful about the provenance of their data. And companies such as Kaggle and GitHub that host datasets and machine learning models should start thinking about ways to verify the quality and security of their datasets.

We still don’t have complete tools to detect adversarial perturbations in training datasets. But securing the pipeline for accessing and managing machine learning training datasets can be a good first step in preventing the kind of data poisoning measures Mehra and his co-authors describe in their paper.

The Adversarial ML Threat Matrix, introduced last October, provides solid guidelines on finding and fixing possible holes in the training and deployment pipeline of machine learning models. But a lot more needs to be done. Another useful tool is a series of deep learning trust metrics developed by AI researchers at the University of Waterloo, which can find classes and areas where a computer vision system is underperforming and might be vulnerable to adversarial attacks.

“Through this work, we want to show that advances in certified adversarial robustness are dependent on the quality of the data used for training the models,” Mehra says. “Current methods for detecting data poisoning attacks may not be sufficient when attacker adds imperceptibly distorted data. We need more sophisticated methods to deal with this and is a direction for our future research.”

The post Adversarial machine learning: The underrated threat of data poisoning appeared first on Artificial Intelligence.

Many fear about the future of artificial intelligence and robotics. There are multiple factors supporting the claim that AI is in fact not a threat to humankind, but rather an advantage. One factor is that humans thrives off of social interaction and human communication, which robots evidently can’t provide. While online chatbots are a useful and efficient form of artificial intelligence, robots lack the necessary emotional connection to humans. 

In addition, while AI can replace certain human occupations, it also has the potential to increase job opportunities for people in the technology field. Lastly, as seen at Walmart, AI can improve the efficiency of employees without necessarily replacing them. Artificial intelligence is an amazing technological advancement and it isn’t a threat to humankind.

Most humans partake in social interaction countless times per day. Whether it’s speaking with family members, meeting with colleagues or ordering food and drink at a restaurant, social interaction keeps humankind moving forward. A study published in 1997 by Lisa F. Berkman and S. Leonard Syme, discovered that “people who lacked social and community ties were more likely to die in the follow-up period than those with more extensive contacts,” emphasizing the need for social interaction. 

People need social interaction to live long, healthy lives — and artificial intelligence can limit this. While AI can easily replace humans in various customer service jobs, the potential for AI to replace humans in the business world and other occupations based on social interaction is much more limited.

In addition to limiting social interaction, some fear that AI will replace many human jobs. Keith Block, author for The World Economic Forum, explains that advancements in the field of technology have historically created jobs. 

His claim is supported by various examples, such as “the advent of diesel, the internal combustion engine and electricity caused societal disruptions as workers moved from farms to cities, yet these innovations brought us Ford’s Model T, refrigeration, mechanised agriculture and an age of mass production that produced a new global middle class.” 

Keith suggests that while the structure of the job market may change, new jobs surrounding artificial intelligence will likely develop, as has happened in the past.

While AI can pose a targeted threat to white collar jobs, it will eventually spark a new wave of job opportunities once the initiatives are in full swing. Box is an “enterprise content management platform” meaning they deal with “sharing and accessing files,” as well as “data retention and governance.” Due to the relevance of AI in that field, their chief product officer Jeetu Patel discussed artificial intelligence in an interview for Forbes, saying it will “enable workforces to spend more energy on deep thinking versus completing low value tasks like data entry. With work that can be automated, enterprises can redistribute talent resources and gain cost savings.” 

Overall, artificial intelligence will benefit humankind greatly without posing a threat to our society. AI will replace some tedious forms of labor will be replaced in the workplace, developing our society to be higher functioning and more successful. While there are many pros and cons to the integration of AI in the workplace, it will benefit humanity more than it will harm it. 

The post Opinion | Artificial intelligence isn’t a threat to humanity appeared first on Artificial Intelligence.

Artificial intelligence (AI) is a bigger threat to national security than terrorism, the newly appointed president of one of the world’s oldest science institutions has warned.

Jim Al-Khalili, the incoming president of the British Science Association and professor of physics and public engagement at the University of Surrey, U.K., told The Telegraph the looming dangers of AI trump those posed even by climate change, antibiotic resistance, world poverty, the threat of pandemics or terrorism.

Citing Russian cyber hackers meddling in the 2016 U.S. election, he argued little would stop “cyber terrorists” from forcing their way into AI-controlled infrastructure, such as power grids, transport networks, and military installations.

“I am certain the most important conversation we should be having is about the future of AI,” he said. “It will dominate what happens with all of these other issues for better or for worse.”

Fears that the rise of automation and AI, known as Fourth Industrial Revolution, will endanger jobs is also warranted, he said. His concerns are mirrored by a November 2017 report by the management consulting firm McKinsey, which estimated 50 percent of current work could be automated as soon as 2030.

Al-Khalili is the latest expert to warn against the unregulated rise of AI. In April, a report by the research organization Rand Corporation concluded advances in technology and AI could see humanity speeding towards an international nuclear war.

This rise of such technology could create a dangerous cycle where governments feel obliged to update their nuclear arsenal, while trusting advice from AI machinery which could be flawed or tampered with.

“The fear that computers, by mistake or malice, might lead humanity to the brink of nuclear annihilation has haunted imaginations since the earliest days of the Cold War,” the report read.

“The danger might soon be more science than fiction. Stunning advances in AI have created machines that can learn and think, provoking a new arms race among the world’s major nuclear powers. It’s not the killer robots of Hollywood blockbusters that we need to worry about; it’s how computers might challenge the basic rules of nuclear deterrence and lead humans into making devastating decisions.”

Similarly, venerated physicist Stephen Hawking cautioned AI could destroy civilization before he passed away earlier this year.

“Computers can, in theory, emulate human intelligence, and exceed it,” he said. “Success in creating effective AI, could be the biggest event in the history of our civilization. Or the worst. We just don’t know. So we cannot know if we will be infinitely helped by AI, or ignored by it and side-lined, or conceivably destroyed by it.”

But Subramanian Ramamoorthy, reader in the school of informatics at the University of Edinburgh, disagrees that AI is the biggest threat facing humanity.

He told Newsweek while the popular discourse around AI is heavily driven by major concerns, the technology could also provide benefits that will improve our day-to-day lives.

“Some obviously good applications range from prosthetic and assistive robotic devices that restore the capabilities of the disabled, to predictive models that stabilize and reduce congestion in energy and traffic networks,” he said.

“Closer to home for me, technologies like self-driving cars have the potential to fundamentally change how our cities look and feel for most of us—positively influencing congestion, accessibility and affordability of mobility. Such machines are powered by AI.”

“That said,” he continued, “AI has indeed enabled new forms of issues.

“However, I am not yet convinced that these problems can’t be overcome through careful thinking at the policy level, for which reason I do not yet consider AI to be ‘the biggest challenge facing humanity.’ There are much bigger issues, having to do with people quite independent of technology enabling them.”

The post FORGET TERRORISM, CLIMATE CHANGE AND PANDEMICS: ARTIFICIAL INTELLIGENCE IS THE BIGGEST THREAT TO HUMANITY appeared first on Artificial Intelligence.