AI in art is giving birth to many artificial intelligence artists

Artificially intelligent systems are gradually taking control of errands recently done by people, and numerous processes including redundant, simple developments have effectively been completely automated. Meanwhile, people keep on being better when it comes to abstract and creative tasks. Over a recent couple of years, we’ve seen the rise of AI in art giving birth to many artificial intelligence artists.

These unpredictable algorithms are making interesting (and in some cases frightful) works of art. They’re producing dazzling visuals, significant poetry, extraordinary music, and surprisingly practical film scripts. However, work by these AI artists are bringing up issues about the idea of artificial intelligence in art and the job of human creativity in future societies.

Creativity appears to be baffling because when we have innovative thoughts it is hard to clarify how we got them and we frequently talk about obscure ideas like “motivation” and “instinct” when we try to explain creativity. The fact that we are not aware of how a creative thought shows itself doesn’t really suggest that a scientific explanation cannot exist. In actuality, we don’t know about how we perform different activities, for example, pattern recognition, language understanding, etc., yet we have better art created by AI ready to duplicate such activities.

In March 2019, an AI artist called AICAN and its maker Ahmed Elgammal took control over a New York exhibition. The exhibition at HG Commentary showed two series of canvas works depicting nerve racking, dream-like faceless pictures.

The gallery was not just credited to a machine, yet rather ascribed to the joint effort of a human and machine. Ahmed Elgammal is the Founder and Head of the Art and Artificial Intelligence Laboratory at Rutgers University. He considers AICAN to not exclusively be an autonomous AI artist, yet in addition a collaborator for artistic endeavors promoting artificial intelligence art.

How did AICAN make these scary faceless portraits? The framework was given 100,000 photographs of Western art from more than five centuries, permitting it to become familiar with the style of art via machine learning. It at that point drew from this historical knowledge and the order to make something new to make a work of art with AI without human mediation.

Recently, Hanson Robotics’ female AI robot named Sophia became the world’s first machine artist to sell her Non-Fungible Token (NFT) digital paintings named ‘Sophia Instantiation’ at an auction on premier marketplace Nifty Gateway. Her AI art pieces, which were bought utilizing Ethereum blockchain exchange, were made in collaboration with the UK-based Italian art specialist Andrea Bonaceto, who sold Beeple’s Everyday at Christie’s for $69 million. NFTs, in the resemblance of the digital money, comprise unique codes and can be stored in records or digi wallets. Artificial intelligence robot Sophia’s auction of NFT AI art pieces denoted the first breakthrough between a human and a robot trading.

Sophia is internationally known for being a worldwide celebrity doing a lot of TV appearances across the world including the Jimmy Fallon Show in the US and the Ivan Urgant show in Russia. Sophia is likewise an Innovation Champion for the United Nations Development Program (UNDP) and the first robot to get citizenship of a country. Recently, her computational works were introduced at the Neurips AI conference in the neural inventiveness workshop, and in the poster session at the AAAS yearly gathering. This will be the first series of AI artworks promoted by Sophia and her maker Dr. David Hanson.

Dr. Hanson says, “We made Sophia herself as a work of art as well as an AI development platform. Her intelligence is a group of algorithms and people working together like a hive. For this show, Sophia made the art totally utilizing neural networks and symbolic AI, responding to her impression of Andrea Bonaceto’s works as well as to information from her “life” experiences, under direction from the Sophia team’s designers and programmers. How she reacted to Andrea’s art just excites me. I’m one pleased dad.”

Sophia digitally made her own picture utilizing artificial intelligence art and did what the roboticist David Hanson portrays as ‘an artistic revolution.’ “The experience of teaming up with Sophia and Hansen on the project at Nifty Gateway has been stunning.  Grown so much as an artist since we began this undertaking”, Bonaceto tweeted. Then, humanoid Sophia plans to “study the most noteworthy bidder’s face” who will buy her work and will add one final emphasis to her AI artwork.

The post AI IN ART IS DEVELOPING AND GETTING CREATIVE RAPIDLY appeared first on Artificial Intelligence.

NVIDIA’s open-source toolkit, NVIDIA NeMo( Neural Models), is a revolutionary step towards the advancement of Conversational AI. Based on PyTorch, it allows one to build quickly, train, and fine-tune conversational AI models.

As the world is getting more digital, Conversational AI is a way to enable communication between humans and computers. The set of technologies behind some fascinating technologies like automated messaging, speech recognition, voice chatbots, text to speech, etc. It broadly comprises three areas of AI research: automatic speech recognition (ASR), natural language processing (NLP), and speech synthesis (or text-to-speech, TTS). 

Conversational AI has shaped the path of human-computer interaction, making it more accessible and exciting. The latest advancements in Conversational AI like NVIDIA NeMo help bridge the gap between machines and humans.

NVIDIA NeMo consists of two subparts: NeMo Core and NeMo Collections. NeMo Core deals with all models generally, whereas NeMo Collections deals with models’ specific domains. In Nemo’s Speech collection (nemo_asr), you’ll find models and various building blocks for speech recognition, command recognition, speaker identification, speaker verification, and voice activity detection. NeMo’s NLP collection (nemo_nlp) contains models for tasks such as question answering, punctuation, named entity recognition, and many others. Finally, in NeMo’s Speech Synthesis (nemo_tts), you’ll find several spectrogram generators and vocoders, which will let you generate synthetic speech.

There are three main concepts in NeMo: model, neural module, and neural type. 

• Models contain all the necessary information regarding training, fine-tuning, neural network implementation, tokenization, data augmentation, infrastructure details like the number of GPU nodes,etc., optimization algorithm, etc.
• Neural modules are a sort of encoder-decoder architecture consisting of conceptual building blocks responsible for different tasks. It represents the logical part of a neural network and forms the basis for describing the model and its training process. Collections have many neural modules that can be reused whenever required.
• Inputs and outputs to Neural Modules are typed with Neural Types. A Neural Type is a pair that contains the information about the tensor’s axes layout and semantics of its elements. Every Neural Module has input_types and output_types properties that describe what kinds of inputs this module accepts and what types of outputs it returns.

Even though NeMo is based on PyTorch, it can also be effectively used with other projects like PyTorch Lightning and Hydra. Integration with Lightning makes it easier to train models with mixed precision using Tensor Cores and can scale training to multiple GPUs and compute nodes. It also has some features like logging, checkpointing, overfit checking, etc. Hydra also allows the parametrization of scripts to keep it well organized. It makes it easier to streamline everyday tasks for users.

The post NVIDIA NeMo: An Open-Source Toolkit For Developing State-Of-The-Art Conversational AI Models In Three Lines Of Code appeared first on Artificial Intelligence.

If a swarm of heavily armed fast boats barreled full speed at an aircraft carrier, the crew would have very little time to react.

But if that crew had artificial intelligence and machine learning at its disposal, that blitz of boats probably wouldn’t pose nearly as much of a problem.
Raytheon Intelligence & Space, one of four businesses that form Raytheon Technologies, is using artificial intelligence and machine learning to improve the intelligence, surveillance and reconnaissance capabilities of the U.S. and allied armed forces. The approach is to synthesize reams of data into actionable intelligence and accurate targeting information at speed and scale, in high-risk environments.

“In multi-domain operations, you don’t have full domain superiority but have to exploit what they call moments of superiority in the battlefield,” said Jim Wright, RI&S’ technical director for Intelligence, Surveillance and Reconnaissance Systems. “Speed is a big issue in this defense strategy, which is challenged by huge amounts of data and a limited number of people to look at it.”

A military customer once told Wright the armed services collect 22 football seasons’ worth of video every day. That’s far too much to sift through manually – especially when operators have to make critical decisions quickly, such as protecting ships in crowded sea lanes.

“We’re looking at how machine learning can augment our existing sensor product lines and the question is: ‘How can we utilize machine learning technology to help military commanders make decisions?’” said Shane Zabel, AI Technology Area director for RI&S. “How do we embed some kind of learning machine to go with the sensors to help better execute the mission?”

Traditionally, operators have control sensors and analyzed data much like you’d think they would – by keeping their eyes locked on screens, pressing buttons and using joysticks to move things around. RI&S is using military and commercial advancements in technology to automate those functions.

“AI/ML is at the core of our technology roadmap across Raytheon Intelligence & Space,” said Barbara Borgonovi, vice president of Intelligence Surveillance and Reconnaissance Systems at RI&S. “We are implementing AI/ML into next-generation ISR capabilities so operators can rapidly make the right decisions in any threat environment.”

The business is also developing smart software called Cognitive Aids to Sensor Processing, Exploitation and Response (CASPERTM), to lighten the operator’s workload and use automation to help make decisions faster.

The aids interpret operator requests, then control sensor and data processing functions. They are being integrated into products like the Multi-Spectral Targeting System, which provides visible and infrared intelligence and targeting information for an array of airborne platforms.

CASPER allows operators to work above the drudgery of data processing and instead focus on decision making, resulting in exponentially faster threat response.

Take the fast boat scenario, for example.

“Much like talking to Alexa or Siri, an operator tells CASPER to scan for fast boats and prioritize by threat to the carrier,” Wright said. “CASPER then takes control of sensor functions, rapidly identifies which boats are threats based on things like their appearance and behavior over space and time, and provides the operator with the threat list and recommended courses of action.

“This enables the operator to focus attention on ensuring recommendations are correct and consistent with policy, making the whole process shorter and safer,” he said.

RI&S is developing advanced automation capabilities for ground station systems, and is advancing these capabilities to the leading edge of the sensor grid.

There are thousands of systems and sensors in today’s battlespace. Automation will also help deliver the right data at the right time to make decisions faster through another transformative solution – Joint All Domain Command and Control (JADC2). JADC2 is a future command and control network that will link capabilities and military platforms across the globe in all domains – air, land, sea, cyber and space.

“Machine learning has really taken off,” Zabel said. “It’s all about harnessing the speed potential AI and ML offer.”

The post How Artificial Intelligence And Machine Learning Will Make ISR Faster appeared first on Artificial Intelligence.

The use of robotics in medtech continues to grow. Whether it’s a cobot working alongside humans to automate manufacturing or a surgical robot in the OR, a single point of failure can cause serious harm. The incorporated software systems must take safety into account.

IEC 61508-3 offers several techniques for developing software for safety-related systems, which the medical device software development community can draw on when designing and implementing risk-control measures as required by ISO 14971.

Developing “safe” software begins with establishing a software coding standard. IEC 61508-3 promotes using well-known techniques, including:

• Using modular code.
• Using preferred design patterns.
• Avoiding reentrance and recursion.
• Avoiding dynamic memory allocations and global data objects.
• Minimizing the use of interrupt service routines and locking mechanisms.
• Avoiding dead wait loops.
• Using deterministic timing patterns.

Keep it simple

There are other suggestions under the “keep it simple” principle around limiting the use of pointers, unions and type casting, and not using automatic type conversions while encouraging the use of parentheses and brackets to clarify intended syntax.

A hazard analysis might identify that your code or data spaces can get corrupted. There are well-known risk-control measures around maintaining code and memory integrity which can be easily adopted. Running code from read-only memory, protected with a cyclic redundancy check (CRC-32) that can be checked at boot time and periodically during runtime, prevents errant changes to the code space and provides a mechanism to detect these failures.

Segregating data into different memory regions that can be protected through virtual memory space and using CRC-32 over blocks of memory regions or even adding a checksum to each item stored in memory allows these CRC/checksums to be checked periodically.

CRC/checksums can be verified on each read access to a stored item and updated atomically on every write access to these protected items. Building tests into the software is an important tool as well. It’s a good idea to perform a power-on self-test (POST) at power-up to make sure the hardware is working and to check that your code and data spaces are consistent and not corrupt.

What else can happen?

Another hazardous situation arises when controlling and monitoring are performed on the same processor or in the same process. What happens to your safety system if your process gets hung up in a loop? Techniques that separate the monitor from the controlling function introduce some complexity to the software system, but this complexity can be offset by ensuring the controlling function implements the minimum safety requirements while the monitor handles the fault and error recovery.

Fault detection systems and error recovery mechanisms are much easier to implement when designed from the start. Poorly designed software can experience unexpected, inconsistent timing, which results in unexpected failures. It’s possible to avoid these failures by controlling latency in the software. State machines, software watchdogs and timer-driven events are common design elements to control this.

Keep an eye on communications

Inter-device and inter-process communications are another area of concern for safety-related systems. The integrity of these communications must be monitored to ensure they are robust. Using CRC-32 on any protocol between two entities is recommended. Separate CRC-32 on the headers and the payload helps to detect corruption of these messages. Protocols should be written and designed with the idea that at any time, your system could reboot due to some fault. Thus, building in retry attempts and stateless protocols is recommended.

Safe operational software verifies the ranges of all inputs at the interface where it is encountered; checks internal variables for consistency; and defines default settings to help recover from an inconsistent setting or to support a factory reset. Software watchdog processes can be put in place to watch the watcher and ensure that processes are running as they should.

By taking these techniques into account, software developers working on medical robotic devices can better address the concerns of safety-related systems.

The post Developing software for safety in medical robotics appeared first on Artificial Intelligence.

While global cybersecurity spending now surpasses $100 billion annually, 64 percent of enterprises were compromised in 2018, according to a study by the Ponemon Institute. What explains this less-than-impressive ROI? The standard answer is that wily cyber-criminals are employing ever-evolving, increasingly sophisticated attack methods, part of a never-ending game of cat-and-mouse in which they all too often outsmart the good guys.

This is undoubtedly true – but the root of the problem is that traditional machine learning-based cybersecurity solutions fail to keep up with the growing sophistication of today’s cyber threats, both those that are created by hackers and AI alike. 

Why does machine learning so often come up short – and how should cybersecurity evolve to meet the scale and complexity of the challenge?

Fighting Yesterday’s War

There’s no question that machine learning has driven significant improvements in cybersecurity. Harnessing massive datasets on prevalent malware and prior attacks, machine learning-based solutions are capable of rapidly identifying and thwarting threats.

The problem? To paraphrase former U.S. Defense Secretary Donald Rumsfeld, it’s not enough to go after the known knowns, it’s the known unknowns and the unknown unknowns that are going to cause the most grief.

As the threat landscape has evolved, machine learning is failing to remain resilient in the face of advanced new malware, created by both hackers and artificial intelligence. Fighting yesterday’s war may work when today’s threats are the same as yesterday’s – but not when novel threats are constantly emerging.

Not only is machine learning regularly failing to identify new malicious threats, but it also routinely misidentifies benign ones, yielding a high rate of false positives and creating unnecessary additional work for enterprises’ security teams.

Machine Learning’s Limitations

Traditional machine learning suffers from several factors that impede its ability to prevent complex and first-seen attacks.

Chief among these is data. Only a fraction of the available data will be fed into an algorithm that trains a machine learning model. A computer scientist with a focus on cybersecurity will curate a set of features that he or she recognizes, and this will be used to train the algorithm. This means most of the data in the file won’t be used for training as the system can only learn from the vector of features identified and defined, leaving most characteristics in the data set untouched.

The brains behind those models are invariably brilliant, but nevertheless fallible. Raw data can’t be fed directly into machine learning systems, so the extraction of the data – which is based on human professionals’ knowledge and expertise – unavoidably limits the system. What’s more, hackers understand this and build malware capable of tricking machine learning systems into thinking it is benign.

Because organizations face resource constraints, they can’t hire an unlimited number of computer scientists with cybersecurity expertise to engage in the labor-intensive task of continuously updating and developing data sets. Even if they could, there’s a limit to the size of the dataset that can be used to train a machine learning model before reaching a learning curve saturation – the threshold beyond which the system no longer improves its accuracy.

Additionally, most machine learning models only support portable executable files, so attacks that use other types of files, or even file-less malware, move freely past these cybersecurity solutions.

Not only do machine learning-based cybersecurity solutions fall short on prevention, but according to the Ponemon Institute, it takes organizations 196 days on average to detect a breach. This lag time – more than half a year – is far longer than organizations can afford, particularly when their assets and their reputation are on the line.

The Next Age of Cybersecurity: AI vs. AI

As with all market evolutions, sheer demand will create enough pressure for providers to improve their cybersecurity services and move beyond outmoded machine learning solutions. Given the massive investments enterprises are making in cybersecurity – and the hefty costs associated with breaches – companies are clamoring for preventative capabilities that both hold down costs and provide dynamic security protection.

The result will be advanced AI-based technologies, such as deep learning, that leverage much more of the available data and files, enabling analyses that yield higher levels of detection and prevention, lower false positive rates, more autonomous features, and smaller staff requirements.

To be sure, there’s no magic-bullet solution that will provide 100 percent protection, once and for all. Cyber threats are constantly changing – and it’s time cybersecurity solutions better reflected that.

The post How Traditional Machine Learning Is Holding Cybersecurity Back appeared first on Artificial Intelligence.

Only a minority of consumers trust the brands they use. And the Internet of Things (IoT) itself has a trust problem in the consumer sector. Privacy concerns and poor user experience have “stymied adoption and created a hesitance among users to trust IoT devices,” wrote William Webb and Matthew Hatton in “The Internet of Things Myth.”

While the adoption of smart-home devices continues to tick upward, privacy and security concerns constrain their use to mainly routine tasks. The most popular smart speaker functionality, for instance, is merely playing music, according to eMarketer research.

Meanwhile, IoT device makers continue to face pushback from consumers and regulators over privacy and security. “We’re in a situation where [IoT manufacturers] are fighting these DDoS [distributed denial of service] attacks and all different types of hacking threats that are out there,” said Dilip Sarangan, senior director of research at Frost & Sullivan.

Add to that is the public’s frustration with how manufacturers implement Internet of Things security and privacy. Last year, an Internet Society survey found that 63% of respondents found connected devices to be “creepy.” Three-quarters of respondents did not trust IoT device markers to respect their preferences in how data is used.

The situation is unlikely to change until IoT manufacturers become savvier in terms of information governance. Here, we examine common pitfalls to avoid when developing an IoT product.

Believing Open-Source Software Is Bulletproof

Headlines about consumer IoT devices’ insecurity have remained prevalent in recent years. Most recently, researchers discovered a series of vulnerabilities known as Ripple20 found in hundreds of millions of IoT devices that extend well beyond the consumer sector. “The Ripple20 vulnerabilities affect a vast array of critical IoT devices, including healthcare systems, power grids, smart home devices and more,” said Natali Tshuva, CEO of Sternum.

The discovery of the Ripple20 vulnerability is not surprising, said Terry Dunlap, a former National Security Agency employee who is now the CEO of ReFirm Laws. Many IoT devices are built with open-source components. If there is a flaw in any of these components, “it’s going to get spread far and wide,” Dunlap said. While open-source software can provide greater oversight than proprietary software, open-source security researchers and developers can’t check for every possible security flaw.

The post Common Internet of Things security pitfalls appeared first on Artificial Intelligence.

Two things often mentioned with deep learning are “data” and “compute resources.” You need a lot of both when developing, training, and testing deep learning models. When developers don’t have a lot of training samples or access to very powerful servers, they use transfer learning to finetune a pre-trained deep learning model for a new task.

At this year’s ICML conference, scientists at IBM Research and Taiwan’s National Tsing Hua University Research introduced “black-box adversarial reprogramming” (BAR), an alternative repurposing technique that turns a supposed weakness of deep neural networks into a strength.

BAR expands the original work on adversarial reprogramming and previous work on black-box adversarial attacks to make it possible to expand the capabilities of deep neural networks even when developers don’t have full access to the model.

Pretrained and finetuned deep learning models

When you want to develop an application that requires deep learning, one option is to create your own neural network from scratch and train it on available or curated examples. For instance, you can use ImageNet, a public dataset that contains more than 14 million labeled images.

There is a problem, however. First, you must find the right architecture for the task, such as the number and sequence of convolution, pooling, and dense layers. You must also decide the number of filters and parameters for each layer, the learning rate, optimizer, loss function, and other hyperparameters. A lot of these decisions require trial-and-error training, which is a slow and costly process unless you have access to strong graphics processors or specialized hardware such as Google’s TPU.

To avoid reinventing the wheel, you can download a tried-and-tested model such as AlexNet, ResNet, or Inception, and train it yourself. But you’ll still need a cluster of GPUs or TPUs to complete the training in an acceptable amount of time. To avoid the costly training process, you can download the pre-trained version of these models and integrate them into your application.

Alternatively, you can use a service such as Clarifia and Amazon Rekognition, which provide application programming interfaces for image recognition tasks. These services are “black-box” models because the developer doesn’t have access to the network layers and parameters and can only interact with them by providing them images and retrieving the resulting label.

Now, suppose you want to create a computer vision algorithm for a specialized task, such as detecting autism from brain scans or breast cancer from mammograms. In this case, a general image recognition model such as AlexNet or a service like Clarifai won’t cut it. You need a deep learning model trained on data for that problem domain.

The first problem you’ll face is gathering enough data. A specialized task might not require 14 million labeled images, but you’ll still need quite a few if you’re training the neural network from scratch.

Transfer learning allows you to slash the number of training examples. The idea is to take a pre-trained model (e.g., ResNet) and retrain it on the data and labels from a new domain. Since the model has been trained on a large dataset, its parameters are already tuned to detect many of the features that will come in handy in the new domain. Therefore, it will take much less time and data to retrain it for the new task.

While it sounds easy, transfer learning is itself a complicated process and does not work well in all circumstances. Based on how close the source and target domains are, you’ll need to freeze and unfreeze layers and add new layers to the model during the transfer learning. You’ll also need to do a lot of hyperparameter tweaking in the process.

In some cases, transfer learning can perform worse than training a neural network from scratch. You also can’t perform transfer learning on API-based systems where you don’t have access to the deep learning model.

Adversarial attacks and reprogramming

Adversarial reprogramming is an alternative technique for repurposing machine learning models. It leverages adversarial machine learning, an area of research that explores how perturbations to input data can change the behavior of neural networks. For example, in the image below, adding a layer of noise to the panda photo on the left causes the award-winning GoogLeNet deep learning model to mistake it for a gibbon. The manipulations are called “adversarial perturbations.”

Adversarial machine learning is usually used to display vulnerabilities in deep neural networks. Researchers often use the term “adversarial attacks” when discussing adversarial machine learning. One of the key aspects of adversarial attacks is that the perturbations must go undetected to the human eye.

At the ICLR 2019 conference, artificial intelligence researchers at Google showed that the same technique can be used to enable neural networks to perform a new task, hence the name “adversarial reprogramming.”

“We introduce attacks that instead reprogram the target model to perform a task chosen by the attacker,” the researchers wrote at the time.

Adversarial reprogramming shares the same basic idea as adversarial attacks: The developer changes the behavior of a deep learning model not by modifying its parameters but by making changes to its input.

There are, however, also some key differences between adversarial reprogramming and attacks (aside from the obvious goal). Unlike adversarial examples, reprogramming is not meant to deceive human observers, therefore the modifications to the input data do not need to be imperceptible to the human eye. Also, while in adversarial attacks, noise maps must be calculated per input, adversarial reprogramming uses a single perturbation map to all inputs.

For instance, a deep learning model (e.g., ResNet) trained on the ImageNet dataset can detect 1,000 common things such as animals, plants, objects, etc. An adversarial program aims to repurpose the AI model for another task, such as the number of white squares in an image (see example above). After running the adversarial program on the images, the deep learning model will be able to distinguish each class. However, since the model has been originally trained for another task, you’ll have to map the output to your target domain. For example, if the model outputs goldfish, then it’s an image with two squares, tiger shark is four squares, etc.

The adversarial program is obtained by starting with a random noise map and making small changes until you achieve the desired outputs.

Basically, adversarial reprogramming creates a wrapper around the deep learning model, modifying every input that goes in with the adversarial noise map and mapping the outputs to the target domain. Experiments by the AI researchers showed that in many cases, adversarial reprogramming can produce better results than transfer learning.

Black-box adversarial learning

While adversarial reprogramming does not modify the original deep learning model, you still need access to the neural network’s parameters and layers to train and tune the adversarial program (more specifically, you need access to gradient information). This means that you can’t apply it to black-box models such as the commercial APIs mentioned earlier.

This is where black-box adversarial reprogramming (BAR) enters the picture. The adversarial reprogramming method developed by researchers at IBM and Tsing Hua University does not need access to the details of deep learning models to change their behavior.

To achieve this, the researchers used Zeroth Order Optimization (ZOO), a technique previously developed by AI researchers at IBM and the University of California Davis. The ZOO paper proved the feasibility of black-box adversarial attacks, where an attacker could manipulate the behavior of a machine learning model by simply observing inputs and outputs and without having access to gradient information.

BAR uses the same technique to train the adversarial program. “Gradient descent algorithms are primary tools for training deep learning models,” Pin-Yu Chen, chief scientist at IBM Research and co-author of the BAR paper, told TechTalks. “In the zeroth-order setting, you don’t have access to the gradient information for model optimization. Instead, you can only observe the model outputs (aka function values) at queries points.” In effect, this means that you can, for example, only provide an image to the deep learning model and observe its results.

“ZOO enables gradient-free optimization by using estimated gradients to perform gradient descent algorithms,” Chen says. The main advantage of this method is that it can be applied to any gradient-based algorithms and is not limited to neural-network-based systems alone.

Another improvement Chen and his colleagues added in BAR is “multi-label mapping”: Instead of mapping a single class from the source domain to the target domain (e.g., goldfish = one square), they found a way to map several source labels to the target (e.g., tench, goldfish, hammerhead = one square).

“We find that multiple-source-labels to one target-label mapping can further improve the accuracy of the target task when compared to one-to-one label mapping,” the AI researchers write in their paper.

To test black-box adversarial reprogramming, the researchers used it to repurpose several popular deep learning models for three medical imaging tasks (autism spectrum disorder classification, diabetic retinopathy detection, and melanoma detection). Medical imaging is an especially attractive use for techniques such as BAR because it is a domain where data is scarce, expensive to come by, and subject to privacy regulations.

In all three tests, BAR performed better than transfer learning and training the deep learning model from scratch. It also did nearly as well as standard adversarial reprogramming.

The AI researchers were also able to reprogram two commercial, black-box image classification APIs (Clarifai Moderation and NSFW APIs) with BAR, obtaining decent results.

“The results suggest that BAR/AR should be a strong baseline for transfer learning, given that only wrapping the inputs and outputs of an intact model can give good transfer learning results,” Chen said.

In the future, the AI researchers will explore how BAR can be applied to other data modalities beyond image-based applications.

The post How to trick deep learning algorithms into doing new things appeared first on Artificial Intelligence.

When Deep Blue defeated world chess champion Garry Kasparov in 1997, it may have seemed artificial intelligence had finally arrived. A computer had just taken down one of the top chess players of all time. But it wasn’t to be.

Though Deep Blue was meticulously programmed top-to-bottom to play chess, the approach was too labor-intensive, too dependent on clear rules and bounded possibilities to succeed at more complex games, let alone in the real world. The next revolution would take a decade and a half, when vastly more computing power and data revived machine learning, an old idea in artificial intelligence just waiting for the world to catch up.

Today, machine learning dominates, mostly by way of a family of algorithms called deep learning, while symbolic AI, the dominant approach in Deep Blue’s day, has faded into the background.

Key to deep learning’s success is the fact the algorithms basically write themselves. Given some high-level programming and a dataset, they learn from experience. No engineer anticipates every possibility in code. The algorithms just figure it.

Now, Alphabet’s DeepMind is taking this automation further by developing deep learning algorithms that can handle programming tasks which have been, to date, the sole domain of the world’s top computer scientists (and take them years to write).

In a paper recently published on the pre-print server arXiv, a database for research papers that haven’t been peer reviewed yet, the DeepMind team described a new deep reinforcement learning algorithm that was able to discover its own value function—a critical programming rule in deep reinforcement learning—from scratch.

Surprisingly, the algorithm was also effective beyond the simple environments it trained in, going on to play Atari games—a different, more complicated task—at a level that was, at times, competitive with human-designed algorithms and achieving superhuman levels of play in 14 games.

DeepMind says the approach could accelerate the development of reinforcement learning algorithms and even lead to a shift in focus, where instead of spending years writing the algorithms themselves, researchers work to perfect the environments in which they train.

Pavlov’s Digital Dog

First, a little background.

Three main deep learning approaches are supervised, unsupervised, and reinforcement learning.

The first two consume huge amounts of data (like images or articles), look for patterns in the data, and use those patterns to inform actions (like identifying an image of a cat). To us, this is a pretty alien way to learn about the world. Not only would it be mind-numbingly dull to review millions of cat images, it’d take us years or more to do what these programs do in hours or days. And of course, we can learn what a cat looks like from just a few examples. So why bother?

While supervised and unsupervised deep learning emphasize the machine in machine learning, reinforcement learning is a bit more biological. It actually is the way we learn. Confronted with several possible actions, we predict which will be most rewarding based on experience—weighing the pleasure of eating a chocolate chip cookie against avoiding a cavity and trip to the dentist.

In deep reinforcement learning, algorithms go through a similar process as they take action. In the Atari game Breakout, for instance, a player guides a paddle to bounce a ball at a ceiling of bricks, trying to break as many as possible. When playing Breakout, should an algorithm move the paddle left or right? To decide, it runs a projection—this is the value function—of which direction will maximize the total points, or rewards, it can earn.

Move by move, game by game, an algorithm combines experience and value function to learn which actions bring greater rewards and improves its play, until eventually, it becomes an uncanny Breakout player.

Learning to Learn (Very Meta)

So, a key to deep reinforcement learning is developing a good value function. And that’s difficult. According to the DeepMind team, it takes years of manual research to write the rules guiding algorithmic actions—which is why automating the process is so alluring. Their new Learned Policy Gradient (LPG) algorithm makes solid progress in that direction.

LPG trained in a number of toy environments. Most of these were “gridworlds”—literally two-dimensional grids with objects in some squares. The AI moves square to square and earns points or punishments as it encounters objects. The grids vary in size, and the distribution of objects is either set or random. The training environments offer opportunities to learn fundamental lessons for reinforcement learning algorithms.

Only in LPG’s case, it had no value function to guide that learning.

Instead, LPG has what DeepMind calls a “meta-learner.” You might think of this as an algorithm within an algorithm that, by interacting with its environment, discovers both “what to predict,” thereby forming its version of a value function, and “how to learn from it,” applying its newly discovered value function to each decision it makes in the future.

Prior work in the area has had some success, but according to DeepMind, LPG is the first  algorithm to discover reinforcement learning rules from scratch and to generalize beyond training. The latter was particularly surprising because Atari games are so different from the simple worlds LPG trained in—that is, it had never seen anything like an Atari game.

Time to Hand Over the Reins? Not Just Yet

LPG is still behind advanced human-designed algorithms, the researchers said. But it outperformed a human-designed benchmark in training and even some Atari games, which suggests it isn’t strictly worse, just that it specializes in some environments.

This is where there’s room for improvement and more research.

The more environments LPG saw, the more it could successfully generalize. Intriguingly, the researchers speculate that with enough well-designed training environments, the approach might yield a general-purpose reinforcement learning algorithm.

At the least, though, they say further automation of algorithm discovery—that is, algorithms learning to learn—will accelerate the field. In the near term, it can help researchers more quickly develop hand-designed algorithms. Further out, as self-discovered algorithms like LPG improve, engineers may shift from manually developing the algorithms themselves to building the environments where they learn.

Deep learning long ago left Deep Blue in the dust at games. Perhaps algorithms learning to learn will be a winning strategy in the real world too.

The post DeepMind’s Newest AI Programs Itself to Make All the Right Decisions appeared first on Artificial Intelligence.

New legislation is needed for “high-risk” applications of artificial intelligence (AI), in the view of a slim majority of companies that responded to a recent European Commission consultation.

The commission on Friday published an overview of the 1,215 responses it received on possible AI legislation, which show a wide mix of opinions on how Europe should regulate the technology.

The dilemma for politicians is deciding whether to draw up brand new rules or modify existing legislation.

Forty two per cent of respondents backed the introduction of a new regulatory framework on AI, while another 33 per cent said it could be dealt with by changing existing legislation. Only 3 per cent say current legislation is sufficient.

“It is interesting to note that respondents from industry and businesswere more likely to agree with limiting new compulsory requirements to high-risk applications [by] a percentage of 54.6 per cent,” the commission noted. High-risk applications could include self-driving cars, facial recognition technology and AI used in healthcare.

The consultation, which ran until June, followed publication of an EU AI white paper in February, spelling out options for AI laws. The paper likened the current situation to “the Wild West”, with AI applications like facial recognition technology coming into use without proper oversight.

Companies and researchers are aware the technology they are developing comes with risks.

The overwhelming majority of responses acknowledged the possibility of AI “breaching fundamental rights” and that use of AI may lead to discriminatory outcomes. “Ninety per cent and 87 per cent of respondents [respectively] find these concerns important or very important,” the commission said.

Contributions arrived from all over the world, including the EU’s 27 member states and India, China, Japan, Syria, Iraq, Brazil, Mexico, Canada, the US and the UK.

The post Opinion almost equally divided on new rules for ‘high-risk’ artificial intelligence appeared first on Artificial Intelligence.

In recent years, a growing number of researchers have been developing artificial neural network (ANN)- based models that can be trained using a technique known as reinforcement learning (RL). RL entails training artificial agents to solve a variety of tasks by giving them “rewards” when they perform well, for instance, when they classify an image correctly.

So far, most ANN-based models were trained employing online RL methods, where an agent that was never exposed to the task it is designed to complete learns by interacting with an online virtual environment. However, this approach can be quite expensive, time-consuming and inefficient.

More recently, some studies explored the possibility of training models offline. In this case, an artificial agent learns to complete a given task by analyzing a fixed dataset, and thus does not actively interact with a virtual environment. While offline RL methods have achieved promising results on some tasks, they do not allow agents to learn actively in real time.

Researchers at UC Berkeley recently introduced a new algorithm that is trained using both online and offline RL approaches. This algorithm, presented in a paper pre-published on arXiv, is initially trained on a large amount of offline data, yet it also completes a series of online training trials.

“Our work focuses on a scenario that that lies between two cases that we face constantly in real-world robotics settings,” Ashvin Nair, one of the researchers who carried out the study, told TechXplore. “Often, when trying to solve robotics problems, researchers have some prior data (for instance, a few expert demonstrations of how to solve the task or some data from the last experiment you performed) and want to leverage the prior data to solve the task partially, but then be able to fine-tune the solution to master it with a small number of interactions.”

While reviewing past RL literature, Nair and his colleagues realized that previously developed models did not perform well when they were first trained offline and then fine-tuned online. This was typically because they learned too slowly or did not make the best use of offline datasets during training.

In their study, the researchers studied the limitations of existing models in depth and then devised an algorithm that could overcome these issues. The algorithm they created can achieve satisfactory performance when pre-trained on large quantities of data offline. This allows it to quickly master the task it is designed to complete at a later stage, when it is actively trained in a virtual online environment.

“Our paper addresses a common problem that was stalling our progress: that we were always making robots learn tasks from scratch rather than being able to use existing datasets for RL,” Nair explained. “It actually came about as a result of realizing that our experiment cycles for a separate idea was taking too long and too much effort to evaluate running on a robot in the real world, and we needed a way to evaluate the idea by pre-training on data we already had and doing only a small amount of extra real-world interaction.”

Nair and his colleagues identified three key limitations of previously developed models trained via RL. First, they observed that on-policy techniques such as advantage weighted regression (AWR) and demonstration augmented policy gradient (DAPG), which are often used to fine-tune models online, typically learn quite slowly compared to off-policy methods.

In addition, the researchers observed that off-policy methods, such as soft actor critic (SAC) approaches, often did not improve much when trained on offline datasets. Finally, they found that techniques to train models offline, such as bootstrap error accumulation reduction (BEAR), behavior regularized actor critic (BRAC) and advantage behavior models (ABM) typically worked well in the offline pre-training stage, but their performance did not improve much when they were trained online. This is primarily because they rely on behavior models, which work well when trying to outline the general distribution of data and learning policies accordingly, but not as well when fine-tuning models in online environments.

“Confronted with these challenges, we developed advantage weighted actor critic (AWAC), which is an off-policy actor-critic algorithm that does not rely on a behavior model to stay close to the data distribution,” Nair said. “Instead, we show that we can derive an algorithm that implicitly stays close to the data by sampling.”

The AWAC algorithm developed by Nair and his colleagues can be pre-trained just as well offline as techniques that are specifically designed for offline training. However, its performance improves further and by a significant margin when it is trained online.

The researchers evaluated their algorithm’s performance on different dexterous manipulation tasks characterized by three key aspects, namely complex discontinuous contacts, very sparse binary rewards and the control of 30 joints. More specifically, their algorithm was trained to control a robot’s movements, allowing it to rotate a pen in its hand, open a door or pick up a ball and move it to a desired location. For each task, Nair and his colleagues trained the algorithm on an offline dataset containing 25 human demonstrations and 500 trajectories of off-policy data, attained using a technique known as behavioral cloning.

“The first task, pen rotation, is relatively simpler and many methods eventually solve the task, but AWAC is the fastest,” Nair said. “Only AWAC solves the second and third task. Prior methods fail, for a myriad of reasons, centered around their inability to obtain a reasonable initial policy to collect good exploration data, or their inability to learn online from interaction data.”

Nair and his colleagues compared their AWAC algorithm to eight other methods trained via offline or online RL and found that it was the only one that could consistently solve the difficult manipulation tasks they tested it on. Their algorithm could also solve simpler MuJoCo benchmark tasks and a pushing task faster than previously developed methods, learning from suboptimal, randomly generated data.

In the future, the algorithm could enable the use of RL to train models on a far wider range of tasks. Other research teams could also draw inspiration from their work and devise similar RL approaches that combine offline and online training.

“Going forward, we plan to use AWAC to speed up experiments and stabilize training on new tasks by taking advantage of existing data,” Nair said. “The direction we are really excited about is to scale up the amount of data that we use for RL so that can start seeing significant generalization across tasks and visual and physical characteristics of objects.”

The post An algorithm that merges online and offline reinforcement learning appeared first on Artificial Intelligence.