Designing algorithms for more challenging data

Machine Learning researchers often have to overcome the ‘sim-to-real’ transfer, where algorithmic feats accomplished in computer simulations can be repeated in test performances. DESIRE has produced a data-driven, robust decision-making algorithm to achieve just that.

Advancements in computing, such as the game AlphaGo, both rely on and generate large amounts of data. To cater for this volume of data, researchers depend on Machine Learning (ML) algorithms developed from techniques such as Reinforcement Learning (RL), alongside Artificial Intelligence (AI) breakthroughs. However, while these algorithms can be effective within simulations, they often prove disappointing in the real world. Such performance failures matter in high-stakes areas such as robotics where, for reasons of practicality and expense, only a limited number of trials can be undertaken. The EU-supported DESIRE project set out to improve the robustness of the optimisation, learning and control algorithms underlying many innovations striving for autonomous control.

Kernel DRO

One of the key problems in the sim-to-real transfer is an ML phenomenon called ‘distribution shift’. Put simply, this is when a discrepancy appears between the distribution of data in the datasets used for training and those used for testing in the real world. “This is usually because the test datasets prove to be too simplistic in their rendering of real-world conditions,” says research fellow Jia-Jie Zhu who received support from the Marie Skłodowska-Curie Actions programme. “Distribution shift has been one of the major problems plaguing learning and control algorithms and a stumbling block to progress,” adds Zhu, from the Max Planck Institute for Intelligent Systems (the project host). The DESIRE project drew upon so-called kernel-based learning methods to reduce this distribution drift. These are computations which make algorithms more reliable by recognising patterns in data, identifying and then organising relations within the data according to predetermined features such as correlations or classifications. This enabled DESIRE to create an algorithm employing kernel distributionally robust optimisation (Kernel-DRO), in which decisions, such as control commands for robots, were robustly determined.

Broad applicability

While DESIRE’s work is theoretical, besides contributing to the literature of mathematical optimisation, control and ML theory, it has a range of very practical implications. Indeed, a strength of the team’s Kernel-DRO solution is precisely this broad applicability. “Many of today’s learning tasks suffer from data distribution ambiguity. We believe that industry or business practitioners looking to improve robustness in their machine learning can easily apply our algorithm,” explains Zhu. To take the work further, Zhu is now aiming to create larger-scale learning algorithms which can cater for more random data inputs, suitable for industrial applications. For example, the principle of data robustness is being applied to model predictive control, a highly effective control method useful for safety-critical applications such as flight control, chemical process control and robotics.

The post Data-Efficient Scalable Reinforcement Learning for Practical Robotic Environments appeared first on Artificial Intelligence.

In a recent paper published on the preprint server Arxiv.org, researchers at Carnegie Mellon, Facebook, and the University of Illinois Urbana-Champaign propose Active Neural Simultaneous Localization and Mapping (Active Neural SLAM), a hierarchical approach for teaching AI agents to explore environments. They say that it leverages the strength of both classical and AI-based path- and goal-planning methods, making it robust against errors and sidestepping the complexities associated with previous approaches.

Techniques like those underpinning Active Neural SLAM could greatly advance the state of the art in robotics. Navigation, which in this context refers not only to coordinate navigation but to pathfinding (i.e., finding paths to objects), is a critical task for autonomous machines. But training those machines to learn about mapping requires a lot of computation.

Active Neural SLAM, then, works with raw sensory inputs such as camera images and exploits regularities in the layouts of environments, enabling it to achieve the same or better performance than existing methods while requiring a fraction of the training data.

The neural SLAM module within Active Neural SLAM comprises a Mapper and a Pose Estimator. The Mapper is responsible for generating a top-down spatial map of a given environment and predicting obstacles and explored areas, while the Pose Estimator anticipates the agent’s pose based on past pose estimates. The spatial map — where each element corresponds to a cell size of 25 square centimeters in the physical world — is ingested along with the agent pose by a global policy to produce various long-term goals. A Planner model then takes the goals, the spatial obstacle map, and the agent pose estimates to compute short-term goals, or the shortest paths from the current location to the long-term goals. Lastly, a local policy outputs navigational actions using camera data and the short-term goals.

In experiments, the researchers paired Facebook’s open source Habitat platform, a modular high-level library for training agents across a variety of tasks, environments, and simulators, with data sets (Gibson and Matterport’s MP3D) consisting of 3D reconstructions of real-world environments like office and home interiors. Agents could make one of three moves — forward 25 centimeters, leftward 10 degrees, or rightward 10 degrees — in the environments and were trained in 994 episodes consisting of 1,000 steps or 10 million frames, such that all of Active Neural SLAM’s components — the Mapper, the Pose Estimator, the global policy, and the local policy — were trained simultaneously.

The team reports that the Active Neural SLAM managed to almost completely explore small scenes in around 500 steps versus the baselines’ 85% to 90% exploration of the same scenes in 1,000 steps. The baseline models also tended to become stuck in areas, indicating that they weren’t able to “remember” explored areas over time — a problem that Active Neural SLAM didn’t exhibit.

Encouraged by these results, the coauthors deployed the trained Active Neural SLAM policy from simulation to a real-world Locobot robot. After adjusting the camera height and vertical field-of-views to match those of the Habitat simulator, they say that it successfully explored the living area in an apartment.

“In the future, [Active Neural SLAM] can be extended to complex semantic tasks such as semantic goal navigation and embodied question answering by using a semantic Neural SLAM module, which creates a … map capturing semantic properties of the objects in the environment,” wrote the coauthors. “The model can also be combined with prior work on localization to relocalize in a previously created map for efficient navigation in subsequent episodes.”

The post Facebook’s AI teaches robots to navigate environments using less data appeared first on Artificial Intelligence.

The post Seven Key Dimensions to Help You Understand Artificial Intelligence Environments appeared first on Artificial Intelligence.