Facebook AI Research (FAIR) research on meta-learning has majorly classified into two types:  First, methods that can learn representation for generalization. Second, methods that can optimize models. We have thoroughly discussed the type first in our previous article MBIRL. For this post, we are going to give a brief introduction to the second type. Last month, at the International Conference on Pattern Recognition, {ICPR}, Italy, January 10-15, 2021, a group of researchers: S. Bechtle,  A. Molchanov, Y. Chebotar, E. Grefenstette, L. Righetti, G. S. Sukhatme, F. Meier submitted a research paper focussing on the automation of “meta-training” processing: Meta Learning via Learned Loss.

Motivation Behind ML³

In meta-learning, the goal is to efficiently optimize the function f_θ  which can be a regressor or classifier that finds the optimal value of θ. L is the loss function and h is the gradient transform. The majority of the work in deep learning is associated with learning the f function directly from data and some meta-learning work focuses on the parameter updation. In ML³ approach, the authors have targeted loss learning. Loss functions are architecture independent and widely used for learning problems so learning a loss function doesn’t require any engineering and optimization and allows the addition of extra information during meta-training.

The key idea of the proposed framework is to develop a pipeline for meta-training that not only can optimize the performance of the model but also generalize for different tasks and model architectures. The proposed framework of learning loss functions efficiently optimize the models for new tasks. The main contribution of the ML³ framework are :

i) It is capable of learning adaptive, high-dimensional functions via back propagation and gradient descent.

ii) The given framework is very flexible as it is capable of storing additional information at the meta-train time and provides generalization by solving regression, classification, model-based reinforcement learning, model-free reinforcement learning.

The Model Architecture of ML³

The task of learning a loss function is based on a bi-level optimization technique i.e., it contains two optimization loops: inner and outer. The inner loop is responsible for training the model or optimizee with gradient descent by using the loss function learners meta-loss function and the outer loop optimized the meta-loss function by minimizing the task loss i.e., regression or classification or reinforcement learning loss. 

The process contains a function f parameterized by θ that takes a variable x and outputs y. It also learns meta-loss network M parameterized by Φ  that takes the input and output of function f and together with task-specific information g (for example ground truth label for regression or classification, final position in MBIRL or the sample reward from model-free reinforcement learning problems) and outputs the meta- loss function L parameterized by both Φ and θ. 

So, to update function f, compute the gradient of Meta Loss L with respect to θ and update the gradient using the learned loss function, as shown below :

Now, to update M, the loss network, formulate a task-specific loss that compares the output of the currently optimal f with the target information since f is updated with L, the task is also functional Φ and perform gradient update on Φ to optimize M. This architecture finally forms a fully differential loss learning framework used for training.

To use the learning loss  at Test time, directly update f by taking the gradient of learned loss L with respect to the parameters of f.

Applications of ML³

1. Regression problems.
2. Classification problems.
3. Shaping Loss during training e.g., Covexifying Loss, exploration signal. ML³ provides a possibility to add additional information during meta-training.
4. Model-based Reinforcement Learning.
5. Model-free Reinforcement Learning.

Requirements & Installation

1. Python=3.7
2. Clone the Github repository via git.
3. Install all the dependencies of ML³ via :

Paper Experiment Demos

This section contains different experiments mentioned in the research paper.

A. Loss Learning for Regression

1. Run Sin function regression experiment by code below:

2. Now, you can visualize the results by the following code:

2.1 Import the required libraries, packages and modules and specify the path to the saved data during meta-training. The code snippet is available here.

2.2 Load the saved data during the experiment.

2.3 Visualize the performance of the meta loss when used to optimize the meta training tasks, as a function of (outer) meta training iterations.

2.4 Evaluating learned meta loss networks on test tasks. Plot the performance of the final meta loss network when used to optimize the new test tasks at meta test time. Here the x-axis represents the number of gradient descent steps. The code snippet is available here.

C. Learning with extra information at the meta-train time

This demo shows how we can add extra information during meta training in order to shape the loss function. For experiment purposes, we have taken the example of sin function. Now, with the code, the script requires two arguments, first one is train\test, the 2nd one indicates whether to use extra information by setting True\False (with\without extra info).

1. For training, the code is given below
2. To test the loss with extra information run:

3. For comparison purposes, we have repeated the above two steps with argument as False. The full code is available here.
4. Comparison of results via visualization.

Similarly, the research experiment for meta learning the loss with an additional goal in the mountain car experiment run can be done. The code lines is available here.

EndNotes

In this write-up we have given an overview Meta Learning via Learned Loss(ML³), a gradient-based bi-level optimization algorithm which is capable of learning any parametric loss function as long as the output is differential with respect to its parameters. These learned loss functions can be used to efficiently optimize models for new tasks.

Note : All the figures/images except the output of the code are taken from official sources of ML³.

• Colab Notebook ML³ Demo

Official Code, Documentation & Tutorial are available at:

• Github 
• Website 
• Research Paper

The post What Is Meta-Learning via Learned Losses (with Python Code) appeared first on Artificial Intelligence.

Machine learning (ML) isn’t a replacement for conventional programming, but it’s one more tool that developers have available when creating solutions. Unfortunately, the scope and variations of ML are complex and changing continually as designers refine existing methodologies and develop new ones. This would be less of an issue if the changes weren’t occurring so frequently and if ML had less of an impact. Often, ML is the edge that companies need to stay ahead of the competition.

New models and approaches are being created in the neural-network space where most of the latest ML technology is found, even as hardware designers rush to incorporate ML accelerators in their latest releases. Few processor, GPU, or networking chips lack some sort of ML acceleration these days. These can often provide increased speeds on the order of multiple magnitudes, making many applications possible.

The advantage of the plethora of options is that tools like Xnor’s AI2Go and H2O.ai’s H20are allowing developers to choose ML models based on applications and criteria. These can be quickly incorporated into an application. They frequently provide services to train models for a particular application, too.

Machine learning (ML) isn’t a replacement for conventional programming, but it’s one more tool that developers have available when creating solutions. Unfortunately, the scope and variations of ML are complex and changing continually as designers refine existing methodologies and develop new ones. This would be less of an issue if the changes weren’t occurring so frequently and if ML had less of an impact. Often, ML is the edge that companies need to stay ahead of the competition.

New models and approaches are being created in the neural-network space where most of the latest ML technology is found, even as hardware designers rush to incorporate ML accelerators in their latest releases. Few processor, GPU, or networking chips lack some sort of ML acceleration these days. These can often provide increased speeds on the order of multiple magnitudes, making many applications possible.

The advantage of the plethora of options is that tools like Xnor’s AI2Go and H2O.ai’s H20are allowing developers to choose ML models based on applications and criteria. These can be quickly incorporated into an application. They frequently provide services to train models for a particular application, too.

Likewise, combining ML models can often be similar to combining filters in DSP applications. Sometimes it’s as simple as connecting a few blocks together sequentially, but other times the interconnections and interactions can be as complex as the ML models themselves. More applications are utilizing multiple models rather than a single model that addresses just one aspect of the application.

Also, ML is being applied to all aspects of application design, deployment, and maintenance in many cases. ML support is part of most cloud services and management systems these days. ML models are often employed in apps and cloud services given the growth of IoT in embedded applications.

ML tools and models aren’t a panacea for embedded developers. They’re also not a solution for all applications. In fact, many embedded and IoT applications will not benefit incorporating ML support. By the same token, ML can be a time and money sink with little payoff, especially when the technology is misunderstood or not applicable.

It’s a good idea for developers and managers to get more educated about ML and the options available. Therefore, they can determine whether or when it might be applicable to your needs. Remember, neural networks are all about probabilities, so their relevance isn’t always a binary choice.

The post Have We Learned Machine Learning Yet? appeared first on Artificial Intelligence.