As known to many, Google has recently released Model Search which is an open-source platform. This caters to developing efficient and best machine learning models automatically. Rather than focusing on a particular domain, Model Search is domain agnostic and flexible beyond imagination. Well, not just that. To our surprise, it is even capable of finding just the right architecture. With this, it will best fit a given dataset and the associated problem. At the same time, it is mastered enough to minimize the time that goes behind in coding, the effort as well as the resources that are put in.

Model Search is built on Tensorflow. It is flexible to the extent that it can run either on a single machine or in a distributed setting. This feature does set it apart from the rest, without any doubt. It is equipped with multiple trainers, a search algorithm, a transfer learning algorithm and a database as well that aims at storing the various evaluated models.

The Architecture of Model Search

Talking about the architecture of Model Search, it is based on four foundational components: They are –

1. Model Trainers:As evident as it can get, these components cater to training and evaluating the various models asynchronously.
2. Search Algorithms: With this component of a search algorithm, it is possible to select the best trained architectures. It doesn’t end there. There’s also an option for the user to add some “mutations” to it which can be sent to the trainers for further evaluation.
3. Transfer Learning Algorithm:Model Search boasts of using transfer learning techniques such as knowledge distillation. This further brings in an advantage of reusing knowledge across different experiments. Model Search enables this using two ways – knowledge distillation or weight sharing. The former allows improving the accuracy by adding a loss term that matches the predictions of the high-performing models. On the other hand, the latter bootstraps some of the network’s parameters from previously trained candidates.
4. Model Database:This is where the results of the experiments can be persisted in such a way that it can be reused on different cycles.

What Makes Model Search So Unique?

• Now, that one aspect which makes people look forward to Model Search is its ability to run training and evaluation experiments for AI models in an adaptive and asynchronous fashion. This ability paves the way for the trainers to share the knowledge that they’ve gained from their experiments. The working is such that at the beginning of every cycle, the search algorithm closely monitors all the completed trials. After this, what follows is deciding what to try next. This platform makes use of beam search while deciding what is to be tried next. The next step that follows is to invoke mutation and assign the resulting model back to a trainer.
• When the users go for a Model Search run, they are in a position to compare the many models found during the search. Well, there’s more to this. The platform also allows the users to create their own search space. With this, it is possible for them to customize the architectural elements in their models.
• The researchers have also claimed that Model Search improves upon production models with minimal iterations.

All in all, The Model Search Code aims to provide the researchers with a flexible, domain-agnostic framework to develop the most efficient and best machine learning model. Also, the framework is so powerful that it can build models with state-of-the-art performance. It also has the capability to deal with well-known problems when provided with a search space composed of standard building blocks.

The post MODEL SEARCH: A PLATFORM FOR FINDING OPTIMAL ML MODELS appeared first on Artificial Intelligence.

Google AI has announced the release of Model Search, a platform that will help researchers develop machine learning (ML) models automatically and efficiently. Model Search isn’t domain-specific, flexible, and well equipped to find the appropriate architecture that best fits a given dataset and problem. At the same time, it minimizes the coding time, effort, and resources. Model Search is built on Tensorflow and can run on both distributed settings or a single machine.

The Success of neural networks often depends on the extent to which they can generalize to various tasks. It is challenging to design Neural networks that can generalize well as the research community’s understanding of this concept is limited. The limitations become complicated when Machine Learning domains are taken into consideration. Techniques like neural architecture search (NAS) use algorithms, reinforcement learning (RL), evolutionary algorithms, and combinatorial search to build a neural network from a given search space. Although these techniques can deliver results better than their manually designed counterparts, these algorithms usually compute heavily and need thousands of models to train before converging and are domain-specific.

These shortcomings can be overcome by using Model Search. The Model Search System is built up of multiple trainers, a search algorithm, and a database to store evaluated models. The system can run both training and evaluation experiments in an adaptive yet asynchronous manner. Each trainer conducts experiments on their own, and all the trainers share knowledge from their experiments. At the starting of every cycle, the search algorithm goes over all the completed trials and then uses beam search to determine what to try next. It then implores mutation over one of the best architectures it finds and assigns the resulting model back to a trainer.

The neural network is built from a set of predefined blocks. This approach is more efficient as it explores only structures and not their fundamental and detailed components, thereby reducing the search space scale. As the framework is built on Tensorflow, blocks can implement any function that takes a tensor as an input. Moreover, the blocks provided can be fully defined neural networks that are already known to work for the given problem. In this case, Model Search can be configured to act as a powerful ensembling machine. The search algorithms used in Model Search are adaptive, greedy, and incremental making them converge faster than RL algorithms.

To improve efficiency and accuracy, Model Search enables transfer learning between various internal experiments in two ways: knowledge distillation or weight sharing. Knowledge distillation allows improving candidates’ accuracy by adding a loss term that matches the high-performing models’ predictions in addition to the ground truth. In contrast, Weight sharing bootstraps some of the network’s parameters from previously trained candidates by copying suitable weights from once trained models and randomly initializing the remaining ones.

The researchers claim that Model Search improves upon production models with minimal iterations. They illustrated Model Search’s capabilities in the speech domain by discovering a model for keyword spotting and language identification. It used fewer than 200 iterations and was found to improve efficiency. The researchers also applied Model Search to find an architecture suitable for image classification on the heavily explored CIFAR-10 imaging dataset. They observed that they were quickly able to reach a benchmark accuracy of 91.83 in only 209 trials as compared to 5807 trials for the RL algorithm.

The Model Search Code aims to provide the researchers with a flexible, domain-agnostic framework for ML model discovery. The framework is powerful enough to build models with state-of-the-art performance on well-known problems when provided with a search space composed of standard building blocks. The code extends access to AutoML solutions to the ever-flourishing research community.

The post Google AI Introduces ‘Model Search’: An Open Source Platform For Finding Optimal Machine learning (ML) Models appeared first on Artificial Intelligence.