As larger neural networks with more layers and nodes are considered, reducing their storage and computational cost becomes critical, especially for some real-time applications such as online learning and incremental learning

In addition, recent years witnessed significant progress in virtual reality, augmented reality, and smart wearable devices, creating challenges in deploying deep learning systems to portable devices with limited resources (e.g. memory, CPU, energy, bandwidth).

Here are a few methods that are part of all compression techniques:

Parameter Pruning And Sharing

• Reducing redundant parameters which are not sensitive to the performance
• Robust to various settings
• Redundancies in the model parameters are explored and the uncritical yet redundant ones are removed

Low-Rank Factorisation

• Uses matrix decomposition to estimate the informative parameters of the deep convolutional neural networks

Transferred/Compact Convolutional Filters

• Special structural convolutional filters are designed to reduce the parameter space and save storage/computation

Knowledge Distillation

• A distilled model is used to train a more compact neural network to reproduce the output of a larger network

Now let’s take a look at a few papers that introduced novel compression models:

1.Deep Neural Network Compression with Single and Multiple Level Quantization

In this paper, the authors propose two novel network quantization approaches single-level network quantization (SLQ) for high-bit quantization and multi-level network quantization (MLQ).

The network quantization is considered from both width and depth level.

2.Efficient Neural Network Compression

In this paper the authors proposed an efficient method for obtaining the rank configuration of the whole network. Unlike previous methods which consider each layer separately, this method considers the whole network to choose the right rank configuration.

3.3LC: Lightweight and Effective Traffic Compression

3LC is a lossy compression scheme developed by the Google researchers that can be used for state change traffic in distributed machine learning (ML) that strikes a balance between multiple goals: traffic reduction, accuracy, computation overhead, and generality. It combines three techniques — value quantization with sparsity multiplication, base encoding, and zero-run encoding.SEE ALSO

DEVELOPERS CORNER

WHAT IS DATAOPS? THINGS AROUND IT THAT YOU NEED TO KNOW

4.Universal Deep Neural Network Compression

This work for the first time, introduces universal DNN compression by universal vector quantization and universal source coding. In particular, this paper examines universal randomised lattice quantization of DNNs, which randomises DNN weights by uniform random dithering before lattice quantization and can perform near-optimally on any source without relying on knowledge of its probability distribution.

5.Compression using Transform Coding and Clustering

The compression (encoding) approach consists of transform and clustering with great encoding efficiency, which is expected to fulfill the requirements towards the future deep model communication and transmission standard. Overall, the framework works towards light weight model encoding pipeline with uniform quantization and clustering has yielded great compression performance, which can be further combined with existing deep model compression approaches towards light-weight models.

6.Weightless: Lossy Weight Encoding

The encoding is based on the Bloomier filter, a probabilistic data structure that saves space at the cost of introducing random errors. The results show that this technique can compress DNN weights by up to 496x; with the same model accuracy, this results in up to a 1.51x improvement over the state-of-the-art.

7.Adaptive Estimators Show Information Compression

The authors developed more robust mutual information estimation techniques, that adapt to hidden activity of neural networks and produce more sensitive measurements of activations from all functions, especially unbounded functions. Using these adaptive estimation techniques, they explored compression in networks with a range of different activation functions. 

8.MLPrune: Multi-Layer Pruning For Neural Network Compression

It is computationally expensive to manually set the compression ratio of each layer to find the sweet spot between size and accuracy of the model. So,in this paper, the authors propose a Multi-Layer Pruning method (MLPrune), which can automatically decide appropriate compression ratios for all layers.

Large number of weights in deep neural networks make the models difficult to be deployed in low memory environments. The above-discussed techniques achieve not only higher model compression but also reduce the compute resources required during inferencing. This enables model deployment in mobile phones, IoT edge devices as well as “inferencing as a service” environments on the cloud.

The post 8 NEURAL NETWORK COMPRESSION TECHNIQUES FOR ML DEVELOPERS appeared first on Artificial Intelligence.

Turns out people are not that great at tracking thousands of variables

American upstart Databricks, established by the original authors of the Apache Spark framework, reckons its open-source machine-learning management engine MLflow is ready for prime time.

The released version 1.0 of the platform focuses on core API components. It improves the handling of metrics and search functionality, and adds support for Hadoop as an artifact store, in addition to the previously supported Amazon S3, Azure Blob Storage, Google Cloud Storage, SFTP, and NFS.

It also adds an experimental Open Neural Network Exchange (ONNX) model flavour, and a CLI command for building a Docker image capable of serving an MLflow model.

And finally, there’s Windows support for the MLflow client – in the unlikely event data scientists decide to opt for something other than Linux.

MLflow enables data scientists to track and distribute experiments, package and share models across frameworks, and deploy them – no matter if the target environment is a personal laptop or a cloud data centre.

The company launched the alpha version of MLflow project last year at the Spark + AI Summit.

Multiple code approaches

The basic machine learning life cycle – taking raw data, preparing it, training your model and deploying it – is full of variables and fraught with complications. It can involve hundreds of different open source tools and frameworks, each with dozens of configurable parameters.

Facebook, Google and Uber have all built their own proprietary tools to deal with this complexity.

MLflow was designed to take some of the pain out of machine learning in organizations that don’t have the coding and engineering muscle of the hyperscalers. It works with every major ML library, algorithm, deployment tool and language.

One of the project’s goals is to improve collaboration between data scientists and engineers that deploy their creations in production.

In a true open source fashion, MLflow users didn’t wait for a stable release to start experimenting: Databricks says the platform has already been deployed at thousands of organizations to manage their machine learning workloads, and the company is offering it as a managed service.

Group effort

Databricks might have started the project, but today, it has more than 100 contributors, including a few from Microsoft.

“People are excited about having an open-source project in this space,” Mattei Zacharia, co-founder and chief technologist of Databricks, told El Reg last year.

“They’re excited about having an ML platform – it’s something that resonates with them, and that many wanted to build already – and having one that is a community effort will be much better than what any company could build on its own.”

The next major addition to MLflow will be a Model Registry that allows users to manage their ML model’s lifecycle from experimentation to deployment and monitoring.

The post Databricks wants one tool to rule all AI systems – coincidentally, its own MLflow tool appeared first on Artificial Intelligence.