AI and ML solutions are already being used by thousands of companies with the goal of improving the healthcare experience. For example, Babylon Health is changing the way we manage and better understand health. Founder, Ali Parsa developed the app in 2013 with a mission of providing accessible and affordable healthcare to every individual on earth. Babylon’s AI system has been designed to understand and recognise the way humans express their medical symptoms and it can interpret symptoms and medical questions through a chatbot interface and match them to the most appropriate service. It can recognise most healthcare issues seen in primary care and provide information on next steps to take.

The conversation around artificial intelligence (AI) and machine learning (ML) in healthcare continues to grow. Research in cutting-edge areas like machine learning continues to demonstrate that computers have the potential to predict outcomes and optimise clinical operations in a wide variety of settings.

Healthcare stands poised for a transformation driven by AI and ML, and fuelled by an abundance of data sources – electronic health records, claims data, genomic sequences, mobile devices, medical imaging, and even embedded sensor data.

• How to build healthcare around IoT

Building a foundation for AI and ML

Data is the fundamental raw material required to power AI and ML systems, and is an essential ingredient that enables healthcare organisations to increase efficiency, improve outcomes, and enhance quality of life for both patients and providers.

While the demands of treating patients and developing new therapies often relegate data collection and analysis to a back burner in healthcare, new tools enable developers to integrate ML and other capabilities easily into the routine process of developing and delivering treatments. Far from being an exclusive province of researchers and technology companies, AI and ML are now accessible to all.

As these use cases expand, success is dependent on several ingredients. First, such initiatives require large quantities of carefully curated, high-quality data, which may be hard to come by in healthcare where data is often complex and unstructured. High-quality data sets are required not only to operate AI and ML-driven systems, but even more importantly, to feed the training models upon which they are built.

Second, these systems need to be optimised for the compute-intensive jobs typically required by AI applications. And finally, IT resources supporting AI applications must comply with industry standards and regulations and adhere to the highest security and privacy standards to protect patient and other sensitive data.

One company that has successfully rooted itself in developing and curating its data is Touch Surgery The company is transforming professional healthcare training through the delivery of a unique platform that links mobile apps with powerful data back-end. Touch Surgery uses cognitive mapping techniques coupled with cutting edge AI and 3D rendering technology to codify surgical procedures. They have partnered with leaders in Virtual Reality and Augmented Reality to work toward a vision of advancing surgical care in the operating room. With over 1 million users, the firm are recording vast amounts of usage data to power their data analytics product, which in turn allows users to learn and practice over 50 surgical procedures, evaluate and measure progress, and connect with physicians across the world.

• New frontiers: Healthcare’s digital path forward

Powering Innovation

A crucial technology that provides storage capacity, compute elasticity, security, and analytic capabilities needed to implement AI and ML – and drive innovation – is cloud computing. Cloud computing platforms make it easy to ingest and process data, whether structured, unstructured, or streaming and simplifies the process of building, training, and deploying machine learning-based models. Healthcare organisations that can use cloud computing to make themselves more efficient and effective will be the most successful in coming years, particularly as the industry shifts to value-based care.

For the National Health Service (NHS), AI and ML are having a huge impact on its ability to cut costs, while improving patient services. The NHS is the UK’s largest employer and health provider. NHS Business Services Authority (NHS BSA), a Special Health Authority and an Arm’s Length Body of the Department of Health and Social Care, provides a range of critical central services to NHS organisations, NHS contractors, patients and the public. As such, the NHS BSA’s call centre staff handle around five million calls per year. The organisation decided to implement a cloud-based contact centre and deep learning chatbot service using Amazon Connect and Amazon Lex to help improve the user experience, reduce call centre load, increase efficiency and cut costs. By moving to the cloud, the NHS BSA has identified around $650,000 in cost savings per annum from a reduction in average call times alone.

Healthcare companies, whether established or new start-ups, are increasingly looking to AI and ML to drive innovation and transformation at their company and across the healthcare industry. These organisations share a common goal of reducing time to discovery and insight, improving care quality and enhancing the patient and provider experience. As the availability and volume of data sources continue to grow, the essential ingredients for AI and ML success will remain the same: high-quality data, cloud computing to remove undifferentiated heavy lifting, and ML services accessible to everyday developers. Once these foundational elements are established, AI and ML have the potential to power more efficient and effective care, enhanced decision making and the ability to drive greater value for patients and providers.

The post The transformation of healthcare with AI and machine learning appeared first on Artificial Intelligence.

As larger neural networks with more layers and nodes are considered, reducing their storage and computational cost becomes critical, especially for some real-time applications such as online learning and incremental learning

In addition, recent years witnessed significant progress in virtual reality, augmented reality, and smart wearable devices, creating challenges in deploying deep learning systems to portable devices with limited resources (e.g. memory, CPU, energy, bandwidth).

Here are a few methods that are part of all compression techniques:

Parameter Pruning And Sharing

• Reducing redundant parameters which are not sensitive to the performance
• Robust to various settings
• Redundancies in the model parameters are explored and the uncritical yet redundant ones are removed

Low-Rank Factorisation

• Uses matrix decomposition to estimate the informative parameters of the deep convolutional neural networks

Transferred/Compact Convolutional Filters

• Special structural convolutional filters are designed to reduce the parameter space and save storage/computation

Knowledge Distillation

• A distilled model is used to train a more compact neural network to reproduce the output of a larger network

Now let’s take a look at a few papers that introduced novel compression models:

1.Deep Neural Network Compression with Single and Multiple Level Quantization

In this paper, the authors propose two novel network quantization approaches single-level network quantization (SLQ) for high-bit quantization and multi-level network quantization (MLQ).

The network quantization is considered from both width and depth level.

2.Efficient Neural Network Compression

In this paper the authors proposed an efficient method for obtaining the rank configuration of the whole network. Unlike previous methods which consider each layer separately, this method considers the whole network to choose the right rank configuration.

3.3LC: Lightweight and Effective Traffic Compression

3LC is a lossy compression scheme developed by the Google researchers that can be used for state change traffic in distributed machine learning (ML) that strikes a balance between multiple goals: traffic reduction, accuracy, computation overhead, and generality. It combines three techniques — value quantization with sparsity multiplication, base encoding, and zero-run encoding.SEE ALSO

DEVELOPERS CORNER

WHAT IS DATAOPS? THINGS AROUND IT THAT YOU NEED TO KNOW

4.Universal Deep Neural Network Compression

This work for the first time, introduces universal DNN compression by universal vector quantization and universal source coding. In particular, this paper examines universal randomised lattice quantization of DNNs, which randomises DNN weights by uniform random dithering before lattice quantization and can perform near-optimally on any source without relying on knowledge of its probability distribution.

5.Compression using Transform Coding and Clustering

The compression (encoding) approach consists of transform and clustering with great encoding efficiency, which is expected to fulfill the requirements towards the future deep model communication and transmission standard. Overall, the framework works towards light weight model encoding pipeline with uniform quantization and clustering has yielded great compression performance, which can be further combined with existing deep model compression approaches towards light-weight models.

6.Weightless: Lossy Weight Encoding

The encoding is based on the Bloomier filter, a probabilistic data structure that saves space at the cost of introducing random errors. The results show that this technique can compress DNN weights by up to 496x; with the same model accuracy, this results in up to a 1.51x improvement over the state-of-the-art.

7.Adaptive Estimators Show Information Compression

The authors developed more robust mutual information estimation techniques, that adapt to hidden activity of neural networks and produce more sensitive measurements of activations from all functions, especially unbounded functions. Using these adaptive estimation techniques, they explored compression in networks with a range of different activation functions. 

8.MLPrune: Multi-Layer Pruning For Neural Network Compression

It is computationally expensive to manually set the compression ratio of each layer to find the sweet spot between size and accuracy of the model. So,in this paper, the authors propose a Multi-Layer Pruning method (MLPrune), which can automatically decide appropriate compression ratios for all layers.

Large number of weights in deep neural networks make the models difficult to be deployed in low memory environments. The above-discussed techniques achieve not only higher model compression but also reduce the compute resources required during inferencing. This enables model deployment in mobile phones, IoT edge devices as well as “inferencing as a service” environments on the cloud.

The post 8 NEURAL NETWORK COMPRESSION TECHNIQUES FOR ML DEVELOPERS appeared first on Artificial Intelligence.

eep learning” is the new buzzword in the field of artificial intelligence. As Natalie Wolchover reported in a recentQuanta Magazine article, “‘deep neural networks’ have learned to converse, drive cars, beat video games and Go champions, dream, paint pictures and help make scientific discoveries.” With such successes, one would expect deep learning to be a revolutionary new technique. But one would be quite wrong. The basis of deep learning stretches back more than half a century to the dawn of AI and the creation of both artificial neural networks having layers of connected neuronlike units and the “back propagation algorithm” — a technique of applying error corrections to the strengths of the connections between neurons on different layers. Over the decades, the popularity of these two innovations has fluctuated in tandem, in response not just to advances and failures, but also to support or disparagement from major figures in the field. Back propagation was invented in the 1960s, around the same time that Frank Rosenblatt’s “perceptron” learning algorithm called attention to the promise of artificial neural networks. Back propagation was first applied to these networks in the 1970s, but the field suffered after Marvin Minsky and Seymour Papert’s criticism of one-layer perceptrons. It made a comeback in the 1980s and 1990s after David Rumelhart, Geoffrey Hinton and Ronald Williams once again combined the two ideas, then lost favor in the 2000s when it fell short of expectations. Finally, deep learning began conquering the world in the 2010s with the string of successes described above.

What changed? Only brute computing power, which made it possible for back-propagation-using artificial neural networks to have far more layers than before (hence the “deep” in “deep learning”). This, in turn, allowed deep learning machines to train on massive amounts of data. It also allowed networks to be trained on a layer by layer basis, using a procedurefirst suggested by Hinton.

Can merely increasing the number of layers in a network produce such a big qualitative difference? Let’s see if we can demonstrate it as we explore a simple neural network in this month’s puzzle questions.

Problem 1

We’re going to create a simple network that converts binary numbers to decimal numbers. Imagine a network with just two layers — an input layer consisting of three units and an output layer with seven units. Each unit in the first layer connects to each unit in the second, as shown in the figure below.

As you can see, there are 21 connections. Every unit in the input layer, at a given point in time, is either off or on, having a value of 0 or 1. Every connection from the input layer to the output layer has an associated weight that, in artificial neural networks, is a real number between 0 and 1. Just to be contrary, and to make the network a touch more similar to an actual neural network, let us allow the weight to be a real number between  –1 and 1 (the negative sign signifying an inhibitory neuron). The product of the input’s value and the connection’s weight is conveyed to an output unit as shown below. The output unit adds up all the numbers it gets from its connections to obtain a single number, as shown in the figure below using arbitrary input values and connection weights for a single output unit. Based on this number, the output number decides what its state is going to be. If it is more than a certain threshold, the unit’s value becomes 1, and if not, then its value becomes 0. We can call a unit that has a value of 1 a “firing” or “activated” unit and a unit with a value of 0 a “quiescent” unit.

Our three input units from top to bottom can have the values 001, 010, 011, 100, 101, 110 or 111, which readers may recognize as the binary numbers from 1 through 7.

Now the question: Is it possible to adjust the connection weights and the thresholds of the seven output units such that every binary number input results in the firing of only one appropriate output unit, with all the others being quiescent? The position of the activated output unit should reflect the binary input value. Thus, if the original input is 001, the leftmost output unit (or bottommost when viewed on a phone, as shown in the top figure) alone should fire, whereas if the original input is 101, the output unit that is fifth from the left (or from the bottom on a phone) alone should fire, and so on.

If you think the above result is not possible, try to adjust the weights to get as close as you can. Can you think of a simple adjustment, using additional connections but not additional units, that can improve your answer?

Problem 2

Now let’s bring in the learning aspect. Assume that the network is in an initial state where every connection weight is 0.5 and every output threshold is 1. The network is then presented with the entire set of all seven input values in serial order (10 times over, if required) and allowed to adjust its weights and thresholds based on the error difference between the observed value and the desired value. As in Problem 1, the connection weights must remain between –1 and 1. Can you create a global, general purpose learning rule that adjusts the connections and thresholds such that the optimum condition of Problem 1 can be reached or approached? For example, a learning rule could say something like “increase the connection weight by 0.2 within the permitted limits, and decrease its threshold by 0.1, if the output unit remains quiescent in a situation where it should have fired.” You can be as creative as you like with the rule, provided the specified limits are followed. Note that the rule must be the same for all the units in a given layer and must be general purpose: It should allow the network to perform better for different sets of inputs and outputs as well.

Problem : 3

Finally, let’s add an extra layer of five units between the input and output layers. Analyzing the resulting three-layer network will probably require computer help, such as using a spreadsheet or a simple simulation program. Following the same rules as above, with extra rules for the hidden layer, how does the three-layer network perform on Problems 1 and 2 compared to the two-layer network?

That’s it for now. Happy puzzling. Hopefully, this exercise will slightly alter the connection weights and thresholds of some of your neurons in constructive ways!

The post How to Win at Deep Learning appeared first on Artificial Intelligence.