A new collaboration has demonstrated fully-autonomous Scanning Probe Microscopy (SPM) operation, applying artificial intelligence (AI) and deep learning to remove the need for constant human supervision.

According to the researchers, the new system, dubbed DeepSPM, bridges the gap between nanoscience, automation and AI, firmly establishing the use of machine learning for experimental scientific research.

“Optimising SPM data acquisition can be very tedious. This optimisation process is usually performed by the human experimentalist and is rarely reported,” said Future Low-Energy Electronics Technologies (FLEET) Chief Investigator Dr Agustin Schiffrin, at Monash University, Australia. “Our new AI-driven system can operate and acquire optimal SPM data autonomously, for multiple straight days and without any human supervision.”

The advance brings advanced SPM methodologies such as atomically-precise nanofabrication and high-throughput data acquisition closer to a fully automated turnkey application, say the researchers. 

The new deep learning approach can also be generalised to other SPM techniques. The researchers have made the entire framework publicly available online as open source, creating an important resource for the nanoscience research community.

“Crucial to the success of DeepSPM is the use of a self-learning agent, as the correct control inputs are not known beforehand,” said Dr Cornelius Krull, project co-leader. “Learning from experience, our agent adapts to changing experimental conditions and finds a strategy to maintain the system stable.” 

The AI-driven system begins with an algorithmic search of the best sample regions and proceeds with autonomous data acquisition. It then uses a convolutional neural network to assess the quality of the data. If the quality of the data is poor, DeepSPM uses a reinforcement learning agent to improve the condition of the probe.

The system can run for several days, acquiring and processing data continuously, while managing SPM parameters in response to varying experimental conditions, without any supervision, highlight the researchers. 

The post Autonomous Scanning Probe Microscopy technique developed using AI appeared first on Artificial Intelligence.

Several technologies have been developed over the past few years that are pushing the boundaries of our scientific knowledge to new heights. As these technologies develop scientists are looking into ways of using them in tandem, to produce more accurate results and new ways of approaching the problems of the modern scientific industry. Two such technologies that can be combined to produce a better understanding of biological systems are localization microscopy and deep learning.

Localization microscopy

Localization microscopy is a recently developed technique that has the potential to transform our understanding of cell function at the molecular level. It provides pin-point accurate information about molecular distributions in biological structures, and analysis with this technique can even provide the absolute number of proteins present in biological systems.

It is a powerful high-resolution imagining technique that has been shown to have the potential to be an extremely useful complementary method for experiments, utilizing conventional light and electron microscopy. This technique provides direct experimental feedback for modeling the complexity of biological interactions.

Localization microscopy works on the principle that the diffraction barrier (the resolution limit of light waves) can be bypassed by photoswitching, photoactivation, or photoconversion of individual fluorophores, which are attached to the structure of interest.

By utilizing precise single-molecule localization in conjunction with advanced reconstruction techniques of multiple images, which contain a handful of active isolated fluorophores, structures of less than half the wavelength of light in diameter can be resolved, compared with techniques such as conventional or confocal fluorescence microscopy. The density of activated molecules is kept low so that the individual fluorophores being imaged in each cycle do not overlap.

Thus, using this technique, detection of a mere few thousand protons from a single fluorophore allows localization of said fluorophore with precision in the nanometer range.

Applying deep learning to produce better, faster results

However, there is one limitation with localization microscopy: the number of images that can be produced in a given time. As the technology relies on multiple images of different activated fluorophores, this is a major stumbling block without the application of other technologies to speed up the process. One such current technology that can resolve this issue is deep learning.

In the field of artificial intelligence (AI), deep learning is a recently developed subset of machine learning. It utilizes artificial neural networks to make it capable of learning from unstructured or unlabelled data. It has been used in many different applications from facial recognition to self-driving cars, and healthcare applications. It is a cutting-edge technology that is fundamentally changing how software and machines interact with the world around us.

Recently, studies have been carried out by researchers to apply deep learning to life science research and complement the advances within the field. One such study that sought to provide better and faster localization microscopy high-throughput images was carried out by a team led by Wei Ouyang, of the Imaging and Modeling unit of the Pasteur Institute in Paris. They developed a system called ANNA-PALM, which was shown to vastly improve super-resolution images of delicate structures.

They used simulations and experimental imaging of mitochondria, microtubules, and nuclear pores to demonstrate that this technology could reconstruct high-quality super-resolution images from up to two orders of magnitude fewer frames than would normally be needed, without compromising spatial resolution.

With ANNA-PALM, they managed to achieve super-resolution reconstruction of widefield images alone (a difficult proposition without the addition of deep learning techniques), and by adding localization data image quality was improved. The system yielded images spanning spatial scales from ~20nm – ~2nm and imaged >1,000 fields of view containing >1,000 cells in ~3h. Acquisition time and sample irradiation were drastically reduced, leading to gentler and faster high-throughput and live-cell super-resolution imaging.

The future

By utilizing cutting-edge technologies, researchers can easily complement existing techniques to produce results that are faster and more accurate than ever before. Through applications such as improving localization microscopy imaging, deep learning is one such technology that is advancing the way we learn about the complexities of the natural world and represents a revolutionary development in current and future scientific studies.

The post Applying Deep Learning to Localization Microscopy appeared first on Artificial Intelligence.