This is another feather in the cap for artificial intelligence.

A part of what we see in science fiction movies will soon become a reality, thanks to artificial intelligence. Every time you saw people talking to holograms in sci-fi movies and thought to yourself “that would be awesome to have”, you just might be closer to that future. Smartphones will soon be able to create photorealistic 3D holograms with an AI model developed by a research team at MIT. This system determines the best way to generate holograms from a sequence of input images. This fascinating technology could have applications for VR and AR headsets. Unlike conventional 3D and VR displays that create the illusion of depth causing nausea and headaches, a holographic display can be viewed by people without straining their eyes.

A major challenge in creating holographic media is maintaining the data that is needed to create holographs. Every holograph constitutes huge amounts of data which creates the “depth” of the holographs. This is why creating a hologram demands lots of computing power. To simplify this process, researchers at MIT applied deep convolutional neural networks to the problem. This approach created a network that is capable of quickly generating holograms based on pictographic data.

Past Vs Present

The traditional method of generating holograms creates many chunks of holograms and then uses scientific simulations to combine the chunks into a complete pictorial representation. This process is power-intensive and time-consuming. But according to the IEEE spectrum, the method designed by the team of researchers at MIT is a lot different. It uses deep learning networks to slice images into chunks that can be recompiled into holograms using fewer “slices” than that of the traditional methods. This is possible because of the convolutional neural network’s ability to analyze images and separate them into discrete chunks. This new method is far less power-intensive.

In order to design this artificial intelligence holographic generator, the MIT team began by creating a database that included approximately 4000 computer-generated images, with a matchable 3D hologram allotted to each of those images. Based on this dataset, the convolutional neural network was trained to learn the way each of those images was connected to its hologram. When the artificial intelligence system was given the unseen data with depth information, it was able to generate new holograms with the given data. For this process, the depth information is supplied to the AI system through the use of a lidar sensor of multi-camera displays that renders it as a computer-generated image. Some iPhones have these components which makes it possible to generate holograms if connected to the right type of display.

The new artificial intelligence hologram system needs less memory than the traditional methods. This system can create colored 3D holograms at a speed of 60 frames per second with a resolution of 1920 x 1080 using approximately 620 KB of memory, all this by running on a single graphics processing unit (GPU). The MIT research team was able to run their new AI technology on an iPhone 11 creating 1 hologram per second. They also tried it on a Google Edge TPU which could create 2 holograms per second. This implies that the artificial intelligence hologram system can have applications in volumetric 3D printing or in the designing of holographic microscopes.

This is just the inception of this technology. In the future, with further advancements, this technology might revolutionize our way of communication and perceiving visual data. It surely is an exciting time for the tech world.

The post ARTIFICIAL INTELLIGENCE IS MAKING 3D HOLOGRAMS POSSIBLE ON SMARTPHONES appeared first on Artificial Intelligence.

A new method called tensor holography could enable the creation of holograms for virtual reality, 3D printing, medical imaging, and more — and it can run on a smartphone.

Despite years of hype, virtual reality headsets have yet to topple TV or computer screens as the go-to devices for video viewing. One reason: VR can make users feel sick. Nausea and eye strain can result because VR creates an illusion of 3D viewing although the user is in fact staring at a fixed-distance 2D display. The solution for better 3D visualization could lie in a 60-year-old technology remade for the digital world: holograms.

Holograms deliver an exceptional representation of 3D world around us. Plus, they’re beautiful. (Go ahead — check out the holographic dove on your Visa card.) Holograms offer a shifting perspective based on the viewer’s position, and they allow the eye to adjust focal depth to alternately focus on foreground and background.

Researchers have long sought to make computer-generated holograms, but the process has traditionally required a supercomputer to churn through physics simulations, which is time-consuming and can yield less-than-photorealistic results. Now, MIT researchers have developed a new way to produce holograms almost instantly — and the deep learning-based method is so efficient that it can run on a laptop in the blink of an eye, the researchers say.

“People previously thought that with existing consumer-grade hardware, it was impossible to do real-time 3D holography computations,” says Liang Shi, the study’s lead author and a PhD student in MIT’s Department of Electrical Engineering and Computer Science (EECS). “It’s often been said that commercially available holographic displays will be around in 10 years, yet this statement has been around for decades.”

Shi believes the new approach, which the team calls “tensor holography,” will finally bring that elusive 10-year goal within reach. The advance could fuel a spillover of holography into fields like VR and 3D printing.

Shi worked on the study, published on March 10, 2021, in Nature, with his advisor and co-author Wojciech Matusik. Other co-authors include Beichen Li of EECS and the Computer Science and Artificial Intelligence Laboratory at MIT, as well as former MIT researchers Changil Kim (now at Facebook) and Petr Kellnhofer (now at Stanford University).

The quest for better 3D

A typical lens-based photograph encodes the brightness of each light wave — a photo can faithfully reproduce a scene’s colors, but it ultimately yields a flat image.

In contrast, a hologram encodes both the brightness and phase of each light wave. That combination delivers a truer depiction of a scene’s parallax and depth. So, while a photograph of Monet’s “Water Lilies” can highlight the paintings’ color palate, a hologram can bring the work to life, rendering the unique 3D texture of each brush stroke. But despite their realism, holograms are a challenge to make and share.

First developed in the mid-1900s, early holograms were recorded optically. That required splitting a laser beam, with half the beam used to illuminate the subject and the other half used as a reference for the light waves’ phase. This reference generates a hologram’s unique sense of depth.  The resulting images were static, so they couldn’t capture motion. And they were hard copy only, making them difficult to reproduce and share.

Computer-generated holography sidesteps these challenges by simulating the optical setup. But the process can be a computational slog. “Because each point in the scene has a different depth, you can’t apply the same operations for all of them,” says Shi. “That increases the complexity significantly.” Directing a clustered supercomputer to run these physics-based simulations could take seconds or minutes for a single holographic image. Plus, existing algorithms don’t model occlusion with photorealistic precision. So Shi’s team took a different approach: letting the computer teach physics to itself.

They used deep learning to accelerate computer-generated holography, allowing for real-time hologram generation. The team designed a convolutional neural network — a processing technique that uses a chain of trainable tensors to roughly mimic how humans process visual information. Training a neural network typically requires a large, high-quality dataset, which didn’t previously exist for 3D holograms.

The team built a custom database of 4,000 pairs of computer-generated images. Each pair matched a picture — including color and depth information for each pixel — with its corresponding hologram. To create the holograms in the new database, the researchers used scenes with complex and variable shapes and colors, with the depth of pixels distributed evenly from the background to the foreground, and with a new set of physics-based calculations to handle occlusion. That approach resulted in photorealistic training data. Next, the algorithm got to work.

By learning from each image pair, the tensor network tweaked the parameters of its own calculations, successively enhancing its ability to create holograms. The fully optimized network operated orders of magnitude faster than physics-based calculations. That efficiency surprised the team themselves.

“We are amazed at how well it performs,” says Matusik. In mere milliseconds, tensor holography can craft holograms from images with depth information — which is provided by typical computer-generated images and can be calculated from a multicamera setup or LiDAR sensor (both are standard on some new smartphones). This advance paves the way for real-time 3D holography. What’s more, the compact tensor network requires less than 1 MB of memory. “It’s negligible, considering the tens and hundreds of gigabytes available on the latest cell phone,” he says.

The research “shows that true 3D holographic displays are practical with only moderate computational requirements,” says Joel Kollin, a principal optical architect at Microsoft who was not involved with the research. He adds that “this paper shows marked improvement in image quality over previous work,” which will “add realism and comfort for the viewer.” Kollin also hints at the possibility that holographic displays like this could even be customized to a viewer’s ophthalmic prescription. “Holographic displays can correct for aberrations in the eye. This makes it possible for a display image sharper than what the user could see with contacts or glasses, which only correct for low order aberrations like focus and astigmatism.”

“A considerable leap”

Real-time 3D holography would enhance a slew of systems, from VR to 3D printing. The team says the new system could help immerse VR viewers in more realistic scenery, while eliminating eye strain and other side effects of long-term VR use. The technology could be easily deployed on displays that modulate the phase of light waves. Currently, most affordable consumer-grade displays modulate only brightness, though the cost of phase-modulating displays would fall if widely adopted.

Three-dimensional holography could also boost the development of volumetric 3D printing, the researchers say. This technology could prove faster and more precise than traditional layer-by-layer 3D printing, since volumetric 3D printing allows for the simultaneous projection of the entire 3D pattern. Other applications include microscopy, visualization of medical data, and the design of surfaces with unique optical properties.

“It’s a considerable leap that could completely change people’s attitudes toward holography,” says Matusik. “We feel like neural networks were born for this task.”

The post Using Artificial Intelligence to Generate 3D Holograms in Real-Time on a Smartphone appeared first on Artificial Intelligence.

As Industry 4.0 takes hold, industrial automation and robotics are replacing many manual tasks in manufacturing. However, when it comes to visual quality inspection, most production lines still employ human workers in the tedious task of examining products and judging defects.

The biggest drawback of manual visual inspection is that humans make mistakes. Tired workers often miss defects that ‘escape’ the quality screens on the production floor and leak into finished goods packages or into integrated systems. When these defects are discovered or surface at a later stage often by end customers, users or consumers, it is too late and very costly to fix. The Cost of Poor Quality (CoPQ) in these cases is significant.  It includes – among other elements – the costs of returned or rejected goods (RMA), scrap, rework and in many cases the negative impact on brand reputation and end customer dissatisfaction.

Israel based Kitov is paving the way towards smart manufacturing, by developing the technology to enable smart computer-driven visual inspection and support manufacturers along their digital transformation path.

KITOV ONE is a Smart 3D, Universal System that can effectively inspect virtually any product. Leveraging advanced 3D computer vision and deep-learning algorithms, KITOV ONE achieves unprecedented levels of detection, eliminating the tedious work and inconsistent results associated with manual inspection. KITOV supports complex 3D structures, numerous materials, and complete inspection specifications.

By imitating human learning processes, KITOV ONE features an intuitive method to teach the system how to optimally inspect almost any product.  Setting up the system does not require programming skills or knowledge of robotics or optics. KITOV ONE software computes and controls the processes of image acquisition and image processing by using pre-set algorithms called detectors. Artificial intelligence capabilities are used to find and classify defects.

“We have developed artificial intelligence (AI) systems for advanced manufacturing that can be intuitively trained within a few hours by a non-expert to automatically plan and perform sophisticated visual inspection tasks on complex 3D products at the highest performance levels.” states Dr. Yossi Rubner, CTO and Founder of Kitov.ai.

By using dashboards and Big Data Analytics Kitov helps manufacturers to identify trends and proactively attend to quality issues early on and by providing powerful insights about manufacturing process and product design can support root cause analysis and elimination of defects.

The post Smart 3D Universal Inspection System Uses Deep Learning appeared first on Artificial Intelligence.

As a source of inspiration, aquatic creatures such as fish, cetaceans, and jellyfish could inspire innovative designs to improve the ways that manmade systems operate in and interact with aquatic environments. Jellyfishes in nature propel themselves through their surroundings by radially expanding and contracting their bell-shaped bodies to push water behind them, which is called jet propulsion.

Contrary to prevailing view that jellyfishes are described as inefficient swimmers, jellyfishes have been proven to be one of the most energetically efficient swimmers. That is, it indicates that jellyfish-like swimming will have a remarkable propulsive advantage if low-energy propulsion is demanded. Therefore, the movements of jellyfish have attracted significant interest over the past decade in the context of bioinspired underwater vehicle.

Recently, researchers from Institute of Automation, Chinese Academy of Sciences in Beijing, China successfully developed a novel robotic jellyfish able to perform three-dimensional jellyfish-like propulsion and maneuvers based on a reinforcement learning-based method.

Combining the latest advancements in mechatronic design, materials, electronics, and control methods, researchers are making an integrated effort to develop smart actuators to fabricate various robotic jellyfishes. In generally, such robotic jellyfishes are often tethered and much slower in speed in comparison with the kind actuated by conventional electric motors. Most of existing robotic jellyfishes cannot freely adjust their three-axis attitude, which has an adverse effect on free-swimming propulsion and plausible applications.

To solve this problem, the research group led by Prof. Junzhi Yu from Institute of Automation, Chinese Academy of Sciences has investigated how a bioinspired motor-driven jellyfish-like robotic system capable of 3D motion is designed and controlled.

The designed robotic jellyfish models after Aurelia aurita (commonly termed moon jellyfish), which has a relatively large displacement and is especially suited for use with large load capacity. It is about 138 mm height and weights about 8.2 kg. As illustrated in Figure 1, the robotic jellyfish is hemispherical in shape and consists of a bell-shaped rigid head, a cylindroid main cavity, four separate six-bar linkage mechanisms, and a soft rubber skin. To enhance the maneuverability of the robotic jellyfish, a barycenter adjustment mechanism assembled inside the cavity is introduced. Through adjusting two clump weights in vertical or horizontal direction or in a combination of the two, the attitude regulation is achieved.

“It is very hard to establish a precise dynamic model for jellyfish-like swimming, since it is a highly nonlinear, strong coupling, and time-varying system.” said by Prof. Junzhi Yu. “Parametric uncertainties and external disturbances in dynamic aquatic environments, at the same time, cause difficulty in deriving control laws by solving the inverse kinematics problem.” Therefore, a reinforcement learning based closed-loop attitude control method is proposed for the robotic jellyfish, which can solve optimal decision control problem through direct interaction with the environment, particularly without the need for dynamic modeling.

Finally, the proposal of the reinforcement learning based attitude control method makes autonomous attitude regulation possible. “In comparison with most of the other robotic jellyfish, the built robot displays a high order of structure flexibility and yaw maneuverability.” Pointed out by Prof. Junzhi Yu. He also stressed that this self-propelled robotic jellyfish with 3D motion has great implications for bioinspired design of jet propulsion system with great agility.

The post A novel robotic jellyfish able to perform 3D jet propulsion and maneuvers appeared first on Artificial Intelligence.