There has been substantial progress in building a Machine Learning (ML) methodology for Earth Observation (EO) data analysis; however, experts worldwide face many challenges while using ML algorithms on EO data.

For ML models to work, two processes work simultaneously. First, tons of data is captured from EO satellites, which is processed to make it application-ready. This data is called application-ready data (ARD), put in Cloud and organized into different datasets called data cubes. Secondly, the training data is collected to train models. Once both datasets are organized, an appropriate ML model is selected to classify, smoothen, and process the data to get valuable insights.  

Using multiple ML algorithms on large volumes of EO data ensures reliable and conclusive results, thereby easing the process to prove or disprove a given hypothesis. While the benefits are many, EO satellite data’s abundant availability makes it tricky to run ML models and algorithms efficiently. Currently, we have a ton of datasets like Sentinel 2, Sentinel 3, Landsat 8, and SkySat, to name a few, which provide more than 2 petabytes (PB) of data every day. Thus, while many ML models operate efficiently on sample models, they fail to represent actual reality.

One of the most critical challenges faced in deploying ML models appropriately is the massive volume of data collected. Prof. Dr. Gilberto Camara, Secretariat Director, GEO, mentioned during the discussion that the data derived from EO satellites should be enough to cover all the categories and details a project requires explicitly. However, data labeling of the number of categories is crucial, which defines the behavior of the classifier modeling the data.

Data labeling

ML requires labels to understand data better, but natures’ diversity limits the application of ML algorithms. The currently available categorization is often found not enough to label data. For a basic understanding, we take the example of using ML algorithms for EO data of forests –

As depicted in the above image, how one defines a forest label is different for different places. Forest is a single label, but it has several variations, ranging from Boreal forest to Tropical forest. One may think this problem can be easily solved by breaking the label down into several small labels. Supposedly, if one were to break the term forest into eight different labels, the problem of finding good samples to train the ML algorithm is multiplied by eight. Hence, if we required 1,000 samples for the forest, in the above scenario, we would require 8,000 samples for the same, which complicates the matter. Thus, to describe nature appropriately, it is essential to ensure whether the labels used to define nature are consistent with the ML models.

Time as an element

In the case of forests, EO data is being used to monitor a forest’s condition – particularly deforestation. Deforestation is not a one-time process but is the result of a series of steps happening over time. In the ML context, it involves working with both Space and time. To understand it better, we can look at the diagram below, which explains how a forest evolves. A forest can grow in any of the ways mentioned below; for instance, it can be conserved throughout time, as in Fig A (1). There can be deforestation, or there can be afforestation in deforested land with time, as in Fig A (3).

To sum it up, one needs to measure what exists in a certain place at a certain point in time and determine the events that have happened in that particular location over time. Hence, to work with Space and time, we require spatial-temporal models. Modeling events and time is key for big EO data analysis, but ML has a hard time dealing with the change.

A solution to the above challenge is to use Geospatial Semantics for EO data analysis. Herein, EO data is organized using a logical view, including indexing and/or ingestion, rather than arranging it in three dimensions: time, longitude, and latitude. The significance of ingesting data is that it can be collected in a query-optimized way. Certain access patterns can be achieved more efficiently, such as spatial analysis or time series analysis.

Other technical challenges

The other technical challenges that data analysts and processors face while feeding the images to the ML models, like:

• Resolution– Different satellites provide different resolutions of images ranging from 500m provided by MODIS to 0.3 m by WorldView. Additionally, different datasets have different formats, such as JPEG2000 and GeoTIFF, among others. Thus, the processor must learn to work with different resolutions and formats. This problem can be partly solved by third-party software like Sentinel Hub, which harmonizes the Earth Observation data in one single format. 

• often partly or fully covered by clouds. The Clouds make it difficult for any algorithm and processor to derive useful insights from the satellite imageries. Therefore, the processor should mask these clouds so that these white spots or shadows do not distort the signals.
• Geometrical accuracy– Satellite images often twitch because geo-referenced points used for georeferencing the image are not perfect. While this has gotten better over the past few years, but it still could not be expected that one pixel will represent one point of the world.

Conclusion

Artificial Intelligence experts face many challenges while applying ML algorithms on EO data, affecting every phase of the data processing and analysis, ranging from collecting training data to deriving valuable insights from it. Recently, MKAI Technical Forum conducted a webinar on using AI on EO data. The webinar discussed how ML models are used to classify, smoothen and post-process the humongous volumes of EO data. A few ways to deal with these challenges would be to build robust and geographically diverse training data sets, include Geospatial Semantics in the process, and harmonize the data using various third-party software available in the market.

The post Challenges of using Machine Learning on Earth Observation data appeared first on Artificial Intelligence.

Machine learning is undeniably a tool that most disciplines like to have in their toolbox. However, scientists are still investigating the limits and barriers to incorporating machine learning into their research. Junior Sonia Reilly spent her summer opening up the machine learning black box to better understand how information flows through neural networks as part of the Undergraduate Research Opportunities Program (UROP). Her project, which investigates how machine learning works with the intention of improving its application to the observation of natural phenomena, was overseen by Sai Ravela in the Department of Earth, Atmospheric and Planetary Sciences (EAPS). As a major in Course 18C (Mathematics with Computer Science), Reilly is uniquely equipped to help investigate these connections.

“In recent years, deep learning has become an immensely popular tool in all kinds of research fields, but the mathematics of how and why it is so effective is still very poorly understood,” says Reilly. “Having that knowledge will enable the design of better-performing learning machines.” To do that, she looks more closely at how the algorithms evolve to produce their final most-probable conclusions, with the end goal of providing insights on information flow, bottlenecks, and maximizing gain from neural networks.

“We don’t want to be drowning in big data. On the contrary, we want to transform big data into perhaps what we might call smart data,” Ravela says of how machine learning must proceed. “The end goal is always a sensing agent that gathers data from our environment, but one that is knowledge-driven and does just enough work to gather just enough information for meaningful inferences.”

For Ravela, who leads the Earth Signals and Systems Group (ESSG), better-performing learning machines means more robust early predictions of potential disasters. His group’s research lies largely in how the Earth works as a system, primarily focusing on climate and natural hazards. They observe natural phenomena to produce effective predictive models for dynamic natural processes, such as hurricanes, clouds, volcanoes, earthquakes, glaciers, and wildlife conservation strategies, as well as making advances in engineering and learning itself.

“In all these projects, it’s impossible to gather dense data in space and time. We show that actively mining the environment through a systems analytic approach is promising,” he says. Ravela recently delivered his group’s latest work — including Reilly’s contributions — to the Association of Computing Machinery’s special interest group on knowledge discovery and data mining (SIGKDD 2019) in early August. He teaches an “infinite course” with a duology of classes taught in spring and fall semesters that provides an overview of machine learning foundations for natural systems science, which anyone can follow along with online.

According to Ravela, if Reilly is to succeed at advancing the mathematical basis for computational learning models, she will be one of the “early pioneers of learning that can be explained,” an achievement that can provide a promising career path.

That is ideal for Reilly’s goals of obtaining a PhD in mathematics after graduating from MIT and remaining a contributor to research that can positively impact the world. She’s starting with cramming as much research as she can manage into her schedule over her final two undergraduate years at MIT, including her experience this summer.

Although this was Reilly’s first UROP experience, it is her second time undertaking a research project that blends mathematics, computer science, and Earth science. Previously, at the Johns Hopkins University Applied Physics Laboratory, Reilly helped develop signal processing techniques and software that would improve the retrieval of useful climate change information from low-quality satellite data.

“I’ve always wanted to be part of an interdisciplinary research environment where I could use my knowledge of math to contribute to the work of scientists and engineers,” Reilly says of working within EAPS. “It’s encouraging to see that type of environment and get a taste of what it would be like to work in one.”

Ravela explains that the ESSG is fond of the mutually beneficial inclusion of UROP students. “For me, UROPs are better than grad student and postdocs if, and only if, one can create the right-sized questions for them to run with. But then they run the fastest and are the most clever of all.” He says he feels the UROP program is invaluable and could be beneficial for all students to incorporate, as it offers a chance to learn about other fields and interdisciplinary research, as well as how to incorporate what they learn into tangible results.

For Reilly, research builds on her foundation obtained from taking classes at MIT, which are a controlled and predictable environment, she says, “but research is nowhere near so linear.” She has relied on her foundation of mathematics and computer science from her courses during her UROP experience while having to learn how to connect and apply them to new fields and to consider topics often outside an undergraduate education. “It often feels like every step I take requires me to learn about an entirely new field of mathematics, and it’s difficult to know where to start. I definitely feel lost sometimes, but I’m also learning an incredible amount.”

The post Computing in Earth science: a non-linear path appeared first on Artificial Intelligence.