Machine learning is gaining popularity across scientific and technical fields, but it’s often not clear to researchers, especially young scientists, how they can apply these methods in their work.

Machine learning (ML), loosely defined as the “ability of computers to learn from data without being explicitly programmed,” has become tremendously popular in technical disciplines over the past decade or so, with applications including complex game playing and image recognition carried out with superhuman capabilities. The Earth and space sciences (ESS) community has also increasingly adopted ML approaches to help tackle pressing questions and unwieldy data sets. From 2009 to 2019, for example, the number of studies involving ML published in AGU journals approximately doubled.

In many ways, ESS present ideal use cases for ML applications because the problems being addressed—like climate change, weather forecasting, and natural hazards assessment—are globally important; the data are often freely available, voluminous, and of high quality; and computational resources required to develop ML models are steadily becoming more affordable. Free computational languages and ML code libraries are also now available (e.g., scikit-learn, PyTorch, and TensorFlow), contributing to making entry barriers lower than ever. Nevertheless, our experience has been that many young scientists and students interested in applying ML techniques to ESS data do not have a clear sense of how to do so.

The Tools of the Trade

An ML algorithm can be thought of broadly as a mathematical function containing many free parameters (thousands or even millions) that takes inputs (features) and maps those features into one or more outputs (targets). The process of “training” an ML algorithm involves optimizing the free parameters to map the features to the targets accurately.

There are two broad categories of ML algorithms relevant in most ESS applications: supervised and unsupervised learning (a third category, reinforcement learning, is used infrequently in ESS). Supervised learning, which involves presenting an ML algorithm with many examples of input-output pairs (called the “training set”), can be further divided, according to the type of target that is being learned, as either categorical (classification; e.g., does a given image show a star cluster or not?) or continuous (regression; e.g., what is the temperature at a given location on Earth?). In unsupervised learning, algorithms are not given a particular target to predict; rather, an algorithm’s task is to learn the natural structure in a data set without being told what that structure is.

Supervised learning is more commonly used in ESS, although it has the disadvantage that it requires labeled data sets (in which each training input sample must be tagged, or labeled, with a corresponding output target), which are not always available. Unsupervised learning, on the other hand, may find multiple structures in a data set, which can reveal unanticipated patterns and relationships, but it may not always be clear which structures or patterns are “correct” (i.e., which represent genuine physical phenomena).

Applications in Earth and Space Sciences

Books and classes about ML often present a range of algorithms that fall into one of the above categories but leave people to imagine specific applications of these algorithms on their own. However, in practice, it is usually not obvious how such approaches (some seemingly simple) may be applied in a rich variety of ways, which can create an imposing obstacle for scientists new to ML. Below we briefly describe various themes and ways in which ML is currently applied to ESS data sets (Figure 1), with the hope that this list—necessarily incomplete and biased by our personal experience—inspires readers to apply ML in their research and catalyzes new and creative use cases.

1. Pattern Identification and Clustering

One of the simplest and most powerful applications of ML algorithms is pattern identification, which works particularly well with very large data sets that cannot be traversed manually and in which signals of interest are faint or highly dimensional. Researchers, for example, applied ML in this way to detect signatures of Earth-sized exoplanets in noisy data making up millions of light curves observed by the Kepler space telescope. Detected signals can be further split into groups through clustering, an unsupervised form of ML, to identify natural structure in a data set.

Conversely, atypical signals may be teased out of data by first identifying and excluding typical signals, a process called anomaly or outlier detection. This technique is useful, for example, in searching for signatures of new physics in particle collider experiments.

2. Time Series and Spatiotemporal Prediction

An important and widespread application of supervised ML is the prediction of time series data from instruments or from an index (or average value) that is intended to encapsulate the behavior of a large-scale system. Approaches to this application often involve using past data in the time series itself to predict future values; they also commonly involve additional inputs that act as drivers of the quantities measured in the time series. A typical example of ML applied to time series in ESS is its use in local weather prediction, with which trends in observed air temperature and pressure data, along with other quantities, can be predicted.

In many instances, however, predicting a single time series of data is insufficient, and knowledge of the temporal evolution of a physical system over regional (or global) spatial scales is required. This spatiotemporal approach is used, for example, in attempts to predict weather across the entire globe as a function of time and 3D space in high-capacity models such as deep neural networks.

3. Emulators and Surrogates

Traditional, physics-based simulations (e.g., global climate models) are often used to model complex systems, but such models can take days or weeks to run on even the most powerful computers, limiting their utility in practice. An alternate solution is to train ML models to act as emulators for physics-based models or to replicate computationally intensive portions within such models. For example, global climate models that run on a coarse grid (e.g., 50- to 100-kilometer resolution) can include subgrid processes, like convection, modeled using ML-based parameterizations. Results with these approaches are often indistinguishable from those produced by the original model alone but can run millions or billions of times faster.

4. Boundary or Driving Conditions

Many physics-based simulations proceed by integrating a set of partial differential equations (PDEs) that rely on time-varying boundary conditions and other conditions that drive interior parts of the simulation. The physics-based model then propagates information from these boundary and driver conditions into the simulation space—imagine, for example, a 3D cube being heated at its boundary faces with time-varying heating rates or with thermal conductivity that varies spatiotemporally within the cube. ML models can be trained to reflect the time-varying parameterizations both within and along the simulation boundaries of a physical model, which again may be computationally cheaper and faster.

5. Interpretability and Knowledge Discovery

If a spatiotemporal ML model of a physical system can be trained to produce accurate results under a variety of input conditions, then the implication is that the model implicitly accounts for all the physical processes that drive that system, and thus, it can be probed to gain insights into how the system works. Certain algorithms (e.g., random forests) can automatically provide a ranking of “feature importance,” giving the user a sense of which input parameters affect the output most and hence an intuition about how the system works.

More sophisticated techniques, such as layerwise relevance propagation, can provide deeper insights into how different features interact to produce a given output at a particular location and time. For example, a neural network trained to predict the evolution of the El Niño–Southern Oscillation (ENSO), which is predominantly associated with changes in sea surface temperature in the equatorial Pacific Ocean, revealed that precursor conditions for ENSO events occur in the South Pacific and Indian Oceans.

6. Accelerating Inversions

A ubiquitous challenge in ESS is to invert observations of a physical entity or process into fundamental information about the entity or the causes of the process (e.g., interpreting seismic data to determine rock properties). Historically, inverse problems are solved in a Bayesian framework requiring multiple runs of a forward model, which can be computationally expensive and often inaccurate. ML offers alternative methods to approach inverse problems, either by using emulators to speed up forward models or by using physics-informed machine learning to discover hidden physical quantities directly. ML models trained on prerun physics-based model outputs can be used for rapid inversion.

7. Creating High-Resolution Global Data Sets

Satellite observations often provide global, albeit low-resolution and sometimes indirect (i.e., proxy-based), measurements of quantities of interest, whereas local measurements provide more accurate and direct observations of those quantities at smaller scales. A popular and powerful use for ML models is to estimate the relationship between global proxy satellite observations and local accurate observations, which enables the creation of estimated global observations on the basis of localized measurements. This approach often includes the use of ML to create superresolution images and other data products.

8. Uncertainty Quantification

Typically, uncertainty in model outputs is quantified using a single metric such as the root-mean-square of the residual (the difference between model predictions and observations). ML models can be trained to explicitly predict the confidence interval, or inherent uncertainty, of this residual value, which not only serves to indicate conditions under which model predictions are trustworthy (or dubious) but can also be used to generate insights about model performance. For instance, if there is a large error at a certain location in a model output under specific conditions, it could suggest that a particular physical process is not being properly represented in the simulation.

9. Physics-Informed Neural Networks

Domain experts analyzing data from a given system, even in relatively small quantities, are often able to extrapolate the behavior of the system—at least conceptually—because of their understanding of and trained intuition about the system based on physical principles. In a similar way, laws and relationships that govern physical processes and conserved quantities can be explicitly encoded into neural network algorithms, resulting in more accurate and physically meaningful models that require less training data.

10. Finding and Solving Governing Equations

In certain applications, the values of terms or coefficients in PDEs that drive a system—and thus that should be represented in a model—are not known. Various ML algorithms were developed recently that automatically determine PDEs that are consistent with the available physical observations, affording a new and powerful discovery tool.

In still newer work, ML methods are being developed to directly solve PDEs. These methods offer accuracy comparable to traditional numerical integrators but can be dramatically faster, potentially allowing large-scale simulations of complex sets of PDEs that have otherwise been unattainable.

Addressing Urgent Challenges

The Earth and space sciences are poised for a revolution centered around the application of existing and rapidly emerging ML techniques to large and complex ESS data sets being collected. These techniques have great potential to help scientists address some of the most urgent challenges and questions about the natural world facing us today. We hope the above list sparks creative and valuable new applications of ML, particularly among students and young scientists, and that it becomes a community resource to which the ESS community can add more ideas.

Acknowledgments

We thank the AGU Nonlinear Geophysics section for promoting interdisciplinary, data-driven research, for supporting the idea of writing this article, and for suggesting Eos as the ideal venue for dissemination. The authors gratefully acknowledge the following sources of support: J.B. from subgrant 1559841 to the University of California, Los Angeles, from the University of Colorado Boulder under NASA Prime Grant agreement 80NSSC20K1580, the Defense Advanced Research Projects Agency under U.S. Department of the Interior award D19AC00009, and NASA/SWO2R grant 80NSSC19K0239 and E.C. from NASA grants 80NSSC20K1580 and 80NSSC20K1275. Some of the ideas discussed in this paper originated during the 2019 Machine Learning in Heliophysics conference.

The post Ten Ways to Apply Machine Learning in Earth and Space Sciences appeared first on Artificial Intelligence.

The Department of Homeland Security has awarded funding worth $1 million each to Alakai Defense Systems and Physical Sciences Inc. to further develop their machine learning platforms to help improve the detection of explosives, narcotics, chemical agents and other threats as part of the second phase of the Small Business Innovation Research program.

“Our impetus for developing these machine-learning modules stems from the Transportation Security Administration’s operational needs for threat signature fusion, the ability to learn, detect and classify new threats without being explicitly programmed, and, ultimately, increase accuracy of detection,” Thoi Nguyen, program manager for the Next Generation Explosive Trace Detection program at DHS’ science and technology directorate, said in a statement published Friday.

Alakai will continue to develop its Agnostic Machine Learning Platform for Spectroscopy designed to detect hazardous chemicals from spectroscopic instruments as part of the two-year SBIR Phase II contract.

PSI will use the SBIR funding to continue to work on its deep learning algorithm meant to detect and classify opioids, narcotics and trace explosives for optical spectroscopic platforms.

DHS said it expects the awardees to come up with a prototype for demonstration and evaluation for Phase III funding. Under the third phase, the companies will seek private funding to bring their technologies to market.

The post Alakai, Physical Sciences to Further Develop Machine Learning Tools Under DHS SBIR Program appeared first on Artificial Intelligence.

Florida State University’s College of Arts and Sciences has announced the launch of a new interdisciplinary graduate program that will welcome its first students in Fall 2021. 

The FSU Interdisciplinary Data Science Master’s Degree Program, or IDS, leverages FSU’s strengths in computer science, mathematics, scientific computing and statistics to prepare students for contemporary careers in data science, one of the fastest growing fields in the United States.  

“The FSU IDS program is innovative and timely, and it will help address the nation’s demand for data science practitioners,” said Sam Huckaba, the college’s dean. “Those involved with building the curriculum have done an outstanding job identifying essential material that will help provide graduates with analytical tools and necessary skills.” 

The new program will be delivered exclusively at FSU’s Tallahassee campus. Students will complete a series of core courses providing a solid starting point in mathematics, machine learning, statistics, data ethics and databases, along with electives that support a specific major area of study — computer science, mathematics, scientific computing or statistics. Nineteen faculty members from across the program’s major disciplines will guide students through coursework and major selections that meet their individual needs and goals. 

“The interdisciplinary data science program is a wonderful addition to our slate of academic offerings,” said Provost Sally McRorie. “Many fields are now requiring a workforce that not only possesses a grasp of big data but also knows how to analyze and generate meaning from that data. At Florida State, we take pride in making sure our students are career ready, and the IDS program will create even more opportunities for our students.” 

Graduates of the FSU Interdisciplinary Data Science Master’s Degree Program will fill a growing demand for a workforce trained in data science and possess sought-after skills to read, analyze, explore, model and draw conclusions from the highly complex, multi-dimensional, rapidly expanding and diverse data universe.  

“The data science market is projected to be worth $103 billion by 2023, and the U.S. Bureau of Labor Statistics projects jobs for computer and information research scientists, and data scientists will experience 14 percent growth through 2028,” said Gordon Erlebacher, interim IDS program director and professor of scientific computing. “Learning how to responsibly collect, analyze, and apply data to a variety of fields will be key to success for FSU students now and into the future.” 

A number of alumni from IDS’ affiliated programs have landed jobs in the data science industry, but the ability to earn a degree in data science at FSU will cement future graduates’ preparation and marketability to employers. Sectors where data science skills will prove indispensable include cybersecurity, data information processing, financial services, epidemiology, public health, survey research, airline and auto industries, real estate, online retail, and more. 

FSU’s IDS program has been designed to appeal to students from a wide range of undergraduate backgrounds. 

“The program will appeal to any student holding a degree in a mathematical sciences discipline, but it also will be of interest to students from other disciplines, such as the physical sciences and engineering, or in fact any discipline, who have satisfied baseline prerequisites,” Huckaba said. “Further, the IDS platform is designed to accommodate major tracks all across campus.” 

While there are other data science education programs in Florida’s State University System and the U.S., the FSU IDS program is unique in its inclusion of ethics and communication as part of the core curriculum.   

“Spending time educating our students on the ethics related to data mining and data extraction and communication skills and strategies will strengthen their sense of responsibility and increase the value they bring to potential employers after graduation,” Erlebacher said. 

The post FSU College of Arts and Sciences announces new interdisciplinary data science program appeared first on Artificial Intelligence.

American geneticist David Reich recently re-opened a debate on the topic of race that went against the dominant liberal scientific consensus. Author of the book titled Who We Are and How We Got Here? (2018), Reich argued in a New York Times article that “while race may be a social construct, differences in genetic ancestry that happen to correlate to many of today’s racial constructs are real”.

For Reich, the reality of biological differences is one we cannot ignore. If there do exist noticeable differences due to height and skin colour among populations, then “it seems a bad bet to argue that there cannot be similar average differences in cognitive or behavioural traits”.

While height and skin colour can be measured and quantified, how are we to measure differences in intelligence? Is intelligence entirely determined by biology?

Catherine Malabou’s new book, Morphing Intelligence: From IQ Measurement to Artificial Brains calls into question this simple correlation of intelligence with genetic heredity. She points out how measuring intelligence in the form of IQ tests is not as objective as one may suppose. The controversial case of Charles Murray’s The Bell Curve, which claimed that Africans have lower average IQ, is just one example that can corroborate this point.

Thus, Malabou begins by questioning the innateness of intelligence. She highlights how the search for the ‘intelligence gene’ was one that quite predictably failed. Genetic mechanisms do not function, she argues, on simple one-to-one correlation between genotype and phenotype. Then she moves on to a conception of brain development where “habit, experience and education play a determining role in the formation and life of neuronal connections”.

In the final part, she discusses the question of artificial intelligence. We still draw a strict hierarchy between automatism and creativity, despite all the evidence to the contrary. For Malabou, this philosophical distinction can no longer be held viable.

Malabou, a French philosopher, has achieved some notoriety in philosophical circles for marrying deconstruction (a French philosophical movement inspired by Jacques Derrida which functions by identifying a hierarchy, say between masculine and feminine, and demonstrating how the former actually depends on the latter for its consistency) with the latest developments in the neurobiological sciences.

What interests Malabou in the obscure and sometimes esoteric biological debates between genetics and epigenetics is a concept she calls plasticity. In her books, she defined define plasticity as the “the potential for neuronal architecture to be shaped by the influences of environment, habit, and education”.

Unlike most philosophers, Malabou has never shied away from a materialism where symbolic processes actually physically affect neuronal configuration, and vice versa. Her earlier book What Should We Do with Our Brain? (2004)put forth the claim that the plasticity of the brain made it not just the recipient and creator of form but also “an agency of disobedience to every constituted form, a refusal to submit to a model”.

Other philosophers have consistently sought to barricade themselves from the biological sciences. Malabou was one of the few who connected this relation to a transformative politics. Her concept of plasticity simultaneously materialised intelligence in the brain without privileging heredity. What this means is that while intelligence depends on brain structure, the structure itself is not encoded in DNA but can be transformed and moulded at any time.

In What Should we do with our Brain?,Malabou privileged the plasticity of the brain over the determinism of the computer. Every move of the computer is supposed to be calculable in advance, leaving it with no degree of freedom. Recent technological developments have, to put it mildly, exploded this thesis.

In her earlier books, Malabou worked to drive a difference between genetics and epigenetics, the first being completely programmed and the second having various degrees of freedom. She claimed that a computer could only be programmed and would never be able to achieve the plasticity of the organic. This opposition was, of course, both forced and false.

Recent developments of synaptic chips by IBM (the aptly titled Project SyNAPSE) have blurred the line between automatism and spontaneity. Thus Malabou discovers that the concept of automatism bears within itself creativity and spontaneity: “An automatism refers to an involuntary movement, one without a soul. But in Greek automatos also means that which moves by itself, spontaneously.”

This book is thus a re-evaluation of the hierarchies she had drawn between deterministic automatism and plastic creativity. What Malabou discovers, through her reading of philosophers like John Dewey, is that the concept of automatism is not entirely deterministic. The automaton, to be successful in the environment it finds itself, must simultaneously be able to interrupt its own automatic routine, in short, be creative.

As Malabou writes, “The subtlety of algorithmic calculation today derives precisely from the fact that it is capable of simulating non-calculation, that is, spontaneity, creative freedom, and the directness of emotion.”

This may lead to a situation where new forms of intelligence come into existence. Such a time is imminent, Malabou writes, and rather than trying to safeguard some quintessentially human domains, she concludes we should strive to be “creative otherwise”. However, it remains unclear what this injunction actually means, perhaps revealing a certain perplexity within Malabou herself.

Morphing Intelligence was originally a series of lectures given at the University of California, Irvine, in 2015. While there has been a lot of progress since then in the neurosciences, what Malabou did not foresee was the failure of Henry Markram’s ‘Human Brain Project‘ to achieve its goals. This project attempted to simulate the workings of an entire brain but it has achieved nothing as spectacular as its outlandish claims. As Ed Yong writes in The Atlantic, “The brain’s intricacies are more unknown than known…. It is hard enough to map and model the 302 neurons of the roundworm C. elegans, let alone the 86 billion neurons within our skulls.”

However, the emergence of radically new forms of intelligence cannot be denied anymore. Morphing Intelligence thus makes us repeat with a sense of urgency Malabou’s original question: what should we do with our brain?

The post Book Review | The Ever-Changing Nature of Human Intelligence appeared first on Artificial Intelligence.