A UW Allen School team recently published an article in Nature Machine Intelligence finding models predicting COVID-19 diagnosis from X-rays are relying on shortcuts.

Several research groups have developed artificial intelligence (AI) models to diagnose COVID-19 based on  chest radiography, with the intention of increasing COVID-19 testing accessibility.

When UW M.D. and Ph.D. students Alex DeGrave and Joseph Janizek heard about this, they immediately thought something in the models might be amiss. 

Janizek said they had been studying AI models predicting pneumonia from chest X-rays and found many of the models were using shortcuts — or aspects of images unrelated to the actual disease — to make the predictions. DeGrave and Janizek thought the COVID-19 diagnosis models might be doing the same thing.  

“We figured that the combination of the high profile of these new studies coming out, and the likelihood of the data being sort of problematic, made it a really good place to kind of apply what we’ve been looking at,” Janizek said.  

Shortcut learning occurs when AI learns to associate things from the training data that are not meaningfully associated in real life. Janizek and DeGrave found the COVID-19 prediction AI models associated having labels on the bottom of the X-ray image with a COVID-19 diagnosis.

DeGrave, Janizek, and their Ph.D. advisor and associate professor Su-In Lee created and trained deep convolutional neural network AI models to replicate what had been done in published studies. The team found the AI performed well on data from the same hospital system as the training data, but when given data from a different hospital system the accuracy was reduced by half.

This trend was something they had noticed in pneumonia models as well, which suggested AI models might be using shortcuts to make diagnosis predictions, Janizek said.

Traditionally, AI functions like a “black box.” The AI model receives large amounts of data to learn from, then users ask the model to make a prediction about a new piece of data. The AI will give an answer, but users typically have no idea why this answer should be reliable.

The team employed a variety of techniques to open up the black box of COVID-19 diagnosis AI models. DeGrave, Janizek, and Lee used saliency maps, which highlighted regions the AI used to determine COVID-19 diagnosis. They also used generative adversarial networks, which involves illustrating what the AI “thinks” is important about the image, in addition to manually modifying the image to see how AI’s COVID-19 diagnosis would change.

“The reason why we used this large set of techniques, three complimentary techniques, is because I think they all overlap each other’s pitfalls nicely,” DeGrave said. “They really complement each other and make the set of experiments much stronger.”

The team found the AI models were using parts of the image, such as annotations, labels, and body positioning, that had nothing to do with COVID-19 to make a COVID-19 diagnosis prediction. These AI models were particularly reliant on shortcut learning because in the limited data available, X-ray images from COVID-19 positive and COVID-19 negative individuals were from different sources.

“Having a problem this severe is fairly unique to COVID,” DeGrave said. “However, there’s a less severe version of the problem that we just see all over the place as well.”

Applying AI to make diagnosis predictions is a popular area of study, but DeGrave said to “be wary also of any other models for any other conditions that were trained in the problematic nature exposed in this paper.”

Janizek said he was surprised when he discovered people were planning to use these problematic models in a clinical setting.

“There needs to be more of these kinds of watchdog type papers, where people are really looking at the reproducibility of existing models and problems that exist out there,” DeGrave said.

The post UW researchers find COVID-19 diagnoses made by artificial intelligence unreliable appeared first on Artificial Intelligence.

Last year, the scientific community built thousands of machine learning models and other artificial intelligence systems to identify COVID-19 on chest X-ray and CT images. Some researchers were sceptical of the results: were the models identifying COVID-19 pathology or were they instead making decisions based on confounders such as arrows and other medically irrelevant features?

To answer this question, two medical students working toward their doctorates in computer science in Su-In Lee’s laboratory at the University of Washington rigorously audited hundreds of machine learning models intended for classifying chest X-rays as COVID-19-positive or COVID-19-negative. Results of their audit are reported in Nature Machine Intelligence.

The domain shift problem

The University of Washington researchers wanted to know whether or not published machine learning (ML) models were generalizable. A generalizable ML model will classify chest X-rays as COVID-19-positive or COVID-19-negative correctly no matter where the chest X-rays came from. A model that isn’t generalizable won’t perform well, for example, when it sees chest X-rays that were acquired at a different hospital.

Computer scientists call this drop in performance domain shift. Machine learning models affected by domain shift pick up on minute, systematic differences between datasets that are stronger and more obvious to the model than subtle indications of COVID-19 infection. These ML models then adopt shortcut learning, training on confounders like arrows and text labels and making spurious associations that emerge even when models are trained and tested on other datasets.

In this way, an ML model that uses shortcut learning will demonstrate domain shift and will not be generalizable, while an ML model that relies on medically relevant features to make decisions is more likely to be generalizable and maintain its performance across datasets.

Auditing, machine learning style

While ML models designed to classify chest X-rays tend to use similar architectures, training methods and optimization schemes, the first hurdle that the University of Washington researchers faced was recreating the published ML models.

“Models can differ in subtle ways…And instead of distributing trained models, researchers give out directions for how they made their models,” says Alex DeGrave, co-first author on the University of Washington study. “There’s a whole range of models that you could end up getting out of that set of directions due to randomness in the [model] training process.”

To reflect possible variations that might emerge during training, co-first authors DeGrave and Joseph Janizek, with their adviser and senior author Su-In Lee, first designed an ML model representative of those introduced in dozens of studies and then made minor adjustments to the representative model. They ultimately created and audited hundreds of models and classified thousands of chest X-rays.

Is it COVID-19 or just an arrow?

After introducing their models to new datasets and observing drops in classification performance indicative of domain shift and shortcut learning, the researchers decided to pinpoint the shortcuts themselves. This is challenging because the decisions made by ML models come from a “black box” – exactly how these models make classification decisions is unknown even to model designers.

DeGrave and Janizek deconstructed this “black box” with saliency maps that highlight regions that a model deems important, applying generative methods that transform images, and by manually editing images. Some saliency maps showed medically relevant areas like the lungs, while others pointed to text or arrows on an image, or to an image’s corners, suggesting that the ML models learned and decided COVID-19 status based on these features rather than pathology.

To validate these results, the researchers applied generative methods to make COVID-19-negative chest X-rays look like COVID-19-positive chest X-rays and vice versa.

“We found that if we went back and fed these [altered] images into the original networks we were auditing, it would typically fool those networks into thinking that they were images from the opposite class,” DeGrave explains. “So that means that the things these generative networks were changing were indeed things that the networks we were auditing looked at.”

The researchers again found that model performance depended upon text markers when they swapped written text on pairs of images (one COVID-19-positive and one COVID-19-negative chest X-ray). The researchers’ experiments also revealed that model architecture had little impact on model performance.

“There’s a lot of focus in the literature, I think, on ‘we have the nicest, most interesting new architecture’. We found that actually has a limited impact…whereas working with the data, and changing the data, collecting better data, had a very sizable impact,” Janizek says.

Building and auditing trustworthy AI systems

The researchers’ results indicate the gravity of shortcut learning. They also point to a need for explainable artificial intelligence, which requires that decisions made by machine learning models be understandable and traceable by humans, going forward.

So, how can researchers build machine learning networks that learn from medically relevant features and are generalizable?

DeGrave and Janizek provide several suggestions. First, researchers should collect data prospectively and with the model’s goal in mind, and datasets should be balanced with good overlap. For example, each institution involved in a study should collect COVID-19-positive and COVID-19-negative data, not one or the other. Second, clinicians should be involved in study design and data collection, and researchers should work with clinicians to identify different kinds of confounders that the ML model might rely on. Third, ML models should be audited before they are applied elsewhere.

These suggestions alone are not enough to overcome shortcut learning, the researchers say, and more research is needed. For now, they hope that this study will spark a broader dialogue about the importance of auditing ML models and the need for explainable artificial intelligence. They also want people to be more aware of the kinds of mistakes machine learning models can make.

“There are methods to explain models and detect shortcuts, there are methods to try to improve models…Researchers need to be really thinking about how all of these methods connect to each other to build not just better methods, but a better ecosystem of methods that connect with each other and make it easy for model developers to build a model that we can trust and rely on,” says Janizek.

The post Machine learning models that detect COVID-19 on chest X-rays are not suitable for clinical use appeared first on Artificial Intelligence.

Artificial intelligence (AI) may offer a way to accurately determine that a person is not infected with COVID-19. An international retrospective study finds that infection with SARS-CoV-2, the virus that causes COVID-19, creates subtle electrical changes in the heart. An AI-enhanced EKG can detect these changes and potentially be used as a rapid, reliable COVID-19 screening test to rule out COVID-19 infection.

The AI-enhanced EKG was able to detect COVID-19 infection in the test with a positive predictive value — people infected — of 37% and a negative predictive value — people not infected — of 91%. When additional normal control subjects were added to reflect a 5% prevalence of COVID-19 — similar to a real-world population — the negative predictive value jumped to 99.2%. The findings are published in Mayo Clinic Proceedings.

COVID-19 has a 10- to 14-day incubation period, which is long compared to other common viruses. Many people do not show symptoms of infection, and they could unknowingly put others at risk. Also, the turnaround time and clinical resources needed for current testing methods are substantial, and access can be a problem.

“If validated prospectively using smartphone electrodes, this will make it even simpler to diagnose COVID infection, highlighting what might be done with international collaborations,” says Paul Friedman, M.D., chair of Mayo Clinic’s Department of Cardiovascular Medicine in Rochester. Dr. Friedman is senior author of the study.

The realization of a global health crisis brought together stakeholders around the world to develop a tool that could address the need to rapidly, noninvasively and cost-effectively rule out the presence of acute COVID-19 infection. The study, which included data from racially diverse populations, was conducted through a global volunteer consortium spanning four continents and 14 countries.

“The lessons from this global working group showed what is feasible, and the need pushed members in industry and academia to partner in solving the complex questions of how to gather and transfer data from multiple centers with their own EKG systems, electronic health records and variable access to their own data,” says Suraj Kapa, M.D., a cardiac electrophysiologist at Mayo Clinic. “The relationships and data processing frameworks refined through this collaboration can support the development and validation of new algorithms in the future.”

The researchers selected patients with EKG data from around the time their COVID-19 diagnosis was confirmed by a genetic test for the SARS-Co-V-2 virus. These data were control-matched with similar EKG data from patients who were not infected with COVID-19.

Researchers used more than 26,000 of the EKGs to train the AI and nearly 4,000 others to validate its readings. Finally, the AI was tested on 7,870 EKGs not previously used. In each of these sets, the prevalence of COVID-19 was around 33%.

To accurately reflect a real-world population, more than 50,000 additional normal EKGs were then added to reach a 5% prevalence rate of COVID-19. This raised the negative predictive value of the AI from 91% to 99.2%.

Zachi Attia, Ph.D., a Mayo Clinic engineer in the Department of Cardiovascular Medicine, explains that prevalence is a variable in the calculation of positive and negative predictive values. Specifically, as the prevalence decreases, the negative predictive value increases. Dr. Attia is co-first author of the study with Dr. Kapa.

“Accuracy is one of the biggest hurdles in determining the value of any test for COVID-19,” says Dr. Attia. “Not only do we need to know the sensitivity and specificity of the test, but also the prevalence of the disease. Adding the extra control EKG data was critical to demonstrating how a variable prevalence of the disease — as we have encountered with regions having widely different rates of disease at different stages of the pandemic — would impact how the test would perform.”

“This study demonstrates the presence of a biological signal in the EKG consistent with COVID-19 infection, but it included many ill patients. While it is a hopeful signal, we must prospectively test this in asymptomatic people using smartphone-based electrodes to confirm that it can be practically used in the fight against the pandemic,” notes Dr. Friedman. “Studies are underway now to address that question.”

This study was designed and conceived by Mayo Clinic investigators, and the work was made possible in part by a philanthropic gift from the Lerer Family Charitable Foundation Inc., and by the voluntary support from participating physicians and hospitals around the world who contributed in an effort to combat the COVID-19 pandemic. Technical support was donated by GE Healthcare, Philips and Epiphany Healthcare for the transfer of EKG data.

The post Artificial Intelligence for Rapid Exclusion of COVID-19 Infection appeared first on Artificial Intelligence.

Machine learning is enabling a smooth shift in this COVID-19 struck world.

Machine learning is one of the most used technologies in this generation. It has varied capabilities that can transform businesses across industries for the better. From being considered as a niche technology, machine learning is now seeing an increased adoption within companies in all sectors.

From a global perspective, brands are leveraging machine learning to accelerate innovation and better customer experience. For example, Nike uses machine learning for personalized product recommendations. In the F&B industry, Dominos maintains its 10 minutes or less pizza delivery time using machine learning technologies. Another widely used example is how automobile giant BMW uses machine learning to analyze data from vehicle subsystems and predicts the performance of vehicle components and recommends when they should be serviced.

In 2020, machine learning became a priority for tech companies in order to achieve revenue growth while reducing costs. In 2021, those companies are now exploring many matured applications of this technology. Disruptive tech organizations have been leading this technology across many areas like process automation, customer experience, and security.

Following the continuing growth trend, these five industries are likely to adopt machine learning to change their business processes in 2021.

Healthcare Industry

The coronavirus global pandemic has highlighted the importance of investing on and optimizing the healthcare systems. Machine learning is being considered as the most promising technology that enables healthcare providers to generate large volumes of data for insightful clinical decisions. Machine learning also enables huge processes in drug discovery, cutting down the long discovery and development time and reducing overall costs. It can also improve healthcare delivery systems to better the overall quality of healthcare under low costs. In the future, machine learning is predicted to be a critical part of clinical trials. Including pharmaceuticals and the biotech industry, machine learning will be having a huge impact in all aspects.

Banking and Finance Sector

The banking sector is already seeing many advanced use cases of machine learning, especially when it comes to fraud detection and automating processes. Machine learning applications will be proactively explored in areas in trading, investment modeling, risk prevention, and customer sentiment analysis. As countries are making digital transactions their primary mode of payment, machine learning is combining predictive analytics to play a pivotal role in helping financial companies to improve transaction efficiencies within the entire transaction lifecycle. Banks and financial institutions will also use machine learning technology to customize their banking products and offerings to stay up to date in the competitive environment.

Media And Entertainment Industry

Media giants like Amazon and Netflix have already popularized the data-based content consumption channels in recent times. When the world got initially struck with the global pandemic, the demand for new consumption models grew and left companies to leverage their artificial intelligence and machine learning capabilities to create value for the customers. In this process, machine learning is going to be crucial for the media and entertainment industry , whether it’s developing better recommendation engines, delivering hyper-targeted services, or presenting the most relevant content in real-time. Predictive modeling will also be key in communicating with the customers on time, anticipating their future demands, and making good investments.

Retail And Commerce Industry

The retail industry saw a big shift owing to the coronavirus pandemic. The pandemic has disrupted many traditional practices of this industry and machine learning has become a key enabler of change. From the perspective of brick and mortar stores or e-commerce companies, machine learning is helping this sector reinvent their supply chain, inventory management, predicting user behaviour, and analyzing trends. Dynamic pricing is emerging as a key machine learning application to help retailers thrive in the competitive market.

Manufacturing Industry

IoT devices have already flooded this industry and it is only going to increase. Machine learning will be critical to bridge the gaps created by huge amounts of data. It will serve as a building block for the industry along with automation, data connectivity, real-time error detection, supply chain visibility, warehousing efficiency, cost reduction, and asset tracking. Keeping traditional processes aside, machine learning will facilitate innovation and efficiency in the coming days.

The post IN 2021, MACHINE LEARNING IS SET TO TRANSFORM THESE 5 INDUSTRIES appeared first on Artificial Intelligence.

Also: PainChek’s app picks up European and Australian regulatory clearances; Digital health access as a social determinant of health.

AI isn’t ready for COVID-19 prime time. A systematic review of published this week in nature machine intelligence warns that new models using machine learning to review chest radiographs and chest computed tomographies for COVID-19 have major methodical deficiencies or underlying biases.

Among the 62 published or pre-print papers outlining these approaches, the authors wrote that not a single one was of potential clinical use. Many were hampered by low-quality data, and in particular the high likelihood of duplicated images across different sources that result in so-called “Frankenstein datasets.” All the proposed models also suffered some degree of bias, they wrote, such as including samples from nonrepresentative populations.

“Despite the huge efforts of researchers to develop machine learning models for COVID-19 diagnosis and prognosis, we found methodological flaws and many biases throughout the literature, leading to highly optimistic reported performance,” the reviewers wrote.

“Higher-quality datasets, manuscripts with sufficient documentation to be reproducible and external validation are required to increase the likelihood of models being taken forward and integrated into future clinical trials to establish independent technical and clinical validation, as well as cost-effectiveness.”

Easy med refills for psychiatric patients. Today marked the launch of New York-based Minded, a digital service that helps those taking psychiatric medications renew, adjust, refill and order delivery of their prescriptions.

The startup, which has raised more than $5 million from investors, aims to cut down the burden and cost of regular visits to a traditional provider for assessment and prescription renewal.

Through its app-based platform, users can instead complete a five-minute online assessment regarding their mental health and a 10-minute video consultation. If appropriate, they can either have their prescription filled at a local pharmacy or delivered to their home for free.

The subscription service costs $30 per month plus $5 for each medication, and includes 24/7 access to the company’s care team and other long-term medication management support.

“Once I found what worked for me, I did not want to go to the doctor every 90 days to pay $300 for a five-minute appointment. I wanted to take the frustrating, time-consuming, and expensive process of renewing my prescription and make it magically simple,” David Ronick, Minded cofounder and CEO, said in a statement. “We’re tackling the critical issues of access and affordability facing millions of Americans.”

Regulatory wins for pain measurement app. PainChek, the maker of a pain assessment and monitoring app for smartphones, announced this week that it’s received a CE Mark and a Therapeutic Goods Administration clearance for its Universal Pain Assessment Solution.

Designed for caretakers and others providers, the tool helps assess pain severity among those who cannot adequately describe it, or otherwise document quantified pain levels for those who can self-report. With these, the company said that it’d be rolling out the app in the U.K. and Australia next month, and then moving onto the rest of Europe and other international markets.

“PainChek can become a single, simple and rapid point-of-care solution for healthcare professionals in assessing and documenting pain across all their patients, in a broad range of settings including the larger home care and hospital care markets,” CEO Philip Daffas said in a statement. “Based on initial market feedback, we expect this novel solution will be well received by our existing users and attract a wider global audience.”

Not everyone has a smartphone. A comment letter published today in NPJ Digital Medicine makes the case that access to digital tools, and subsequently mobile health technologies, is increasingly important for healthcare stakeholders to view as another social determinant of health (SDOH).

Economic access, Internet connectivity and general tech literacy are becoming core issues as care delivery is digitized and novel tools are built using software or devices, they wrote. As such, they recommended that health systems adopt “a digital-inclusion-informed strategy regarding mobile health” that not only takes access into account, but works to assess and support patients as they learn digital skills.

“Mobile health technologies hold significant promise to increase the efficiency of care and improve health outcomes. Yet, we must be cognizant of their potential to increase health disparities,” they wrote.

The post Machine learning analyses of lung imaging for COVID-19 falls short, Minded launches to streamline psychiatric med refills and more digital health news briefs appeared first on Artificial Intelligence.

Two important research breakthroughs—one in the US and the other in UK— leverage advanced artificial intelligence technologies to fight the pandemic

Clinicians, academicians and government entities around the world have leaned on new-age technologies such as artificial intelligence (AI), machine learning (ML) and data science, to track and fight the coronavirus pandemic. While AI has greatly facilitated exchanges of views and information between the scientific community, we take a look at two interesting research breakthroughs that have deployed this niche technology to fight the virus.

A team of scientists at the University of Liverpool, UK, has used AI to work out where the next novel coronavirus could emerge, BBC reports. The researchers used a combination of fundamental biology and machine learning. Their computer algorithm predicted many more potential hosts of new virus strains than have previously been detected.

The scientists say their findings could help to target the surveillance for new diseases – possibly helping prevent the next pandemic before its starts.

Using AI, another research team at the University of Southern California’s Viterbi School of Engineering developed a method to speed up the analysis of vaccines and zero in on the best potential preventive medical therapy. The method is easily adaptable to analyse potential mutations of the virus, ensuring the best possible vaccines are quickly identified —solutions that give humans a big advantage over the evolving contagion. Their machine-learning model can accomplish vaccine design cycles that once took months or years in a matter of seconds and minutes, the study says.

“This AI framework, applied to the specifics of this virus, can provide vaccine candidates within seconds and move them to clinical trials quickly to achieve preventive medical therapies without compromising safety,” said Paul Bogdan, associate professor of electrical and computer engineering at USC Viterbi and corresponding author of the study. “Moreover, this can be adapted to help us stay ahead of the coronavirus as it mutates around the world.”

The AI-assisted method predicted 26 potential vaccines that would work against the coronavirus. From those, the scientists identified the best 11 from which to construct a multi-epitope vaccine, which can attack the spike proteins that the coronavirus uses to bind and penetrate a host cell.

Moreover, the engineers can construct a new multi-epitope vaccine for a new virus in less than a minute and validate its quality within an hour. By contrast, current processes to control the virus require growing the pathogen in the lab, deactivating it and injecting the virus that caused a disease. The process is time-consuming and takes more than one year; meanwhile, the disease spreads.

USC’s AI-assisted method will be especially useful during this stage of the pandemic as the coronavirus begins to mutate in populations around the world. Some scientists are concerned that the mutations may minimise the effectiveness of vaccines which are now being distributed.

The post Tech tactics: How Artificial Intelligence is aiding the fight against Covid-19 appeared first on Artificial Intelligence.

DataRobot’s AI platform makes it easy to build predictive models to identify infection hotspots

A new data-driven project is launching with the ambitious aim of resolving the COVID-19 pandemic in 60 days. 

ContagionNET has been designed by DataRobot, a global enterprise AI company that enables easy building and deployment of predictive models. Aimed at solving the data sharing problem that has plagued the COVID-19 pandemic response, the not-for-profit initiative will help determine when a patient is at their most contagious.

ContagionNET combines affordable antigen checks performed at home, anonymous data collection, and an AI platform. With these tools DataRobot are aiming to identify the most contagious areas in a community and encourage people to take preventative action. The goal is to identify people earlier than traditional testing (i.e., when they are most infectious) to disrupt the chain of transmission. 

ContagionNET can also predict how many people need to participate on a county-by-county basis by measuring the impact of different participation levels and testing frequency. This localised approach allows those with the highest viral load to self-isolate earlier, preventing further spread of the virus. 

“The reality is that those most likely to spread COVID-19 aren’t usually aware they are contagious” said Sally Embrey, DataRobot’s VP of Public Health and Health Technologies. 

“With the proliferation of frequent, easy-to-use at home antigen checks, ContagionNET is poised to inform the most contagious of their risk and help them make lifestyle modifications that will significantly reduce spread of the virus.”

DataRobot has been involved in pandemic response since early 2020. Working with the US government, DataRobot’s forecasting helped vaccine manufacturers select the right participants and prioritise enrollment in the highest risk locations over each vaccine trial period. 

The organisation has also used modeling to improve testing distribution, equity, and diagnostic reporting, all of which have informed its approach to ContagionNET. 

The post DataRobot launches solution to defeat Covid-19 appeared first on Artificial Intelligence.

Machine learning tools analyzed tweets about COVID-19 and showed that social media can significantly influence people’s health beliefs.

 Using machine learning, researchers found that people’s biases about COVID-19 and its treatments are exacerbated when they read tweets from other users, a study published in JMIR showed.

The analysis also revealed that scientific events, like scientific publications, and non-scientific events, like speeches from politicians, equally influence health belief trends on social media.

The rapid spread of COVID-19 has resulted in an explosion of accurate and inaccurate information related to the pandemic – mainly across social media platforms, researchers noted.

“In the pandemic, social media has contributed to much of the information and misinformation and bias of the public’s attitude toward the disease, treatment and policy,” said corresponding study author Yuan Luo, chief Artificial Intelligence officer at the Institute for Augmented Intelligence in Medicine at Northwestern University Feinberg School of Medicine.

“Our study helps people to realize and re-think the personal decisions that they make when facing the pandemic. The study sends an ‘alert’ to the audience that the information they encounter daily might be right or wrong, and guide them to pick the information endorsed by solid scientific evidence. We also wanted to provide useful insight for scientists or healthcare providers, so that they can more effectively broadcast their voice to targeted audiences.”

Researchers set out to evaluate individuals’ COVID-19-related health beliefs on Twitter. The team retrospectively collected COVID-19-related tweets using the Twitter API. In total, they gathered 92,687,660 tweets corresponding to 8,967,986 users from January 6 to June 21, 2020. To train the machine learning model, the team randomly selected 5,000 of the tweets for annotation.

Researchers used machine learning to review each tweet doubly to determine if they met any of the four core constructs of the health belief model (HBM), a framework developed to investigate people’s beliefs about health problems. The HBM’s four core constructs include perceived susceptibility, perceived severity, perceived benefits, and perceived barriers.

The results showed that the machine learning tools achieved areas under the receiver operating characteristic curve of 0.86 for the classification of all four HBM constructs. The team pointed out that fluctuations in the number of health belief-related tweets could reflect dynamics in case and death statistics, systematic interventions, and public events.

Specifically, researchers found that scientific events and non-scientific events were comparable in their ability to influence health belief trends on social media.

“Our findings demonstrated that trends in health beliefs were correlated with dynamics in positive case and mortality rates. Additionally, we observed a decline in perceived disease susceptibility during government-issued lockdowns, while perceived severity appeared unaltered. Lastly, our study identified top news events, scientific and nonscientific, that may play a role in altering health beliefs,” researchers said.

“These findings lay the groundwork to better understand how the general public’s COVID-19-related health beliefs are influenced by case and mortality rates, government policies, current news, and significant events.”

The group noted that this study is unique in that integrates machine learning algorithms with classic epidemiology models to retrospectively investigate the contents on social media and its effects. Researchers also worked to improve the machine learning model’s interpretability, allowing other investigators to understand how this algorithm works.

“We identified the fluctuating trends of public attitudes from the tweets, then aligned the important scientific and non-scientific events that are associated with these trends,” Luo said. “As a result, we are offering insights people can take action on.”

The research team is currently integrating machine learning and deep learning to understand how social media can impact the general public’s attitude toward COVID-19 vaccines. The overall aim of this effort is to identify specific public concerns and inform targeted vaccination campaigns to maximize inoculation impact. Additionally, the group is exploring the use of social media to detect gender and racial disparities during and beyond the pandemic.

The team expects that their study can help inform public health strategies for reducing the spread of COVID-19 misinformation.

“The excessive information disseminated on social media platforms and other sources is closely related to the dynamics of the general public’s health beliefs. The dynamics of the pandemic, news, scientific and nonscientific events, and even the related tweets already published on social media platforms may influence the health beliefs of the general public on social media to some extent,” researchers concluded.

“Our findings provide clues and evidence for more effective management of the infodemic associated with the COVID-19 pandemic.”

Data analytics tools and social media platforms have played a significant role in tracking the attitude of the public throughout the pandemic. In May 2020, a team from Penn Medicine showed that public health officials can use natural language processing techniques to track surges in interest in COVID-19 topics on online forums like Reddit.

“Public health priorities do not always align with community priorities, and the success of public health efforts often depends on having a plan to address community concerns,” said Daniel Stokes, a research fellow with the Center for Emergency Care Policy and the Center for Digital Health at Penn Medicine.

“Having a source like Reddit that is directly tied to people’s thoughts could prove invaluable in crafting plans that meet people where they are.”

The post Machine Learning Shows Social Media Greatly Affects COVID-19 Beliefs appeared first on Artificial Intelligence.

A new machine learning approach to COVID-19 testing has produced encouraging results in Greece. The technology, named Eva, dynamically used recent testing results collected at the Greek border to detect and limit the importation of asymptomatic COVID-19 cases among arriving international passengers between August and November 2020, which helped contain the number of cases and deaths in the country.

The findings of the project are explained in a paper titled “Deploying an Artificial Intelligence System for COVID-19 Testing at the Greek Border,” authored by Hamsa Bastani, a Wharton professor of operations, information and decisions and affiliated faculty at Analytics at Wharton; Kimon Drakopoulos and Vishal Gupta from the University of Southern California; Jon Vlachogiannis from investment advisory firm Agent Risk; Christos Hadjicristodoulou from the University of Thessaly; and Pagona Lagiou, Gkikas Magiorkinis, Dimitrios Paraskevis and Sotirios Tsiodras from the University of Athens.

The analysis showed that Eva on average identified 1.85 times more asymptomatic, infected travelers than what conventional, random surveillance testing would have achieved. During the peak travel season of August and September, the detection of infection rates was up to two to four times higher than random testing.

“Our work paves the way for leveraging [artificial intelligence] and real-time data for public health goals, such as border control during a pandemic,” the paper stated. With the rapid spread of a new coronavirus strain, Eva also holds the promise of maximizing the already overburdened testing infrastructure in most countries.

“The main issue was, given the fixed budget for tests, whether we could conduct the tests in a smarter way with dynamic surveillance to identify more infected travelers,” said Bastani. One of the biggest challenges governments face in dealing with COVID-19 is the inability of the testing infrastructure at their national borders to realistically check every arriving passenger. Such comprehensive testing would be both costly and time-consuming, which is why most countries screen either arriving passengers from specific countries or conduct random testing for COVID-19.

“The main issue was, given the fixed budget for tests, whether we could conduct the tests in a smarter way with dynamic surveillance to identify more infected travelers.”–Hamsa Bastani

Eva also allowed Greece to identify when a country was exhibiting a spike in COVID-19 infections a median of nine days earlier than what would have been possible with machine learning-based algorithms using only publicly available data.

The underlying technology of Eva is a “contextual bandit algorithm,” a machine-learning framework built for “sequential decision-making,” taking into account various practical challenges like time-varying information and port-specific testing budgets, Bastani explained. The algorithm balances the need to maintain high-quality surveillance estimates of COVID-19 prevalence across countries and the allocation of limited testing results to catch likely infected travelers. Eva is the first instance of that technology being applied to address a public health challenge, although such algorithms have found use in online advertising and A/B testing, she added.

Overcoming Data Challenges

Eva is an advancement over conventional border control policies because it does not rely on publicly reported data, which has a number of issues.

Publicly reported data is of “poor quality” chiefly because different countries follow different reporting protocols and testing strategies. It is common to focus testing resources on symptomatic patients, but the resulting prevalence rate may not be reflective of the asymptomatic population that is likely to travel. There is often also a reporting delay due to poor infrastructure, said Bastani. “We can tell, based on the data we’re actively collecting at borders, that a country’s COVID cases are spiking typically nine days before you will see that reflected in the public data.”

“Testing is usually targeted towards symptomatic individuals rather than asymptomatic individuals,” Bastani said in an interview with the Wharton Business Daily radio show on SiriusXM last July, as the Greek deployment was getting underway. (Listen to the podcast from that episode above). “You can imagine tourists who are coming in are probably asymptomatic.” That underscores the criticality of not relying on publicly reported data, but using data that accurately reflects the prevalence of asymptomatic COVID-19 travelers across countries.

Eva’s algorithm overcomes the poor quality of public data by dynamically collecting testing results at the Greek border, thereby maintaining high-quality surveillance estimates of the prevalence in each country. “By adaptively adjusting border policies nine days earlier, Eva prevented additional infected travelers from arriving,” the paper noted, referring to the Greece deployment. “That is a long period of time in which a lot of high-risk people would probably have come in and infected other citizens,” said Bastani.

It is common for border control policies to use publicly reported data, but such data are often unreliable and inconsistent across countries, said Bastani. The inconsistencies arise from censorship of testing data by some countries, and even varying definitions of a COVID-19 death, she added. She pointed to the recent discovery of undercounting of COVID-19 deaths in nursing homes in New York City as an example of flawed data. “That issue is exacerbated when you compare death counts in different countries because in some places they’re accounting very accurately and in other places they’re not.”

Greece is the first country to design border controls based on the dynamic random surveillance testing approach that Eva uses. The model specifies the infrastructure required to collect COVID-19 test results, using those to form estimates and to inform future testing decisions in a dynamic feedback loop.

“No country should just be relying on public data; they should be actively monitoring who is coming to their borders, testing at least a subset of them, and using that to make informed decisions about border control.”–Hamsa Bastani

In using the Eva model, Greece required every individual or family planning to enter the country to fill out 24 hours before arrival a digitized “Passenger Locator Form,” where they provided some basic information about themselves such as other countries they have visited in the past year. All those who submitted those forms received a QR code that allowed tracking. Eva’s algorithm processes the information in the forms to identify those who need to get tested for COVID-19. Greece’s border control authorities processed an average of 38,500 forms each day; some 18% of those who submitted the forms did not eventually show up.

Keeping COVID-19 at Bay

Eva’s targeted testing that allowed for adaptive border control policies helped Greece keep its case count “very low pretty much all of the summer” said Bastani. The country was able to maintain some economic activity, unlike many others that had to completely shut down, she noted. Greece imposed a second lockdown and travel restrictions in November after a spike in COVID-19 cases.

The Greek government acknowledged Eva’s accomplishments in a press conference last July. “The AI system developed by Bastani, Drakopoulos, Gupta, and Vlachogiannis has been an asset both for preparing the opening of the country to visitors from all over the world, as well as for allowing flexibility in decision-making regarding our COVID-19 strategy,” said Nikos Hardalias, Greece’s civil protection and deputy minister for crisis management, who heads the COVID-19 Response Taskforce for the country.

Free-to-use Technology

Eva is an open-source technology, which means Bastani and her team will provide it free of cost to any country that might want it. They have made presentations to COVID task forces in several countries in the European Union. Adapting it to other countries would involve designing passenger locator forms that are appropriate for different immigration processes and dovetailing back-end resources such as testing labs.

Bastani made a strong pitch for governments to capture private data such as that generated by the passenger locator forms used in the Greece deployment, and customize them to suit their specific situations. “No country should just be relying on public data; they should be actively monitoring who is coming to their borders, testing at least a subset of them, and using that to make informed decisions about border control,” she said. “That said, if a country doesn’t have the resources to do that, it’s probably better to use a policy that mimics another country that is doing that rather than relying only on public data.”

Bastani and her colleagues are working on refining Eva to incorporate more passenger-specific information than they used in the Greece deployment. Europe’s General Data Protection Regulation limited the scope of data they could use with Eva; they used only anonymized and aggregated data with limited demographic information. Other countries with less stringent data protection regulations could gather a wider range of data, such as on occupation, Bastani said. “We know that certain occupations carry a much higher COVID-19 risk than others.”

Eva could also be trained to incorporate pooling to mitigate constraints faced by testing labs, she added. Overloaded labs could share their samples with other labs that may have spare capacity at any given point in time, she explained. In much the same way, Eva could also use dynamic data to help determine optimal staffing levels at labs and other locations in the testing infrastructure, she added.

The post How Artificial Intelligence Can Slow the Spread of COVID-19 appeared first on Artificial Intelligence.

The overwhelming increase in critically ill COVID-19 patients, who urgently require intensive care units (ICU) and emergency departments (ED), has challenged healthcare systems worldwide.

The unprecedented and large number of patients, especially in the regions where the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic has hit badly, has created an immediate need for novel approaches to dealing with the issue.  One such approach is the application of artificial intelligence (AI). Now, in a new research paper published on the medRxiv* server, an international team of scientists has systematically reviewed and critically appraised the current evidence on AI applications for COVID-19 in intensive care and emergency settings, focusing on methods, reporting standards, and clinical utility.

Artificial intelligence uses computational methods to replicate human intelligence. Two branches of AI, namely, deep learning and machine learning, are involved in the automatic development of computer programs through experience. In medical research, various regression models such as logistic, linear, or Cox regressions are used to develop AI-based applications. These models are the simplest form of machine learning. However, more recently, advanced and complex forms of AI, i.e., machine learning, including neural networks, random forest models, and support vector machines, are becoming more popular in medical research.

Prior research has shown that AI could assist with the automated monitoring of patients in intensive care and emergency settings, prognostication, and optimal allocation of staff. Previous systematic reviews have also revealed the issues concerning the quality of COVID-19 prediction models developed for the disease’s diagnosis and prognosis. The research concluded that the machine learning studies’ limitations are inadequate sample size and insufficient validation of predictions. In the current scenario where the rate of hospitalization is exceptionally high, researchers mainly focus on the use of AI for optimization of ICU bed usage.

Presently, not much information is available about AI’s role as a decisive technology in the clinical management of COVID-19 patients in ICU and emergency settings. Thereby, scientists have systematically analyzed the existing documents involving the application of AI for COVID-19 patients admitted to the intensive care unit of a hospital. They have also focused on the clinical efficacy, various methods, and reporting standards associated with the emergency settings.

For this study, a thorough review of the literature, available in IEEE Xplore, Scopus, Embase, ACM Digital Library, CINAHL, and PubMed, was conducted from inception pandemic to 1st October 2020. The literature review was done across papers published in different languages. All the articles were associated with the application of AI for COVID-19 patients, healthcare resources in the intensive care unit, emergency, prehospital settings, and health care workers. For the predictive modeling studies, researchers have used various tools such as the prediction model risk of bias assessment tool (PROBAST) and a modified transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD).

Among the fourteen studies that were analyzed, eleven developed predictive AI-based diagnostic models. Two of the three remaining studies showed the development of lung segmentation software (based on deep learning) used for prognosis, and the remaining study was associated with the optimization in the ICU.

All of these studies were assessed to be at a high risk of bias. Some of the common drawbacks of these studies were poor handling of missing data, weak validation of models, small sample sizes, and failure to account for censored participants. Among the studies, the most common source of bias that commonly prevailed was the inadequate sample size. A small sample size leads to the risk of over-fitting and model optimism. Missing data also leads to significant error in the model, and ideally, the percentage of the missing variable must be reported. In the case of diagnostic studies, bias was introduced while using the reverse transcription-polymerase chain reaction (RT-PCR) test as many a time a false-negative report arises in the diagnostic model and validation study. Additionally, poor reporting on model calibration, development of proper guidelines, and lack of accurate reports of predictor assessment have failed to validate the research in clinical settings.

The current systematic review has shown that despite the rapid development of novel technologies to contain the COVID-19 pandemic, there is a shortage in AI-based applications for clinical applicability. A valuable improvement in the development and deployment of AI applications in emergency settings could help combat the current situation, which requires optimal emergency resources usage. Integration of new AI-specific reporting guidelines such as SPIRIT-AI and CONSORT-AI into research would help develop novel AI-based applications. Such applications would help the health care system to fight the COVID-19 pandemic and other future pandemics. Researchers have emphasized the need for interdisciplinary collaboration between AI developers and medical experts.

The post Applications of Artificial Intelligence in COVID-19 ICU and ER situations appeared first on Artificial Intelligence.