Assistive technologies such as handheld tablets and eye-tracking devices are increasingly helping give voice to individuals with paralysis and speech impediments who otherwise would not be able to communicate. Now, researchers are directly harnessing electrical brain activity to help these individuals.

In a study published Wednesday in the New England Journal of Medicine, researchers at the University of California, San Francisco, describe an approach that combines a brain-computer interface and machine learning models that allowed them to generate text from the electrical brain activity of a patient paralyzed because of a stroke.

Other brain-computer interfaces, which transform brain signals into commands, have used neural activity while individuals attempted handwriting movements to produce letters. In a departure from previous work, the new study taps into the speech production areas of the brain to generate entire words and sentences that show up on a screen.

This may be a more direct and effective way of producing speech and helping patients communicate than using a computer to spell out letters one by one, said David Moses, a UCSF postdoctoral researcher and first author of the paper.Related: 

Virtual vocal tract creates speech from brain signals, a potential aid for ALS and stroke patients

The study was conducted in a single 36-year-old patient with anarthria, a condition that renders people unable to articulate words because they lose control of muscles tied to speech, including in the larynx, lips, and tongue. The anarthria was brought on by a stroke more than 15 years ago that paralyzed the man.

The researchers implanted an array of electrodes in the patient’s brain, in the area that controls the vocal tracts, known as the sensorimotor cortex. They measured the electrical activity in the patient’s brain while he was trying to say a word and used a machine learning algorithm to then match brain signals with specific words. With this code, the scientists prompted the patient with sentences and asked him to read them, as though he were tying to say them out loud. The algorithm interpreted what the patient was trying to say with 75% accuracy.

Although the experiment was only conducted in one patient and only included asking the patient to try to say up to 50 words, the study shows that “the critical neural signals [for speech production] exist and that they can be leveraged for this application,” said Vikash Gilja, an associate professor at the University of California, San Diego, who was not involved in the study.

To Moses and his team, this study represents a proof of concept. “We started with a small vocabulary to prove in principle that this is possible,” he said, and it was. “Moving forward, if someone was trying to get brain surgery to get a device that could help them communicate, they would want to be able to express sentences made up of more than just those 50 words.”

STAT spoke with Moses to learn more about the development of the technology and how it could be applied in the future. This interview has been edited for length and clarity.

What problems were you seeking to address?

It’s kind of easy for us to take speech for granted. We have met people who are unable to speak because of paralysis, and it can be an extremely devastating condition for them to be in. It hadn’t been understood before if the brain signals that normally control the vocal tract can be recorded by an implanted neural device and translated into attempted speech.

Can you describe how the technology works? What information goes into it and how is that analyzed to produce words?

This is in no way mind reading; our system is able to generate words based on the person’s attempts to speak. While he’s trying to say the words that he’s presented [with], we record his brain activity, use machine learning models to detect subtle patterns, and understand how those patterns are associated with words. Then we use those models with a natural language model to decode actual sentences when he is trying to speak.

What’s the importance of including the natural language model?

You could imagine when you’re typing on your phone and it figures, “Oh, this might not be what you want to say,” that can be very helpful. Even with the results that we report, it’s imperfect. It helps to be able to use the language model and the structure of English to improve your predictions.

What was surprising to you about what you learned during this study?

One of the very pleasant surprises was that you were able to see these functional patterns of brain activity that have remained intact for someone who hasn’t spoken in over a decade. As long as someone [can imagine] producing the sounds of what their vocal tract would normally do, it’s possible for us to be able to record that activity and identify these patterns.

How did you feel when you realized that the system was in fact producing the words that the patient was trying to say?

My first thought was “OK, that’s just one sentence. It could have been a fluke.” But then when we saw that it was working sentence after sentence. It was extremely thrilling and rewarding. I know that the participant also felt this way, because you can tell from looking at him that he was getting very excited.

What are the next steps to improving the system?

We need to validate this in more than one person. And we want to know how far this technology can go. Can this, for example, be used to help someone who’s locked in completely — who only has eye movements and cannot move any other muscles. If we show that it can work reliably in people with that level of paralysis, then I think that that’s a strong indicator that this is really a viable approach.

How do you envision this technology being applied in the future?

The ultimate goal really for us is to completely restore speech to someone who’s lost it. That would mean any sound someone wants to make, the system is able to produce that sound for them by synthesizing their voice. You could even restore some personal aspects of the speech, such as intonation, pitch, and accent. It’s going to be a lot of effort and we have a lot of work to do, but I think this is a really strong start.

The post Researchers use machine learning to translate brain signals from a paralyzed patient into text appeared first on Artificial Intelligence.

Patient safety, data privacy, and health equity are key considerations for the use of chatbots powered by artificial intelligence in healthcare, according to a viewpoint piece published in JAMA.

With the emergence of COVID-19 and social distancing guidelines, more healthcare systems are exploring and deploying automated chatbots, the authors noted. However, there are several key considerations organizations should keep in mind before implementing these tools.

“We need to recognize that this is relatively new technology and even for the older systems that were in place, the data are limited,” said the viewpoint’s lead author, John D. McGreevey III, MD, an associate professor of Medicine in the Perelman School of Medicine at the University of Pennsylvania.

“Any efforts also need to realize that much of the data we have comes from research, not widespread clinical implementation. Knowing that, evaluation of these systems must be robust when they enter the clinical space, and those operating them should be nimble enough to adapt quickly to feedback.”

The authors outlined 12 different focus areas that leaders should consider when planning to implement a chatbot or conversational agent (CA) in clinical care. For chatbots that use natural language processing, the messages these agents send to patients are extremely significant, as are patient’s reactions to them.

“It is important to recognize the potential, as noted in the NAM report, that CAs will raise questions of trust and may change patient-clinician relationships. A most basic question is to what extent CAs should extend the capabilities of clinicians (augmented intelligence) or replace them (artificial intelligence),” the authors said.

“Likewise, determining the scope of the authority of CAs requires examination of appropriate clinical scenarios and the latitude for patient engagement.”

The authors considered the example of someone telling a chatbot something as serious as “I want to hurt myself.” In this case, the patient safety element is brought to the forefront, as someone would need to be monitoring the chatbot often.

This hypothetical situation also raises the question of whether patients would take a response from a chatbot seriously, as well as who is responsible if the chatbot fails in its task.

“Even though technologies to determine mood, tone, and intent are becoming more sophisticated, they are not yet universally deployed in CAs nor validated for most populations,” the authors said.

“Moreover, there is no mention of CAs in the US Food and Drug Administration’s (FDA) proposed regulatory framework for AI or machine learning for software as a medical device nor is there a user’s guide for deploying these platforms in clinical settings.”

The authors also noted that regulatory organizations like the FDA should develop frameworks for appropriate classification and oversight of CAs in healthcare. For example, policymakers could classify CAs as low risk versus higher risk.

“Low-risk CAs might be less automated, structured for a specialized task, and have relatively minor consequences if they fail. A CA that guides patients to appointments might be one such example,” the authors wrote.

“In contrast, higher-risk CAs would involve more automation (natural language processing, machine learning), unstructured, open-ended dialogue with patients, and have potentially serious patient consequences in the event of system failure. Examples of higher-risk CAs might be those that advise patients after hospital discharge or offer recommendations to patients about titrating medications.”

Additionally, the authors noted that in partnerships between vendors and healthcare organizations to use CAs, all should be mindful of converging incentives and work to balance these goals with attention to each of the domains.

“Given the potential of CAs to benefit patients and clinicians, continued innovation should be supported. However, hacking of CA systems (as with other medical systems) represents a cybersecurity threat, perhaps allowing individuals with malicious intent to manipulate patient-CA interactions and even offer harmful recommendations, such as quadrupling an anticoagulant dose,” the authors stated.

The authors stated that ultimately, the successful and effective deployment of chatbots in healthcare will depend on the industry’s ability to assess these tools.

“Conversational agents are just beginning in clinical practice settings, with COVID-19 spurring greater interest in this field. The use of CAs may improve health outcomes and lower costs. Researchers and developers, in partnership with patients and clinicians, should rigorously evaluate these programs,” the authors concluded.

“Further consideration and investigation involving CAs and related technologies will be necessary, not only to determine their potential benefits but also to establish transparency, appropriate oversight, and safety.”

Healthcare leaders will need to ensure they continually evaluate the capacity of these tools to improve care delivery.

“It’s our belief that the work is not done when the conversational agent is deployed,” McGreevey said. “These are going to be increasingly impactful technologies that deserve to be monitored not just before they are launched, but continuously throughout the life cycle of their work with patients.”

The post Patient Safety, Data Privacy Key for Use of AI-Powered Chatbots appeared first on Artificial Intelligence.

A team at Johns Hopkins University has received a $195,000 Rapid Response Research grant from the National Science Foundation to use machine learning to detect which COVID-19 patients are at high risk of heart damage.

For more coronavirus updates, visit our resource page, updated twice daily by Xtelligent Healthcare Media.

Evidence has shown that COVID-19 can have a negative impact on the cardiovascular system, leaving patients at risk for adverse events such as heart failure, sustained abnormal heartbeats, heart attacks, and death. Because of the increased risk for these complications, there is a significant need to identify COVID-19 patients at high risk for heart problems, but these predictive capabilities don’t currently exist.

With this grant, Johns Hopkins researchers will aim to develop these capabilities using machine learning.

“This project will provide clinicians with early warning signs and ensure that resources are allocated to patients with the greatest need,” said Natalia Trayanova, the Murray B. Sachs Professor in the Department of Biomedical Engineering at The Johns Hopkins University Schools of Engineering and Medicine and the project’s principal investigator.

The first phase of the one-year project just received IRB approval for Suburban Hospital and Sibley Memorial Hospital within the Johns Hopkins Health System (JHHS).

In this first phase, researchers will collect data from more than 300 COVID-19 patients admitted to JHHS, including cardiac-specific laboratory tests, continuously-obtained vital signs, and imaging data like CT scans echocardiography. The team will use this data to train the machine learning algorithm.

Researchers will then test the algorithm using data from COVID-19 patients with heart injury at JHHS, other hospitals nearby, and maybe some in New York City. The overarching goal is to create a predictive risk score that can determine which patients are at high risk of developing adverse cardiac events up to 24 hours ahead of time. For new patients, the model will perform a baseline prediction that is updated each time new health data becomes available.

According to the research team, this will be the first approach to predict COVID-19-related cardiovascular outcomes. While similar studies exist, previous research has focused on predictions of general COVID-19 mortality or a patient’s need for ICU care.

This new machine learning approach will analyze multiple sources of data to produce a risk score that is continually updated as researchers acquire new data.

The project will also help providers understand how COVID-19-related heart injury could lead to heart dysfunction and sudden cardiac death. The study will also help clinicians determine which biomarkers are most predictive of adverse clinical outcomes. After creating and testing the algorithm, researchers will make the tool widely available for healthcare institutions to implement.

“As a clinician, major knowledge gaps exist in the ideal approach to risk stratify COVID-19 patients for new heart problems that are common and may be life-threatening. These patients have varying clinical presentations and a very unpredictable hospital course,” said Allison G. Hays, Associate Professor of Medicine in the Johns Hopkins University School of Medicine’s Division of Cardiology and the project’s clinical collaborator.

“This project aims to help clinicians quickly risk stratify patients using real time clinical data, with the goal of widely disseminating this knowledge to help medical practitioners around the world in their approach to treating and monitoring patients suffering from COVID-19.”

This project will help researchers obtain information critical to fighting COVID-19.

“By predicting who’s at risk for developing the worst outcomes, healthcare professionals will be able to undertake the best routes of therapy or primary prevention and save lives,” said Trayanova.

The post Machine Learning Helps Detect Heart Damage in COVID-19 Patients appeared first on Artificial Intelligence.