New AI institute to focus on the speech language pathology needs of children



The University of Illinois is part of a nine-university consortium led by the University of Buffalo that has been awarded a $20 million grant by the National Science Foundation to establish a national institute that develops artificial intelligence systems that identify and assist young children with speech and/or language processing challenges. The award will establish the AI Institute for Exceptional Education to advance foundational AI technologies, human-centered AI design, and learning science that improve educational outcomes for young children. 

The institute will help address the nationwide shortage of speech-language pathologists and provide services to children ages 3 to 10 who are at increased risk of falling behind in their academic and socio-emotional development – issues exacerbated by the COVID-19 pandemic.

Pamela Hadley, professor and head of the Department of Speech and Hearing Science at the University of Illinois Urbana-Champaign, is one of the co-principal investigators for the grant. 

“In light of the shortage of speech-language pathologists nationwide, there is a pressing need to develop health technologies that can help identify young children at-risk for speech and language disorders at younger ages and do so more efficiently,” said Hadley, a fellow of the American Speech-Language Hearing Association. “Our multidisciplinary team will enhance automatic speech recognition systems, improving early identification and interventions for children with developmental language disorder and other conditions that affect speech and language. Our team will also create advanced artificial intelligence systems that will support tailored interventions for children on the caseloads of speech-language pathologists. By doing so, we will create educational environments that help children thrive socially and academically.”

Institute will help underserved students

The AI Institute for Exceptional Education will focus on serving the millions of children nationwide who, under the Individuals with Disabilities Education Act, require speech and language services.
Specially, it will develop two advanced AI solutions: the AI Screener for early identification of potential speech and/or language disorders; and the AI Orchestrator, which will act as a virtual teaching assistant by providing students with ability-based interventions.

The AI Screener will listen to and observe children in the classroom, collecting samples of children’s speech, facial expressions, gestures and other data. It will create weekly summaries of these interactions that catalogue each child’s vocabulary, pronunciation, video snippets and more. These summaries will help teachers monitor their students’ speech and language abilities and, if needed, suggest a formal evaluation with a speech-language pathologist.

The AI Orchestrator is an app that will help speech-language pathologists, most of whom have caseloads so large that they must provide group-based interventions for children instead of individualized care. The app addresses this by recommending personalized content tailored to students’ needs. It continues to monitor students’ progress and adjusts lesson plans to ensure that the interventions are working.

“The AI Institute for Exceptional Education follows 18 already established NSF-led AI Institutes, an ecosystem of AI research and education in pursuit of transformational advances in AI research and development of AI-powered innovation,” NSF Program Director James Donlon said. “We are happy to welcome this new team to the AI Institutes program.”

Institute comprises top research universities

The institute will consist of more than 30 researchers from nine universities including the University of Buffalo; Stanford University; the University of Washington; Cornell University; the University of Nevada, Reno; the University of Texas at El Paso; Penn State University; and the University of Oregon.

Other investigators at Illinois are Heng Ji (Computer Science), Mark Hasegawa-Johnson (Electrical and Computer Engineering), Yun Huang (Information Science), Hedda Meadan-Kaplansky (Special Education), and Windi Krok (Speech and Hearing Science).

“We are eager to see how this team advances AI research to develop better solutions for children with specific speech-language needs, as well as their families and the U.S. schools who serve them. This project is a great example of how we can harness the opportunities that AI technologies can offer to enhance the services that our nation can offer the American people,” NSF Program Director Fengfeng Ke said.

Editor’s note:

To reach Vince Lara-Cinisomo, email vinlara@illinois.edu.
 

Related news

Toy Talk promotes language development



Research shows that the more language-rich interactions children have with their parents, the faster they learn words and the better they understand them. Toys can help facilitate language-rich interactions.

The quantity and quality of interactions between parents and children are critical in early language development. Research has shown that the more language-rich interactions children have with their parents, the faster they learn words and the better they understand them. The quality of the interaction is also important, especially in terms of the responsiveness to children’s attempts to communicate.

Responsive Labeling, Self-talk, Parallel-talk

Language interventionists have typically relied upon three main language modeling strategies when working with parents to increase their responsiveness. The rest, responsive labeling, occurs when the parent labels an object that the child is playing with, saying, for example, “That’s a baby.” In self-talk, parents describe their own actions with the toy, for example, “I’m rocking the baby.” Parallel talk involves the parent describing the child’s actions with the toy, for example, “You’re feeding the baby.” Research has shown that these language modeling strategies lead to increases in the vocabulary used by toddlers and the length of sentences they produce. Dr. Pamela Hadley and Dr. Matthew Rispoli, associate professors in the Department of Speech and Hearing Science, were concerned that the language modeling strategies did not do enough to increase toddlers’ development of syntax, or the way words are combined to form sentences.

“These strategies—responsive labeling, self-talk, and parallel talk—actually reduce the diversity of the words in the input to the child, especially in the number of different words that appear as sentence subjects,” Dr. Hadley said. “They promote pronoun subjects such as it, that, you, and I to the exclusion of vast numbers of possible noun subjects.”

Toy Talk

Pam Hadley and Matt Rispoli

To increase the number of different words appearing as sentence subjects during interactions with children, Drs. Hadley and Rispoli designed a new language modeling strategy they call toy talk. The strategy shifts parent-child talk during play from the interpersonal space, or what the parent and child are doing, to descriptive talk about the toy itself, such as its location, properties, or actions in the play environment. Parents also are taught to give the object its name.

“Consider a child holding a bottle to a doll’s mouth,” Dr. Hadley said. “Instead of responding with ‘That’s a bottle,’ which is labeling, or ‘You’re feeding the baby,’ which is parallel talk, the parent could say, ‘The baby likes her juice’ or ‘The juice is gone.’ That’s toy talk.” Both toy talk sentences have noun subjects rather than pronouns, a subtle shift, she notes, but one that creates opportunities for parents to produce more diverse sentences.

It sounds simple but, perhaps surprisingly, toy talk sentences with nouns in the subject position are rare in naturally-occurring conversations between adults and young children, Dr. Rispoli noted. “It is much more common for adults to ask children questions—‘Are you feeding the baby?’—or to direct their behavior—‘Give the baby more juice’—or to make descriptive statements using pronoun subjects—‘It’s all gone,’” he said.

Toy Talk Benefits

The challenge of language acquisition has been described as putting words together. “But maybe the challenge is pulling words apart,” he said. “When children consistently hear phrases such as ‘It’s a doll,’ ‘That’s a horse,’ and so on, the subject and the verb get chunked together. The child may not understand that ‘itsa’ and ‘thatsa’ are actually three separate words.”

With funding from the National Institute of Child Health and Human Development, Drs. Hadley and Rispoli evaluated the effectiveness of toy talk in a study that taught parents of toddlers how to use toy talk in both group and individualized coaching sessions over a three-month period. Their study demonstrated that not only did parents’ use of toy talk sentences increase following the instruction but also that their use of toy talk predicted children’s rate of growth in the production of diverse simple sentences and other crucial elements of syntactic development over the following six months.

“We think toy talk works, in part, because the diversity of noun subjects in parents’ input makes it easier for children to identify the boundary between a subject and a verb,” Dr. Hadley said. She and Dr. Rispoli emphasized that toy talk is not a replacement for other language modeling strategies. “Rather, it should be integrated with other strategies to interpret and expand children’s communication attempts and to model diverse combinations of words within simple sentence structure,” she said.

Because toy talk represents a relatively minor modification of familiar language modeling strategies, both scholars believe it can be incorporated rapidly into existing clinical practice.

Related news

Department of Speech and Hearing Science
901 South Sixth Street
M/C 482
Champaign, IL 61820
217-333-2230