The app first administers caregiver consent forms and survey questions and then uses the phone's 'selfie' camera to collect videos of young children's reactions while they watch movies designed to elicit autism risk behaviors, such as patterns of emotion and attention, on the device's screen.
The videos of the child's reactions are sent to the study's servers, where automatic behavioral coding software tracks the movement of video landmarks on the child's face and quantifies the child's emotions and attention. For example, in response to a short movie of bubbles floating across the screen, the video coding algorithm looks for movements of the face that would indicate joy.
In this study, children whose parents rated their child as having a high number of autism symptoms showed less frequent expressions of joy in response to the bubbles.
Autism screening in young children is presently done in clinical settings, rather than the child's natural environment, and highly trained people are needed to both administer the test and analyze the results. "That's not scalable," said New York University's Helen Egger, M.D., one of the co-leaders of the study.
This study, from informed consent to data collection and preliminary analysis, was conducted with an app available for free from Apple Store and based on Apple's ResearchKit open source development platform.
In one year, there were more than 10,000 downloads of the app, and 1,756 families with children aged one to six years participated in the study. Parents completed 5,618 surveys and uploaded 4,441 videos. Usable data were collected on 88 percent of the uploaded videos, demonstrating for the first time the feasibility of this type of tool for observing and coding behavior in natural environments.
"This demonstrates the feasibility of this approach," said Geraldine Dawson, Ph.D., Director of the Duke Center for Autism and Brain Development and co-leader of the study. "Many caregivers were willing to participate, the data were high quality and the video analysis algorithms produced results consistent with the scoring we produce in our autism program here at Duke."
An app-based approach can reach into underserved areas better and make it much easier to track an individual child's changes over time, said Guillermo Sapiro, Edmund T. Pratt, Jr. School Professor of Electrical and Computer Engineering at Duke and a co-leader of the study.
"This technology has the potential to transform how we screen and monitor children's development," Sapiro said.
The reported project was a 12-month study. The entire test took about 20 minutes to complete, with only a few minutes involving the child.
The app also included a widely used questionnaire that screens for autism. Based on the questionnaire, participating families received some feedback from the app about what the child's apparent risk for autism might be. If parents reported a high level of autism symptoms on the questionnaire, they were encouraged to seek further consultation with their health care providers.
Helen L Egger, Geraldine Dawson, Jordan Hashemi, Kimberly LH Carpenter, Steven Espinosa, Kathleen Campbell, Samuel Brotkin, Jana Schaich-Borg, Qiang Qiu, Mariano Tepper, Jeffrey P Baker, Richard A Bloomfield Jr, Guillermo Sapiro.
Automatic emotion and attention analysis of young children at home: a ResearchKit autism feasibility study.
npj Digital Medicinevolume 1, Article number: 20 (2018). doi: 10.1038/s41746-018-0024-6.