SBIR-STTR Award

The heterogeneity of the more than 1.3% of Americans who suffer from severe physical impairments (SPIs) preclude the use of common augmentative or alternative communication (AAC) solutions such as manual signs, gestures or dexterous interaction with a touchscreen for communication. While efforts to develop alternative access methods through eye or head tracking have provided some communication advancements for these individuals, all current technologies suffer from the same fundamental limitation: existing AAC devices require patients to conform to generic communication access methods and interfaces rather than the device conforming to the user. Consequently, AAC users are forced to settle for interventions that require excessive training and cognitive workload only to deliver extremely slow information transfer rates (ITRs) and recurrent communication errors that ultimately deprive them of the fundamental human right of communication. To meet this health need, we propose the first smart-AAC system designed using individually adaptive access methods and AAC interfaces to accommodate the unique manifestations of motor impairments specific to each user. Preliminary research by our team of speech researchers at Madonna Rehabilitation Hospital (Communication Center Lab) and Boston University (STEPP Lab), utilizing wearable sensors developed by our group (Altec, Inc) have already demonstrated that metrics based on surface electromyographic (sEMG) and accelerometer measures of muscle activity and movement for head-mediated control can be combined with optimizable AAC interfaces to improve ITRs when compared with traditional unoptimized AAC devices. Leveraging this pilot work, our team is now proposing a Phase I project to demonstrate the proof-of-concept that a single sEMG/IMU hybrid sensor worn on the forehead can provide improvements in ITR and communication accuracy when integrated with an AAC interface that is optimized through machine learning algorithms. The prototype system will be tested and compared to a conventional (non-adaptable) interface in subjects with SPI at a collaborative clinical site. Assistance by our speech and expert-AAC collaborators will ensure that all phases of technology development are patient-centric and usable in the context of clinical care. In Phase II we will build upon this proof-of-concept to design a smart-AAC system with automated optimization software that achieves dynamic learning which adapts to intra-individual changes in function through disease progression or training as well as inter-individual differences in motor impairments for a diverse set of users with spinal cord injury, traumatic brain injury, cerebral palsy, ALS, and other SPIs. The innovation is the first and only AAC technology that combines advancements in wearable-sensor access with interfaces that are autonomously optimized to the user, thereby reducing the resources and training needed to achieve effective person-centric communication in SPI, through improved HMI performance and reduced workload.

Public Health Relevance Statement:
This project addresses the fundamental mission of NIDCD (National Institute for Deafness and Communication Disorders) to provide a direct means of assisting communication for people with severe physical impairments caused by stroke, high level spinal cord injury, neural degeneration, or neuromuscular disease. Leveraging wearable access technology (which has barely been explored for AAC users), we will develop a first-of-its-kind adaptive tablet interface tailored to individual users through advanced movement classification algorithms. Through these efforts, we aim to provide an improved Human Machine Interface (HMI) that is able to accommodate varying degrees of inter- and intra-subject residual motor function and context dependent impairments to provide individuals with SPI the opportunity for improved societal integration and quality of life.

Project Terms:
Boston; Cerebral Palsy; youngster; childrens'; children; Children (0-21); Child Youth; 0-11 years old; Child; Communication; Communicative Disorders; Communication Disorders; Communication impairment; deafness; Diagnosis; Eyeball; Eye; Facial Muscles; Lack of Energy; Fatigue; Forehead; Gestures; Goals; Head; Health; Heterogeneity; Hospitals; Human Rights; Hybrids; Institutes; Learning; Linguistic; Linguistics; Manuals; Methods; Mission; body movement; Movement; muscular; Muscle Tissue; Muscle; Persons; neuronal degeneration; neurological degeneration; neurodegenerative; neurodegeneration; neural degeneration; Neuron Degeneration; Nerve Degeneration; neuromuscular disorder; neuromuscular degenerative disorder; myoneural disorder; Neuromuscular Diseases; Patients; QOL; Quality of life; Recurrent; Recurrence; rehabilitative therapy; rehabilitative; Rehabilitation; Medical Rehabilitation; Rehabilitation therapy; Research; Researchers; Investigators; Research Personnel; Research Resources; Resources; biological signal transduction; Signaling; Signal Transduction Systems; Intracellular Communication and Signaling; Cell Signaling; Cell Communication and Signaling; Signal Transduction; Software; Computer software; Speech; Traumatic Myelopathy; Spinal cord injuries; Spinal cord injured; Spinal Trauma; Spinal Cord Trauma; Spinal cord injury; cerebrovascular accident; cerebral vascular accident; brain attack; Cerebrovascular Stroke; Cerebrovascular Apoplexy; Cerebral Stroke; Brain Vascular Accident; Apoplexy; Stroke; Tablets; Technology; Testing; Time; Translating; Universities; man machine interface; human machine interface; human computer interface; User-Computer Interface; Work; Measures; Work Load; Workload; Mediating; population diversity; heterogeneous population; diverse populations; Population Heterogeneity; Custom; base; sensor; improved; Surface; Residual state; Residual; Phase; Variant; Variation; Series; Ensure; Training; Intuition; Individual; Disease Progression; United States National Aeronautics and Space Administration; National Aeronautics and Space Administration; NASA; Head Movements; Pattern; System; American; Performance; technology development; tech development; kinematics; kinematic model; novel; Communication Methods; Devices; Imaging problem; Traumatic Brain Injury; traumatic brain damage; Brain Trauma; mathematical model; mathematical modeling; mathematic model; Math Models; Intervention; interventional strategy; Intervention Strategies; Motor Manifestations; Address; Motor; Cognitive; Text; Development; developmental; Image; imaging; National Institute on Deafness and Other Communication Disorders; NIDCD; design; designing; touchscreen; touchscreen panel; touch screen panel; touch screen; touch panel; alternative communication; clinical research site; clinical site; Individual Differences; innovation; innovative; innovate; Impairment; two-dimensional; 2-dimensional; rehabilitation engineering; rehab engineering; motor impairment; movement limitation; movement impairment; prototype; clinical care; rehabilitation science; signal processing; Accelerometer; activity tracker; activity monitor; accelerometry; communication device; experimental study; experimental research; experiment; sensor technology; sensing technology; wearable device; wearable sensor; body worn sensor; body sensor; machine learning algorithm; classification algorithm

Nearly 5 million Americans require augmentative and alternative communication (AAC) methods to meet their daily communication needs. Some of these high-need individuals have motor impairments so severe (due to conditions such as brainstem stroke, traumatic brain injury, Guillain Barré syndrome, and cerebral palsy, among other disabilities) that they do not have the manual dexterity to control AAC technology and require alternative access methods (such as eye-tracking, head-tracking, or switch-scanning). Existing solutions, however, require extensive maintenance, frequent re-calibrations, and manual interface modifications that must be carried out with continued assistance from a caregiver or by compensating via their own residual motor activity. The excessive workload of adapting to these alternative communication methods are among the leading causes of AAC abandonment, ultimately depriving this population of their fundamental right to communication. To meet the critical communication needs of individuals with severe motor impairments, we propose the first AAC device comprising a versatile access method that automatically learns and customizes a keyboard interface to the residual motor function of the individual. In Phase I, we established the feasibility of developing a personalized keyboard interface (limited to A-Z, space) based on an individual's cursor movement and target selection abilities using a combined surface electromyographic (sEMG) and inertial (IMU) access method placed on their forehead. When evaluated amongst individuals with and without severe motor impairments, our AAC solution achieved greater information transfer rates (ITRs) over the standard QWERTY keyboard. Having successfully demonstrated this proof-of-concept, we are collaborating with speech researchers and clinicians at Boston University, MA (STEPP Lab for Sensorimotor Rehabilitation Engineering) and Madonna Rehabilitation Hospital, NE (Institute for Rehabilitation Science and Engineering) to advance our Phase I system into a pre- commercial MyAACTM system comprising versatile access method and personalized, comprehensive communication software. We will achieve this by developing hardware to support streamlined access across multiple points on the body (Aim 1), designing automated algorithms to rapidly create an expanded AAC interface, inclusive of letters, numbers, symbols, emojis, and word completion options, that is personalized based on the residual motor function of user-specific access points (Aim 2), creating software for point-of-care use of the access technology and interface, and evaluating the resulting MyAACTM system for communication efficacy in individuals with severe motor impairments (Aim 3). Our milestone will be to demonstrate that MyAACTM improves ITR and user experience over conventional AAC devices. The final MyAACTM deliverable will be easily integrated with existing AAC tablets and mobile devices to provide those in need of alternative communication methods with an automatically customized, efficient, and intuitive solution to restore communication access in their daily lives.

Public Health Relevance Statement:
This project addresses the fundamental mission of NIDCD (National Institute for Deafness and Communication Disorders) to provide novel means of assisting communication for people with severe motor impairments caused by congenital and/or acquired disabilities such as stroke, high-level spinal cord injury, neural degeneration, or neuromuscular disease. Leveraging advances in wearable access technology, which has not been explored sufficiently for AAC users, we are developing MyAACTM, the first AAC device that enables versatile access and automatically learns and customizes a keyboard interface to an individual's residual motor capabilities. MyAACTM represents a unique AAC device that provides alternative communication access by accommodating the residual motor function of individuals with severe motor impairments for improved societal integration and quality of life.

Project Terms:
Algorithms; Boston; Calibration; Cerebral Palsy; Communication; Communication impairment; Communication Disorders; Communicative Disorders; Computers; deafness; Family; foot; Forehead; Frustration; Gestures; Goals; Guillain Barré Syndrome; AIDP; Acute Autoimmune Neuropathy; Acute Infective Polyneuritis; Acute Inflammatory Demyelinating Polyradiculoneuropathy; Acute Inflammatory Polyneuropathy; Acute Inflammatory Polyradiculoneuropathy; Guillaine-Barre Syndrome; Landry's paralysis; Landry-Guillain-Barre Syndrome; acute idiopathic polyneuritis; acute post-infectious polyneuropathy; acute postinfectious polyneuropathy; Hand; Head; Hospitals; Institutes; Learning; Maintenance; Manuals; Methods; Mission; Motor Activity; Locomotor Activity; Movement; body movement; Persons; Neuron Degeneration; neural degeneration; neurodegeneration; neurodegenerative; neurological degeneration; neuronal degeneration; Nerve Degeneration; myoneural disorder; neuromuscular degenerative disorder; neuromuscular disorder; Neuromuscular Diseases; QOL; Quality of life; Medical Rehabilitation; Rehabilitation; rehab therapy; rehabilitative; rehabilitative therapy; Rehabilitation therapy; Investigators; Researchers; Research Personnel; Socialization; Software; Computer software; Speech; Spinal Cord Trauma; Spinal Trauma; Spinal cord injured; Traumatic Myelopathy; Spinal cord injury; Apoplexy; Brain Vascular Accident; Cerebral Stroke; Cerebrovascular Apoplexy; Cerebrovascular Stroke; brain attack; cerebral vascular accident; cerebrovascular accident; Stroke; Survey Instrument; Surveys; Target Populations; Technology; Testing; Genetic Transcription; Gene Transcription; RNA Expression; Transcription; Universities; Work; Workload; Work Load; Caregivers; Care Givers; Custom; base; sensor; improved; Surface; Clinical; Residual state; Residual; Phase; Evaluation; disability; Intuition; Individual; Letters; Life; Scanning; System; Brainstem Infarctions; Brainstem Stroke; Brain Stem Infarctions; preference; American; experience; Performance; eye tracking; visual tracking; user friendly computer software; user friendly software; novel; Participant; Communication Methods; Brain Trauma; traumatic brain damage; Traumatic Brain Injury; Drops; Address; AAC Device; AAC Intervention; Augmentative and Alternative Communication; Motor; Modification; point of care; National Institute on Deafness and Other Communication Disorders; NIDCD; design; designing; alternative communication; Population; rehabilitation engineering; rehab engineering; handheld mobile device; mobile device; usability; motor impairment; movement impairment; movement limitation; data exchange; data transfer; data transmission; rehabilitation science; dexterity; Tablet Computer; tablet device; Caregiver support; Care giver support; communication device; wireless communication; automated algorithm; automatic algorithm; wireless