Dementia is the progressive decline in cognitive functioning (memory, attention, language and problem solving) due to damage or disease in the brain beyond what occurs with normal aging. In later stages, affected suffers may not know where or who they are, or what day it is. The prevalence of dementia, closely associated with aging, is on the rise globally and is estimated at 1% among those aged 60-65, 6% of those 75-79, and 45% of those 95 and older.
To help people with dementia remain independent longer and enjoy a higher quality of life, researchers at the University of Dundee School of Computing in Scotland joined forces with the Department of Occupational Science and Occupational Therapy at the University of Toronto in Canada, to develop a system that would assist such people in what for non-sufferers would be routine tasks, such as handwashing. The cognitive assistance device uses a Dragonfly2 IEEE-1394a digital camera from Point Grey Research to observe behavior and detemine whether assitance is required.
The project team, led by Dr. Alex Mihailidis, developed a system that uses a combination of computer vision and artificial intelligence techniques to decide when and how to prompt and encourage specific behaviour.
Dishwashing system schematic
Cameras are ideal sensors for this type of system because image data provides a broad spectrum of information. The challenge is to extract the right set of features from the data, to allow the system to understand the observed activity and make correct decisions based on those observations.
Tap and towel use prompted by system
The system uses the Dragonfly2 IEEE-1394a digital camera to infer water flow based on user behavior. It is capable of processing 640x350 frames at 19 Hz, or 320x240 frames at 40 Hz. The Dragonfly2 camera, with 1/3” Sony® CCD and on-board color processing, was chosen for its speed and image quality. It also offers automatic synchronization of multiple cameras on the same IEEE-1394 bus, auto-iris output, look up table support, on board memory channels and more.
Dragonfly2 camera mounted to ceiling
Video from the Dragonfly2 camera mounted above a sink is input to a system to track objects of interest (e.g. a hand and towel tracker.) The tracker reports the positions of the hands and towel to a belief monitor that tries to estimate where in the task the user is currently—what have they accomplished so far, and what their internal mental state is.
The belief about the user's state is then passed to the “policy” which maps the belief to an action for the system to take, usually an audio or video prompt, or call for human assistance. For example, if the person is standing in front of the sink but has not turned the water on, the system would provide the verbal prompt “turn on the water now.” Over time, the system estimates whether the person’s attitude (e.g. “has not moved” for 60 seconds vs 30 seconds previously) is changing, which could indicate a higher level of dementia.
The research team’s computerized cognitive assistance device is now being tested in clinical trials in Toronto, and they are investigating broader implementation to support other daily living activites such as toothbrushing and toileting. The system has the potential to significantly improve the lives of older adults living with cognitive disabilities by reducing the feelings of dependance that come from relying on caregivers for private daily tasks.