Jie Gu, Associate Professor from Northwestern University, examines AI-empowered neural processing for intelligent human-machine interface and biomedical devices
Most conventional wearable devices rely on motion detection or image classifications to capture users’ activities. However, they lack the ability to decode neural signals generated by the human body.
Neural signals, such as EEG, ECG, and EMG, offer a rich amount of information on a person’s physiological and psychological activities. Recognition and use of such signals present many new opportunities for applications in medical and daily commercial usage. Recently, artificial intelligence (AI) has been applied to neural signal processing, leading to a new generation of intelligent human-machine interfaces and biomedical devices.
Neural signals for physiological tracking and emotion detection
Traditionally, the use of human neural signals such as EEG, ECG, and EMG has found broad applications for human healthcare. For instance, in rehabilitation, patients’ EMG signals are decoded into the users’ motion intents, which are further used to control prosthetic arms. ECG signals have been widely used for cardiac disease diagnosis such as Arrhythmia. EEG signals have been used to detect anomalies in brain activities such as seizures.
Health monitoring, such as heart rate tracking, has been available through existing wearable devices for commercial use. Interestingly, EEG signals and skin conductance can provide helpful information on human emotional states. These can be detected during gaming or video watching to determine the users’ mental status. They have also been used for depression studies.
While neural signals are highly beneficial and informative for tracking human physical and psychological status, their processing and classification are not trivial due to the highly noisy signal representations, human motion artefacts, and highly user-dependent characteristics. This typically requires very sophisticated signal processing methods and feature extraction operations. Significant progress has recently been made in using AI to perform neural signal processing tasks with high accuracy, low latency, and strong adaptability.
Neural processing empowered by AI for intelligent human-machine interface
Human-machine interface with AI has developed into a large commercial market in recent years, mainly driven by the ongoing blooms of Virtual Reality (VR) and Mixed Reality (MR).
For instance, Meta has developed an EMG-based wristband assisted by AI technology for human gesture and motion tracking. Similarly, AI has been applied to EEG signals through commercial EEG headcaps for a broad range of “mental imagery” activities, e.g., mind control of joysticks. Lately, significant interest has been drawn in “mind-to-text” conversion, where EEG signals are collected and translated into natural language through advanced AI models, which usually include a cascade of neural network modules such as CNNs and transformers.
Similarly, human emotions are also well-tracked by the classification of EEG and ECG signals using advanced AI models. A deep perception of human neural signals opens up brand new opportunities for machines or computers to understand and communicate with the human “mind” or “body” for intelligent assistance in daily human activities.
AI-empowered neural processing for biomedical applications
Using AI for biomedical devices has boomed in recent years. Applications that require neural signal processing especially observe high adoption of AI algorithms, which replace many traditional signal processing efforts.
For instance, AI algorithms have been applied to EEG for real-time seizure detection. Similarly, a large body of research has applied CNN for cardiac disease diagnosis with very high accuracy.
Due to the complexity of biological neural signals and the large number of neural channels commonly used for classification, AI techniques are a natural fit for in-vivo biomedical applications. Recently, Medtronic has just announced an AI healthcare revolution by investing in AI for real-time medical devices. It is perceivable that AI-empowered neural processing will become the common computing platform for future biomedical devices.
Challenges of supporting AI for intelligent human-machine interface and biomedical devices
While AI models using CNN or the recent transformers are extremely powerful, one significant challenge is their deployment on small wearable or even implantable electronic devices, which are usually 100 times smaller and less powerful than CPU or GPU.
A small, embedded microprocessor that is commonly used in commercial biomedical devices is not able to process the modern AI models within a targeted timeframe for real-time neural signal processing, e.g., milliseconds for EMG-based gesture classification.
It is well known that such conventional processors using Von-Neumann architecture are very inefficient for AI computing. As a result, there has been a recent surge of edge AI devices using dedicated neural network accelerators to process AI tasks on local small devices. This type of computing device provides 100-1000 times better efficiency than existing microprocessors. It is reasonable to expect that the AI accelerator-powered edge AI device will become the hardware choice for the upcoming AI healthcare revolution.
AI-empowered microelectronic chips at Northwestern University
One of the prominent groups in the development of AI hardware for neural processing is Prof. Jie Gu’s team at Northwestern University. They have developed state-of-the-art AI-empowered neural processor chips that use special neural accelerator architecture to bolster real-time AI computing for neural signal processing.
For instance, a tiny system-on-chip (SoC) solution at a dimension of 2mm by 2mm built from Gu’s lab can provide both analogue front-end sensing and dedicated CNN/LSTM/MLP AI computing within 1~5ms latency using only 20~50μW, which uses 1,000 times less power than the off-the-shelf microprocessors on the market. The chips are used for a variety of biomedical applications, such as motion intent detection for prosthetic devices, arrhythmia detection for cardiac diagnosis, etc.
Recently, a special cognitive chip has been developed to support VR/MR applications. The chip seamlessly integrates into the Meta Quest2 VR headset. It can classify users’ EEG signals in real-time with electrodes hidden inside the headset without cumbersome wearing as conventional EEG caps. The integrated VR headset can perform four typical mind control jobs to control the VR scenes, including mental imagery, where the user’s imagination is used to select the VR user menu; emotion detection, where the user’s emotions are classified and used to control game speed, motor imagery where users can use the imagination of limb movements to issue computer command, and finally steady-state visual evoked potential (SSVEP) where special flashing signals on the screen are detected from EEG signals to issue users’ command. All the above points to the tremendous potential AI-based neural processing powered by novel AI accelerator chips brings to our daily lives, workplaces, and healthcare.
AI: Future directions and new opportunities
AI has become the “brain” of engineering. By embedding AI into human interface devices, e.g., biomedical devices or wearable devices, we will be able to understand better human desires, activities, emotions, and health status. We expect AI-empowered neural processing to become the new computing platform for future human- facing devices in both healthcare and daily assistance. This will lead to abundant applications on personal mobile devices, human robotics, VR/MR systems, gaming systems, online recommendation systems, and biomedical devices. For the next step, enabling online learning and user adoption will be a crucial feature to be developed to make such devices widely acceptable for general users and patients.
References
- Wei, Y, et al., (2022) A 65nm Implantable Gesture Classification SoC for Rehabilitation with Enhanced Data Compression and Encoding for Robust Neural Network Operation Under Wireless Power Condition, 2022 IEEE Custom Integrated Circuits Conference (CICC), [online] https://doi.org/10.1109/CICC53496.2022.9772838
- Wei, Y, et al., (2021) A Gesture Classification SoC for Rehabilitation With ADC-Less Mixed-Signal Feature Extraction and Training Capable Neural Network Classifier, IEEE Journal of Solid-State Circuits, [online] 56, 3, 876-886, https://doi.org/10.1109/JSSC.2020.3041288
- Wei, Y, et al., (2020) A Wearable Bio-signal Processing System with Ultra-low-power SoC and Collaborative Neural Network Classifier for Low Dimensional
Data Communication. Annual Int Conf IEEE Eng
Med Biol Soc. 2020, [online] 4002-4007. https://doi.org/10.1109/EMBC44109.2020.9176647 - Zhong, Z, et al., (2024) A Sub-1μJ/class Headset-Integrated Mind Imagery and Control SoC for VR/MR Applications with Teacher-Student CNN and General-Purpose Instruction Set Architecture, International Solid-State Circuits Conference (ISSCC), 2024. [online] https://doi.org/10.1109/ISSCC49657.2024.10454317
- Wei, Y, et al., (2023) Human Activity Recognition SoC for AR/VR with Integrated Neural Sensing, AI Classification and Chained Infrared Communication for Multi-chip Collaboration, VLSI Symposium on Circuits and Technology (VLSI), 2023. [online] 10.23919/VLSITechnologyandCir57934.2023.10185392
This work is licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International.
