The SEMEOTICONS project presents the Wize Mirror, a multisensory platform that analyses your face for signs of cardio-metabolic risk and guides behavioural change
Cardiovascular diseases (CVD) are the leading causes of mortality worldwide. It is estimated that 17.5 million people died from CVD in 2012, representing 31% of all global deaths. The increase of CVD on a global scale is putting a strain on social resources and health systems. Studies published in 2014 demonstrated that the total cost of CVDs in six European countries (France, Germany, Italy, Spain, Sweden and the UK) was around €102.1bn – roughly the size of the gross domestic product of a mid-sized European economy such as Hungary. Costs from CVDs in the 6 study countries are expected to increase to €122.6bn by the end of the decade, i.e. €20.5bn over 6 years. Prevention is the way forward, cardio-metabolic risk factors (hypertension, dyslipidemia, insulin resistance, type 2 diabetes mellitus, and obesity) are mainly related to individual lifestyle, including diet, physical activity, tobacco and alcohol consumption, stress and psychological status. Awareness and empowerment of individuals to self-monitor their health status and take action upon their lifestyle is the winning strategy to preserve health and reduce the incidence and burden of disease.
SEMEOTICONS’ Wize Mirror may help people to shift from passive recipients of care towards management of their own health. While you sit in front of the mirror – either as a part of your daily morning routine at home, or at the gym or the pharmacy – the mirror detects signs of cardio-metabolic risk on your face, and advises you on how to reduce the risk through behavioural changes. It’s not magic, it’s science and technology.
Medical semeiotics goes digital
The rationale behind the mirror is medical semeiotics, which has been used by medical doctors since Aristotle’s times to deduce one’s health status via physical face signs and facial expressions. The Wize Mirror is a multisensory platform that does the same, thanks to a contactless sensing framework and an advanced processing module.
The sensing framework includes a multispectral imaging (MSI) system for skin tissue analysis, made up of 5 compact monochrome cameras with band pass filters at selected wavelengths, and 2 computer controlled LED light sources (white light and ultraviolet); low-cost 3D optical sensors for face reconstruction and bio-morphometric analysis; high-resolution visible cameras for emotional analysis; and a portable device for breath analysis. According to a semeiotic model of the face for cardio-metabolic risk, the MSI system analyses the face skin to quantify AGE (Advanced Glycation End-products) deposits, which are linked to inflammation and atherosclerosis; measure skin cholesterol concentration; and evaluate endothelial dysfunction – a mechanism which may lead to coronary artery disease – in facial skin microcirculation after thermal stimulation by a remote skin heather. The 3D optical sensors support face detection and recognition, 3D head pose tracking, face segmentation and labelling, and 3D reconstruction to detect and monitor over time morphological changes due to weight gain, since overweight and obesity are major factors for cardio-metabolic risk. The visible cameras record video sequences, which support expression analysis (head, eye and mouth movements) and colorimetric analysis (skin pallor, redness, jaundice) to account for stress, anxiety and fatigue; other contactless estimates of physiological parameters include respiratory rate, heart rate and heart rate variability. Finally, the gas sensing device analyses the breath composition and gives quantitative feedback about noxious habits such as smoking and alcohol consumption.
The processing framework calculates a composite wellness index out of the acquired data. This index measures the health status of a person, related to his/her cardio-metabolic risk. Besides the objective measurements gathered from the sensors, the wellness index also takes into account the subjective evaluation of perceived health status. A sensible graphical representation shows the user an estimate of his/her global wellness, along with estimates on the physical, emotional and lifestyle components. The wellness index is traced over time and reported in a health diary for the user (or the practitioner) to consult.
The Wize Mirror comes as the result of three years of research and development by ten European partners, who met technological, scientific, and clinical challenges. From a technological standpoint, the challenge was to develop a non-intrusive platform, integrated into daily-life settings. To face this challenge touch-less data acquisition was mandatory together with synchronisation and real-time processing of multimodal data, end-user experience optimisation, and hardware and software integration. From the scientific perspective, the mirror was built on intelligent methods translating face signs into accurate and sensible computational measures, and their semantic integration towards the evaluation of holistic wellness. Clinical research was meant to develop a semeiotic face model of cardio-metabolic risk, and validate the clinical significance of the mirror against traditional, commercially available diagnostic techniques. Three mirror prototypes underwent validation in clinical sites in Italy and France between July and October 2016. A human study involved 66 volunteers followed for three months, on a bi-monthly basis. The study was designed to assess reproducibility of measures and the potential influence of environmental conditions, on one side; to prospectively assess how the mirror output reflects changes in body composition and metabolism, on the other.
Target: Improving lifestyle
Upon the assessment of cardio-metabolic risk, the Wize Mirror offers personalised user guidance, to lifestyle improvement for risk reduction. We believe that the key to successful lifestyle intervention is represented by sustained, long-term engagement. Indeed, the increasing adoption of supportive smart devices (e.g., in September 2013 one of ten US consumers, over the age of 18 owned a modern activity tracker), is counterbalanced by the fact that most of the consumers stop using the device within six months. To drive long-term engagement, the Wize Mirror guidance is based on the promotion of behavioural changes towards both physical and emotional wellness. Most of the people not only want to lose weight or look better, but wish to improve their quality of life and overall wellness. Therefore, the mirror offers recommendations on different target areas related to lifestyle (diet, training, tobacco, and alcohol) and on stress and anxiety management. Also, modulators based on the user profile (e.g., initial health condition, reported self-efficacy, emotional strength) are aimed towards tuning the intervention and to improving adherence to recommendations.
Stimulating initial adoption and periodic utilisation by providing a pleasant user experience, also play a key role in the long-term impact of the mirror. One of the volunteers enrolled in the study at the end of three full months of mirror interface stated: “I wish the study could last three more months: for the first time I was able to reduce my weekly alcohol consumption and improve my dietary habits having a lot of fun.”
The road ahead: From academia to market
The Wize Mirror is an innovative technological product which integrates advanced software and hardware solutions. To make a true impact outside academy, it is mandatory for the Wize Mirror to enter the mass market. The first target is the well-being sector. The well-being industry has been advancing rapidly in recent years: collectively, the wellbeing industry was valued globally at $3,4 trillion in 2013. Nevertheless, the smart mirrors market is still in its earliest stage, and most companies are focusing on automotive, defense and commercial/retail/entertainment, rather than on wellness. The Wize Mirror target market includes gyms, fitness centres, spas, and hotels, which will use the mirror to offer personalised coaching to their clients. High schools could also represent a potential target, after customisation to youngsters. The residential sector can be initially addressed as a niche market, with the Wize Mirror as a high-end technology product, and become the main target sector when the mirror reaches the cost of a big television set.
In the future, more sophisticated mirror versions could be released as medical-device grade instruments, and become part of today’s medical toolbox. As a matter of fact, a device like the Wize Mirror could help physicians in improving counselling on cardio-metabolic risk factors in the general population, especially for primary prevention. The possibility of close follow-ups by means of an easily manageable technique which periodically provides information on maintaining a good lifestyle by improving diet or exercise, as well as reminding the user to modify dangerous habits such as alcohol consumption and smoking, is an additional time-saving support to busy health professionals.
We believe that the Wize Mirror is a practical and innovative eHealth solution which offers a fresh perspective on educational programs and lifestyle intervention for the prevention of cardio-metabolic diseases, with potential impact on the society at large.
Sara Colantonio
Project coordinator
Tel: +39 050 621 3141
Giuseppe Coppini
Scientific project coordinator
Tel: +39 050 621 3480
Maria-Aurora Morales
Clinical project coordinator
Tel: +39 050 621 3228
Institute of Information Science and Technologies
Institute of Clinical Physiology
National Research Council of Italy
Please note: this is a commercial profile
