Shinichi Tokuno from the University of Tokyo walks us through voice analysis technology, specifically focussing on social impact analysis using voice biomarkers
We have previously described the monitoring and screening of individual mental health using voice biomarkers. However, if this technology is applied to the masses, it should be possible to observe the anxiety levels of an entire society in instances such as natural disasters. In other words, this technology could aid in establishing the impact of the incident in question on society. Let us review some examples.
Kumamoto Earthquake: Psychological impact
Figure 1 shows changes in voice biomarkers that have been analysed from voice calls made on smartphones before, during, and after a series of seismic activities centred around the Kumamoto Prefecture that occurred from April 14, 2016, onwards. Among the indicators shown in the graph, ‘vitality’ is indicative of the vitality (energy level) of each call, while ‘mental activity’ is indicative of the mental health state of the callers for over two weeks (refer to our article in the October 2019 issue for more details). This particular earthquake had some unique features: in the first six days, there were a total of 20 earthquakes with a magnitude of ‘5’ or greater, especially in the first two days, there were three successive earthquakes, each with a magnitude of ‘6’ or greater.
The graph also shows depressive tendencies that were measured using audio for five days before the earthquake, five days during which the aftershocks continued, and five days after the aftershocks began to subside. Since the only data from individuals who made at least one call in each period were extracted, the sample size was small; however, the characteristics of each region were still present. It can also be seen in the graph that the red area (earthquake-affected region) around the epicentre of the earthquake fell sharply during the period in which the earthquake continued to fall sharply during the aftershocks, then in this area, peoples anxiety recovered somewhat later on. This graphical depiction shows that depression gradually increased in the region around the disaster area. Although no major changes were seen in other regions before and after the earthquake, the areas damaged by the Great East Japan Earthquake that occurred five years ago experienced changes that were different than other unaffected areas. These figures may have captured psychological changes, such as PTSD; however, it should be noted that this would be a mere deduction because no other tests have been conducted to confirm this.
Mental health during COVID-19
Figure 2 shows the psychological impact of the COVID-19 ‘stay-at-home’ orders on people in Japan. As noted in the graph, although each episode associated with the pandemic led to a short-term decline in mental activity, those associated with refraining from going out showed a significant decline.
The graph shows the weekly means and their standard errors. (i) The first case of infection reported in Japan. (ii) Charter plane brings 206 Japanese people back from Wuhan. (iii) The WHO declares a public health emergency; the quarantine of the Diamond Princess cruise ship begins. (iv) Elementary, middle, and high schools closed. (v) The Governor of Tokyo mentions lockdown. (vi) State of emergency declared (voluntary isolation at home). (vii) State of emergency extended. (viii) State of emergency lifted.
More specifically, a temporary decrease in mental activity levels was documented, but the changes were not significant. The temporary decreases occurred after (1) a report on the first case of infection in Japan, (2) a report declaring a state of emergency by the World Health Organization (WHO), (3) the start of quarantine at the Diamond Princess, and (4) a request to close elementary, middle, and high schools. However, the Tokyo Governor’s mention of Tokyo’s lockdown triggered a continuous decline in mental activity values, which was further accelerated by the declaration of a state of emergency (i.e., the “stay-at-home” orders) and the subsequent extension of this state of emergency. However, at this point, the mental activity values started to recover and by the time the state of emergency was lifted, the values had returned to normal.
From the information presented in the graph, there is evidence that people react strongly to events that directly affect them; especially when their freedoms are taken away. Additionally, even if people do not currently have certain freedoms, the graph shows a recovery trend at the stage when life can begin to return to “normal”.
Similar analyses of the masses, as presented here, may help inform the development of government and corporate policies. It should be noted that since this particular analysis was completed, there has been a resurgence in the number of people infected with COVID-19 in Japan, which is, again, making people feel anxious. We are, therefore, continuing to measure mental activity, and are closely watching future developments.
References
- Tokuno S, et al. Psychological impact of Kumamoto earthquake by voice analysis using a smart phone application, Neuroscience 2016 (San Diego) 2016.11.12-16.
- Tokuno, S. Pathophysiological Voice Analysis for Diagnosis and Monitoring of Depression. Understanding Depression (pp. 83-95). Springer, Singapore. (2018).
- Tokuno S, et al. Social Impact Analysis by Smart Phone Voice. International Workshop EDEN (Exploratory Domains of Econophysics News) – XI (Pitesity, Virtual), 2020.10.10.
Please note: This is a commercial profile
