Application of Facial Expression Analysis Technology in Violence Risk Assessment of Individuals with Mental Disorders in Supervised Settings

doi:10.12116/j.issn.1004-5619.2023.131101

Abstract

Abstract:

Objective To explore the association between violent behaviors and emotions in individuals with mental disorders, to evaluate the application value of facial expression analysis technology in violence risk assessment of individuals with mental disorders in supervised settings, and to provide a reference for violence risk assessment. Methods Thirty-nine male individuals with mental disorders in supervised settings were selected, the participant risk of violence, cognitive function, psychiatric symptoms and severity were assessed using the Modified Overt Aggression Scale （MOAS）, the Historical, Clinical, Risk Management-Chinese version（HCR-CV）, the Positive and Negative Syndrome Scale （PANSS） and the Brief Psychiatric Rating Scale （BPRS）. An emotional arousal was performed on the participants and the intensity of their emotions and facial expression action units was recorded before, during and after the arousal. One-way analysis of variance （ANOVA） was used to compare the differences in the intensity of emotions and facial expression action units before, during and after the arousal. Pearson correlation analysis was used to calculate the correlations between the intensity of the seven basic emotional facial expressions and the scores of the assessment scales. Results The intensity difference of sadness, surprise and fear in different time periods was statistically significant （P<0.05）. The intensity of the left medial eyebrow lift action unit was found significantly different before and after the emotional arousal （P<0.05）. The intensity of anger was positively correlated with the Modified Overt Aggression Scale score throughout the experiment （P<0.05）. Conclusion Eye action units such as eyebrow lifting, eyelid tightening and upper eyelid lifting can be used as effective action units to identify sadness, anger and other negative emotions associated with violent behaviors. Facial expression analysis technology can be used as an auxiliary tool to assess the potential risk of violence in individuals with mental disorders in supervised settings.

Key words: forensic psychiatry, mental disorder, emotions, facial expression, violence risk assessment, supervised setting

CLC Number:

Xin-di LING, Hao-zhe LI, Shu-jian WANG, Wen LI, Wei-xiong CAI. Application of Facial Expression Analysis Technology in Violence Risk Assessment of Individuals with Mental Disorders in Supervised Settings[J]. Journal of Forensic Medicine, 2024, 40(3): 261-268.

Figures/Tables 2

References 41

1	KRUG E G， MERCY J A， DAHLBERG L L， et al. World report on violence and health[EB/OL].（2002-10-01）[2024-01-30]. .
2	WHITING D， LICHTENSTEIN P， FAZEL S. Violence and mental disorders： A structured review of associations by individual diagnoses， risk factors， and risk assessment[J]. Lancet Psychiatry，2021，8（2）：150-161. doi：10.1016/S2215-0366（20）30262-5 .
3	HOSIE J， SIMPSON K， DUNNE A， et al. A study of the relationships between rumination， anger rumination， aggressive script rehearsal， and aggressive behavior in a sample of incarcerated adult males[J]. J Clin Psychol，2022，78（9）：1925-1939. doi：10.1002/jclp.23341 .
4	BERKOWITZ L. Aversively stimulated aggression. Some parallels and differences in research with animals and humans[J]. Am Psychol，1983，38（11）：1135-1144. doi：10.1037//0003-066x.38.11.1135 .
5	CARVER C S， HARMON-JONES E. Anger is an approach-related affect： Evidence and implications[J]. Psychol Bull，2009，135（2）：183-204. doi：10.1037/a0013965 .
6	HARMON-JONES E， HARMON-JONES C， ABRAM-SON L， et al. PANAS positive activation is associated with anger[J]. Emotion，2009，9（2）：183-196. doi：10.1037/a0014959 .
7	CARVER C S. Negative affects deriving from the behavioral approach system[J]. Emotion，2004，4（1）：3-22. doi：10.1037/1528-3542.4.1.3 .
8	WILKOWSKI B M， MEIER B P. Bring it on： Angry facial expressions potentiate approach-motivated motor behavior[J]. J Pers Soc Psychol，2010，98（2）：201-210. doi：10.1037/a0017992 .
9	ANDERSON C A， BUSHMAN B J. Human aggression[J]. Annu Rev Psychol，2002，53：27-51. doi：10.1146/annurev.psych.53.100901.135231 .
10	BLAIR R J R. Traits of empathy and anger： Implications for psychopathy and other disorders associated with aggression[J]. Philos Trans R Soc Lond B Biol Sci，2018，373（1744）：20170155. doi：10.1098/rstb.2017.0155 .
11	POND R S， DEWALL C N， LAMBERT N M， et al. Repulsed by violence： Disgust sensitivity buffers trait， behavioral， and daily aggression[J]. J Pers Soc Psychol，2012，102（1）：175-188. doi：10.1037/a0024296 .
12	RIHMER Z， RIHMER A. Depression and suicide — The role of underlying bipolarity[J]. Psychiatr Hung，2019，34（4）：359-368.
13	HOUSTON R J， STANFORD M S. Characterization of aggressive behavior and phenytoin response[J]. Aggr Behav，2006，32（1）：38-43. doi：10.1002/ab.20104 .
14	BANKS S J， BELLEROSE J， DOUGLAS D， et al. Bilateral skin conductance responses to emotional faces[J]. Appl Psychophysiol Biofeedback，2012，37（3）：145-152. doi：10.1007/s10484-011-9177-7 .
15	HAGGARD E A， ISAACS K S. Micromomentary facial expressions as indicators of ego mechanisms in psychotherapy[M]// GOTTSCHALK L A， AUERBACH A H. Methods of research in psychotherapy. Boston， MA： Springer，1966：154-165.
16	YAN W J， WU Q， LIANG J， et al. How fast are the leaked facial expressions： The duration of micro-expressions[J]. J Nonverbal Behav，2013，37（4）：217-230. doi：10.1007/s10919-013-0159-8 .
17	EKMAN P. Darwin， deception， and facial expression[J]. Ann N Y Acad Sci，2003，1000：205-221. doi：10.1196/annals.1280.010 .
18	SUN Y J， AYAZ H， AKANSU A N. Multimodal affective state assessment using fNIRS + EEG and spontaneous facial expression[J]. Brain Sci，2020，10（2）：85. doi：10.3390/brainsci10020085 .
19	MATSUMOTO D， HWANG H C. Microexpressions differentiate truths from lies about future malicious intent[J]. Front Psychol，2018，9：2545. doi：10.3389/fpsyg.2018.02545 .
20	ALVARI G， FURLANELLO C， VENUTI P. Is smiling the key? Machine learning analytics detect subtle patterns in micro-expressions of infants with ASD[J]. J Clin Med，2021，10（8）：1776. doi：10.3390/jcm10081776 .
21	EKMAN P， FRIESEN W V. Measuring facial movement[J]. Environ Psychol Nonverbal Behav，1976，1：56-75. doi：10.1007/BF01115465 .
22	HASHEMI J， DAWSON G， CARPENTER K L H， et al. Computer vision analysis for quantification of autism risk behaviors[J]. IEEE Trans Affect Comput，2021，12（1）：215-226. doi：10.1109/TAFFC.2018.2868196 .
23	DAS A， MOCK J， HUANG Y F， et al. Interpretable self-supervised facial micro-expression learning to predict cognitive state and neurological disorders[J]. Proc AAAI Conf Artif Intell，2021，35（1）：818-826.
24	GE Y， ZHAO G Z， ZHANG Y L， et al. A standardised database of Chinese emotional film clips[J]. Cogn Emot，2019，33（5）：976-990. doi：10.1080/02699931.2018.1530197 .
25	MO F， ZHANG Z H， CHEN T， et al. MFED： A database for masked facial expression[J]. IEEE Access，2021，9：96279-96287. doi：10.1109/ACCESS. 2021.3091289 .
26	CHIU M H， LIAW H L， YU Y R， et al. Facial micro-expression states as an indicator for conceptual change in students’ understanding of air pressure and boiling points[J]. Brit J Educational Tech，2019，50（1）：469-480. doi：10.1111/bjet.12597 .
27	TSIKANDILAKIS M， BALI P， DERRFUSS J， et al. Anger and hostility： Are they different? An analytical exploration of facial-expressive differences， and physiological and facial-emotional responses[J]. Cogn Emot，2020，34（3）：581-595. doi：10.1080/02699931.2019.1664415 .
28	SUAREZ E C， LEWIS J G， KUHN C. The relation of aggression， hostility， and anger to lipopolysaccharide-stimulated tumor necrosis factor （TNF）-α by blood monocytes from normal men[J]. Brain Behav Immun，2002，16（6）：675-684. doi：10.1016/s0889-1591（02）00019-3 .
29	FLEISCHMAN A， WERBELOFF N， YOFFE R， et al. Schizophrenia and violent crime： A population-based study[J]. Psychol Med，2014，44（14）：3051-3057. doi：10.1017/S0033291714000695 .
30	CADORET R J， YATES W R， TROUGHTON E， et al. Genetic-environmental interaction in the ge-nesis of aggressivity and conduct disorders[J]. Arch Gen Psychiatry，1995，52（11）：916-924. doi：10.1001/archpsyc.1995.03950230030006 .
31	BERKOWITZ L. On the formation and regulation of anger and aggression. A cognitive-neoassociationistic analysis[J]. Am Psychol，1990，45（4）：494-503. doi：10.1037//0003-066x.45.4.494 .
32	REAGU S， JONES R， KUMARI V， et al. Angry affect and violence in the context of a psychotic illness： A systematic review and meta-analysis of the literature[J]. Schizophr Res，2013，146（1/2/3）：46-52. doi：10.1016/j.schres.2013.01.024 .
33	章雪利，胡峻梅. 修订版外显攻击行为量表用于精神疾病患者危险行为评估[J].法医学杂志，2011，27（5）：342-345. doi：10.3969/j.issn.1004-5619.2011.05.007 .
	ZHANG X L， HU J M. Application of modified overt aggression scales on risk behavior assessment of patients with mental illness[J]. Fayixue Zazhi，2011，27（5）：342-345.
34	FULLAM R， DOLAN M. Emotional information processing in violent patients with schizophrenia： Association with psychopathy and symptomatology[J]. Psychiatry Res，2006，141（1）：29-37. doi：10.1016/j.psychres.2005.07.013 .
35	NOLAN K A， CZOBOR P， ROY B B， et al. Characteristics of assaultive behavior among psychiatric inpatients[J]. Psychiatr Serv，2003，54（7）：1012-1016. doi：10.1176/appi.ps.54.7.1012 .
36	BOZIKAS V P， KOSMIDIS M H， ANEZOULAKI D， et al. Relationship of affect recognition with psychopathology and cognitive performance in schizophrenia[J]. J Int Neuropsychol Soc，2004，10（4）：549-558. doi：10.1017/S1355617704104074 .
37	CHEUNG P， SCHWEITZER I， CROWLEY K， et al. Aggressive behaviour in schizophrenia： Role of state versus trait factors[J]. Psychiatry Res，1997，72（1）：41-50. doi：10.1016/s0165-1781（97）00090-5 .
38	BENTON A L. Differential behavioral effects in frontal lobe disease[J]. Neuropsychologia，1968，6（1）：53-60. doi：10.1016/0028-3932（68）90038-9 .
39	WEBER R， LI J T， SOLADIÉ C， et al. A survey on databases of facial macro-expression and micro-expression[C]// International Joint Conference on Computer Vision， Imaging and Computer Graphics. Cham： Springer，2019：298-325.10.1007/978-3-030-26756-8_15
40	SAFFARYAZDI N， WASIM S T， DILEEP K， et al. Using facial micro-expressions in combination with EEG and physiological signals for emotion recognition[J]. Front Psychol，2022，13：864047. doi：10.3389/fpsyg.2022.864047 .
41	ZENG X M， ZHAO X C， WANG S Y， et al. Affection of facial artifacts caused by micro-expressions on electroencephalography signals[J]. Front Neurosci，2022，16：1048199. doi：10.3389/fnins.2022.1048199 .

量表	得分
MOAS	0.180±0.823
HCR-CV	6.385±4.592
暴力历史（H）	5.282±3.713
临床（C）	1.103±1.314
风险评估（R）	0.000±0.000
PANSS	38.718±8.370
阳性（P因子）	7.718±1.146
阴性（N因子）	10.231±3.688
一般精神病理（G因子）	20.769±4.556
BPRS	25.718±5.010

量表	得分
MOAS	0.180±0.823
HCR-CV	6.385±4.592
暴力历史（H）	5.282±3.713
临床（C）	1.103±1.314
风险评估（R）	0.000±0.000
PANSS	38.718±8.370
阳性（P因子）	7.718±1.146
阴性（N因子）	10.231±3.688
一般精神病理（G因子）	20.769±4.556
BPRS	25.718±5.010

时间段	愉快	悲伤	愤怒	惊奇	惧怕	厌恶	轻蔑
F01	0.011±0.022	0.073±0.082^1）	0.051±0.061	0.077±0.075^1）	0.146±0.080^1）	0.025±0.042	0.028±0.060
F02	0.014±0.026	0.062±0.069^1）	0.047±0.065	0.063±0.074^1）	0.170±0.119^1）	0.030±0.045	0.028±0.062
M01	0.009±0.027	0.198±0.204	0.050±0.080	0.016±0.025	0.029±0.072	0.015±0.040	0.034±0.103
M02	0.012±0.042	0.199±0.221	0.049±0.068	0.016±0.019	0.027±0.053	0.021±0.046	0.028±0.094
M03	0.012±0.037	0.226±0.234	0.051±0.078	0.014±0.017	0.024±0.044	0.021±0.051	0.021±0.059
M04	0.012±0.040	0.234±0.241	0.058±0.099	0.013±0.018	0.022±0.050	0.027±0.077	0.020±0.041
M05	0.006±0.017	0.249±0.243	0.051±0.096	0.012±0.017	0.023±0.060	0.019±0.052	0.028±0.057
M06	0.006±0.018	0.262±0.240	0.055±0.091	0.013±0.027	0.015±0.044	0.021±0.068	0.030±0.095
M07	0.005±0.016	0.247±0.225	0.054±0.090	0.012±0.022	0.016±0.044	0.021±0.066	0.028±0.072
M08	0.005±0.017	0.256±0.234	0.054±0.087	0.013±0.026	0.013±0.030	0.019±0.065	0.036±0.093
P01	0.015±0.032	0.082±0.095^1）	0.041±0.062	0.080±0.073^1）	0.144±0.104^1）	0.019±0.032	0.025±0.065
P02	0.015±0.027	0.066±0.076^1）	0.038±0.058	0.071±0.071^1）	0.166±0.133^1）	0.028±0.037	0.026±0.056

时间段	愉快	悲伤	愤怒	惊奇	惧怕	厌恶	轻蔑
F01	0.011±0.022	0.073±0.082^1）	0.051±0.061	0.077±0.075^1）	0.146±0.080^1）	0.025±0.042	0.028±0.060
F02	0.014±0.026	0.062±0.069^1）	0.047±0.065	0.063±0.074^1）	0.170±0.119^1）	0.030±0.045	0.028±0.062
M01	0.009±0.027	0.198±0.204	0.050±0.080	0.016±0.025	0.029±0.072	0.015±0.040	0.034±0.103
M02	0.012±0.042	0.199±0.221	0.049±0.068	0.016±0.019	0.027±0.053	0.021±0.046	0.028±0.094
M03	0.012±0.037	0.226±0.234	0.051±0.078	0.014±0.017	0.024±0.044	0.021±0.051	0.021±0.059
M04	0.012±0.040	0.234±0.241	0.058±0.099	0.013±0.018	0.022±0.050	0.027±0.077	0.020±0.041
M05	0.006±0.017	0.249±0.243	0.051±0.096	0.012±0.017	0.023±0.060	0.019±0.052	0.028±0.057
M06	0.006±0.018	0.262±0.240	0.055±0.091	0.013±0.027	0.015±0.044	0.021±0.068	0.030±0.095
M07	0.005±0.016	0.247±0.225	0.054±0.090	0.012±0.022	0.016±0.044	0.021±0.066	0.028±0.072
M08	0.005±0.017	0.256±0.234	0.054±0.087	0.013±0.026	0.013±0.030	0.019±0.065	0.036±0.093
P01	0.015±0.032	0.082±0.095^1）	0.041±0.062	0.080±0.073^1）	0.144±0.104^1）	0.019±0.032	0.025±0.065
P02	0.015±0.027	0.066±0.076^1）	0.038±0.058	0.071±0.071^1）	0.166±0.133^1）	0.028±0.037	0.026±0.056

[1]	Wen LI, Hao-zhe LI, Chen CHEN, Wei-xiong CAI. Research Progress and Application Prospect of Facial Micro-Expression Analysis in Forensic Psychiatry [J]. Journal of Forensic Medicine, 2023, 39(5): 493-500.
[2]	Tian-yu NAN. Progress of Keystroke Dynamics in Deception Research [J]. Journal of Forensic Medicine, 2022, 38(6): 783-787.
[3]	Long-rui ZHAO, Jian-bo ZHANG, Wei HAN, Li ZHU, Teng CHEN, Fang-lin GUAN. Application Prospect of Integrative Omics in Forensic Identification of Methamphetamine-Associated Psychosis [J]. Journal of Forensic Medicine, 2022, 38(5): 650-656.
[4]	Hui-yu FAN, Hao-zhe LI, Qing-ting ZHANG. The Legal Correspondence of Sexual Self-Defense Capability Assessment [J]. Journal of Forensic Medicine, 2022, 38(5): 606-610.
[5]	Ruo-chen CAO, Xia-can CHEN, Lu YIN, Hao-lan HUANG, Wei-zhi WAN, Yan LI, Jun-mei HU. An Epidemiologic Survey and Violent Behavior Analysis of Antisocial Personality Disorder in Young Men in Chengdu [J]. Journal of Forensic Medicine, 2022, 38(2): 239-245.
[6]	Sheng-yu ZHANG, Hao-zhe LI, Chen CHEN, Qin-ting ZHANG. Analysis of Post-Concussion Syndrome Disability Re-appraisal [J]. Journal of Forensic Medicine, 2021, 37(5): 661-665.
[7]	DU Yu, JIA Qiong, YAO Lei, et al.. Forensic Analysis of 105 Autopsy Cases of Psychiatric Patients [J]. Journal of Forensic Medicine, 2021, 37(1): 54-57.
[8]	WANG Shu-jian, ZHENG Yu-zi, HE Yong, et al.. Analysis of 123 Cases of Forensic Psychiatry Testimony of Individuals Suspected of Road Traffic Offences [J]. Journal of Forensic Medicine, 2020, 36(6): 784-790,796.
[9]	LIANG Wan-ying, ZHANG Yu-qing. Research Status and Limitations of Sadomasochism [J]. Journal of Forensic Medicine, 2020, 36(6): 828-840.
[10]	CHEN Yang , HU Li-ping , LIU Lin-lin , et al.. Analysis of Correlation between Gene Polymorphisms of CRHBP in Yunnan Han Population and Schizophrenia and Aggressive Behavior [J]. Journal of Forensic Medicine, 2020, 36(5): 672-676.
[11]	SONG Hong-fei, DENG Xi, YANG Ning,et al. Impulse Control and ERP Characteristics of Patients with Mental Disorder Caused by Traumatic Brain Injury [J]. 法医学杂志, 2020, 36(3): 326-332.
[12]	GAO Ying, ZHANG Zeng, MIAO Qing-shan, et al. Analysis of Consistency between Forensic Psychiatry Identification Diagnosis and Clinical Diagnosis of 1 369 Cases [J]. 法医学杂志, 2020, 36(3): 333-336，340.
[13]	XU Shuo, LIANG Yi-ying, HONG Bin-xue,et al. Research Progress on the Relationship between Genetic Polymorphism and Violent Behavior of Patients with Schizophrenia [J]. 法医学杂志, 2020, 36(3): 379-388.
[14]	XU Ya-jun, WANG Qiang, SU Li,et al.. Analysis of Scores of SCL-90 of Patients with Traumatic Subdural Effusion [J]. 法医学杂志, 2020, 36(2): 223-228.
[15]	GE Fen-fen, YANG Xiao-qing, CHEN Yu-xing,et al.. Application of Eye Tracker in Lie Detection [J]. 法医学杂志, 2020, 36(2): 229-232，238.