Comparison among Four Deep Learning Image Classification Algorithms in AI-based Diatom Test

doi:10.12116/j.issn.1004-5619.2021.411001

Abstract

Abstract: Objective

To select four algorithms with relatively balanced complexity and accuracy among deep learning image classification algorithms for automatic diatom recognition, and to explore the most suitable classification algorithm for diatom recognition to provide data reference for automatic diatom testing research in forensic medicine.

Methods

The “diatom” and “background” small sample size data set （20 000 images） of digestive fluid smear of corpse lung tissue in water were built to train, validate and test four convolutional neural network （CNN） models, including VGG16, ResNet50, InceptionV3 and Inception-ResNet-V2. The receiver operating characteristic curve （ROC） of subjects and confusion matrixes were drawn, recall rate, precision rate， specificity， accuracy rate and F1 score were calculated， and the performance of each model was systematically evaluated.

Results

The InceptionV3 model achieved much better results than the other three models with a balanced recall rate of 89.80%, a precision rate of 92.58%. The VGG16 and Inception-ResNet-V2 had similar diatom recognition performance. Although the performance of diatom recall and precision detection could not be balanced, the recognition ability was acceptable. ResNet50 had the lowest diatom recognition performance, with a recall rate of 55.35%. In terms of feature extraction, the four models all extracted the features of diatom and background and mainly focused on diatom region as the main identification basis.

Conclusion

Including the Inception-dependent model, which has stronger directivity and targeting in feature extraction of diatom. The InceptionV3 achieved the best performance on diatom identification and feature extraction compared to the other three models. The InceptionV3 is more suitable for daily forensic diatom examination.

Key words: forensic pathology, artificial intelligence, drowning, deep learning, diatom test, convolutional neural network

CLC Number:

DF795.1

Yong-zheng ZHU, Ji ZHANG, Qi CHENG, Hui-xiao YU, Kai-fei DENG, Jian-hua ZHANG, Zhi-qiang QIN, Jian ZHAO, Jun-hong SUN, Ping HUANG. Comparison among Four Deep Learning Image Classification Algorithms in AI-based Diatom Test[J]. Journal of Forensic Medicine, 2022, 38(1): 31-39.

Figures/Tables 8

Fig. 1 Model training data set after data enhancement

Fig. 2 Flowchart of four DL image classification models

Fig. 3 The accuracy and loss curves of four models after the 2nd order smoothing on training and validation

Tab. 1 Optimal performance of four models

最优模型	训练集		验证集		迭代次数
最优模型	准确率/%	损失函数值	准确率/%	损失函数值	迭代次数
VGG16	99.99	0.001	96.25	0.074	89
ResNet50	99.99	0.001	94.00	0.143	70
InceptionV3	99.77	0.012	98.91	0.042	45
Inception-ResNet-V2	99.65	0.043	97.50	0.176	46

Tab. 2 Diatom identification performance parameters of four models

模型	RR/%	PR/%	S/%	AR/%	F1分数
VGG16	84.65	78.96	77.45	81.05	0.82
ResNet50	55.35	81.04	87.05	71.20	0.66
InceptionV3	89.80	92.58	92.80	91.30	0.91
Inception-ResNet-V2	72.35	87.22	89.40	80.88	0.79

Fig. 4 The ROC curves and AUC values of 4 optimal models

Fig. 5 The confusion matrixes， recall rate and specificity of 4 optimal models

Fig. 6 Heat map analysis of pennataes and centricaes identificated with 4 models by Grad-CAM and GBP methods

References 34

1	World Health Organization. Global report on drowning： Preventing a leading killer[R/OL]. （2014-09-17）[2021-09-21]. .
2	SZPILMAN D， MORGAN P J. Management for the drowning patient[J]. Chest，2021，159（4）：1473-1483. doi：10.1016/j.chest.2020.10.007 .
3	李棨，马开军，张晓东，等. 水中尸体肺组织硅藻检验与死因分析[J].法医学杂志，2011，27（5）：324-326，333. doi：10.3969/j.issn.1004-5619.2011.05.002 .
	LI Q， MA K J， ZHANG X D， et al. Diatom test in lung tissue of corpses in water and causes of death[J]. Fayixue Zazhi，2011，27（5）：324-326，333.
4	CARBALLEIRA R， VIEIRA D N， FEBRERO-BANDE M， et al. A valid method to determine the site of drowning[J]. Int J Legal Med，2018，132（2）：487-497. doi：10.1007/s00414-017-1708-1 .
5	PIETTE M H A， DE LETTER E A. Drowning： Still a difficult autopsy diagnosis[J]. Forensic Sci Int，2006，163（1/2）：1-9. doi：10.1016/j.forsciint.2004.10.027 .
6	LUNETTA P， PENTTILÄ A， HÄLLFORS G. Scanning and transmission electron microscopical evidence of the capacity of diatoms to penetrate the alveolo-capillary barrier in drowning[J]. Int J Legal Med，1998，111（5）：229-237. doi：10.1007/s004140050159 .
7	ZHANG P， KANG X， ZHANG S， et al. The length and width of diatoms in drowning cases as the evidence of diatoms penetrating the alveoli-capillary barrier[J]. Int J Legal Med，2020，134（3）：1037-1042. doi：10.1007/s00414-019-02164-4 .
8	LUDES B， COSTE M， NORTH N， et al. Diatom analysis in victim’s tissues as an indicator of the site of drowning[J]. Int J Legal Med，1999，112（3）：163-166. doi：10.1007/s004140050224 .
9	SHEN X， LIU Y， XIAO C， et al. Analysis of false-positive results of diatom test in the diagnosis of drowning-would not be an impediment[J]. Int J Legal Med，2019，133（6）：1819-1824. doi：10.1007/s00414-019-02021-4 .
10	BORTOLOTTI F， TAGLIARO F， MANETTO G. Objective diagnosis of drowning by the “diatom test” - A critical review[J]. Forensic Sci Rev，2004，16（2）：135-148.
11	周圆圆，曹永杰，杨越，等. 基于人工智能硅藻自动化识别系统的实际案例应用[J].法医学杂志，2020，36（2）：239-242. doi：10.12116/j.issn.1004-5619.2020.02.017 .
	ZHOU Y Y， CAO Y J， YANG Y， et al. Application of artificial intelligence automatic diatom identification system in practical cases[J]. Fayixue Zazhi，2020，36（2）：239-242.
12	KRIZHEVSKY A， SUTSKEVER I， HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Commun ACM，2017，60（6）：84-90. doi：10.1145/3065386 .
13	ARESTA G， ARAÚJO T， KWOK S， et al. BACH： Grand challenge on breast cancer histology images[J]. Med Image Anal，2019，56：122-139. doi：10.1016/j.media.2019.05.010 .
14	HA Q S， LIU B， LIU F X. Identifying melanoma images using EfficientNet ensemble： Winning solution to the SIIM-ISIC melanoma classification challenge[J/OL]. （2020-10-11）[2021-09-21]. arXiv：2010. 05351v1.
15	ZHOU Y， CAO Y， HUANG J， et al. Research advances in forensic diatom testing[J]. Forensic Sci Res，2020，5（2）：98-105. doi：10.1080/20961790. 2020.1718901 .
16	HU H， GU J Y， ZHANG Z， et al. Relation networks for object detection[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City：IEEE，2018：3588-3597. doi：10.1109/CVPR.2018.00378 .
17	GIRSHICK R， DONAHUE J， DARRELL T， et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus：IEEE，2014：580-587. doi：10.1109/CVPR.2014.81 .
18	GIRSHICK R. Fast R-CNN[C]//2015 IEEE International Conference on Computer Vision. Santiago：IEEE，2015：1440-1448. doi：10.1109/ICCV. 2015.169 .
19	REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： Towards real-time object detection with region proposal networks[J]. IEEE Trans Pattern Anal Mach Intell，2017，39（6）：1137-1149. doi：10.1109/TPAMI.2016.2577031 .
20	YU W M， XUE Y， KNOOPS R， et al. Automated diatom searching in the digital scanning electron microscopy images of drowning cases using the deep neural networks[J]. Int J Leg Med，2021，135（2）：497-508. doi：10.1007/s00414-020-02392-z .
21	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J/OL].（2015-04-10）[2021-09-21]. arXiv：.
22	SZEGEDY C， IOFFE S， VANHOUCKE V. Inception-V 4， inception-ResNet and the impact of residual connections on learning[J/OL]. （2016-08-23）[2021-09-21]. arXiv：.
23	ZHOU Y Y， ZHANG J， HUANG J， et al. Digital whole-slide image analysis for automated diatom test in forensic cases of drowning using a convolutional neural network algorithm[J]. Forensic Sci Int，2019，302：109922. doi：10.1016/j.forsciint.2019. 109922 .
24	ZHANG J， ZHOU Y， VIEIRA D N， et al. An efficient method for building a database of diatom populations for drowning site inference using a deep learning algorithm[J]. Int J Legal Med，2021，135（3）：817-827. doi：10.1007/s00414-020-02497-5 .
25	黄平，张吉，邓恺飞，等. 一种基于硅藻种属分类的溺水地点鉴定系统及方法：1102916.7[P]. 2021-07-20.
	HUANG P， ZHANG J， DENG K F， et al. A drowning site identification system and method based on the classification of diatom species：1102916.7[P]. 2021-07-02.
26	司法鉴定科学研究院. 智能化硅藻鉴识系统[简称：DiatomNet]V1.0：2021SR0337617[P]. 2020-12-23.
	Accademy of Forensic Science. Intelligent diatom discrimination system[Abbreviation： DiatomNet]V1.0：2021SR0337617[P]. 2020-12-23.
27	福建省公安厅刑侦总队. 人体组织器官中硅藻硝酸破机法检验： [S]. 北京：中国标准出版社，2009.
	Criminal Investigation Corps of Fujian Provincial Public Security Department. Detection the diatoms in human tissues and organs by digesting organic matter with nitric acid： [S]. Beijing： China Standards Publishing Press，2009.
28	DENG J， DONG W， SOCHER R， et al. ImageNet： A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition.Miami： IEEE，2009：248-255. doi：10.1109/CVPR.2009.5206848 .
29	SELVARAJU R R， COGSWELL M， DAS A， et al. Grad-CAM： Visual explanations from deep networks via gradient-based localization[J]. 2017 IEEE Int Conf Comput Vis ICCV，2017：618-626. doi：10.1109/ICCV.2017.74 .
30	SZEGEDY C， VANHOUCKE V， IOFFE S， et al. Rethinking the inception architecture for computer vision[J]. 2016 IEEE Conf Comput Vis Pattern Recognit CVPR，2016：2818-2826. doi：10.1109/CVPR.2016.308 .
31	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas： IEEE，2016：770-778. doi：10.1109/CVPR.2016.90 .
32	KRAUSE L M K， KOC J， ROSENHAHN B， et al. Fully convolutional neural network for detection and counting of diatoms on coatings after short-term field exposure[J]. Environ Sci Technol，2020，54（16）：10022-10030. doi：10.1021/acs.est.0c01982 .
33	RUIZ-SANTAQUITERIA J， BUENO G， DENIZ O， et al. Semantic versus instance segmentation in microscopic algae detection[J]. Eng Appl Artif Intel，2020，87. doi：10.1016/j.engappai.2019.103271 .
34	BUENO G， DENIZ O， RUIZ-SANTAQUITERIA J， et al. Lights and pitfalls of convolutional neural networks for diatom identification[C]//Optics， Photonics and Digital Technologies for Imaging Applications V. Strasbourg：SPIE Photonics Europe，2018. doi：10.1117/12.2309488 .

[1]	Yong ZENG, Dong-hua ZOU, Ying FAN, Qing XU, Lu-yang TAO, Yi-jiu CHEN, Zheng-dong LI. Research Progress and Forensic Application of Human Vascular Finite Element Modeling and Biomechanics [J]. Journal of Forensic Medicine, 2023, 39(5): 471-477.
[2]	Yu-xin SUN, Xiao-juan GONG, Xiu-li HAO, Yu-xin TIAN, Yi-ming CHEN, Bao ZHANG, Chun-xia YAN. Screening of Genes Co-Associated with Sudden Infant Death Syndrome and Infectious Sudden Death in Infancy and Bioinformatics Analysis of Their Regulatory Networks [J]. Journal of Forensic Medicine, 2023, 39(5): 433-440.
[3]	Yu YANG, Fan-zhang LEI, Yu-you DONG, Jian-long MA, Qi-qiang SHI, Xue-song YE. Retrospective Analysis of Death Cases of Oral Diphenidol Hydrochloride Poisoning [J]. Journal of Forensic Medicine, 2023, 39(4): 393-398.
[4]	Jie BAI, Jing SUN, Xiao-guang CHENG, Fan LIU, Hua LIU, Xu WANG. Construction and Application of Rib Fracture Diagnosis Model Based on YOLOv3 Algorithm [J]. Journal of Forensic Medicine, 2023, 39(4): 343-349.
[5]	Qing-qing XIANG, Li-fang CHEN, Qin SU, Yu-kun DU, Pei-yan LIANG, Xiao-dong KANG, He SHI, Qu-yi XU, Jian ZHAO, Chao LIU, Xiao-hui CHEN. Research Progress on Microbial Community Succession in the Postmortem Interval Estimation [J]. Journal of Forensic Medicine, 2023, 39(4): 399-405.
[6]	Qin SU, Qian-ling CHEN, Wei-bin WU, Qing-qing XIANG, Cheng-liang YANG, Dong-fang QIAO, Zhi-gang LI. Metabonomics Analysis of Brain Stem Tissue in Rats with Primary Brain Stem Injury Caused Death [J]. Journal of Forensic Medicine, 2023, 39(4): 373-381.
[7]	Xu-dong ZHANG, Yao-ru JIANG, Xin-rui LIANG, Tian TIAN, Qian-qian JIN, Xiao-hong ZHANG, Jie CAO, Qiu-xiang DU, Jun-hong SUN. Postmortem Interval Estimation Using Protein Chip Technology Combined with Multivariate Analysis Methods [J]. Journal of Forensic Medicine, 2023, 39(2): 115-120.
[8]	Yong-gang MA, Yong-jie CAO, Yi-hua ZHAO, Xin-jun ZHOU, Bin HUANG, Gao-chao ZHANG, Ping HUANG, Ya-hui WANG, Kai-jun MA, Feng CHEN, Dong-chuan ZHANG, Ji ZHANG. Sex Estimation of Medial Aspect of the Ischiopubic Ramus in Adults Based on Deep Learning [J]. Journal of Forensic Medicine, 2023, 39(2): 129-136.
[9]	Wu LONG, Peng-fei QU, Lin MA, Rui WANG, Yan-mei XI, Yu-hua LI, Sheng-jie NIE, Ting DUAN, Jin-liang DU, Xue TANG, Jing-feng ZHAO, Pu-ping LEI, Yue-bing WANG. Cytotoxicity of 4 Wild Mushrooms in a Case of Yunnan Sudden Unexplained Death [J]. Journal of Forensic Medicine, 2023, 39(2): 121-128.
[10]	Jun-wei GAO, Yang LU, Yan-jun LI, Dong-hua ZOU, Guang-long HE, Yan-bin WANG. Survey on the Construction Status of Forensic Virtual Autopsy Laboratory and the Applicability of Laboratory Accreditation [J]. Journal of Forensic Medicine, 2023, 39(2): 186-192.
[11]	Fang-Fang LIU, Hui WU, Wei WANG, Ying XIE. Changes of 1，5-AG in Vitreous Humor of Rabbit Cadavers with Hyperglycemic Metabolism [J]. Journal of Forensic Medicine, 2023, 39(1): 13-17.
[12]	Zhi-ling TIAN, Ruo-lin WANG, Jian-hua ZHANG, Ping HUANG, Zhi-qiang QIN, Zheng-dong LI, He-wen DONG, Dong-hua ZOU, Mao-wen WANG, Zhuo LI, Lei WAN, Xiao-tian YU, Ning-guo LIU. Comparison of CT Values between Thrombus and Postmortem Clot Based on Cadaveric Pulmonary Angiography [J]. Journal of Forensic Medicine, 2023, 39(1): 7-12.
[13]	Hong-xia HAO, Ya-hui WANG, Zhi-lu ZHOU, Tai-ang LIU, Jin CHEN, Yu-heng HE, Lei WAN, Wen-tao XIA. Research Progress of Age Estimation in the Living by Knee Joint MRI [J]. Journal of Forensic Medicine, 2023, 39(1): 66-71.
[14]	Jia-yi WU, You-jia YU, Kai LI, Xin YIN, Han-ting FAN, Rong LI, Zhi-wei ZHANG, Wei TANG, Hui-jie HUANG, Feng CHEN. Report on Cardiac Gross Pathologic Measurements of Sudden Cardiac Death in Adults [J]. Journal of Forensic Medicine, 2023, 39(1): 1-6.
[15]	Hui WU, Fang-fang LIU, Jun-da WU, Ying XIE. Research Progress on Estimation of Postmortem Interval Based on Ocular Tissues Structure [J]. Journal of Forensic Medicine, 2023, 39(1): 50-56.