Comparison among Four Deep Learning Image Classification Algorithms in AI-based Diatom Test

doi:10.12116/j.issn.1004-5619.2021.411001

Abstract

Abstract: Objective

To select four algorithms with relatively balanced complexity and accuracy among deep learning image classification algorithms for automatic diatom recognition, and to explore the most suitable classification algorithm for diatom recognition to provide data reference for automatic diatom testing research in forensic medicine.

Methods

The “diatom” and “background” small sample size data set （20 000 images） of digestive fluid smear of corpse lung tissue in water were built to train, validate and test four convolutional neural network （CNN） models, including VGG16, ResNet50, InceptionV3 and Inception-ResNet-V2. The receiver operating characteristic curve （ROC） of subjects and confusion matrixes were drawn, recall rate, precision rate， specificity， accuracy rate and F1 score were calculated， and the performance of each model was systematically evaluated.

Results

The InceptionV3 model achieved much better results than the other three models with a balanced recall rate of 89.80%, a precision rate of 92.58%. The VGG16 and Inception-ResNet-V2 had similar diatom recognition performance. Although the performance of diatom recall and precision detection could not be balanced, the recognition ability was acceptable. ResNet50 had the lowest diatom recognition performance, with a recall rate of 55.35%. In terms of feature extraction, the four models all extracted the features of diatom and background and mainly focused on diatom region as the main identification basis.

Conclusion

Including the Inception-dependent model, which has stronger directivity and targeting in feature extraction of diatom. The InceptionV3 achieved the best performance on diatom identification and feature extraction compared to the other three models. The InceptionV3 is more suitable for daily forensic diatom examination.

Key words: forensic pathology, artificial intelligence, drowning, deep learning, diatom test, convolutional neural network

CLC Number:

DF795.1

Yong-zheng ZHU, Ji ZHANG, Qi CHENG, Hui-xiao YU, Kai-fei DENG, Jian-hua ZHANG, Zhi-qiang QIN, Jian ZHAO, Jun-hong SUN, Ping HUANG. Comparison among Four Deep Learning Image Classification Algorithms in AI-based Diatom Test[J]. Journal of Forensic Medicine, 2022, 38(1): 31-39.

Add to citation manager EndNote|Ris|BibTeX

URL: http://www.fyxzz.cn/EN/10.12116/j.issn.1004-5619.2021.411001

http://www.fyxzz.cn/EN/Y2022/V38/I1/31

Figures/Tables 8

Fig. 1 Model training data set after data enhancement

Fig. 2 Flowchart of four DL image classification models

Fig. 3 The accuracy and loss curves of four models after the 2nd order smoothing on training and validation

Tab. 1 Optimal performance of four models

最优模型	训练集		验证集		迭代次数
最优模型	准确率/%	损失函数值	准确率/%	损失函数值	迭代次数
VGG16	99.99	0.001	96.25	0.074	89
ResNet50	99.99	0.001	94.00	0.143	70
InceptionV3	99.77	0.012	98.91	0.042	45
Inception-ResNet-V2	99.65	0.043	97.50	0.176	46

Tab. 2 Diatom identification performance parameters of four models

模型	RR/%	PR/%	S/%	AR/%	F1分数
VGG16	84.65	78.96	77.45	81.05	0.82
ResNet50	55.35	81.04	87.05	71.20	0.66
InceptionV3	89.80	92.58	92.80	91.30	0.91
Inception-ResNet-V2	72.35	87.22	89.40	80.88	0.79

Fig. 4 The ROC curves and AUC values of 4 optimal models

Fig. 5 The confusion matrixes， recall rate and specificity of 4 optimal models

Fig. 6 Heat map analysis of pennataes and centricaes identificated with 4 models by Grad-CAM and GBP methods

References 34

1	World Health Organization. Global report on drowning： Preventing a leading killer[R/OL]. （2014-09-17）[2021-09-21]. .
2	SZPILMAN D， MORGAN P J. Management for the drowning patient[J]. Chest，2021，159（4）：1473-1483. doi：10.1016/j.chest.2020.10.007 .
3	李棨，马开军，张晓东，等. 水中尸体肺组织硅藻检验与死因分析[J].法医学杂志，2011，27（5）：324-326，333. doi：10.3969/j.issn.1004-5619.2011.05.002 .
	LI Q， MA K J， ZHANG X D， et al. Diatom test in lung tissue of corpses in water and causes of death[J]. Fayixue Zazhi，2011，27（5）：324-326，333.
4	CARBALLEIRA R， VIEIRA D N， FEBRERO-BANDE M， et al. A valid method to determine the site of drowning[J]. Int J Legal Med，2018，132（2）：487-497. doi：10.1007/s00414-017-1708-1 .
5	PIETTE M H A， DE LETTER E A. Drowning： Still a difficult autopsy diagnosis[J]. Forensic Sci Int，2006，163（1/2）：1-9. doi：10.1016/j.forsciint.2004.10.027 .
6	LUNETTA P， PENTTILÄ A， HÄLLFORS G. Scanning and transmission electron microscopical evidence of the capacity of diatoms to penetrate the alveolo-capillary barrier in drowning[J]. Int J Legal Med，1998，111（5）：229-237. doi：10.1007/s004140050159 .
7	ZHANG P， KANG X， ZHANG S， et al. The length and width of diatoms in drowning cases as the evidence of diatoms penetrating the alveoli-capillary barrier[J]. Int J Legal Med，2020，134（3）：1037-1042. doi：10.1007/s00414-019-02164-4 .
8	LUDES B， COSTE M， NORTH N， et al. Diatom analysis in victim’s tissues as an indicator of the site of drowning[J]. Int J Legal Med，1999，112（3）：163-166. doi：10.1007/s004140050224 .
9	SHEN X， LIU Y， XIAO C， et al. Analysis of false-positive results of diatom test in the diagnosis of drowning-would not be an impediment[J]. Int J Legal Med，2019，133（6）：1819-1824. doi：10.1007/s00414-019-02021-4 .
10	BORTOLOTTI F， TAGLIARO F， MANETTO G. Objective diagnosis of drowning by the “diatom test” - A critical review[J]. Forensic Sci Rev，2004，16（2）：135-148.
11	周圆圆，曹永杰，杨越，等. 基于人工智能硅藻自动化识别系统的实际案例应用[J].法医学杂志，2020，36（2）：239-242. doi：10.12116/j.issn.1004-5619.2020.02.017 .
	ZHOU Y Y， CAO Y J， YANG Y， et al. Application of artificial intelligence automatic diatom identification system in practical cases[J]. Fayixue Zazhi，2020，36（2）：239-242.
12	KRIZHEVSKY A， SUTSKEVER I， HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Commun ACM，2017，60（6）：84-90. doi：10.1145/3065386 .
13	ARESTA G， ARAÚJO T， KWOK S， et al. BACH： Grand challenge on breast cancer histology images[J]. Med Image Anal，2019，56：122-139. doi：10.1016/j.media.2019.05.010 .
14	HA Q S， LIU B， LIU F X. Identifying melanoma images using EfficientNet ensemble： Winning solution to the SIIM-ISIC melanoma classification challenge[J/OL]. （2020-10-11）[2021-09-21]. arXiv：2010. 05351v1.
15	ZHOU Y， CAO Y， HUANG J， et al. Research advances in forensic diatom testing[J]. Forensic Sci Res，2020，5（2）：98-105. doi：10.1080/20961790. 2020.1718901 .
16	HU H， GU J Y， ZHANG Z， et al. Relation networks for object detection[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City：IEEE，2018：3588-3597. doi：10.1109/CVPR.2018.00378 .
17	GIRSHICK R， DONAHUE J， DARRELL T， et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus：IEEE，2014：580-587. doi：10.1109/CVPR.2014.81 .
18	GIRSHICK R. Fast R-CNN[C]//2015 IEEE International Conference on Computer Vision. Santiago：IEEE，2015：1440-1448. doi：10.1109/ICCV. 2015.169 .
19	REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： Towards real-time object detection with region proposal networks[J]. IEEE Trans Pattern Anal Mach Intell，2017，39（6）：1137-1149. doi：10.1109/TPAMI.2016.2577031 .
20	YU W M， XUE Y， KNOOPS R， et al. Automated diatom searching in the digital scanning electron microscopy images of drowning cases using the deep neural networks[J]. Int J Leg Med，2021，135（2）：497-508. doi：10.1007/s00414-020-02392-z .
21	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J/OL].（2015-04-10）[2021-09-21]. arXiv：.
22	SZEGEDY C， IOFFE S， VANHOUCKE V. Inception-V 4， inception-ResNet and the impact of residual connections on learning[J/OL]. （2016-08-23）[2021-09-21]. arXiv：.
23	ZHOU Y Y， ZHANG J， HUANG J， et al. Digital whole-slide image analysis for automated diatom test in forensic cases of drowning using a convolutional neural network algorithm[J]. Forensic Sci Int，2019，302：109922. doi：10.1016/j.forsciint.2019. 109922 .
24	ZHANG J， ZHOU Y， VIEIRA D N， et al. An efficient method for building a database of diatom populations for drowning site inference using a deep learning algorithm[J]. Int J Legal Med，2021，135（3）：817-827. doi：10.1007/s00414-020-02497-5 .
25	黄平，张吉，邓恺飞，等. 一种基于硅藻种属分类的溺水地点鉴定系统及方法：1102916.7[P]. 2021-07-20.
	HUANG P， ZHANG J， DENG K F， et al. A drowning site identification system and method based on the classification of diatom species：1102916.7[P]. 2021-07-02.
26	司法鉴定科学研究院. 智能化硅藻鉴识系统[简称：DiatomNet]V1.0：2021SR0337617[P]. 2020-12-23.
	Accademy of Forensic Science. Intelligent diatom discrimination system[Abbreviation： DiatomNet]V1.0：2021SR0337617[P]. 2020-12-23.
27	福建省公安厅刑侦总队. 人体组织器官中硅藻硝酸破机法检验： [S]. 北京：中国标准出版社，2009.
	Criminal Investigation Corps of Fujian Provincial Public Security Department. Detection the diatoms in human tissues and organs by digesting organic matter with nitric acid： [S]. Beijing： China Standards Publishing Press，2009.
28	DENG J， DONG W， SOCHER R， et al. ImageNet： A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition.Miami： IEEE，2009：248-255. doi：10.1109/CVPR.2009.5206848 .
29	SELVARAJU R R， COGSWELL M， DAS A， et al. Grad-CAM： Visual explanations from deep networks via gradient-based localization[J]. 2017 IEEE Int Conf Comput Vis ICCV，2017：618-626. doi：10.1109/ICCV.2017.74 .
30	SZEGEDY C， VANHOUCKE V， IOFFE S， et al. Rethinking the inception architecture for computer vision[J]. 2016 IEEE Conf Comput Vis Pattern Recognit CVPR，2016：2818-2826. doi：10.1109/CVPR.2016.308 .
31	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas： IEEE，2016：770-778. doi：10.1109/CVPR.2016.90 .
32	KRAUSE L M K， KOC J， ROSENHAHN B， et al. Fully convolutional neural network for detection and counting of diatoms on coatings after short-term field exposure[J]. Environ Sci Technol，2020，54（16）：10022-10030. doi：10.1021/acs.est.0c01982 .
33	RUIZ-SANTAQUITERIA J， BUENO G， DENIZ O， et al. Semantic versus instance segmentation in microscopic algae detection[J]. Eng Appl Artif Intel，2020，87. doi：10.1016/j.engappai.2019.103271 .
34	BUENO G， DENIZ O， RUIZ-SANTAQUITERIA J， et al. Lights and pitfalls of convolutional neural networks for diatom identification[C]//Optics， Photonics and Digital Technologies for Imaging Applications V. Strasbourg：SPIE Photonics Europe，2018. doi：10.1117/12.2309488 .

[1]	Jie YANG, Yi-lei HU, Jin-ming WANG, Zheng-dong LI. Analysis of the Injury Mechanism of Traumatic Craniocerebral Injury from Falls Based on Finite Element Simulation [J]. Journal of Forensic Medicine, 2026, 42(1): 1-7.
[2]	Qiong JIA, Kai-jun JIN, Shi-qi LIU, Yi-meng ZHANG, Yun-le MENG, Hao NIE, Sheng HU, Fan YANG, Xing-chun ZHAO, Jian YE. Research Progress on the Methodological System for Decomposition Stage Classification in Forensic Medicine [J]. Journal of Forensic Medicine, 2026, 42(1): 34-42.
[3]	Xin NIU, Zhe CAO. Forensic Significance of Mesenteric Injury in Child Abuse Cases [J]. Journal of Forensic Medicine, 2026, 42(1): 43-48.
[4]	Zhen WANG, Tai-qi ZANG. Comprehensive Identification of the Vulnerant by Examining Marks on Clothing and Bodies Combining with Trace Comparison： A Case Report [J]. Journal of Forensic Medicine, 2026, 42(1): 69-72.
[5]	Jia MENG, Ling-yu LI, Wu-lin HUANG, Tong ZHOU, Meng-qin ZOU, Jian-wei PENG, Bo LI, Xin HUANG, Qing-bo ZHANG. Multidisciplinary Identification in a Hit-and-Run Traffic Accident： A Case Report [J]. Journal of Forensic Medicine, 2026, 42(1): 73-76.
[6]	Cong-da SHEN, Shi-fan WU, Yi WU, Wen-dong MAO, Xiang-tao MA. Fatal Rupture of the Abdominal Aorta Caused by Trampling： A Case Report [J]. Journal of Forensic Medicine, 2026, 42(1): 77-78.
[7]	Jun-jiang CHEN, Lan-jiang LI, Pu-ping LEI, Shang-wen WANG, Xin-biao LIAO, Li-fang CHEN, Huan ZHANG. Forensic Identification of a Fatality from Confined Container Deflagration Complicated by Tertiary Blast Injury： A Case Report [J]. Journal of Forensic Medicine, 2026, 42(1): 79-83.
[8]	Jian YE, Yong-liang HU, Ya-ling XIE, Yu CAI, Wen-hao LI. Medical Damage Identification of CT Guided Pulmonary Nodule Localization Needle Escapes to the Left Ventricle： A Case Report [J]. Journal of Forensic Medicine, 2026, 42(1): 83-86.
[9]	Ya-ning SUN, Dan-yang LI, Qing XIA, Yan GAO, Zong-shi XU, Wen-tao XIA, Hui-ming ZHOU, Xiao-meng HAN, Xiao-ying YU, Yan-liang SHENG. Frontiers and Perspectives of Artificial Intelligence in Forensic Research on CT Imaging Diagnosis of Rib Fractures [J]. Journal of Forensic Medicine, 2025, 41(6): 593-600.
[10]	Ning-guo LIU. Innovative Development and Future Prospects in Forensic Pathology [J]. Journal of Forensic Medicine, 2025, 41(6): 517-530.
[11]	Hu ZHAO. Current Status and Challenges of Forensic Psychiatry Research [J]. Journal of Forensic Medicine, 2025, 41(6): 531-536.
[12]	Yu-mei WANG. Postmortem Submersion Interval Estimation of Bodies Recovered from the Jingzhou Reach of the Yangtze River by Aquatic Decomposition Score [J]. Journal of Forensic Medicine, 2025, 41(6): 552-558.
[13]	Jin-xuan YANG, Dong-an TIAN, Xin-yu TANG, Zhan-ying ZHAO, Xi-yue JING, Dong-mei WANG, Xiao-fei TIAN. The Role of Iron Dysregulation in White Matter Injury Induced by Carbon Mono- xide Poisoning [J]. Journal of Forensic Medicine, 2025, 41(6): 545-551.
[14]	Li-liang LI, Jian-hua ZHANG, Feng CHEN, Rui ZHAO, Ya-dong GUO, Zhen-yuan WANG, Yi-wu ZHOU, Da-wei GUAN, Bin CONG. Expert Consensus on Cause of Death Determinations in Antipsychotics-Related Deaths with Negative Autopsy Findings [J]. Journal of Forensic Medicine, 2025, 41(6): 537-544.
[15]	Ji CHEN, Yu-rong ZHAO, Xin HUANG, Yi-ling QU, Yan-fang LU, Yu XING, Han ZHANG, Jian-ye ZENG, Shi-lin LI, Su-hua ZHANG. Effect of Temperature on Microbial Succession in Different Tissues of Cadavers and Estimation of Postmortem Interval [J]. Journal of Forensic Medicine, 2025, 41(5): 456-467.