Construction and Application of YOLOv3-Based Diatom Identification Model of Scanning Electron Microscope Images

doi:10.12116/j.issn.1004-5619.2021.410903

Abstract

Abstract: Objective

To construct a YOLOv3-based model for diatom identification in scanning electron microscope images, explore the application performance in practical cases and discuss the advantages of this model.

Methods

A total of 25 000 scanning electron microscopy images were collected at 1 500× as an initial image set, and input into the YOLOv3 network to train the identification model after experts’ annotation and image processing. Diatom scanning electron microscopy images of lung, liver and kidney tissues taken from 8 drowning cases were identified by this model under the threshold of 0.4, 0.6 and 0.8 respectively, and were also identified by experts manually. The application performance of this model was evaluated through the recognition speed, recall rate and precision rate.

Results

The mean average precision of the model in the validation set and test set was 94.8% and 94.3%, respectively, and the average recall rate was 81.2% and 81.5%, respectively. The recognition speed of the model is more than 9 times faster than that of manual recognition. Under the threshold of 0.4, the mean recall rate and precision rate of diatoms in lung tissues were 89.6% and 87.8%, respectively. The overall recall rate in liver and kidney tissues was 100% and the precision rate was less than 5%. As the threshold increased, the recall rate in all tissues decreased and the precision rate increased. The F1 score of the model in lung tissues decreased with the increase of threshold, while the F1 score in liver and kidney tissues with the increase of threshold.

Conclusion

The YOLOv3-based diatom electron microscope images automatic identification model works at a rapid speed and shows high recall rates in all tissues and high precision rates in lung tissues under an appropriate threshold. The identification model greatly reduces the workload of manual recognition, and has a good application prospect.

Key words: forensic pathology, artificial intelligence, deep learning, diatom test, YOLOv3, scanning electron microscope, image recognition

CLC Number:

DF795.1

Ji CHEN, Xiao-rong LIU, Jia-wen YANG, Ye-qiu CHEN, Cheng WANG, Meng-yuan OU, Jia-yi WU, You-jia YU, Kai LI, Peng CHEN, Feng CHEN. Construction and Application of YOLOv3-Based Diatom Identification Model of Scanning Electron Microscope Images[J]. Journal of Forensic Medicine, 2022, 38(1): 46-52.

Add to citation manager EndNote|Ris|BibTeX

URL: http://www.fyxzz.cn/EN/10.12116/j.issn.1004-5619.2021.410903

http://www.fyxzz.cn/EN/Y2022/V38/I1/46

Figures/Tables 6

References 19

1	刘超. 溺死法医诊断学[M].广州：中山大学出版社，2018：52-53.
	LIU C. Diagnosis of drowning[M]. Guangzhou： Sun Yat-sen University Press，2018：52-53.
2	PACHAR J V， CAMERON J M. The diagnosis of drowning by quantitative and qualitative diatom analysis[J]. Med Sci Law，1993，33（4）：291-299. doi：10.1177/002580249303300404 .
3	LUDES B， COSTE M， NORTH N， et al. Diatom analysis in victim’s tissues as an indicator of the site of drowning[J]. Int J Legal Med，1999，112（3）：163-166. doi：10.1007/s004140050224 .
4	李峥，杨佳雯，陈冶秋，等. 硅藻检验推测溺亡位置1例[J].法医学杂志，2020，36（6）：878-880. doi：10.12116/j.issn.1004-5619.2020.06.024 .
	LI Z， YANG J W， CHEN Y Q， et al. Diatom test speculated the location of drowning： A case report[J]. Fayixue Zazhi，2020，36（6）：878-880.
5	赵建，袁自闯，张彦吉，等. 两种硅藻检验方法的比较[J].中国法医学杂志，2015，30（1）：62-65. doi：10.13618/j.issn.1001-5728.2015.01.018 .
	ZHAO J， YUAN Z C， ZHANG Y J， et al. The comparative study of two diatom test methods[J]. Zhongguo Fayixue Zazhi，2015，30（1）：62-65.
6	ZHAO J， LIU C， HU S， et al. Microwave digestion - vacuum filtration - automated scanning electron microscopy as a sensitive method for forensic diatom test[J]. Int J Legal Med，2013，127（2）：459-463. doi：10.1007/s00414-012-0756-9 .
7	LI Y， HUANG Z， DONG X， et al. Forensic age estimation for pelvic X-ray images using deep learning[J]. Eur Radiol，2019，29（5）：2322-2329. doi：10.1007/s00330-018-5791-6 .
8	CHANDRA S S， BRAN LORENZANA M， LIU X， et al. Deep learning in magnetic resonance image reconstruction[J]. J Med Imaging Radiat Oncol，2021，65（5）：564-577. doi：10.1111/1754-9485.13276 .
9	YANG H B， LIU P， HU Y Z， et al. Research on underwater object recognition based on YOLOv3[J]. Microsyst Technol，2021，27（4）：1837-1844. doi：10.1007/s00542-019-04694-8 .
10	ZHOU Y Y， ZHANG J， HUANG J， et al. Digital whole-slide image analysis for automated diatom test in forensic cases of drowning using a convolutional neural network algorithm[J]. Forensic Sci Int，2019，302：109922. doi：10.1016/j.forsciint.2019. 109922 .
11	周圆圆，曹永杰，杨越，等. 基于人工智能硅藻自动化识别系统的实际案例应用[J].法医学杂志，2020，36（2）：239-242. doi：10.12116/j.issn.1004-5619.2020.02.017 .
	ZHOU Y Y， CAO Y J， YANG Y， et al. Application of artificial intelligence automatic diatom identification system in practical cases[J]. Fayixue Zazhi，2020，36（2）：239-242.
12	YU W M， XUE Y， KNOOPS R， et al. Automated diatom searching in the digital scanning electron microscopy images of drowning cases using the deep neural networks[J]. Int J Leg Med，2021，135（2）：497-508. doi：10.1007/s00414-020-02392-z .
13	REDMON J， DIVVALA S， GIRSHICK R， et al. You only look once： Unified， real-time object detection[C]//2016 IEEE Conf Comput Vis Pattern Recognit. Las Vegas： IEEE，2016：779-788. doi：10.1109/CVPR. 2016.91 .
14	REDMON J， FARHADI A. YOLO9000： Better， faster， stronger[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu： IEEE，2017：6517-6525. doi：10.1109/CVPR.2017.690 .
15	REDMON J， FARHADI A. YOLOv3： An incremental improvement[J/OL]. （2018-04-08）[2021-08-25]. arXiv：.
16	邓杰航，何冬冬，卓家鸿，等. 复杂背景干扰下硅藻图像的深度网络识别与定位[J].南方医科大学学报，2020，40（2）：183-189. doi：10.12122/j.issn.1673-4254.2020.02.08 .
	DENG J H， HE D D， ZHUO J H， et al. Deep learning network-based recognition and localization of diatom images against complex background[J]. Nanfang Yike Daxue Xuebao，2020，40（2）：183-189.
17	PEDRAZA A， BUENO G， DENIZ O， et al. Automated diatom classification （Part B）： A deep learning approach[J]. Appl Sci，2017，7（5）：460. doi：10.3390/app7050460 .
18	KLOSTER M， LANGENKÄMPER D， ZUROWIETZ M， et al. Deep learning-based diatom taxonomy on virtual slides[J]. Sci Rep，2020，10：14416. doi：10.1038/s41598-020-71165-w .
19	DENG J H， GUO W Q， ZHAO Y W， et al. Identification of diatom taxonomy by a combination of region-based full convolutional network，online hard example mining，and shape priors of diatoms[J]. Int J Leg Med，2021，135（6）：2519-2530. doi：10.1007/s00414-021-02664-2 .

阈值	验证集			测试集
阈值	RR/%	PR/%	F1分数	RR/%	PR/%	F1分数
0.05	99.2	58.9	0.739	99.5	57.6	0.729
0.15	98.8	79.2	0.879	98.8	76.1	0.859
0.25	98.8	80.5	0.887	98.6	77.4	0.867
0.35	97.9	81.4	0.889	97.2	78.1	0.866
0.45	77.9	95.1	0.856	77.5	92.9	0.845
0.55	73.2	99.3	0.843	73.6	98.6	0.843
0.65	72.4	99.6	0.838	72.9	99.5	0.841
0.75	71.8	99.7	0.835	72.3	99.8	0.839
0.85	70.3	99.8	0.825	70.8	100.0	0.829
0.95	51.4	100.0	0.679	53.3	100.0	0.695

阈值	验证集			测试集
阈值	RR/%	PR/%	F1分数	RR/%	PR/%	F1分数
0.05	99.2	58.9	0.739	99.5	57.6	0.729
0.15	98.8	79.2	0.879	98.8	76.1	0.859
0.25	98.8	80.5	0.887	98.6	77.4	0.867
0.35	97.9	81.4	0.889	97.2	78.1	0.866
0.45	77.9	95.1	0.856	77.5	92.9	0.845
0.55	73.2	99.3	0.843	73.6	98.6	0.843
0.65	72.4	99.6	0.838	72.9	99.5	0.841
0.75	71.8	99.7	0.835	72.3	99.8	0.839
0.85	70.3	99.8	0.825	70.8	100.0	0.829
0.95	51.4	100.0	0.679	53.3	100.0	0.695

组织	阈值	完整硅藻			碎片硅藻			总体
组织	阈值	RR/%	PR/%	F1分数	RR/%	PR/%	F1分数	RR/%	PR/%	F1分数
肺	0.4	94.0±6.7	85.9±27.9	0.862±0.225	75.8±12.9	78.2±29.7	0.732±0.219	89.6±6.6	87.8±25.0	0.865±0.181
	0.6	81.5±16.1	88.4±24.5	0.812±0.175	39.1±15.0	79.6±29.1	0.463±0.162	69.7±11.8	89.8±21.9	0.760±0.130
	0.8	71.5±16.8	90.0±22.8	0.774±0.178	24.5±8.3	80.2±29.0	0.345±0.121	58.4±11.8	91.0±20.8	0.699±0.139
肝	0.4	100.0±0.0	1.1±1.2	0.022±0.024	100.0±0.0	1.7±1.2	0.034±0.023	100.0±0.0	2.5±2.0	0.049±0.039
	0.6	94.4±13.6	1.2±1.1	0.024±0.021	92.7±13.7	1.9±1.4	0.037±0.027	92.3±10.9	2.8±2.2	0.053±0.039
	0.8	66.7±42.2	1.5±1.6	0.029±0.030	76.0±34.3	3.4±2.7	0.064±0.050	75.6±27.7	4.4±3.4	0.082±0.061
肾	0.4	100.0±0.0	1.7±2.0	0.033±0.039	100.0±0.0	2.4±3.2	0.047±0.057	100.0±0.0	3.6±4.6	0.067±0.078
	0.6	100.0±0.0	2.0±1.9	0.039±0.036	91.0±17.6	2.8±2.9	0.053±0.052	93.8±11.8	4.2±4.3	0.077±0.073
	0.8	83.3±40.8	3.6±4.3	0.066±0.077	80.6±22.1	4.7±6.1	0.085±0.099	80.3±26.5	6.8±8.5	0.118±0.129

组织	阈值	完整硅藻			碎片硅藻			总体
组织	阈值	RR/%	PR/%	F1分数	RR/%	PR/%	F1分数	RR/%	PR/%	F1分数
肺	0.4	94.0±6.7	85.9±27.9	0.862±0.225	75.8±12.9	78.2±29.7	0.732±0.219	89.6±6.6	87.8±25.0	0.865±0.181
	0.6	81.5±16.1	88.4±24.5	0.812±0.175	39.1±15.0	79.6±29.1	0.463±0.162	69.7±11.8	89.8±21.9	0.760±0.130
	0.8	71.5±16.8	90.0±22.8	0.774±0.178	24.5±8.3	80.2±29.0	0.345±0.121	58.4±11.8	91.0±20.8	0.699±0.139
肝	0.4	100.0±0.0	1.1±1.2	0.022±0.024	100.0±0.0	1.7±1.2	0.034±0.023	100.0±0.0	2.5±2.0	0.049±0.039
	0.6	94.4±13.6	1.2±1.1	0.024±0.021	92.7±13.7	1.9±1.4	0.037±0.027	92.3±10.9	2.8±2.2	0.053±0.039
	0.8	66.7±42.2	1.5±1.6	0.029±0.030	76.0±34.3	3.4±2.7	0.064±0.050	75.6±27.7	4.4±3.4	0.082±0.061
肾	0.4	100.0±0.0	1.7±2.0	0.033±0.039	100.0±0.0	2.4±3.2	0.047±0.057	100.0±0.0	3.6±4.6	0.067±0.078
	0.6	100.0±0.0	2.0±1.9	0.039±0.036	91.0±17.6	2.8±2.9	0.053±0.052	93.8±11.8	4.2±4.3	0.077±0.073
	0.8	83.3±40.8	3.6±4.3	0.066±0.077	80.6±22.1	4.7±6.1	0.085±0.099	80.3±26.5	6.8±8.5	0.118±0.129

[1]	Jie YANG, Yi-lei HU, Jin-ming WANG, Zheng-dong LI. Analysis of the Injury Mechanism of Traumatic Craniocerebral Injury from Falls Based on Finite Element Simulation [J]. Journal of Forensic Medicine, 2026, 42(1): 1-7.
[2]	Qiong JIA, Kai-jun JIN, Shi-qi LIU, Yi-meng ZHANG, Yun-le MENG, Hao NIE, Sheng HU, Fan YANG, Xing-chun ZHAO, Jian YE. Research Progress on the Methodological System for Decomposition Stage Classification in Forensic Medicine [J]. Journal of Forensic Medicine, 2026, 42(1): 34-42.
[3]	Xin NIU, Zhe CAO. Forensic Significance of Mesenteric Injury in Child Abuse Cases [J]. Journal of Forensic Medicine, 2026, 42(1): 43-48.
[4]	Zhen WANG, Tai-qi ZANG. Comprehensive Identification of the Vulnerant by Examining Marks on Clothing and Bodies Combining with Trace Comparison： A Case Report [J]. Journal of Forensic Medicine, 2026, 42(1): 69-72.
[5]	Jia MENG, Ling-yu LI, Wu-lin HUANG, Tong ZHOU, Meng-qin ZOU, Jian-wei PENG, Bo LI, Xin HUANG, Qing-bo ZHANG. Multidisciplinary Identification in a Hit-and-Run Traffic Accident： A Case Report [J]. Journal of Forensic Medicine, 2026, 42(1): 73-76.
[6]	Cong-da SHEN, Shi-fan WU, Yi WU, Wen-dong MAO, Xiang-tao MA. Fatal Rupture of the Abdominal Aorta Caused by Trampling： A Case Report [J]. Journal of Forensic Medicine, 2026, 42(1): 77-78.
[7]	Jun-jiang CHEN, Lan-jiang LI, Pu-ping LEI, Shang-wen WANG, Xin-biao LIAO, Li-fang CHEN, Huan ZHANG. Forensic Identification of a Fatality from Confined Container Deflagration Complicated by Tertiary Blast Injury： A Case Report [J]. Journal of Forensic Medicine, 2026, 42(1): 79-83.
[8]	Jian YE, Yong-liang HU, Ya-ling XIE, Yu CAI, Wen-hao LI. Medical Damage Identification of CT Guided Pulmonary Nodule Localization Needle Escapes to the Left Ventricle： A Case Report [J]. Journal of Forensic Medicine, 2026, 42(1): 83-86.
[9]	Ya-ning SUN, Dan-yang LI, Qing XIA, Yan GAO, Zong-shi XU, Wen-tao XIA, Hui-ming ZHOU, Xiao-meng HAN, Xiao-ying YU, Yan-liang SHENG. Frontiers and Perspectives of Artificial Intelligence in Forensic Research on CT Imaging Diagnosis of Rib Fractures [J]. Journal of Forensic Medicine, 2025, 41(6): 593-600.
[10]	Ning-guo LIU. Innovative Development and Future Prospects in Forensic Pathology [J]. Journal of Forensic Medicine, 2025, 41(6): 517-530.
[11]	Hu ZHAO. Current Status and Challenges of Forensic Psychiatry Research [J]. Journal of Forensic Medicine, 2025, 41(6): 531-536.
[12]	Yu-mei WANG. Postmortem Submersion Interval Estimation of Bodies Recovered from the Jingzhou Reach of the Yangtze River by Aquatic Decomposition Score [J]. Journal of Forensic Medicine, 2025, 41(6): 552-558.
[13]	Jin-xuan YANG, Dong-an TIAN, Xin-yu TANG, Zhan-ying ZHAO, Xi-yue JING, Dong-mei WANG, Xiao-fei TIAN. The Role of Iron Dysregulation in White Matter Injury Induced by Carbon Mono- xide Poisoning [J]. Journal of Forensic Medicine, 2025, 41(6): 545-551.
[14]	Li-liang LI, Jian-hua ZHANG, Feng CHEN, Rui ZHAO, Ya-dong GUO, Zhen-yuan WANG, Yi-wu ZHOU, Da-wei GUAN, Bin CONG. Expert Consensus on Cause of Death Determinations in Antipsychotics-Related Deaths with Negative Autopsy Findings [J]. Journal of Forensic Medicine, 2025, 41(6): 537-544.
[15]	Ji CHEN, Yu-rong ZHAO, Xin HUANG, Yi-ling QU, Yan-fang LU, Yu XING, Han ZHANG, Jian-ye ZENG, Shi-lin LI, Su-hua ZHANG. Effect of Temperature on Microbial Succession in Different Tissues of Cadavers and Estimation of Postmortem Interval [J]. Journal of Forensic Medicine, 2025, 41(5): 456-467.