Construction and Application of YOLOv3-Based Diatom Identification Model of Scanning Electron Microscope Images

doi:10.12116/j.issn.1004-5619.2021.410903

Abstract

Abstract: Objective

To construct a YOLOv3-based model for diatom identification in scanning electron microscope images, explore the application performance in practical cases and discuss the advantages of this model.

Methods

A total of 25 000 scanning electron microscopy images were collected at 1 500× as an initial image set, and input into the YOLOv3 network to train the identification model after experts’ annotation and image processing. Diatom scanning electron microscopy images of lung, liver and kidney tissues taken from 8 drowning cases were identified by this model under the threshold of 0.4, 0.6 and 0.8 respectively, and were also identified by experts manually. The application performance of this model was evaluated through the recognition speed, recall rate and precision rate.

Results

The mean average precision of the model in the validation set and test set was 94.8% and 94.3%, respectively, and the average recall rate was 81.2% and 81.5%, respectively. The recognition speed of the model is more than 9 times faster than that of manual recognition. Under the threshold of 0.4, the mean recall rate and precision rate of diatoms in lung tissues were 89.6% and 87.8%, respectively. The overall recall rate in liver and kidney tissues was 100% and the precision rate was less than 5%. As the threshold increased, the recall rate in all tissues decreased and the precision rate increased. The F1 score of the model in lung tissues decreased with the increase of threshold, while the F1 score in liver and kidney tissues with the increase of threshold.

Conclusion

The YOLOv3-based diatom electron microscope images automatic identification model works at a rapid speed and shows high recall rates in all tissues and high precision rates in lung tissues under an appropriate threshold. The identification model greatly reduces the workload of manual recognition, and has a good application prospect.

Key words: forensic pathology, artificial intelligence, deep learning, diatom test, YOLOv3, scanning electron microscope, image recognition

CLC Number:

DF795.1

Ji CHEN, Xiao-rong LIU, Jia-wen YANG, Ye-qiu CHEN, Cheng WANG, Meng-yuan OU, Jia-yi WU, You-jia YU, Kai LI, Peng CHEN, Feng CHEN. Construction and Application of YOLOv3-Based Diatom Identification Model of Scanning Electron Microscope Images[J]. Journal of Forensic Medicine, 2022, 38(1): 46-52.

Figures/Tables 6

References 19

1	刘超. 溺死法医诊断学[M].广州：中山大学出版社，2018：52-53.
	LIU C. Diagnosis of drowning[M]. Guangzhou： Sun Yat-sen University Press，2018：52-53.
2	PACHAR J V， CAMERON J M. The diagnosis of drowning by quantitative and qualitative diatom analysis[J]. Med Sci Law，1993，33（4）：291-299. doi：10.1177/002580249303300404 .
3	LUDES B， COSTE M， NORTH N， et al. Diatom analysis in victim’s tissues as an indicator of the site of drowning[J]. Int J Legal Med，1999，112（3）：163-166. doi：10.1007/s004140050224 .
4	李峥，杨佳雯，陈冶秋，等. 硅藻检验推测溺亡位置1例[J].法医学杂志，2020，36（6）：878-880. doi：10.12116/j.issn.1004-5619.2020.06.024 .
	LI Z， YANG J W， CHEN Y Q， et al. Diatom test speculated the location of drowning： A case report[J]. Fayixue Zazhi，2020，36（6）：878-880.
5	赵建，袁自闯，张彦吉，等. 两种硅藻检验方法的比较[J].中国法医学杂志，2015，30（1）：62-65. doi：10.13618/j.issn.1001-5728.2015.01.018 .
	ZHAO J， YUAN Z C， ZHANG Y J， et al. The comparative study of two diatom test methods[J]. Zhongguo Fayixue Zazhi，2015，30（1）：62-65.
6	ZHAO J， LIU C， HU S， et al. Microwave digestion - vacuum filtration - automated scanning electron microscopy as a sensitive method for forensic diatom test[J]. Int J Legal Med，2013，127（2）：459-463. doi：10.1007/s00414-012-0756-9 .
7	LI Y， HUANG Z， DONG X， et al. Forensic age estimation for pelvic X-ray images using deep learning[J]. Eur Radiol，2019，29（5）：2322-2329. doi：10.1007/s00330-018-5791-6 .
8	CHANDRA S S， BRAN LORENZANA M， LIU X， et al. Deep learning in magnetic resonance image reconstruction[J]. J Med Imaging Radiat Oncol，2021，65（5）：564-577. doi：10.1111/1754-9485.13276 .
9	YANG H B， LIU P， HU Y Z， et al. Research on underwater object recognition based on YOLOv3[J]. Microsyst Technol，2021，27（4）：1837-1844. doi：10.1007/s00542-019-04694-8 .
10	ZHOU Y Y， ZHANG J， HUANG J， et al. Digital whole-slide image analysis for automated diatom test in forensic cases of drowning using a convolutional neural network algorithm[J]. Forensic Sci Int，2019，302：109922. doi：10.1016/j.forsciint.2019. 109922 .
11	周圆圆，曹永杰，杨越，等. 基于人工智能硅藻自动化识别系统的实际案例应用[J].法医学杂志，2020，36（2）：239-242. doi：10.12116/j.issn.1004-5619.2020.02.017 .
	ZHOU Y Y， CAO Y J， YANG Y， et al. Application of artificial intelligence automatic diatom identification system in practical cases[J]. Fayixue Zazhi，2020，36（2）：239-242.
12	YU W M， XUE Y， KNOOPS R， et al. Automated diatom searching in the digital scanning electron microscopy images of drowning cases using the deep neural networks[J]. Int J Leg Med，2021，135（2）：497-508. doi：10.1007/s00414-020-02392-z .
13	REDMON J， DIVVALA S， GIRSHICK R， et al. You only look once： Unified， real-time object detection[C]//2016 IEEE Conf Comput Vis Pattern Recognit. Las Vegas： IEEE，2016：779-788. doi：10.1109/CVPR. 2016.91 .
14	REDMON J， FARHADI A. YOLO9000： Better， faster， stronger[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu： IEEE，2017：6517-6525. doi：10.1109/CVPR.2017.690 .
15	REDMON J， FARHADI A. YOLOv3： An incremental improvement[J/OL]. （2018-04-08）[2021-08-25]. arXiv：.
16	邓杰航，何冬冬，卓家鸿，等. 复杂背景干扰下硅藻图像的深度网络识别与定位[J].南方医科大学学报，2020，40（2）：183-189. doi：10.12122/j.issn.1673-4254.2020.02.08 .
	DENG J H， HE D D， ZHUO J H， et al. Deep learning network-based recognition and localization of diatom images against complex background[J]. Nanfang Yike Daxue Xuebao，2020，40（2）：183-189.
17	PEDRAZA A， BUENO G， DENIZ O， et al. Automated diatom classification （Part B）： A deep learning approach[J]. Appl Sci，2017，7（5）：460. doi：10.3390/app7050460 .
18	KLOSTER M， LANGENKÄMPER D， ZUROWIETZ M， et al. Deep learning-based diatom taxonomy on virtual slides[J]. Sci Rep，2020，10：14416. doi：10.1038/s41598-020-71165-w .
19	DENG J H， GUO W Q， ZHAO Y W， et al. Identification of diatom taxonomy by a combination of region-based full convolutional network，online hard example mining，and shape priors of diatoms[J]. Int J Leg Med，2021，135（6）：2519-2530. doi：10.1007/s00414-021-02664-2 .

阈值	验证集			测试集
阈值	RR/%	PR/%	F1分数	RR/%	PR/%	F1分数
0.05	99.2	58.9	0.739	99.5	57.6	0.729
0.15	98.8	79.2	0.879	98.8	76.1	0.859
0.25	98.8	80.5	0.887	98.6	77.4	0.867
0.35	97.9	81.4	0.889	97.2	78.1	0.866
0.45	77.9	95.1	0.856	77.5	92.9	0.845
0.55	73.2	99.3	0.843	73.6	98.6	0.843
0.65	72.4	99.6	0.838	72.9	99.5	0.841
0.75	71.8	99.7	0.835	72.3	99.8	0.839
0.85	70.3	99.8	0.825	70.8	100.0	0.829
0.95	51.4	100.0	0.679	53.3	100.0	0.695

阈值	验证集			测试集
阈值	RR/%	PR/%	F1分数	RR/%	PR/%	F1分数
0.05	99.2	58.9	0.739	99.5	57.6	0.729
0.15	98.8	79.2	0.879	98.8	76.1	0.859
0.25	98.8	80.5	0.887	98.6	77.4	0.867
0.35	97.9	81.4	0.889	97.2	78.1	0.866
0.45	77.9	95.1	0.856	77.5	92.9	0.845
0.55	73.2	99.3	0.843	73.6	98.6	0.843
0.65	72.4	99.6	0.838	72.9	99.5	0.841
0.75	71.8	99.7	0.835	72.3	99.8	0.839
0.85	70.3	99.8	0.825	70.8	100.0	0.829
0.95	51.4	100.0	0.679	53.3	100.0	0.695

组织	阈值	完整硅藻			碎片硅藻			总体
组织	阈值	RR/%	PR/%	F1分数	RR/%	PR/%	F1分数	RR/%	PR/%	F1分数
肺	0.4	94.0±6.7	85.9±27.9	0.862±0.225	75.8±12.9	78.2±29.7	0.732±0.219	89.6±6.6	87.8±25.0	0.865±0.181
	0.6	81.5±16.1	88.4±24.5	0.812±0.175	39.1±15.0	79.6±29.1	0.463±0.162	69.7±11.8	89.8±21.9	0.760±0.130
	0.8	71.5±16.8	90.0±22.8	0.774±0.178	24.5±8.3	80.2±29.0	0.345±0.121	58.4±11.8	91.0±20.8	0.699±0.139
肝	0.4	100.0±0.0	1.1±1.2	0.022±0.024	100.0±0.0	1.7±1.2	0.034±0.023	100.0±0.0	2.5±2.0	0.049±0.039
	0.6	94.4±13.6	1.2±1.1	0.024±0.021	92.7±13.7	1.9±1.4	0.037±0.027	92.3±10.9	2.8±2.2	0.053±0.039
	0.8	66.7±42.2	1.5±1.6	0.029±0.030	76.0±34.3	3.4±2.7	0.064±0.050	75.6±27.7	4.4±3.4	0.082±0.061
肾	0.4	100.0±0.0	1.7±2.0	0.033±0.039	100.0±0.0	2.4±3.2	0.047±0.057	100.0±0.0	3.6±4.6	0.067±0.078
	0.6	100.0±0.0	2.0±1.9	0.039±0.036	91.0±17.6	2.8±2.9	0.053±0.052	93.8±11.8	4.2±4.3	0.077±0.073
	0.8	83.3±40.8	3.6±4.3	0.066±0.077	80.6±22.1	4.7±6.1	0.085±0.099	80.3±26.5	6.8±8.5	0.118±0.129

组织	阈值	完整硅藻			碎片硅藻			总体
组织	阈值	RR/%	PR/%	F1分数	RR/%	PR/%	F1分数	RR/%	PR/%	F1分数
肺	0.4	94.0±6.7	85.9±27.9	0.862±0.225	75.8±12.9	78.2±29.7	0.732±0.219	89.6±6.6	87.8±25.0	0.865±0.181
	0.6	81.5±16.1	88.4±24.5	0.812±0.175	39.1±15.0	79.6±29.1	0.463±0.162	69.7±11.8	89.8±21.9	0.760±0.130
	0.8	71.5±16.8	90.0±22.8	0.774±0.178	24.5±8.3	80.2±29.0	0.345±0.121	58.4±11.8	91.0±20.8	0.699±0.139
肝	0.4	100.0±0.0	1.1±1.2	0.022±0.024	100.0±0.0	1.7±1.2	0.034±0.023	100.0±0.0	2.5±2.0	0.049±0.039
	0.6	94.4±13.6	1.2±1.1	0.024±0.021	92.7±13.7	1.9±1.4	0.037±0.027	92.3±10.9	2.8±2.2	0.053±0.039
	0.8	66.7±42.2	1.5±1.6	0.029±0.030	76.0±34.3	3.4±2.7	0.064±0.050	75.6±27.7	4.4±3.4	0.082±0.061
肾	0.4	100.0±0.0	1.7±2.0	0.033±0.039	100.0±0.0	2.4±3.2	0.047±0.057	100.0±0.0	3.6±4.6	0.067±0.078
	0.6	100.0±0.0	2.0±1.9	0.039±0.036	91.0±17.6	2.8±2.9	0.053±0.052	93.8±11.8	4.2±4.3	0.077±0.073
	0.8	83.3±40.8	3.6±4.3	0.066±0.077	80.6±22.1	4.7±6.1	0.085±0.099	80.3±26.5	6.8±8.5	0.118±0.129

[1]	Yong ZENG, Dong-hua ZOU, Ying FAN, Qing XU, Lu-yang TAO, Yi-jiu CHEN, Zheng-dong LI. Research Progress and Forensic Application of Human Vascular Finite Element Modeling and Biomechanics [J]. Journal of Forensic Medicine, 2023, 39(5): 471-477.
[2]	Yu-xin SUN, Xiao-juan GONG, Xiu-li HAO, Yu-xin TIAN, Yi-ming CHEN, Bao ZHANG, Chun-xia YAN. Screening of Genes Co-Associated with Sudden Infant Death Syndrome and Infectious Sudden Death in Infancy and Bioinformatics Analysis of Their Regulatory Networks [J]. Journal of Forensic Medicine, 2023, 39(5): 433-440.
[3]	Yu YANG, Fan-zhang LEI, Yu-you DONG, Jian-long MA, Qi-qiang SHI, Xue-song YE. Retrospective Analysis of Death Cases of Oral Diphenidol Hydrochloride Poisoning [J]. Journal of Forensic Medicine, 2023, 39(4): 393-398.
[4]	Jie BAI, Jing SUN, Xiao-guang CHENG, Fan LIU, Hua LIU, Xu WANG. Construction and Application of Rib Fracture Diagnosis Model Based on YOLOv3 Algorithm [J]. Journal of Forensic Medicine, 2023, 39(4): 343-349.
[5]	Qing-qing XIANG, Li-fang CHEN, Qin SU, Yu-kun DU, Pei-yan LIANG, Xiao-dong KANG, He SHI, Qu-yi XU, Jian ZHAO, Chao LIU, Xiao-hui CHEN. Research Progress on Microbial Community Succession in the Postmortem Interval Estimation [J]. Journal of Forensic Medicine, 2023, 39(4): 399-405.
[6]	Qin SU, Qian-ling CHEN, Wei-bin WU, Qing-qing XIANG, Cheng-liang YANG, Dong-fang QIAO, Zhi-gang LI. Metabonomics Analysis of Brain Stem Tissue in Rats with Primary Brain Stem Injury Caused Death [J]. Journal of Forensic Medicine, 2023, 39(4): 373-381.
[7]	Xu-dong ZHANG, Yao-ru JIANG, Xin-rui LIANG, Tian TIAN, Qian-qian JIN, Xiao-hong ZHANG, Jie CAO, Qiu-xiang DU, Jun-hong SUN. Postmortem Interval Estimation Using Protein Chip Technology Combined with Multivariate Analysis Methods [J]. Journal of Forensic Medicine, 2023, 39(2): 115-120.
[8]	Yong-gang MA, Yong-jie CAO, Yi-hua ZHAO, Xin-jun ZHOU, Bin HUANG, Gao-chao ZHANG, Ping HUANG, Ya-hui WANG, Kai-jun MA, Feng CHEN, Dong-chuan ZHANG, Ji ZHANG. Sex Estimation of Medial Aspect of the Ischiopubic Ramus in Adults Based on Deep Learning [J]. Journal of Forensic Medicine, 2023, 39(2): 129-136.
[9]	Wu LONG, Peng-fei QU, Lin MA, Rui WANG, Yan-mei XI, Yu-hua LI, Sheng-jie NIE, Ting DUAN, Jin-liang DU, Xue TANG, Jing-feng ZHAO, Pu-ping LEI, Yue-bing WANG. Cytotoxicity of 4 Wild Mushrooms in a Case of Yunnan Sudden Unexplained Death [J]. Journal of Forensic Medicine, 2023, 39(2): 121-128.
[10]	Jun-wei GAO, Yang LU, Yan-jun LI, Dong-hua ZOU, Guang-long HE, Yan-bin WANG. Survey on the Construction Status of Forensic Virtual Autopsy Laboratory and the Applicability of Laboratory Accreditation [J]. Journal of Forensic Medicine, 2023, 39(2): 186-192.
[11]	Fang-Fang LIU, Hui WU, Wei WANG, Ying XIE. Changes of 1，5-AG in Vitreous Humor of Rabbit Cadavers with Hyperglycemic Metabolism [J]. Journal of Forensic Medicine, 2023, 39(1): 13-17.
[12]	Zhi-ling TIAN, Ruo-lin WANG, Jian-hua ZHANG, Ping HUANG, Zhi-qiang QIN, Zheng-dong LI, He-wen DONG, Dong-hua ZOU, Mao-wen WANG, Zhuo LI, Lei WAN, Xiao-tian YU, Ning-guo LIU. Comparison of CT Values between Thrombus and Postmortem Clot Based on Cadaveric Pulmonary Angiography [J]. Journal of Forensic Medicine, 2023, 39(1): 7-12.
[13]	Hong-xia HAO, Ya-hui WANG, Zhi-lu ZHOU, Tai-ang LIU, Jin CHEN, Yu-heng HE, Lei WAN, Wen-tao XIA. Research Progress of Age Estimation in the Living by Knee Joint MRI [J]. Journal of Forensic Medicine, 2023, 39(1): 66-71.
[14]	Jia-yi WU, You-jia YU, Kai LI, Xin YIN, Han-ting FAN, Rong LI, Zhi-wei ZHANG, Wei TANG, Hui-jie HUANG, Feng CHEN. Report on Cardiac Gross Pathologic Measurements of Sudden Cardiac Death in Adults [J]. Journal of Forensic Medicine, 2023, 39(1): 1-6.
[15]	Hui WU, Fang-fang LIU, Jun-da WU, Ying XIE. Research Progress on Estimation of Postmortem Interval Based on Ocular Tissues Structure [J]. Journal of Forensic Medicine, 2023, 39(1): 50-56.