Performance Evaluation Of Optimizers On Fine-Tuned VGG-19 Model For Thyroid Nodules Classification
Main Article Content
Abstract
The task of thyroid nodules classification involves the intricate analysis of patterns and features present in ultrasound images. Convolutional Neural Networks (CNNs) are being harnessed for thyroid malignancy detection due to their exceptional capability to process complex and high-dimensional medical imaging data effectively. This study introduces a meticulously fine-tuned VGG-19 CNN model, designed to cater specifically to the multi-classification of thyroid nodules within pre-processed ultrasound (US) images. Additionally, the model's efficacy is evaluated across a spectrum of optimization techniques. Experimental results underscore the model's effectiveness, showcasing accuracy rates of 0.6562, 0.8094, 0.8294, and 0.9201 when employing SGD, ADAgrad, RMSprop, and ADAM optimizers, respectively, spanning 150 epochs. Importantly, the ADAM optimizer emerges as the key contributor to the optimal testing loss, signifying its crucial role in refining the model's performance.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Nguyen, Q. T., Lee, E. J., Huang, M. G., Park, Y. I., Khullar, A., & Plodkowski, R. A. (2015). Diagnosis and treatment of patients with thyroid cancer. American health & drug benefits, 8(1), 30–40.
Cancer of the Thyroid - Cancer Stat Facts. (n.d.). SEER. https://seer.cancer.gov/statfacts/html/thyro.html
Thyroid cancer - Symptoms and causes. (2023, April 26). Mayo Clinic.
https://www.mayoclinic.org/diseases-conditions/thyroid-cancer/symptoms-causes/syc-20354161
Kitahara, C. M., Kӧrmendiné Farkas, D., Jørgensen, J. O. L., Cronin-Fenton, D., & Sørensen, H. T. (2018, March 23). Benign Thyroid Diseases and Risk of Thyroid Cancer: A Nationwide Cohort Study. The Journal of Clinical Endocrinology & Metabolism, 103(6), 2216–2224.
https://doi.org/10.1210/jc.2017-02599
Anari, S., Tataei Sarshar, N., Mahjoori, N., Dorosti, S., & Rezaie, A. (2022, August 25). Review of Deep Learning Approaches for Thyroid Cancer Diagnosis. Mathematical Problems in Engineering, 2022, 1–8. https://doi.org/10.1155/2022/5052435
Zhang, X., Lee, V. C. S., Rong, J., Liu, F., & Kong, H. (2022). Multi-channel convolutional neural network architectures for thyroid cancer detection. PloS one, 17(1), e0262128.
https://doi.org/10.1371/journal.pone.0262128
Zhang, Q., Zhang, M., Chen, T., Sun, Z., Ma, Y., & Yu, B. (2019, January). Recent advances in convolutional neural network acceleration. Neurocomputing, 323, 37–51.
https://doi.org/10.1016/j.neucom.2018.09.038
Dogo, E. M., Afolabi, O. J., & Twala, B. (2022, November 23). On the Relative Impact of Optimizers on Convolutional Neural Networks with Varying Depth and Width for Image Classification. Applied Sciences, 12(23), 11976. https://doi.org/10.3390/app122311976
Liang, X., Yu, J., Liao, J., & Chen, Z. (2020, January 10). Convolutional Neural Network for Breast and Thyroid Nodules Diagnosis in Ultrasound Imaging. BioMed Research International, 2020, 1–9. https://doi.org/10.1155/2020/1763803
Wang, Y., Yue, W., Li, X., Liu, S., Guo, L., Xu, H., Zhang, H., & Yang, G. (2020). Comparison Study of Radiomics and Deep Learning-Based Methods for Thyroid Nodules Classification Using Ultrasound Images. IEEE Access, 8, 52010–52017. https://doi.org/10.1109/access.2020.2980290
Xie, J., Guo, L., Zhao, C., Li, X., Luo, Y., & Jianwei, L. (2020, December 1). A Hybrid Deep Learning and Handcrafted Features based Approach for Thyroid Nodule Classification in Ultrasound Images. Journal of Physics: Conference Series, 1693(1), 012160. https://doi.org/10.1088/1742-6596/1693/1/012160
Liu, Z., Zhong, S., Liu, Q., Xie, C., Dai, Y., Peng, C., Chen, X., & Zou, R. (2021, January 6). Thyroid nodule recognition using a joint convolutional neural network with information fusion of ultrasound images and radiofrequency data. European Radiology, 31(7), 5001–5011. https://doi.org/10.1007/s00330-020-07585-z
Vadhiraj, V. V., Simpkin, A., O’Connell, J., Singh Ospina, N., Maraka, S., & O’Keeffe, D. T. (2021, May 24). Ultrasound Image Classification of Thyroid Nodules Using Machine Learning Techniques. Medicina, 57(6), 527. https://doi.org/10.3390/medicina57060527
Li, W., Cheng, S., Qian, K., Yue, K., & Liu, H. (2021, May 27). Automatic Recognition and Classification System of Thyroid Nodules in CT Images Based on CNN . Computational Intelligence and Neuroscience, 2021, 1–11. https://doi.org/10.1155/2021/5540186
Hang, Y. (2021, July 22). Thyroid Nodule Classification in Ultrasound Images by Fusion of Conventional Features and Res-GAN Deep Features. Journal of Healthcare Engineering, 2021, 1–7. https://doi.org/10.1155/2021/9917538
Peng, S., Liu, Y., Lv, W., Liu, L., Zhou, Q., Yang, H., Ren, J., Liu, G., Wang, X., Zhang, X., Du, Q., Nie, F., Huang, G., Guo, Y., Li, J., Liang, J., Hu, H., Xiao, H., Liu, Z., . . . Xiao, H. (2021, April). Deep learning-based artificial intelligence model to assist thyroid nodule diagnosis and management: a multicentre diagnostic study. The Lancet Digital Health, 3(4), e250–e259. https://doi.org/10.1016/s2589-7500(21)00041-8
Qi, Q., Huang, X., Zhang, Y., Cai, S., Liu, Z., Qiu, T., Cui, Z., Zhou, A., Yuan, X., Zhu, W., Min, X., Wu, Y., Wang, W., Zhang, C., & Xu, P. (2023, April). Ultrasound image-based deep learning to assist in diagnosing gross extrathyroidal extension thyroid cancer: a retrospective multicenter study. EClinicalMedicine, 60, 102007. https://doi.org/10.1016/j.eclinm.2023.102007
Liu, Y., Lai, F., Lin, B., Gu, Y., Chen, L., Chen, G., Xiao, H., Luo, S., Pang, Y., Xiong, D., Li, B., Peng, S., Lv, W., Alexander, E. K., & Xiao, H. (2023, June). Deep learning to predict cervical lymph node metastasis from intraoperative frozen section of tumour in papillary thyroid carcinoma: a multicentre diagnostic study. E Clinical Medicine, 60, 102007. https://doi.org/10.1016/j.eclinm.2023.102007
Ajilisa, O. A., Jagathy Raj, V. P., & Sabu, M. K. (2023, July 24). A Deep Learning Framework for the Characterization of Thyroid Nodules from Ultrasound Images Using Improved Inception Network and Multi-Level Transfer Learning. Diagnostics, 13(14), 2463. https://doi.org/10.3390/diagnostics13142463
Yang, Z., Yu, Y., You, C., Steinhardt, J., & Ma, Y. (2020, February 26). Rethinking Bias-Variance Trade-off for Generalization of Neural Networks. arXiv.org. https://arxiv.org/abs/2002.11328v3
Keskar, N. S., & Socher, R. (2017, December 20). Improving Generalization Performance by Switching from Adam to SGD. arXiv.org. https://arxiv.org/abs/1712.07628v1
Choi, D., Shallue, C. J., Nado, Z., Lee, J., Maddison, C. J., & Dahl, G. E. (2019, October 11). On Empirical Comparisons of Optimizers for Deep Learning. arXiv.org. https://arxiv.org/abs/1910.05446v3
Nado, Z., Gilmer, J. M., Shallue, C. J., Anil, R., & Dahl, G. E. (2021, February 12). A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes. arXiv.org. https://arxiv.org/abs/2102.06356v3
Pedraza, L., Vargas, C., Narváez, F., Durán, O., Muñoz, E., & Romero, E. (2015, January 28). An open access thyroid ultrasound image database. SPIE Proceedings. https://doi.org/10.1117/12.2073532
Song, R., Zhang, L., Zhu, C., Liu, J., Yang, J., & Zhang, T. (2020). Thyroid Nodule Ultrasound Image Classification Through Hybrid Feature Cropping Network. IEEE Access, 8, 64064–64074. https://doi.org/10.1109/access.2020.2982767
Simonyan, K., & Zisserman, A. (2014, September 4). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.org. https://arxiv.org/abs/1409.1556v6
Bansal, M., Kumar, M., Sachdeva, M., & Mittal, A. (2021, September 17). Transfer learning for image classification using VGG19: Caltech-101 image data set. Journal of Ambient Intelligence and Humanized Computing, 14(4), 3609–3620. https://doi.org/10.1007/s12652-021-03488-z
Ruder, S. (2016, September 15). An overview of gradient descent optimization algorithms. arXiv.org. https://arxiv.org/abs/1609.04747v2
Duchi, J., Hazan, E., & Singer, Y. (2011). Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research, 12(7).
Dogo, E. M., Afolabi, O. J., & Twala, B. (2022, November 23). On the Relative Impact of Optimizers on Convolutional Neural Networks with Varying Depth and Width for Image Classification. Applied Sciences, 12(23), 11976. https://doi.org/10.3390/app122311976
Kingma, D. P., & Ba, J. (2014, December 22). Adam: A Method for Stochastic Optimization. arXiv.org. https://arxiv.org/abs/1412.6980v9