Abstract

A bi-level analysis of computed tomography (CT) images of malignant pleural mesothelioma (MPM) is presented in this paper, starting with a deep learning-based system for classification, followed by a three-dimensional (3D) reconstruction method. MPM is a highly aggressive cancer caused by asbestos exposure, and accurate diagnosis and determination of the tumor’s volume are crucial for effective treatment. The proposed system employs a bi-level approach, utilizing machine learning and deep learning techniques to classify CT lung images and subsequently calculate the tumor’s volume. The study addresses challenges related to deep neural networks, such as the requirement for large and diverse datasets, hyperparameter optimization, and potential data bias. To evaluate performance, two convolutional neural network (CNN) architectures, Inception-v3 and ResNet-50, were compared in terms of their features and performance. Based on CT images, the second stage incorporates 3D volume reconstruction. The process is carried out by cropping, registering, filtering, and segmenting images. This study demonstrated the efficacy of the developed system by combining CNN optimizations with 3D image reconstruction. It is intended to improve the accuracy of MPM diagnosis and to assist in the determination of chemotherapy doses, both of which may result in improved outcomes for patients.

Graphical Abstract Figure
Graphical Abstract Figure
Close modal

References

1.
Boutin
,
C.
,
Schlesser
,
M.
,
Frenay
,
C.
, and
Astoul
,
P.
,
1998
, “
Malignant Pleural Mesothelioma
,”
Eur. Respir. J.
,
12
(
4
), pp.
972
981
.
2.
Zauderer
,
M. G.
,
Martin
,
A.
,
Egger
,
J.
,
Rizvi
,
H.
,
Offin
,
M.
,
Rimner
,
A.
,
Adusumilli
,
P. S.
, et al.,
2021
, “
The Use of a Next-Generation Sequencing-Derived Machine-Learning Risk-Prediction Model (OncoCast-MPM) for Malignant Pleural Mesothelioma: A Retrospective Study
,”
Lancet Digital Health
,
3
(
9
), pp.
e565
e576
.
3.
Schillebeeckx
,
E.
,
van Meerbeeck
,
J. P.
, and
Lamote
,
K.
,
2021
, “
Clinical Utility of Diagnostic Biomarkers in Malignant Pleural Mesothelioma: A Systematic Review and Meta-Analysis
,”
Eur. Respir. Rev.
,
30
(
162
), p.
210057
.
4.
Lim
,
J. H.
,
Choi
,
J. Y.
,
Im
,
Y.
,
Yoo
,
H.
,
Jhun
,
B. W.
,
Jeong
,
B. -H.
,
Park
,
H. Y.
, et al.,
2020
, “
Prognostic Value of SUVmax on 18F-Fluorodeoxyglucose PET/CT Scan in Patients With Malignant Pleural Mesothelioma
,”
PLoS One
,
15
(
2
), p.
e0229299
.
5.
Bianco
,
A.
,
Valente
,
T.
,
De Rimini
,
M. L.
,
Sica
,
G.
, and
Fiorelli
,
A.
,
2018
, “
Clinical Diagnosis of Malignant Pleural Mesothelioma
,”
J. Thoracic Dis.
,
10
(
Suppl. 2
), p.
S253
.
6.
Mark
,
M.
,
Rusakiewicz
,
S.
,
Früh
,
M.
,
Hayoz
,
S.
,
Grosso
,
F.
,
Pless
,
M.
,
Zucali
,
P.
, et al.,
2022
, “
Long-Term Benefit of Lurbinectedin as Palliative Chemotherapy in Progressive Malignant Pleural Mesothelioma (MPM): Final Efficacy and Translational Data of the SAKK 17/16 Study
,”
ESMO Open
,
7
(
3
), p.
100446
.
7.
Liu
,
F.
,
Zhao
,
B.
,
Krug
,
L. M.
,
Ishill
,
N. M.
,
Lim
,
R. C.
,
Guo
,
P.
,
Gorski
,
M.
, et al.,
2010
, “
Assessment of Therapy Responses and Prediction of Survival in Malignant Pleural Mesothelioma Through Computer-Aided Volumetric Measurement on Computed Tomography Scans
,”
J. Thoracic Oncol.
,
5
(
6
), pp.
879
884
.
8.
Labby
,
Z.
,
Nowak
,
A.
,
Dignam
,
J.
,
Straus
,
C.
,
Kindler
,
H.
, and
Armato III
,
S.
,
2013
, “
Disease Volumes as a Marker for Patient Response in Malignant Pleural Mesothelioma
,”
Ann. Oncol.
,
24
(
4
), pp.
999
1005
.
9.
Frauenfelder
,
T.
,
Tutic
,
M.
,
Weder
,
W.
,
Götti
,
R.
,
Stahel
,
R.
,
Seifert
,
B.
, and
Opitz
,
I.
,
2011
, “
Volumetry: An Alternative to Assess Therapy Response for Malignant Pleural Mesothelioma
?”
Eur. Respir. J.
,
38
(
1
), pp.
162
168
.
10.
Sensakovic
,
W. F.
,
Armato III
,
S. G.
,
Straus
,
C.
,
Roberts
,
R. Y.
,
Caligiuri
,
P.
,
Starkey
,
A.
, and
Kindler
,
H. L.
,
2011
, “
Computerized Segmentation and Measurement of Malignant Pleural Mesothelioma
,”
Med. Phys.
,
38
(
1
), pp.
238
244
.
11.
Chen
,
M.
,
Helm
,
E.
,
Joshi
,
N.
,
Gleeson
,
F.
, and
Brady
,
M.
,
2017
, “
Computer-Aided Volumetric Assessment of Malignant Pleural Mesothelioma on CT Using a Random Walk-Based Method
,”
Int. J. Comput. Assist. Radiol. Surg.
,
12
(
4
), pp.
529
538
.
12.
Brahim
,
W.
,
Mestiri
,
M.
,
Betrouni
,
N.
, and
Hamrouni
,
K.
,
2018
, “
Malignant Pleural Mesothelioma Segmentation for Photodynamic Therapy Planning
,”
Comput. Med. Imaging Graph.
,
65
, pp.
79
92
.
13.
Anderson
,
O.
,
Kidd
,
A. C.
,
Goatman
,
K. A.
,
Weir
,
A. J.
,
Voisey
,
J.
,
Dilys
,
V.
,
Siebert
,
J. P.
, and
Blyth
,
K. G.
,
2020
, “
Fully Automated Volumetric Measurement of Malignant Pleural Mesothelioma From Computed Tomography Images by Deep Learning: Preliminary Results of an Internal Validation
,”
Bioimaging
,
2
, pp.
64
73
.
14.
Hu
,
X.
, and
Yu
,
Z.
,
2019
, “
Diagnosis of Mesothelioma With Deep Learning
,”
Oncol. Lett.
,
17
(
2
), pp.
1483
1490
.
15.
Gudmundsson
,
E.
,
Straus
,
C. M.
, and
Armato III
,
S. G.
,
2018
, “
Deep Convolutional Neural Networks for the Automated Segmentation of Malignant Pleural Mesothelioma on Computed Tomography Scans
,”
J. Med. Imaging
,
5
(
3
), p.
034503
.
16.
Gudmundsson
,
E.
,
Straus
,
C. M.
,
Li
,
F.
, and
Armato III
,
S. G.
,
2020
, “
Deep Learning-Based Segmentation of Malignant Pleural Mesothelioma Tumor on Computed Tomography Scans: Application to Scans Demonstrating Pleural Effusion
,”
J. Med. Imaging
,
7
(
1
), p.
012705
.
17.
Hassan
,
H.
,
Ren
,
Z.
,
Zhao
,
H.
,
Huang
,
S.
,
Li
,
D.
,
Xiang
,
S.
,
Kang
,
Y.
,
Chen
,
S.
, and
Huang
,
B.
,
2022
, “
Review and Classification of AI-Enabled COVID-19 CT Imaging Models Based on Computer Vision Tasks
,”
Comput. Biol. Med.
,
141
, pp.
105
123
.
18.
Khan
,
A.
,
Sohail
,
A.
,
Zahoora
,
U.
, and
Qureshi
,
A. S.
,
2020
, “
A Survey of the Recent Architectures of Deep Convolutional Neural Networks
,”
Artif. Intell. Rev.
,
53
(
8
), pp.
5455
5516
.
19.
Bhatt
,
P. M.
,
Malhan
,
R. K.
,
Rajendran
,
P.
,
Shah
,
B. C.
,
Thakar
,
S.
,
Yoon
,
Y. J.
, and
Gupta
,
S. K.
,
2021
, “
Image-Based Surface Defect Detection Using Deep Learning: A Review
,”
ASME J. Comput. Inf. Sci. Eng.
,
21
(
4
), p.
040801
.
20.
Murugesan
,
M.
,
Kaliannan
,
K.
,
Balraj
,
S.
,
Singaram
,
K.
,
Kaliannan
,
T.
, and
Albert
,
J. R.
,
2022
, “
A Hybrid Deep Learning Model for Effective Segmentation and Classification of Lung Nodules From CT Images
,”
J. Intell. Fuzzy Syst.
,
42
(
3
), pp.
2667
2679
.
21.
Nguyen
,
P. C.
,
Choi
,
J. B.
,
Udaykumar
,
H.
, and
Baek
,
S.
,
2023
, “
Challenges and Opportunities for Machine Learning in Multiscale Computational Modeling
,”
ASME J. Comput. Inf. Sci. Eng.
,
23
(
6
), p.
060808
.
22.
Szegedy
,
C.
,
Vanhoucke
,
V.
,
Ioffe
,
S.
,
Shlens
,
J.
, and
Wojna
,
Z.
,
2016
, “
Rethinking the Inception Architecture for Computer Vision
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Las Vegas, NV
,
June 27–30
, pp.
2818
2826
.
23.
Alzubaidi
,
L.
,
Zhang
,
J.
,
Humaidi
,
A. J.
,
Al-Dujaili
,
A.
,
Duan
,
Y.
,
Al-Shamma
,
O.
,
Santamaría
,
J.
,
Fadhel
,
M. A.
,
Al-Amidie
,
M.
, and
Farhan
,
L.
,
2021
, “
Review of Deep Learning: Concepts, CNN Architectures, Challenges, Applications, Future Directions
,”
J. Big Data
,
8
(
53
), pp.
1
74
.
24.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
, and
Sun
,
J.
,
2015
, “
Delving Deep Into Rectifiers: Surpassing Human-Level Performance on Imagenet Classification
,”
Proceedings of the IEEE International Conference on Computer Vision
,
Santiago, Chile
,
Dec. 7–13
, pp.
1026
1034
.
25.
Wu
,
Z.
,
Shen
,
C.
, and
Van Den Hengel
,
A.
,
2019
, “
Wider Or Deeper: Revisiting the Resnet Model for Visual Recognition
,”
Pattern Recognit.
,
90
, pp.
119
133
.
26.
Zhuang
,
F.
,
Qi
,
Z.
,
Duan
,
K.
,
Xi
,
D.
,
Zhu
,
Y.
,
Zhu
,
H.
,
Xiong
,
H.
, and
He
,
Q.
,
2020
, “
A Comprehensive Survey on Transfer Learning
,”
Proc. IEEE
,
109
(
1
), pp.
43
76
.
27.
Morid
,
M. A.
,
Borjali
,
A.
, and
Del Fiol
,
G.
,
2021
, “
A Scoping Review of Transfer Learning Research on Medical Image Analysis Using ImageNet
,”
Comput. Biol. Med.
,
128
, p.
104115
.
28.
Takase
,
T.
,
Oyama
,
S.
, and
Kurihara
,
M.
,
2018
, “
Effective Neural Network Training With Adaptive Learning Rate Based on Training Loss
,”
Neural Netw.
,
101
, pp.
68
78
.
29.
Rosenwasser
,
T.
,
Lain
,
R.
, and
Cohen
,
M. W.
,
2022
, “
Aortic Enlargement Detection Using Chest X-Rays to Identify Potential Marfan Syndrome
,”
Procedia Comput. Sci.
,
207
, pp.
2125
2133
.
30.
Ghidotti
,
A.
,
Landi
,
D.
,
Regazzoni
,
D.
, and
Rizzi
,
C.
,
2022
, “
Evaluation of Clinical and Technical Parameters to Customize Total Knee Arthroplasty Implants
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
6
), p.
061006
.
31.
Lesniak
,
K.
,
Terpenny
,
J.
,
Tucker
,
C. S.
,
Anumba
,
C.
, and
Bilén
,
S. G.
,
2017
, “
Immersive Distributed Design Through Real-Time Capture, Translation, and Rendering of Three-Dimensional Mesh Data
,”
J. Comput. Inf. Sci. Eng.
,
17
(
3
), p.
031010
.
32.
Simmons-Ehrhardt
,
T.
,
2021
, “
Open Osteology: Medical Imaging Databases as Skeletal Collections
,”
Forensic Imaging
,
26
, p.
200462
.
33.
Zhang
,
R.
,
Zhou
,
W.
,
Li
,
Y.
,
Yu
,
S.
, and
Xie
,
Y.
,
2013
, “
Nonrigid Registration of Lung CT Images Based on Tissue Features
,”
Comput. Math. Methods Med.
,
2013
, p.
834192
.
34.
Marinelli
,
M.
,
Martinez-Möller
,
A.
,
Jensen
,
B.
,
Positano
,
V.
,
Weismüller
,
S.
,
Navab
,
N.
,
Landini
,
L.
,
Schwaiger
,
M.
, and
Nekolla
,
S. G.
,
2010
, “
Registration of Myocardial PET and SPECT for Viability Assessment Using Mutual Information
,”
Med. Phys.
,
37
(
6
), pp.
2414
2424
.
35.
Gudmundsson
,
E.
,
Labby
,
Z.
,
Straus
,
C. M.
,
Sensakovic
,
W. F.
,
Li
,
F.
,
Rose
,
B.
,
Cunliffe
,
A.
,
Kindler
,
H. L.
, and
Armato
,
S. G.
,
2019
, “
Dynamic Contrast-Enhanced CT for the Assessment of Tumour Response in Malignant Pleural Mesothelioma: A Pilot Study
,”
Eur. Radiol.
,
29
, pp.
682
688
.
36.
Soille
,
P.
,
2004
, “Geodesic Transformations,”
Morphological Image Analysis: Principles and Applications
,
Springer
,
Berlin
, Vol. 2, No. 3, pp.
170
171
.
37.
Atapour-Abarghouei
,
A.
, and
Breckon
,
T. P.
,
2018
, “
A Comparative Review of Plausible Hole Filling Strategies in the Context of Scene Depth Image Completion
,”
Comput. Graph.
,
72
, pp.
39
58
.
38.
Banik
,
S.
,
Rangayyan
,
R. M.
, and
Boag
,
G. S.
,
2009
, “
Landmarking and Segmentation of 3D CT Images
,”
Synthesis Lectures Biomed. Eng.
,
4
(
1
), pp.
1
170
.
You do not currently have access to this content.