Clinical evaluation of semi-automatic landmark-based lesion tracking software for CT-scans
© Dankerl et al.; licensee BioMed Central Ltd. 2014
Received: 1 November 2013
Accepted: 9 January 2014
Published: 22 April 2014
To evaluate a semi-automatic landmark-based lesion tracking software enabling navigation between RECIST lesions in baseline and follow-up CT-scans.
The software automatically detects 44 stable anatomical landmarks in each thoraco/abdominal/pelvic CT-scan, sets up a patient specific coordinate-system and cross-links the coordinate-systems of consecutive CT-scans. Accuracy of the software was evaluated on 96 RECIST lesions (target- and non-target lesions) in baseline and follow-up CT-scans of 32 oncologic patients (64 CT-scans). Patients had to present at least one thoracic, one abdominal and one pelvic RECIST lesion. Three radiologists determined the deviation between lesions’ centre and the software’s navigation result in consensus.
The initial mean runtime of the system to synchronize baseline and follow-up examinations was 19.4 ± 1.2 seconds, with subsequent navigation to corresponding RECIST lesions facilitating in real-time. Mean vector length of the deviations between lesions’ centre and the semi-automatic navigation result was 10.2 ± 5.1 mm without a substantial systematic error in any direction. Mean deviation in the cranio-caudal dimension was 5.4 ± 4.0 mm, in the lateral dimension 5.2 ± 3.9 mm and in the ventro-dorsal dimension 5.3 ± 4.0 mm.
The investigated software accurately and reliably navigates between lesions in consecutive CT-scans in real-time, potentially accelerating and facilitating cancer staging.
Radiological staging of cancer patients, especially the time consuming comparison of RECIST (Response Evaluation Criteria in Solid Tumors) lesions in baseline and follow-up CT-scans became a major part of daily radiological routine.
In the era of digital PACS (Picture Achieving And Communication System) and state of the art medical imaging with multidetector computed tomography the amount of image data markedly increased. Several authors addressed the ever increasing image overload, the workflow difficulties of handling huge data volumes and the time-consuming radiological reading and interpretation process [1, 2]. Unified standards such as the RECIST-criteria  for reading and reporting CTs of oncological patients aim to improve patient care and treatment. However, RECIST-based reporting is known to have the inherent disadvantage of being time-consuming and results show low inter-reader comparability .
Previous research that engaged in solving radiological image overload has mainly been directed at improving radiologist’s user interface devices, which enable better navigation in digital data. They conclude that hardware other than the standard mouse, such as a 3D mouse, joystick or infrared controller similar to the Nintendo Wii controller can improve the workflow and optimize image reviewing [5–7]. However, few research has been directed at improving radiological reading software. Modern software could assist radiologists in fulfilling their task of quickly, reliably and comfortably browsing through patient examinations and comparing relevant findings in multiple examinations. Existing radiological software such as in current PACS or 3D image analysis systems feature image synchronisation by picture number, by table position information or by manually assigning corresponding images. These means do not offer a solution as to where in a certain image to look for a finding. The software investigated in this research, is a lesion tracking device, which automatically navigates to lesions in the follow-up CT-examination in real time. The software, however, does not indicate if navigated to lesions change their size in the follow-up examination, instead localizes their anatomical position while leaving the diagnosis up to the radiologist.
The aim of this study was to evaluate the feasibility, accuracy and reliability of an automatic landmark-based lesion tracking software enabling navigation between RECIST-lesions in baseline and follow-up CT-scans.
Institutional Review board approval was obtained for this study and all procedures were in accordance with the Helsinki Declaration.
Patient selection and scan technique
We searched the Radiological Information System (RIS) for consecutive oncologic patients that presented at least one lesion in the thorax, one lesion in the abdomen and one lesion in the pelvis and that received at least two computed tomography (CT)-scans within 6 months. 32 patients (26x lymphoma, 4x melanoma, 2x prostate cancer; 21 males 11 females; mean age 51.2 ± 15.3 years, range 21 to 76) were enrolled for experimental analysis. Two consecutive CT-scans were read according to the RECIST guidelines.
CT-examinations were indicated by clinical needs in all cases. Patients were examined with a 64-row multidetector CT-scanner (Somatom 64, Siemens AG, Erlangen, Germany). The tube voltage was 120 kV, the pitch was 0.75 and the collimation was 0.6 mm. Further, the mean CTDIvol value in investigated scans was 11.2 ± 2.4 (range 6.7 – 16.3). A weight-dependent dose of 350 mg per ml iodine i.v. contrast agent (Imeron 350®, Bracco Imaging, Konstanz, Germany) was administered to all patients with a flow-rate of 3 ml/sec.. Images were acquired in the portal-venous contrast agent phase. For every patient a data-set with 1 mm slice thickness and a soft tissue kernel was calculated. If clinically indicated contrast agent was administered orally or rectally in accordance with guidelines (1.5 liters of water containing 60 ml of Gastrografin®, Bayer, Leverkusen, Germany). All CT-scans were performed in the Department of Radiology of the University Hospital Erlangen.
Lesion tracking software
The software was developed within the scope of the Theseus-Medico research project. This is a 5-year non-commercial, nationwide multi-centre research project started in 2007, with physicians, university health care professionals, technicians and computer scientists as joint venture.
Experimental setup and evaluation
To determine the accuracy of the lesion tracking software we evaluated the deviation between the centre of each lesion (reference standard) and the location the automated lesion tracking software navigated to in the follow-up examination. The centre of 96 RECIST target and non-target lesions (such as metastasized lymph nodes or metastases of the lung or parenchymal abdominal organs) was marked in every baseline and follow-up CT-scan. In a consensus assessment of three readers (with 16, 4 and 3 years of work experience) the centre of each lesion and the deviation to the navigation result of the software was documented (Figure 2). To increase the power and validity of our study we investigated various individuals and independent lesions in different body regions with continuous instead of categorized measurement ).
The lesion tracking software was evaluated on a standard computer (Dual Core Xeon 2.66 GHz; Windows XP, 32 Bit) and dedicated diagnostic displays (Coronis Fusion 6MP LED, Barco, Kortrijk, Belgium).
System’s runtime was computed and mean runtime ± standard deviation was calculated.
Mean deviation ± standard deviation was calculated for every spatial direction (cranial, caudal, ventral, dorsal, lateral-left and lateral-right). To investigate a possible systematic error differences between mean deviations were calculated. Furthermore the unsigned mean deviation ± standard deviation for each of the three spatial dimensions (cranio-caudal, ventro-dorsal and lateral) was calculated and therefrom mean vector length ± standard deviation of deviations between lesions’ centre and the semi-automatic navigation result was calculated.
The mean runtime the lesion tracking software required to initially synchronize baseline and follow-up examinations was 19.4 ± 1.2 seconds. Thereafter, the navigation between corresponding RECIST lesions in baseline and follow-up examinations was performed in real-time.
Mean runtime (± standard deviation) the lesion tracking software required to initially synchronize baseline and follow-up CT-scans is shown in seconds
Software’s initial runtime (sec)
19.4 ± 1.2
5.1 ± 4.1 (1.0 – 14.7)
5.2 ± 3.7 (0.7 – 14.2)
5.5 ± 4.2 (0.8 – 14.3)
5.1 ± 3.8 (0.7 – 13.8)
4.7 ± 3.4 (0.8 – 12.8)
5.9 ± 4.3 (1.2 – 11.9)
5.2 ± 3.9 (0.7 – 14.7)
5.3 ± 4.0 (0.7 – 14.3)
5.4 ± 4.0 (0.8 – 12.8)
Mean length of the deviation vector (mm)
10.2 ± 5.1 (2.4 – 21.9)
The calculated unsigned mean deviation was 5.4 ± 4.0 mm (range 0.8 – 12.8) in the cranio-caudal dimension, 5.2 ± 3.9 mm (range 0.7 – 14.7) in the lateral dimension and 5.3 ± 4.0 mm (range 0.7 – 14.3) in the ventro-dorsal dimension. This resulted in a mean length of the deviation vector of 10.2 ± 5.1 mm (range 2.4 – 21.9).
The investigated lesion tracking software demonstrated a deviation vector of roughly 1 cm length on average, when semi-automatically navigating from lesions in the baseline CT-examinations to the corresponding locations in follow-up scans. The average deviation vector composes of divergences in all three spatial directions. It is the three-dimensional distance between the location the software automatically navigated to in the follow-up scan and the reference standard. This does not correspond to an absolute two-dimensional deviation (e.g. in the axial plain) of 10 mm – in fact the average two-dimensional deviation error (deviations in each direction) in our study was never greater than 5.9 mm. We hold this deviation to be clinically tolerable as lesions which are to be classified as RECIST-target lesions (which have to be compared in baseline and follow-up CT-scans) have to be of a certain size. Lung nodules or liver lesions have to be 10 mm in largest diameter, while lymph nodes even have to measure 15 mm in the short axis (RECIST 1.1 ). As no single of our spatial deviations was greater than 5.9 mm, we conclude that navigation to target lesions with the presented software is very accurate and of clinically tolerable error.
The semi-automatic navigation between corresponding lesions in consecutive CT-examinations presumably saves time during tumor response evaluation. Moreover, only few navigational results demonstrated larger (of about 15 mm) deviation errors. In these cases, the lesions were located in or close to the abdominal wall and thereby relatively far away from the majority of software detected landmarks (mostly located in the spine and the pelvis). This deviation error can even be reduced when more landmarks are implemented into the system. Another possibility for deviational errors might be an uncertain reference point of the lesions in baseline and follow-up due to changes in lesion dimension after chemotherapy. In order to minimize this error, 3 radiological readers determined the reference points in consensus. If certain landmarks cannot be detected (e.g. in osteochondrosis between lumbar vertebra 5 and sacral vertebra 1, or scoliosis) within a certain computational time this landmark is ignored by the algorithm and the graphical network is constructed with the remaining landmarks. Thereby the software system remains stable and functioning even if it cannot detect all landmarks. In our patients the software did not fail to detect landmarks.
To investigate the clinical feasibility of this software, we examined the software utilizing consecutive routine clinical CT-scans, as requested by van Ginneken . The only inclusion criterion was that patients show at least one lesion in the thorax, in the abdomen and in the pelvis to prove accuracy of the software in different body regions. The software apprehended all CT-examinations and no scan was excluded due to software inability. Therefore, the presented lesion tracking software proofed to be feasible and reliable in clinical routine CT-scans.
After the initial synchronization of the data sets, which takes about 20 seconds and can be performed as a preprocessing step, subsequent navigation between lesions in different CT-scans is realized in real time. Therefore, the software facilitates handling of large data sets without delay. Andriole et al. address the increasing image overload and the thereby resulting increased workload for radiologists [1, 2]. However they do not offer a solution from a software standpoint. Addressing this issue, usage of a robust lesion tracking tool is an important point.
The presented software tool is unique in various ways. It automatically detects stable anatomical landmarks throughout the body and sets up a patient specific graphical network between the landmarks, comparable to the global positioning system (GPS) navigation. Beside navigation in matching CT-scans, GPS-like navigation potentially enables comparing findings in non-matching body regions (e.g. comparing a thoracic with a thoraco/abdominal CT-scan). At present, commercial tools by various vendors, for instance for the follow-up of lung lesions are available. However, these tools mainly rely on deforming of images based upon Hounsfield Unit (HU) similarities. According to Seifert et al.  this strategy has the drawback that changes in organ size, e.g. splenomegaly, or topographical changes like nephrectomy cannot be apprehended. Moreover, synchronous navigation to findings in organs of non-matching body areas is a big obstacle for such software . As an example a liver lesion included in a baseline thoracic/upper abdomen CT-scan and a follow-up abdominal/pelvic CT-scan cannot be synchronized by HU similarities or table position. Presented landmark based navigation software possibly overcomes these limitations due to its underlying algorithm .
Another approach for navigation assistive software is to just align axial CT-slices according to picture number. Here different patient positioning on the CT-table and different scanning start points result in poor matches. Further these pieces of software do not indicate which explicit finding on an axial slice was evaluated in the previous examination if there are multiple lesions on the same slice. In contrast, the evaluated software navigates to a certain finding and specifically indicates it by placing a crosshair on the corresponding finding. Finally, the landmark-based approach requires less computational power and consequently less run-time than systems relying on matching HU similarities .
CAD systems have two common goals, to improve accuracy and increase radiologists’ productivity . Recently, various computer-aided detection (CAD) systems for multiple clinical tasks have been developed, such as CAD for lung cancer , pulmonary embolism , breast cancer , colon cancer , liver cancer , prostate cancer  and coronary artery stenoses . Due to its landmark based framework the software fulfills two of Kauczor’s demands for a CAD system : It generates a 3D map and registrates different imaging series facilitating orientation and navigation in CT-scans.
In order to compare consecutive oncological measurements the same, and if possible thinnest slice thickness should be used . This calls for radiological reading with a 3D radiological image analysis system (e.g. MicroView® (Parallax Innovations, Ilderton, Canada), Aquarius® (TeraRecon, Foster City, CA, USA) or SyngoVia® (Siemens AG, Erlangen, Germany)). These, however do not feature image numbers. This even complicates the evaluation of findings in baseline and follow-up studies when having to navigate manually between lesions (e.g. by scrolling through the examinations). For this workflow an additional automatic or semi-automatic navigation solution seems even more needed than with conventional PACS reading (e.g. reconstructed 5 mm slice thickness data sets).
Our study faces some limitations that suggest directions for future work. One limitation was that the software’s proposed time saving effect for oncological reading was not assessed. This is owed to the fact that the aim of our research was to evaluate the clinical feasibility, accuracy and reliability of the software.
Another limitation was that in this study we concentrated on navigation within full-body (thoraco/abdominal/pelvic) CT-scans. However, we did not investigate the technically possible  navigation between non-matching body regions.
One further plan is to combine the features of presented lesion tracking software with features of automatic lesion volumetry software.
The presented and evaluated lesion tracking software reliably handled clinical routine CT-scans and accurately enabled semi-automated real-time lesion tracking with a clinically tolerable accuracy. The software presumably facilitates the handling of large CT data sets, eases the visualization and evaluation of tumor response in consecutive CT-scans, and potentially speeds up radiological reading.
Institutional review board approval was obtained for this study. The need for informed consent was waived. All procedures were in accordance with the Helsinki Declaration.
The presented software was developed and clinically tested within the scope of the nationwide governmental funded German research project Theseus-Medico (project-number: 01MQ07026).
- Andriole KP, Morin RL, Arenson RL, Carrino JA, Erickson BJ, Horii SC, Piraino DW, Reiner BI, Seibert JA, Siegel E: Addressing the coming radiology crisis-the Society for Computer Applications in Radiology transforming the radiological interpretation process (TRIP) initiative. J Digit Imaging 2004, 17(4):235–243. 10.1007/s10278-004-1027-1PubMed CentralPubMedView ArticleGoogle Scholar
- Andriole KP, Wolfe JM, Khorasani R, Treves ST, Getty DJ, Jacobson FL, Steigner ML, Pan JJ, Sitek A, Seltzer SE: Optimizing analysis, visualization, and navigation of large image data sets: one 5000-section CT scan can ruin your whole day. Radiology 2011, 259(2):346–362. 10.1148/radiol.11091276PubMedView ArticleGoogle Scholar
- Eisenhauer EA, Therasse P, Bogaerts J, Schwartz LH, Sargent D, Ford R, Dancey J, Arbuck S, Gwyther S, Mooney M, Rubinstein L, Shankar L, Dodd L, Kaplan R, Lacombe D, Verweij J: New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur. J. Cancer 2009, 45(2):228–247. 10.1016/j.ejca.2008.10.026PubMedView ArticleGoogle Scholar
- Fabel M, Bolte H, von Tengg-Kobligk H, Bornemann L, Dicken V, Delorme S, Kauczor HU, Heller M, Biederer J: Semi-automated volumetric analysis of lymph node metastases during follow-up–initial results. Eur Radiology 2011, 21(4):683–692. 10.1007/s00330-010-1966-5View ArticleGoogle Scholar
- Goyal N, Jain N, Rachapalli V: Ergonomics in radiology. Clin Radiol 2009, 64(2):119–126. 10.1016/j.crad.2008.08.003PubMedView ArticleGoogle Scholar
- Weiss DL, Siddigui KM, Scopelliti J: Radiologist assessment of PACS user interface devices. J Am Coll Radiol 2006, 3(4):265–273. 10.1016/j.jacr.2005.10.016PubMedView ArticleGoogle Scholar
- Lidén M, Andersson T, Geijer H: Alternative user interface devices for improved navigation of CT datasets. J Digit Imaging 2011, 24(1):126–134. 10.1007/s10278-009-9252-2PubMed CentralPubMedView ArticleGoogle Scholar
- Seifert S, Barbu A, Zhou SK, David L, Johannes F, Martin H, Suehling M, Cavallaro A, Comaniciu D: Hierarchical Parsing and Semantic Navigation of Full Body CT Data. Proceedings SPIE 7259, Medical Imaging 2009. doi: 10.1117/12.812214Google Scholar
- Feulner J, Zhou SK, Angelopoulou E, Seifert S, Cavallaro A, Hornegger J, Comaniciu D: Comparing axial CT slices in quantized N-dimensional SURF descriptor space to estimate the visible body region. Comput Med Imaging Graph 2011, 35(3):227–236. 10.1016/j.compmedimag.2010.11.004PubMedView ArticleGoogle Scholar
- Eng J: Sample Size Estimation: How Many Individuals Should Be Studied? Radiology 2003, 227(2):309–313. 10.1148/radiol.2272012051PubMedView ArticleGoogle Scholar
- Curran SD, Muellner AU, Schwartz LH: Imaging response assessment in oncology. Cancer Imaging 2006, 6: 126–130. 10.1102/1470-7330.2006.9039View ArticleGoogle Scholar
- Van Ginneken B, Schaefer-Prokop CM, Prokop M: Computer-aided Diagnosis: How to Move from the Laboratory to the Clinic. Radiology 2011, 261(3):719–732. 10.1148/radiol.11091710PubMedView ArticleGoogle Scholar
- Kotter E, Langer M: Computer aided detection and diagnosis in radiology. Eur Radiol 2011, 21(3):590–592. 10.1007/s00330-010-2046-6PubMedView ArticleGoogle Scholar
- Roos JE, Paik D, Olsen D, Liu EG, Chow LC, Leung AN, Mindelzun R, Choudhury KR, Naidich DP, Napel S, Rubin GD: Computer-aided detection (CAD) of lung nodules in CT scans: radiologist performance and reading time with incremental CAD assistance. Eur Radiol 2010, 20(3):549–557. 10.1007/s00330-009-1596-yPubMed CentralPubMedView ArticleGoogle Scholar
- Blackmon KN, Florin C, Bogoni L, McCain JW, Koonce JD, Lee H, Bastarrika G, Thilo C, Costello P, Salganicoff M, Joseph SU: Computer-aided detection of pulmonary embolism at CT pulmonary angiography: can it improve performance of inexperienced readers? Eur Radiol 2011, 21(6):1214–1223. 10.1007/s00330-010-2050-xPubMedView ArticleGoogle Scholar
- Dorrius MD, der Weide MC, van Ooijen PM, Pijnappel RM, Oudkerk M: Computer-aided detection in breast MRI: a systematic review and meta-analysis. Eur Radiol 2011, 21(8):1600–1608. 10.1007/s00330-011-2091-9PubMed CentralPubMedView ArticleGoogle Scholar
- Neri E, Faggioni L, Regge D, Vagli P, Turini F, Cerri F, Picano E, Giusti S, Bartolozzi C: CT Colonography: Role of a second reader CAD paradigm in the initial training of radiologists. Eur J Radiol 2011, 80(2):303–309. 10.1016/j.ejrad.2010.07.014PubMedView ArticleGoogle Scholar
- Napel SA, Beaulieu CF, Rodriguez C, Cui J, Xu J, Gupta A, Korenblum D, Greenspan H, Ma Y, Rubin DL: Automated retrieval of CT images of liver lesions on the basis of image similarity: method and preliminary results. Radiology 2010, 256(1):243–252. 10.1148/radiol.10091694PubMed CentralPubMedView ArticleGoogle Scholar
- Sung YS, Kwon HJ, Park BW, Cho G, Lee CK, Cho KS, Kim JK: Prostate Cancer Detection on Dynamic Contrast-Enhanced MRI: Computer-Aided Diagnosis Versus Single Perfusion Parameter Maps. AJR Am J Roentgenol 2011, 197(5):1122–1129. 10.2214/AJR.10.6062PubMedView ArticleGoogle Scholar
- Arnoldi E, Gebregziabher M, Schoepf UJ, Goldenberg R, Ramos-Duran L, Zwerner PL, Nikolaou K, Reiser MF, Costello P, Thilo C: Automated computer-aided stenosis detection at coronary CT angiography: initial experience. Eur Radiol 2010, 20(5):1160–1167. 10.1007/s00330-009-1644-7PubMedView ArticleGoogle Scholar
- Kauczor HU: Multimodal imaging and computer assisted diagnosis for functional tumour characterisation. Cancer Imaging 2005, 5: 46–50. 10.1102/1470-7330.2005.0013PubMed CentralPubMedView ArticleGoogle Scholar
- Tan Y, Guo P, Mann H, Marley SE, Juanita Scott ML, Schwartz LH, Ghiorghiu DC, Zhao B: Assessing the effect of CT slice interval on unidimensional, bidimensional and volumetric measurements of solid tumours. Cancer Imaging 2012, 12: 497–505. 10.1102/1470-7330.2012.0046PubMed CentralPubMedView ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.