Abstract
The impact of image pattern recognition on accessing large databases of medical images has recently been explored, and content-based image retrieval (CBIR) in medical applications (IRMA) is researched. At the present, however, the impact of image retrieval on diagnosis is limited, and practical applications are scarce. One reason is the lack of suitable mechanisms for query refinement, in particular, the ability to (1) restore previous session states, (2) combine individual queries by Boolean operators, and (3) provide continuous-valued query refinement. This paper presents a powerful user interface for CBIR that provides all three mechanisms for extended query refinement. The various mechanisms of man–machine interaction during a retrieval session are grouped into four classes: (1) output modules, (2) parameter modules, (3) transaction modules, and (4) process modules, all of which are controlled by a detailed query logging. The query logging is linked to a relational database. Nested loops for interaction provide a maximum of flexibility within a minimum of complexity, as the entire data flow is still controlled within a single Web page. Our approach is implemented to support various modalities, orientations, and body regions using global features that model gray scale, texture, structure, and global shape characteristics. The resulting extended query refinement has a significant impact for medical CBIR applications.
REFERENCES
Tagare HD, Jaffe CC, Dungan J: Medical image databases—A content-based retrieval approach. J Am Med Inform Assoc 4:184–198, 1997
Müller H, Michoux N, Bandon D, Geissbuhler A: A review of content-based image retrieval systems in medical applications—Clinical benefits and future directions. Int J Med Inform 73(1):1–23, 2004
Deselaers T, Müller H, Clough P, Ney H, Deserno TM: The CLEF 2005 automatic medical image annotation task. International Journal of Computer Vision 2007 (in press) DOI 10.1007/s11263-006-0007-y
Müller H, Deselaers T, Deserno TM, Clough P, Kim E, Hersh W: Overview of the ImageCLEFmed 2006 medical retrieval and annotation tasks. Lect Notes Comput Sci (in press)
Deserno TM, Güld MO, Deselaers T, Keysers D, Schubert H, Spitzer K, Ney H, Wein BB: Automatic categorization of medical images for content-based retrieval and data mining. Comput Med Imaging Graph 29(2):143–155, 2005
Zhou W, Smalheiser NR, Yu C: A tutorial on information retrieval: Basic terms and concepts. J Biomed Discov Collab 1:2, 2006
Celetyno A, Di Sciascio E: Feature integration and relevance feedback analysis in image similarity evaluation. J Electron Imaging 7(2):308–317, 1998
Rui Y, Huang TS, Ortega M, Mehrotra S: Relecance feedback—A power tool for interactive content-based image retrieval. IEEE Trans Circuits Syst Video Technol 8(5):644–655, 1998
Smeulders AWM, Worring M, Santini S, Gupta A, Jain R: Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380, 2000
Naster C, Mitschke M, Meilhac C: Efficient query refinement for image retrieval. Proceedings of the IEEE Computer Society Conference on Comput Vis Pattern Recognit 547–552, 1998
Müller H, Müller W, Squire DM, Marchand-Maillet S, Pun T: Strategies for positive and negative relevance feedback in image retrieval. Proceedings 15th International Conference on Pattern Recognition 1043–1046, 2000
Bartolini I, Ciaccia P, Waas F: FeedbackBypass—A new approach to interactive similarity query processing. Proceedings 27th International Conference on Very Large Data Bases 201–210, 2001
Laaksonen J, Koskela M, Laakso S, Oja E: Self-organizing maps as a relevance feedback technique in content-based image retrieval. Pattern Anal Appl 4(2–3):140–152, 2001
Chen JY, Bouman CA, Dalton JC: Active browsing using similarity pyramids. Proceedings SPIE 3656:144–154, 1998
He XF, King O, Ma WY, Li MJ, Zhang HJ: Learning a semantic space from relevance feedback for image retrieval. IEEE Trans Circuits Syst Video Technol 13(1):39–48, 2003
Crestani F, De la Fuente P, Vegas J: Design of a graphical user interface for focussed retrieval of structured documents. Proceedings 8th Symposium on String Processing and Information Retrieval 246–249, 2001
Vegas J, De la Fuente P, Crestani F: A graphical user interface for structured document retrieval. Lect Notes Comput Sci 2291:268–283, 2002
Fox KL, Frieder O, Knepper MM, Snowberg EJ: SENTINEL—A multiple engine information retrieval and visualization system. J Am Soc Inf Sci 50(7):616–625, 1999
Meiers T, Sikora T, Keller I: Hierarchical image database browsing environment with embedded relevance feedback. Proceedings 2002 International Conference on Image Processing 2:593–596, 2002
Zhu XQ, Zhang HJ, Liu WY, Hu CH, Wu L: New query refinement and semantics integrated image retrieval system with semiautomatic annotation scheme. J Electron Imaging 10(4):850–860, 2001
Deserno TM, Güld MO, Thies C, Fischer B, Spitzer K, Keysers D, Ney H, Kohnen M, Schubert H, Wein BB: Content-based image retrieval in medical applications. Methods Inf Med 43(4):354–361, 2004
Güld MO, Thies C, Fischer B, Deserno TM: A generic concept for the implementation of medical image retrieval systems. Int J Med Inf 76(2–3):252–259, 2007
Deserno TM, Plodowski B, Spitzer K, Wein BB, Ney H, Seidl T: Extended query refinement for content-based access to large medical image databases. Proceedings SPIE 5371:90–98, 2004
Berlage T; A selective undo mechanism for graphical user interfaces based on command objects. ACM Trans Comput Hum Interact 1(3):269–294, 1994
Kreuseler M, Nocke T, Schumann H: A history mechanism for visual data mining. Proceedings IEEE symposium on information visualization (INFOVIS) 49–56, 2004
Rui Y, Huang TS, Ortega M, Mehrotra S: Relevance feedback: A power tool for interactive content-based image retrieval. IEEE Trans Circuits Syst Video Technol 8(5):644–655, 1998
Nielsen J, Mack RL Eds.: Usability Inspections Methods. Wiley, Chichester, UK, 1994
Acknowledgement
This work was performed within the image retrieval in medical applications (IRMA) project, which is supported by the German Research Community (Deutsche Forschungsgemeinschaft, DFG), grants Le 1108/4 and Le 1108/6. For further information, visit http://irma-project.org. The authors would also like to acknowledge the helpful comments of the reviewers that improved this manuscript.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Deserno, T.M., Güld, M.O., Plodowski, B. et al. Extended Query Refinement for Medical Image Retrieval. J Digit Imaging 21, 280–289 (2008). https://doi.org/10.1007/s10278-007-9037-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10278-007-9037-4