Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Machine Vision and Applications
Published in: Machine Vision and Applications 2-3/2015

01-04-2015 | Original Paper

Auto-inpainting heritage scenes: a complete framework for detecting and infilling cracks in images and videos with quantitative assessment

Authors: Milind G. Padalkar, Manjunath V. Joshi

Published in: Machine Vision and Applications | Issue 2-3/2015

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The need for preservation of cultural heritage has necessitated the research on digitally repairing the photographs of damaged monuments. In this paper, we first propose a technique for automatically detecting the cracked regions in photographs of monuments. Unlike the usual practice of manually selecting the mask for inpainting, the detected regions are supplied to an inpainting algorithm. Thus, the process of digitally repairing the cracked regions that physical objects have, using inpainting, is completely automated. The detection of cracked regions is based on comparison of patches, for which we use a measure derived from the edit distance, which is a popular string metric used in the area of text mining. Further, we extend this method to perform inpainting of video frames by making use of the scale-invariant feature transform and homography. We consider the camera to move while capturing video of the heritage site, as such videos are typically captured by novices, hobbyists and tourists. Finally, we also propose a video quality measure to quantify the temporal consistency of the inpainted video. Experiments have been carried out on videos captured from the heritage site at Hampi, India.
previous article A novel technique to extract accurate cell contours applied for segmentation of phytoplankton images
next article Geometric and radiometric estimation in a structured-light 3D scanner

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Footnotes
1
The Details of selecting a suitable tolerance value \({\delta _{t}}\) are given in Sect. 6.
 
2
For active contour segmentation technique, we have used the implementation available at http://​www.​mathworks.​in/​matlabcentral/​fileexchange/​23847-sparse-field-methods-for-active-contours.
 
3
An implementation for extraction and matching of SIFT keypoints and corresponding descriptor is available at http://​www.​cs.​ubc.​ca/​~lowe/​keypoints/​.
 
4
For fitting homography to keypoints using RANSAC, we used the code available at http://​www.​csse.​uwa.​edu.​au/​~pk/​Research/​MatlabFns/​Robust/​ransacfithomogra​phy.​m.
 
5
For decomposition of estimated homography, we have used the implementation available at http://​cs.​gmu.​edu/​~kosecka/​examples-code/​homography2Motio​n.​m.
 
6
The details of selecting threshold \({\delta _{r}}\) are given in Sect. 6.
 
Literature
1.
Amano, T.: Correlation based image defect detection. In: Proceedings of the 18th International Conference on Pattern Recognition. ICPR ’06, vol. 01, pp. 163–166. IEEE Computer Society, Washington, DC, USA (2006)
2.
Ammouche, A., Riss, J., Breysse, D., Marchand, J.: Image analysis for the automated study of microcracks in concrete. Cement Concrete Comp. 23(23), 267–278 (2001) (Special theme issue on image analysis)
3.
Bendels, G.H., Guthe, M., Klein, R.: Free-form modelling for surface inpainting. In: Proceedings of the 4th International Conference on Computer Graphics. Virtual reality, visualisation and interaction in Africa, AFRIGRAPH ’06, pp. 49–58. ACM, New York, NY, USA (2006)
4.
Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques. SIGGRAPH ’00, pp. 417–424. ACM Press/Addison–Wesley Publishing Co., New York, NY, USA (2000)
5.
Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2007)CrossRef
6.
Chan, T., Vese, L.: Active contours without edges. Image Process. IEEE Trans. 10(2), 266–277 (2001)CrossRefMATH
7.
Chang, R.C., Sie, Y.L., Chou, S.M., Shih, T.K.: Photo defect detection for image inpainting. In: Proceedings of the Seventh IEEE International Symposium on Multimedia. ISM ’05, pp. 403–407. IEEE Computer Society, Washington, DC, USA (2005)
8.
Cornelis, B., Rui, T., Gezels, E., Dooms, A., Pi++urica, A., Platia, L., Cornelis, J., Martens, M., Mey, M.D., Daubechies, I.: Crack detection and inpainting for virtual restoration of paintings: the case of the ghent altarpiece. Sig. Process. 93(3), 605–619 (2013) (Image processing for digital art work)
9.
Criminisi, A., Pérez, P., Toyama, K.: Object removal by exemplar-based inpainting. Comput. Vis. Pattern Recognit. IEEE Comput. Soc. Conf. 2, 721 (2003)
10.
Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13, 1200–1212 (2004)CrossRef
11.
El-Hakim, S.F.: Semi-automatic 3d reconstruction of occluded and unmarked surfaces from widely separated views. In: International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Greece, pp. 143–148 (2002)
12.
El-Hakim, S.F., MacDonald, G., Lapointe, J.F., Gonzo, L., Jemtrud, M.: On the digital reconstruction and interactive presentation of heritage sites through time. In: VAST 2006: the 7th International Symposium on Virtual Reality, Archaeology and Intelligent Cultural Heritage, Nicosia, Cyprus, 2006. Proceedings, pp. 243–250. Eurographics Association (2006)
13.
Farias, M., Mitra, S.: No-reference video quality metric based on artifact measurements. In: Image Processing, 2005. ICIP 2005. IEEE International Conference on, vol. 3, pp. III-141–144 (2005)
14.
Faugeras, O., Lustman, F.: Motion and structure from motion in a piecewise planar environment. Tech. Rep. RR-0856, INRIA (1988)
15.
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)CrossRefMathSciNet
16.
Grossauer, H.: A combined pde and texture synthesis approach to inpainting. Eur. Conf. Comput. Vis. 4, 214–224 (2004)
17.
Hartley, R., Zisserman, A.: Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, New York (2003)
18.
19.
de Kadt, C., Gain, J., Marais, P.: Revisiting district six: a case study of digital heritage reconstruction from archival photographs. In: Proceedings of the 6th International Conference on Computer Graphics. Virtual reality, visualisation and interaction in Africa, AFRIGRAPH ’09, pp. 13–21. ACM, New York, NY, USA (2009)
20.
Kang, H.W., Shin, S.Y.: Creating walk-through images from a video sequence of a dynamic scene. Presence: telepresence, and automotive. Environments 13, 638–655 (2004)
21.
Kenney, J.F.: Mathematicals of Statistics. Van Nostrand (1954)
22.
23.
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)CrossRef
24.
Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, vol. 2, IJCAI’81, pp. 674–679 (1981)
25.
Ma, Y., Soatto, S., Kosecka, J., Sastry, S.S.: An invitation to 3-D vision: from images to geometric model. Springer (2003)
26.
Masnou, S., Morel, J.M.: Level lines based disocclusion. Image processing, 1998. ICIP 98. In: Proceedings 1998 International Conference on pp. 259–263 (1998)
27.
28.
Oliveira, M.M., Bowen, B., Mckenna, R., Sung Chang, Y.: Fast digital image inpainting. In: Proceedings of the International Conference on Visualization, Imaging and Image Processing VIIP 2001, pp. 261–266, ACTA Press (2001)
29.
Padalkar, M.G., Vora, M.V., Joshi, M.V., Zaveri, M.A., Raval, M.S.: Identifying vandalized regions in facial images of statues for inpainting. New Trends in Image Analysis and Processing-ICIAP 2013. Lecture Notes in Computer Science, vol. 8158, pp. 208–217, Springer, Berlin (2013)
30.
Padalkar, M.G., Zaveri, M.A., Joshi, M.V.: Svd based automatic detection of target regions for image inpainting. In: Park, J.I., Kim, J. (eds.) Computer vision - ACCV 2012 Workshops. Lecture Notes in Computer Science, vol. 7729, pp. 61–71. Springer, Berlin (2013)
31.
Parmar, C.M., Joshi, M.V., Raval, M.S., Zaveri, M.A.: Automatic image inpainting for the facial images of monuments. In: Proceedings of Electrical Engineering Centenary Conference 2011, pp. 415–420. IISc Bangalore, India (2011)
32.
Patwardhan, K.A., Sapiro, G., Bertalmío, M.: Video inpainting under constrained camera motion. IEEE Trans. Image Process. 16(2), 545–553 (2007)CrossRefMathSciNet
33.
Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22(3), 313–318 (2003)CrossRef
34.
Ringot, E., Bascoul, A.: About the analysis of microcracking in concrete. Cement Concrete Comp. 23(23), 261–266 (2001) (Special theme issue on image analysis)
35.
Rother, C., Kolmogorov, V., Blake, A.: Grabcut: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23(3), 309–314 (2004)CrossRef
36.
37.
Saad, M.A., Bovik, A.C.: Blind quality assessment of videos using a model of natural scene statistics and motion coherency. In: Asilomar Conference on Signals, Systems, and Computers, pp. 332–336 (2012)
38.
39.
Shibata, T., Iketani, A., Senda, S.: Image in painting based on probabilistic structure estimation. In: Proceedings of the 10th Asian conference on Computer vision, vol. III. ACCV’10, pp. 109–120. Springer, Berlin (2011)
40.
Shih, T.K., Tang, N.C., Yeh, W.S., Chen, T.J., Lee, W.: Video inpainting and implant via diversified temporal continuations. In: Proceedings of the 14th Annual ACM International Conference on Multimedia. MULTIMEDIA ’06, pp. 133–136. ACM, New York, NY, USA (2006)
41.
Sikora, T.: MPEG digital video-coding standards. Sig. Process. Mag. IEEE 14(5), 82–100 (1997)CrossRef
42.
Tamaki, T., Suzuki, H., Yamamoto, M.: String-like occluding region extraction for background restoration. Int. Conf. Pattern Recognit. 3, 615–618 (2006)
43.
Turakhia, N., Shah, R., Joshi, M.: Automatic crack detection in heritage site images for image inpainting. In: Eighth Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), p. 68 (2012)
44.
45.
Wang, Z., Zhou, F., Qi, F.: Inpainting thick image regions using isophote propagation. In: Image Processing 2006 IEEE International Conference on, pp. 689–692 (2006). doi:10.​1109/​ICIP.​2006.​312428
46.
Wexler, Y., Shechtman, E., Irani, M.: Space-time completion of video. Pattern Anal. Mach. Intell. IEEE Trans. 29(3), 463–476 (2007)CrossRef
47.
Wu, J., Ruan, Q.: Object removal by cross isophotes exemplar-based inpainting. In: Proceedings of the 18th International Conference on Pattern Recognition. ICPR ’06, pp. 810–813. IEEE Computer Society, Washington, DC, USA (2006)
48.
Zou, Q., Cao, Y., Li, Q., Mao, Q., Wang, S.: Cracktree: Automatic crack detection from pavement images. Pattern Recogn. Lett. 33(3), 227–238 (2012)CrossRef
Metadata
Title
Auto-inpainting heritage scenes: a complete framework for detecting and infilling cracks in images and videos with quantitative assessment
Authors
Milind G. Padalkar
Manjunath V. Joshi
Publication date
01-04-2015
Publisher
Springer Berlin Heidelberg
Published in
Machine Vision and Applications / Issue 2-3/2015
Print ISSN: 0932-8092
Electronic ISSN: 1432-1769
DOI
https://doi.org/10.1007/s00138-015-0661-6

Other articles of this Issue 2-3/2015

Machine Vision and Applications 2-3/2015 Go to the issue

Original Paper

Region-based approach for the spectral clustering Nyström approximation with an application to burn depth assessment

Original Paper

Coarse-grained ancient coin classification using image-based reverse side motif recognition

Original Paper

Improved best match search method in depth recovery with descent images

Original Paper

Gaze direction estimation by component separation for recognition of Eye Accessing Cues

Original Paper

Geometric and radiometric estimation in a structured-light 3D scanner

Original Paper

Posed and spontaneous expression recognition through modeling their spatial patterns

Premium Partner

    Image Credits