Automatic registration and fast volume reconstruction from serial histology sections

https://doi.org/10.1016/j.cviu.2011.02.009Get rights and content

Abstract

The scope of this research is to propose a novel method for automatic registration and fast volume reconstruction of serial tissue sections based on common-configured computer, on the premise that the algorithm can achieve accurate result with fast speed and little interaction as much as possible. The whole flowchart comprises the four main parts, of which image registration and image reconstruction are of great significance, materialized the following innovative ideas. Firstly, mutual information measure is combined with morphological gradient information to contain fewer erroneous maxima and lead to the global maximum when registration. Secondly, the hybrid optimizer combined Particle Swarm Optimization (PSO) with Powell algorithm is proposed to restrain local maxima of mutual information function and to improve the registration accuracy. Thirdly, the multiresolution data structure based on Mallat decomposition not only improves the behavior of registration function, but also accelerates the algorithm speed. Finally, an improved Shear–Warp algorithm is proposed based on the sorted volumetric dataset for registered image stack, which can skip all of the transparent voxels and achieve faster speed. Experimental results demonstrate that the novel registration and reconstruction algorithms are effective and efficient to achieve the virtual three-dimensional gekko’s cervical spinal cord successfully and the reconstruction result can be rotated, scaled, incised and measured arbitrarily, which is valuable to be applied to the other kinds of serial histology sections.

Highlights

► The method is given to registration and volume reconstruction of tissue sections. ► The ultimate goal is to achieve results with fast speed and little interaction. ► The innovative aspects are improved MI, hybrid optimizer and improved S-W algorithm. ► The reconstructed 3D gekko’s cervical spinal cord can be operated arbitrarily.

Introduction

Automatic registration and three-dimensional (3D) reconstruction is a powerful strategy allowing the evaluation of significant structures while retaining their localization both within the organ and with respect to each other. This technology has achieved great progress in histology and histopathology of the biomedical research field with the development of slim tissue sections and high-performance microscopes. In fundamental researches, registration and reconstruction of serial tissue sections can display ultrastructure inside organs or cells distinctly through rotation, incision and measurement operations, which is of great importance in morphology, zootomy and cytochemistry. In clinic diagnoses, this technology is often applied to find out pathogeny, character of foci, heteroplasia and concrescence.

However, there are still two puzzles in performing 3D reconstruction of tissue structures from serial sections. The first obstacle is the proper registration of microscope images accurately and fast. In the digital image acquisition stage, each section may be exposed to independent amounts of scaling and/or non-linear deformation as a result of cutting, folding, drying, and optical distortions inevitably. In addition, when positioning the sections onto the glass slides, non-linear distortion should be considered such as rotational and translational offsets between the consecutive section images. Although linear registration has been intensively researched in the automatic registration of medical image datasets, e.g. MRI, CT and PET images, a particular registration method for serial sections is still a different task, yet to be adequately resolved. The second puzzle is to achieve interactive rendering speed. In visualization of scientific computing, two avenues can be taken to achieve 3D reconstruction: surface rendering and volume rendering. Compared to surface rendering technology, volume rendering converts the whole dataset to ultimate three-dimensional object to display, and always proves more efficient and keeps in the foreground especially when the object is complex or large, or when the isosurface is interactively varied. It is common that there are a great large number of sections to be reconstructed, therefore, rendering speed is a key factor to be considered.

A few previous studies have been reported with regard to the registration and reconstruction of the serial section images [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11]. Brey et al. presented a method for automated alignment and quantitative three-dimensional analysis of the microvascular structures based on Automated Image Registration (AIR) software [1]. Duerstock et al. accomplished an isocontouring algorithm to extract surfaces of the regions of interests from the injured spinal cord [2], [3]. Tao Ju used image registration algorithm to correct tissue distortions and to build the surface representing the partitioning of anatomical regions of an adult mouse brain [4]. Yong Chen et al. used atomic force microscopy (AFM) to scan serial thin sections of a mouse embryonic stem cell and reconstructed to observe the in situ inner ultrastructures [5]. Mosaliganti et al. developed a mixture of automated and semi-automated enhancements for the mutual information-based registration algorithm using Insight Segmentation and Registration Toolkit (ITK) framework [6]. Fiala and Andrey designed the visualization editor for serial section stacks, named Reconstruct and Free-D, respectively [7], [8]. Bin Ma et al. accomplished an automatic rigid registration program, named Image-Reg, with the results compared with several commercial software packages, e.g. Image Pro Plus, Image J and 3D Studio Max [9], [10]. Sundaram presented an approach toward the quantification of pulmonary deformation via non-rigid registration of serial MR images of the lung based on the ITK framework [11].

However, there are several evident deficiencies in the above references, summarized as follows:

  • (1)

    Use manpower work [2], [3] or commercial software [1], [6], [10], [11] to realize registration of the serial sections. It is tedious and fallible for the operator to register so many serial sections by manual. Meanwhile, commercial software, e.g. Image Pro Plus, Image J, PhotoShop and 3D Studio Max, are all universal image processing tools, not especially for medical serial sections, which lead to unsatisfactory results on occasions.

  • (2)

    Use surface tiling or isocontouring method to realize three-dimensional surface reconstruction [2], [3], [4], [5], [10]. The key advantage of direct volume rendering over surface rendering is the potential to show the structures of the value distribution throughout the volume rather than just at the selected boundary surfaces of variable values.

  • (3)

    Use SGI graph station [2], [3] or high performance computers [4], [9], leading to cost inflation, which is not compatible with the practical situations of the grass-root hospitals in our country.

  • (4)

    For those software developed for serial sections, special concern should still be taken to avoid dependency upon platform and operating system, to minimize constraints of input data characteristics or formats, and to maximize flexibility and interactivity [7], [8], [9], [10].

The scope of this research is to propose a novel method for registration and reconstruction of the serial tissue sections based on common-configured computer, on the premise that the algorithm can achieve the accurate result with fast speed and little interaction as much as possible. The structure of the remaining part is as follows. To begin with, the whole algorithm flowchart is introduced in Section 2. Sections 3 Improved mutual information, 4 Hybrid multiresolution optimizer, 5 Improved Shear–Warp algorithm propose three innovative aspects of our algorithm, respectively: mutual information combined with morphological gradient, hybrid multiresolution optimizer combined with PSO and Powell algorithms, and improved Shear–Warp algorithm based on the sorted volumetric data structure. Computational steps and experimental results are demonstrated and discussed in Section 6. Finally, Section 7 summarizes and draws conclusions.

Section snippets

Algorithm flowchart

Fig. 1 shows the whole flowchart of our algorithm, including the four main parts: Image Acquisition, Image Preprocessing, Image Registration and Image Reconstruction.

Image acquisition: Serial sections are fixed and stained first, then digitalization are performed using Lecia microscope with its own software Qwin. After acquisition, the stack of equal-sized images is imported into Photoshop software to save as TIFF format (image size 1300 × 1030).

Image preprocessing: The manual nature of the

Mutual information

Mutual information (MI) is an important concept of information theory, which measures the statistical dependence between two variables. It was first proposed as a registration measure in medical image registration in 1995, independently by Viola and Wells [12] and by Collignon [13]. Now MI has been accepted by many researchers as one of the most accurate and robust retrospective registration methods [6], [14], [15]. The MI of two images means that the amount of information which one image

Hybrid multiresolution optimizer

Several kinds of optimization methods have been proposed and applied in image registration up to date. A popular method is Powell routine, which optimizes each transformation parameter in turn. It does not require calculating function derivatives, but is relatively sensitive to local optima in the registration function. The other popular is the Simplex method, which does not require derivatives either and considers all degrees of freedom simultaneously. The third method is Simulated Annealing,

Improved Shear–Warp algorithm

Shear–Warp algorithm was proposed by Lacroute and Levoy [28], [29] and is still considered to be the fastest volume rendering algorithm up to date. This algorithm is based on a Shear–Warp factorization of the viewing transformation, with the schematic diagram shown in Fig. 3. The volume data is first projected by a shear into a two-dimensional intermediate image in the sheared object space, where all viewing rays are parallel to the major axis perpendicular to the viewing direction. Then, the

Experimental results and discussion

The novel algorithm for automatic registration and fast reconstruction of serial histology sections is briefly summarized as follows:

  • STEP1: Determine a suitable pyramid grade number by the size of dataset and implement wavelet transform on the reference and floating image, respectively.

  • STEP2: Use PSO algorithm to search optimum based on the nearest neighbor interpolation.

  • STEP3: Use Powell algorithm to achieve more accurate registration result by taking the prior grade result as the

Conclusions

In summary, we have investigated automatic registration and fast reconstruction of cryo-sectioning without any gold standard for evaluation, which is the first step in visualization and is of great importance in neuropathology. The whole flowchart of the novel algorithm includes image registration and image reconstruction mainly, in which materialized the following innovative ideas.

In general, the accuracy and speed are the two kernel criteria to evaluate a registration algorithm’s performance.

Acknowledgments

The author acknowledge the support of National Natural Science Foundation of China (No. 61005054), Natural Science Foundation of Jiangsu Educational Committee (No. 08KJD520021), Nantong Municipal Natural Science and Technological Application Foundation (No. K2008036), and Scientific Research Start-Up Foundation for PhD of Nantong University (No. 08B15).

The author is grateful to the insight work provided by Prof. Fei Ding and Mrs. Mi Shen, Key Lab of Neuroregeneration of Nantong University.

The

Min Tang, female, was born in Nantong, Jiangsu Province, P. R. China in 1977. She received her bachelor’s degree from Nantong University in 1999 and her PhD degree from Nanjing University of Aeronautics & Astronautics in 2007. Now she is a associate professor in School of Electronics and Information in Nantong University. Her major research fields include image processing, analysis and visualization.

References (30)

  • Ma Bin et al.

    Serial sectioning and three-dimensional reconstruction of mouse Peyer’s patch [J]

    Micron

    (2008)
  • Paul Viola et al.

    Alignment by maximization of mutual information [J]

    International Journal of Computer Vision

    (1997)
  • A. Collignon et al.

    Automated multimodality image registration based on information theory [A]

  • J.P.W. Pluim et al.

    Mutual information based registration of medical images: a survey [J]

    IEEE Transactions on Medical Imaging

    (2003)
  • Lin Feng et al.

    Advances in medical image registration based on mutual information [J]

    Journal of Biomedical Engineering

    (2005)
  • Cited by (0)

    Min Tang, female, was born in Nantong, Jiangsu Province, P. R. China in 1977. She received her bachelor’s degree from Nantong University in 1999 and her PhD degree from Nanjing University of Aeronautics & Astronautics in 2007. Now she is a associate professor in School of Electronics and Information in Nantong University. Her major research fields include image processing, analysis and visualization.

    View full text