Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 5/2022

01-04-2022 | Original Article

Deep learning for biomechanical modeling of facial tissue deformation in orthognathic surgical planning

Authors: Nathan Lampen, Daeseung Kim, Xi Fang, Xuanang Xu, Tianshu Kuang, Hannah H. Deng, Joshua C. Barber, Jamie Gateno, James Xia, Pingkun Yan

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 5/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Purpose

Orthognathic surgery requires an accurate surgical plan of how bony segments are moved and how the face passively responds to the bony movement. Currently, finite element method (FEM) is the standard for predicting facial deformation. Deep learning models have recently been used to approximate FEM because of their faster simulation speed. However, current solutions are not compatible with detailed facial meshes and often do not explicitly provide the network with known boundary type information. Therefore, the purpose of this proof-of-concept study is to develop a biomechanics-informed deep neural network that accepts point cloud data and explicit boundary types as inputs to the network for fast prediction of soft-tissue deformation.

Methods

A deep learning network was developed based on the PointNet++ architecture. The network accepts the starting facial mesh, input displacement, and explicit boundary type information and predicts the final facial mesh deformation.

Results

We trained and tested our deep learning model on datasets created from FEM simulations of facial meshes. Our model achieved a mean error between 0.159 and 0.642 mm on five subjects. Including explicit boundary types had mixed results, improving performance in simulations with large deformations but decreasing performance in simulations with small deformations. These results suggest that including explicit boundary types may not be necessary to improve network performance.

Conclusion

Our deep learning method can approximate FEM for facial change prediction in orthognathic surgical planning by accepting geometrically detailed meshes and explicit boundary types while significantly reducing simulation time.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Literature
1.
go back to reference Alanko OM, Svedström-Oristo AL, Tuomisto MT (2010) Patients’ perceptions of orthognathic treatment, well-being, and psychological or psychiatric status: a systematic review. Acta Odontol Scand 68(5):249–260CrossRef Alanko OM, Svedström-Oristo AL, Tuomisto MT (2010) Patients’ perceptions of orthognathic treatment, well-being, and psychological or psychiatric status: a systematic review. Acta Odontol Scand 68(5):249–260CrossRef
2.
go back to reference Shafi MI, Ayoub A, Ju X, Khambay B (2013) The accuracy of three-dimensional prediction planning for the surgical correction of facial deformities using Maxilim. Int J Oral Maxillofac Surg 42(7):801–806CrossRef Shafi MI, Ayoub A, Ju X, Khambay B (2013) The accuracy of three-dimensional prediction planning for the surgical correction of facial deformities using Maxilim. Int J Oral Maxillofac Surg 42(7):801–806CrossRef
3.
go back to reference Xia JJ, Gateno J, Teichgraeber JF (2009) New clinical protocol to evaluate craniomaxillofacial deformity and plan surgical correction. Int J Oral Maxillofac Surg 67(10):2093–2106CrossRef Xia JJ, Gateno J, Teichgraeber JF (2009) New clinical protocol to evaluate craniomaxillofacial deformity and plan surgical correction. Int J Oral Maxillofac Surg 67(10):2093–2106CrossRef
4.
go back to reference Kim D, Kuang T, Rodrigues YL, Gateno J, Shen SG, Wang X, Deng H, Yuan P, Alfi DM, Liebschner MA, Xia JJ (2019) A new approach of predicting facial changes following orthognathic surgery using realistic lip sliding effect. MICCAI 11768:336–344PubMed Kim D, Kuang T, Rodrigues YL, Gateno J, Shen SG, Wang X, Deng H, Yuan P, Alfi DM, Liebschner MA, Xia JJ (2019) A new approach of predicting facial changes following orthognathic surgery using realistic lip sliding effect. MICCAI 11768:336–344PubMed
5.
go back to reference Xia JJ, Gateno J, Teichgraeber JF, Yuan P, Chen KC, Li J, Zhang X, Tang Z, Alfi DM (2015) Algorithm for planning a double-jaw orthognathic surgery using a computer-aided surgical simulation (CASS) protocol. Int J Oral Maxillofac Surg 44(12):1431–1440CrossRef Xia JJ, Gateno J, Teichgraeber JF, Yuan P, Chen KC, Li J, Zhang X, Tang Z, Alfi DM (2015) Algorithm for planning a double-jaw orthognathic surgery using a computer-aided surgical simulation (CASS) protocol. Int J Oral Maxillofac Surg 44(12):1431–1440CrossRef
6.
go back to reference Kim D, Kuang T, Rodrigues YL, Gateno J, Shen SG, Wang X, Stein K, Deng HH, Liebschner MA, Xia JJ (2021) A novel incremental simulation of facial changes following orthognathic surgery using FEM with realistic lip sliding effect. Med Image Anal 72:102095CrossRef Kim D, Kuang T, Rodrigues YL, Gateno J, Shen SG, Wang X, Stein K, Deng HH, Liebschner MA, Xia JJ (2021) A novel incremental simulation of facial changes following orthognathic surgery using FEM with realistic lip sliding effect. Med Image Anal 72:102095CrossRef
7.
go back to reference Ullah R, Turner PJ, Khambay BS (2015) Accuracy of three-dimensional soft tissue predictions in orthognathic surgery after Le Fort I advancement osteotomies. Br J Oral Maxillofac Surg 53(2):153–157CrossRef Ullah R, Turner PJ, Khambay BS (2015) Accuracy of three-dimensional soft tissue predictions in orthognathic surgery after Le Fort I advancement osteotomies. Br J Oral Maxillofac Surg 53(2):153–157CrossRef
8.
go back to reference Knoops PG, Borghi A, Ruggiero F, Badiali G, Bianchi A, Marchetti C, Rodriguez-Florez N, Breakey RW, Jeelani O, Dunaway DJ, Schievano S (2018) A novel soft tissue prediction methodology for orthognathic surgery based on probabilistic finite element modelling. PloS one 13(5):e0197209CrossRef Knoops PG, Borghi A, Ruggiero F, Badiali G, Bianchi A, Marchetti C, Rodriguez-Florez N, Breakey RW, Jeelani O, Dunaway DJ, Schievano S (2018) A novel soft tissue prediction methodology for orthognathic surgery based on probabilistic finite element modelling. PloS one 13(5):e0197209CrossRef
9.
go back to reference Kim D, Ho DCY, Mai H, Zhang X, Shen SG, Shen S, Yuan P, Liu S, Zhang G, Zhou X, Gateno J (2017) A clinically validated prediction method for facial soft-tissue changes following double-jaw surgery. Med Phys 44(8):4252–4261CrossRef Kim D, Ho DCY, Mai H, Zhang X, Shen SG, Shen S, Yuan P, Liu S, Zhang G, Zhou X, Gateno J (2017) A clinically validated prediction method for facial soft-tissue changes following double-jaw surgery. Med Phys 44(8):4252–4261CrossRef
10.
go back to reference Faure F, Duriez C, Delingette H, Allard J, Gilles B, Marchesseau S, Talbot H, Courtecuisse H, Bousquet G, Peterlik I, Cotin S (2012) SOFA: a multi-model framework for interactive physical simulation. Stud Mechanobiol Tissue Eng Biomater 11:283–321CrossRef Faure F, Duriez C, Delingette H, Allard J, Gilles B, Marchesseau S, Talbot H, Courtecuisse H, Bousquet G, Peterlik I, Cotin S (2012) SOFA: a multi-model framework for interactive physical simulation. Stud Mechanobiol Tissue Eng Biomater 11:283–321CrossRef
11.
go back to reference Johnsen SF, Taylor ZA, Clarkson MJ, Hipwell J, Modat M, Eiben B, Han L, Hu Y, Mertzanidou T, Hawkes DJ, Ourselin S (2015) NiftySim: A GPU-based nonlinear finite element package for simulation of soft tissue biomechanics. Int J Comput Assist Radiol Surg 10:1077CrossRef Johnsen SF, Taylor ZA, Clarkson MJ, Hipwell J, Modat M, Eiben B, Han L, Hu Y, Mertzanidou T, Hawkes DJ, Ourselin S (2015) NiftySim: A GPU-based nonlinear finite element package for simulation of soft tissue biomechanics. Int J Comput Assist Radiol Surg 10:1077CrossRef
12.
go back to reference Mendizabal A, Márquez-Neila P, Cotin S (2020) Simulation of hyperelastic materials in real-time using deep learning. Med Image Anal 59:101569CrossRef Mendizabal A, Márquez-Neila P, Cotin S (2020) Simulation of hyperelastic materials in real-time using deep learning. Med Image Anal 59:101569CrossRef
13.
go back to reference Bouletreau P, Makaremi M, Ibrahim B, Louvrier A, Sigaux N (2019) Artificial intelligence: applications in orthognathic surgery. J. Stomatol Oral Maxillofac Surg 120(4):347–354CrossRef Bouletreau P, Makaremi M, Ibrahim B, Louvrier A, Sigaux N (2019) Artificial intelligence: applications in orthognathic surgery. J. Stomatol Oral Maxillofac Surg 120(4):347–354CrossRef
14.
go back to reference Phellan R, Hachem B, Clin J, Mac-Thiong J, Duong L (2021) Real-time biomechanics using the finite element method and machine learning: review and perspective. Med Phys 48(1):7–18CrossRef Phellan R, Hachem B, Clin J, Mac-Thiong J, Duong L (2021) Real-time biomechanics using the finite element method and machine learning: review and perspective. Med Phys 48(1):7–18CrossRef
15.
go back to reference Pfeiffer M, Riediger C, Weitz J, Speidel S (2019) Learning soft tissue behavior of organs for surgical navigation with convolutional neural networks. Int J Comput Assist Radiol Surg 14:1147–1155CrossRef Pfeiffer M, Riediger C, Weitz J, Speidel S (2019) Learning soft tissue behavior of organs for surgical navigation with convolutional neural networks. Int J Comput Assist Radiol Surg 14:1147–1155CrossRef
16.
go back to reference Mendizabal, A., Tagliabue, E., Brunet, JN., Dall’Alba, D., Fiorini, P., Cotin, S. (2020). Physics-Based Deep Neural Network for Real-Time Lesion Tracking in Ultrasound-Guided Breast Biopsy. In: Miller, K., Wittek, A., Joldes, G., Nash, M., Nielsen, P. (eds) Computational Biomechanics for Medicine. MICCAI MICCAI 2019 2018. Springer, Cham. https://doi.org/10.1007/978-3-030-42428-2_4 Mendizabal, A., Tagliabue, E., Brunet, JN., Dall’Alba, D., Fiorini, P., Cotin, S. (2020). Physics-Based Deep Neural Network for Real-Time Lesion Tracking in Ultrasound-Guided Breast Biopsy. In: Miller, K., Wittek, A., Joldes, G., Nash, M., Nielsen, P. (eds) Computational Biomechanics for Medicine. MICCAI MICCAI 2019 2018. Springer, Cham. https://​doi.​org/​10.​1007/​978-3-030-42428-2_​4
17.
go back to reference Saeed SU, Taylor ZA, Pinnock MA, Emberton M, Barratt DC, Hu Y (2021) Prostate motion modelling using biomechanically-trained deep neural networks on unstructured nodes. arXiv preprint arXiv:2007.04972 Saeed SU, Taylor ZA, Pinnock MA, Emberton M, Barratt DC, Hu Y (2021) Prostate motion modelling using biomechanically-trained deep neural networks on unstructured nodes. arXiv preprint arXiv:​2007.​04972
18.
go back to reference Fu Y, Lei Y, Wang T, Patel P, Jani AB, Mao H, Curran WJ, Liu T, Yang X (2021) Biomechanically constrained non-rigid MR-TRUS prostate registration using deep learning based 3D point cloud matching. Med Image Anal 67:101845CrossRef Fu Y, Lei Y, Wang T, Patel P, Jani AB, Mao H, Curran WJ, Liu T, Yang X (2021) Biomechanically constrained non-rigid MR-TRUS prostate registration using deep learning based 3D point cloud matching. Med Image Anal 67:101845CrossRef
19.
go back to reference Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++ (2017) Deep hierarchical feature learning on point sets in a metric space. Adv Neural Inf Process Syst 30:5105–5114 Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++ (2017) Deep hierarchical feature learning on point sets in a metric space. Adv Neural Inf Process Syst 30:5105–5114
20.
go back to reference Zhang X, Kim D, Sheng S, yuan P, Liu S, Tang Z, Zhang G, Zhou X, Gateno J, Liebschner MA, Xia JJ (2018) An eFTD-VP framework for efficiently generating patient-specific anatomically detailed facial soft tissue FE mesh for craniomaxillofacial surgery simulation. Biomech Model Mechanobiol 17(2):387CrossRef Zhang X, Kim D, Sheng S, yuan P, Liu S, Tang Z, Zhang G, Zhou X, Gateno J, Liebschner MA, Xia JJ (2018) An eFTD-VP framework for efficiently generating patient-specific anatomically detailed facial soft tissue FE mesh for craniomaxillofacial surgery simulation. Biomech Model Mechanobiol 17(2):387CrossRef
21.
go back to reference Odot A, Haferssas R, Cotin S (2021) DeepPhysics: a physics aware deep learning framework for real-time simulation. arXiv preprint arXiv:2109.09491 Odot A, Haferssas R, Cotin S (2021) DeepPhysics: a physics aware deep learning framework for real-time simulation. arXiv preprint arXiv:​2109.​09491
22.
go back to reference Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378:686–707 Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378:686–707
23.
go back to reference Foti S, Koo B, Dowrick T, Ramalhinho J, Allam M, Davidson B, Stoyanov D, Clarkson MJ (2020) Intraoperative liver surface completion with graph convolutional VAE. Lect. Notes Comput. Sci 12443 LNCS:198–207 Foti S, Koo B, Dowrick T, Ramalhinho J, Allam M, Davidson B, Stoyanov D, Clarkson MJ (2020) Intraoperative liver surface completion with graph convolutional VAE. Lect. Notes Comput. Sci 12443 LNCS:198–207
24.
go back to reference Maas SA, Ellis BJ, Ateshian GA, Weiss JA (2012) FEBio: finite elements for biomechanics. J Biomech Eng 134(1) Maas SA, Ellis BJ, Ateshian GA, Weiss JA (2012) FEBio: finite elements for biomechanics. J Biomech Eng 134(1)
Metadata
Title
Deep learning for biomechanical modeling of facial tissue deformation in orthognathic surgical planning
Authors
Nathan Lampen
Daeseung Kim
Xi Fang
Xuanang Xu
Tianshu Kuang
Hannah H. Deng
Joshua C. Barber
Jamie Gateno
James Xia
Pingkun Yan
Publication date
01-04-2022
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 5/2022
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-022-02596-1

Other articles of this Issue 5/2022

International Journal of Computer Assisted Radiology and Surgery 5/2022 Go to the issue

Premium Partner