Computational Planning in Facial Surgery

Abstract

This article reflects the research of the last two decades in computational planning for cranio-maxillofacial surgery. Model-guided and computer-assisted surgery planning has tremendously developed due to ever increasing computational capabilities. Simulators for education, planning, and training of surgery are often compared with flight simulators, where maneuvers are also trained to reduce a possible risk of failure. Meanwhile, digital patient models can be derived from medical image data with astonishing accuracy and thus can serve for model surgery to derive a surgical template model that represents the envisaged result. Computerized surgical planning approaches, however, are often still explorative, meaning that a surgeon tries to find a therapeutic concept based on his or her expertise using computational tools that are mimicking real procedures. Future perspectives of an improved computerized planning may be that surgical objectives will be generated algorithmically by employing mathematical modeling, simulation, and optimization techniques. Planning systems thus act as intelligent decision support systems. However, surgeons can still use the existing tools to vary the proposed approach, but they mainly focus on how to transfer objectives into reality. Such a development may result in a paradigm shift for future surgery planning.

Keywords

model-guided surgery - computer-aided planning - anthropometry - digital patient models - surgical template models - statistical shape models

Reconstructive facial surgery aims at the restoration of facial malformations caused by trauma or tumor-related morphological changes. In contrast, aesthetic or cosmetic surgery aims at the “improvement” of a facial appearance with respect to a particular objective. Facial surgery in cases of congenital dysmorphisms are somehow in between having no unique objective rather being oriented on establishing a natural function and a natural as well as harmonious facial appearance. Thus, we can distinguish between reconstruction and construction of a face. In the former case, the surgical intention is to restore a previous state as close as possible, and the latter case is focused on changing and improving a facial appearance with respect to some idealized target. Typical objectives are symmetry, facial proportions within a normal range, gender specific and cultural attributes, social trends, and many more. An objective could try to preserve individual characteristics and personality or to obtain a “new” face. Hence, a surgical procedure must be thoroughly planned in advance, where different options are tested, and possible outcomes are visually communicated to the patient in a highly conceivable manner to agree on a desired outcome. Within this article, computational planning approaches in cranio-maxillofacial surgery are reviewed. A survey on existing solutions is given and future perspectives of an improved computerized planning of facial surgery are presented.

Facial Surgery Planning—The Past and the Present

Preoperative preparation of a treatment concept for the surgical correction of craniofacial deformities requires a comprehensive knowledge about normally developed anatomical structures (anatomical atlases). Anthropometric and cephalometric studies provide helpful insights to normal proportions of the body or face and the individual (age related), inter-individual (e.g., gender), and inter-cultural variability.[1] [2] [3] For a successful reconstruction or establishment of normal function, taking a harmonious shape of the head or face into account while retaining individual characteristics, careful treatment planning using all available data and resources is essential ([Fig. 1]).

Fig. 1 Complex congenital craniofacial dysplasia (left) malocclusion of Angle class III, (center) hemifacial microsomia, (right) malocclusion of Angle class II.

Based on a graphical analysis of lateral cephalometric radiographs ([Fig. 2], center), first computerized planning tools were used in the 1970s. Repositioning of bony structures were performed in a facial profile view by exemption and displacement of two-dimensional (2D) image segments.[4] Since the mid-1980s, dedicated analysis and planning software was developed that enables a 2D planning based on digitized lateral cephalograms or cephalometric landmarks that are measured on a patient's head, simultaneously using empirically derived ratios to estimate a resulting soft-tissue profile.[5] With the availability of 2D image processing software, it became also possible to distort a profile photo according to a displacement of image features so that a possible postoperative outcome was visually communicated in a rough approximation. The validity of these methods, however, becomes highly limited with increasing complexity of changes of the facial skeleton, and hence the visual communication of a possible facial outcome becomes less convincing or even questionable.

Fig. 2 Facial profile analysis based on lateral cephalograms.

A well-recognized method for 3D planning of complex facial surgery—including osteotomies, mobilization, and relocation of bone segments—is based on life size resin models of the bony structures that can be produced by so-called rapid prototyping techniques from segmented computed tomography (CT) data (see [Fig. 3]). However, the effort and cost to create such models were rather high in the beginning. With the increasing availability of computerized 3D models and 3D rapid prototyping techniques, a quick and meanwhile also affordable way of manufacturing physical facsimiles exists.[6] Based on such physical representations, model surgery can be performed in a very intuitive way including haptics as well as assessment of access paths with respect to instrumentation.

Fig. 3 Model surgery based on life-sized resin models of the bony skull.

Although model surgery based on 3D resin models allows for an intuitive surgery planning, merely one treatment option can be evaluated with the help of a single model, because of the destructive nature of the planning process. To assess different therapeutic concepts, several models become necessary. Such explorative approaches are best performed using computer-aided planning (CAP) tools taking all relevant data and parameters into account. By using CAP methods, mistakes can be made undone, and one can start over and over again or reuse an existing plan as a starting point for an assessment of different strategies. CAP also allows to choose or to design proper instrumentation, as for instance cutting guides, splints, osteosynthesis plates, bone screws, augmentation implants, and so forth. Even soft tissues can be taken into account and the outcome of a facial appearance after model surgery can be simulated.

Since the early 1990s, for the aforementioned reasons, various research groups are engaged in the computerized 3D planning of craniofacial surgery.[7] [8] [9] [10] [11] [12] [13] The foundation of such a planning is tomographic image data. Roentgen CT or cone beam CT/DVT depicts bony structures with high structural detail. A complete head can be captured and digitized in less than a minute. However, the measurement is based on ionizing radiation; thus, it cannot be used extensively for preoperative planning and postoperative assessment. MRI, in contrast, allows a good classification of different soft-tissue structures and does not produce any ionizing radiation, though measurements are more expensive than CT and take considerably longer time. In addition, the spatial resolution of MRI is not as high as of CT and geometric distortions may occur, degrading the quantitative value of MRI measurements. Nevertheless, first attempts to employ MRI as bone imaging method for craniofacial surgery planning have been made.[14] Medical image data from different imaging modalities can also be combined to enrich the information. With advanced imaging or measuring techniques, as well as advanced algorithms for reconstructing the original object from such measurements, faithful and highly detailed digital 3D models of patients' heads can be generated. The aim is to enable surgeons to computationally plan complex surgical procedures based on such digital patient models (DPM) in an intuitive, reliable, and cost-effective manner.

Digital Patient Models

The fundamental prerequisite for a model-based 3D treatment planning in medicine is the faithful reconstruction of individual anatomical structures from medical image data. Such patient models must represent all relevant details, while unwanted imaging artifacts need to be reduced or eliminated within the reconstruction process. For CAP, such models need to be available in digital form for being three-dimensionally visualized on a computer display and for being interactively manipulated or modified with computerized tools. DPMs representing the bony anatomy are the basis for the planning of bone cuts and segment relocations in view of a functional rehabilitation. For simulation of mechanical processes or other biophysical phenomena, adequate volumetric representations of the relevant anatomical structures are required. For the simulation of soft-tissue deformation being induced by changes of the facial skeleton, for instance, a spatial grid needs to be generated between skin and bone surface ([Fig. 4]), being the basis for a finite element analysis.[15] Such models must have an appropriate discretization and geometric quality to become suited for a numerical solution of partial differential equations (PDEs), as for instance in accordance with the 3D theory of elasticity.[16]

Fig. 4 Digital patient model derived from tomographic image data.

Computer graphic renderings of such DPMs can also be used for visual communication and patient information in the same way as 2D photographs are being used ([Fig. 4]). In addition to tomography, stereo-photogrammetry allows to measure the outer facial morphology in high detail together with an acquisition of color images depicting photographic details of the skin ([Fig. 5]). Combining tomographic and photogrammetric imaging enables us to three-dimensionally reconstruct the head and face including a photo-realistic facial appearance.

Fig. 5 Visualizations of computerized 3D models of the face, high-resolution geometry and photographic texture acquired with our stereo-photogrammetric setup.

Under the assumption that measurements will become increasingly precise, less invasive, as well as easy and cheap to accomplish in the future, and faithful 3D models can be automatically generated out of those measurements, such models will become an indispensable part of a model-based planning approach that is very much likely to be fully integrated into surgical planning workflows. Latest research even goes in the direction of 3D reconstruction of anatomical structures from a single 2D radiograph,[17] which—among others—might be beneficial for an enhanced cephalometric analysis based on lateral cephalograms.

Facial Proportions

Based on individual 3D patient models, a qualitative evaluation of facial symmetry and facial profile can be performed by means of computer graphics visualization of bone and skin surfaces. The possibility of an arbitrary orientation of the patient model in any three-dimensional (3D) coordinate system allows for a cephalometric analysis in a standardized manner, in profile view, en-face, or in 3D space.[18] Deviations from ideal proportions can be easily identified and quantitatively evaluated ([Fig. 6], left). By means of anatomical landmarks that can be defined on top of the 3D model and reference planes (midsagittal, Frankfort horizontal, occlusal, etc.), which can be constructed thereof, linear and angular measurements are possible that allow for an assessment and a quantitative analysis of facial symmetry and proportions ([Fig. 6], right).

Fig. 6 Computerized profile and en-face analysis based on digital 3D patient models.

Computerized Osteotomy Planning

By knowing the deviation of an individual patient's skull or face from a desired objective, precise specifications for a surgical correction can be computed. Given those quantitative specifications, the mobilization of bone segments and the determination of displacement vectors and/or rotation parameters for each mobilized bone segment can be computationally derived.[19]

A well-established method for planning complex correctional osteotomies with subsequent relocation of mobilized bone segments is to draw osteotomy lines on top of a life size resin model of the patient's skull and to cut this model accordingly ([Fig. 3] and [Fig. 7], top left). To become an accepted planning tool in clinical practice, a computerized 3D planning must be performed in a similar and intuitive manner. Hence, a reasonable computerized planning approach must allow to draw osteotomy lines on top of a DPM of the skull, while the model can be freely rotated. Such osteotomy planning can be easily achieved using dedicated input devices such as stylus and a touch-sensitive display as provided with modern tablet computers ([Fig. 8]).[20] An even more advanced approach automatically projects osteotomy lines for standardized procedures onto the respective skull model. Proposed osteotomy lines can afterward be interactively modified to meet individual requirements.

Fig. 7 Osteotomy planning based on a resin model, and on digital 3D skull models, (a) conventional, (b) high, and (c) quadrangular Le Fort I osteotomy.

Fig. 8 Intuitive osteotomy planning based on a computerized model.[20]

In [Fig. 7], an example of the planning of a bimaxillary osteotomy for combined upper and lower jaw displacement is depicted. After osteotomy lines have been defined, the resulting, arbitrarily shaped cut surfaces are automatically generated from the drawn contours and visualized, revealing the course of the cut within the bone.[20] On top of these surfaces, the original image data can be visualized ([Fig. 7], bottom). That way, the planned osteotomy can be assessed with respect to internal structures, such as nerves, vessels, roots of teeth, and so forth, without having those structures explicitly segmented from medical image data.

Orthognathic Surgery—Remodeling the Maxillofacial Skeleton

Since anatomical planes or local coordinate systems can be easily derived from anatomical landmarks, geometric deviations of a patient's anatomy with respect to symmetry can be assessed. Mirroring of parts of a healthy contralateral side may serve as a target for the adjustment of the position and orientation of malpositioned or malformed bony structures. Mobilized segments of the digital skull model can be freely positioned to assess a modified facial skeleton with respect to natural proportions, symmetry, functional rehabilitation,[19] as well as a harmonious facial appearance under consideration of facial soft tissue.[21] [22] The spatial transformation of mobilized bone segments can be interactively applied via standard techniques of 3D computer graphics and human–computer interaction (HCI). Parts can be translated in 3D space or rotated around a predefined center or axis ([Fig. 9]). Transformations can be limited to particular degrees of freedom as well as analyzed regarding colliding adjacent structures, or even be restricted via collision prevention techniques ([Fig. 10]). Based on a computerized approach, various osteotomies with segment relocations can be assessed in view of a resulting spatial arrangement of the facial skeleton.

Fig. 9 Relocation of mobilized bone segments. Maxillary advancement and rotation (top), maxillary advancement and mandibular setback (bottom).

Fig. 10 Analysis of dental occlusion for orthodontic treatment planning.[25]

During the preoperative planning phase, it is crucial that a surgeon has all options to modify the model at his or her own will. Helpful tools may simplify the separation of parts, spatial alignment, and measurement to assist in the task of an explorative but goal-oriented design of a regular facial skeleton. At any time, a modified skull is vividly visualized in three dimensions and can be interactively rotated to understand complex spatial relationships. As a specialty of computerized planning, parts of the DPM can be removed or visualized in a semitransparent manner to reveal hidden structures to assess a new arrangement and to identify possible complications that may result from a modification. By actively performing these remodeling steps, a surgeon already becomes better prepared.

Orthodontics—Dentition and Dental Occlusion

Surgical and orthodontic treatment planning belong together in maxillofacial surgery to achieve an optimal functional and aesthetically pleasing result.[23] Teeth need to be reconfigured to achieve a proper dental occlusion in accordance with the new skeletal arrangement and especially the incisors have an impact on the facial appearance. Hence, the computational planning of orthodontic treatments and its integration into facial surgery planning concepts is of importance.[24]

An orthodontist's primary goal is to achieve/obtain a proper dental occlusion based on a sufficient dentition. Although the occlusal plane can be roughly determined from tomographic data, detailed geometric information about dental fissures and occlusal surfaces cannot be derived from tomography. A combination of tomographic data and digitized dental casts enables us to integrate a highly detailed model of the dentition into the DPM[25] ([Fig. 10], left). The combined model can then be used for a spatial rearrangement of jaw segments allowing an improved geometric analysis of tooth contacts ([Fig. 10], right). The contact zones, as shown in [Fig. 10], are useful indicators for planning an orthodontic treatment together with the orthognathic surgery to consider normal dental occlusion.

Because dental occlusion is such an important factor for functional rehabilitation, advanced CAP systems for orthognathic surgery planning must provide a computerized analogue to a dental articulator as shown in [Fig. 3]. Otherwise, orthodontic planning needs to be done in a conservative manner based on dental casts, and the parameters of the articulator must be transferred into the CAP system for a respective adjustment of the jaw segments of the DPM.[26] Even more advanced CAP systems for orthodontic treatment planning could automatically propose proper positions for the jaw segments, or even individual teeth. Using a mathematical modeling, simulation, and optimization (MSO) approach, they could also provide decision support for an optimal bracket design, taking forces and moments into account that are acting on the respective configuration of the teeth.

By assessing the modified DPM, a decision for an appropriate surgical procedure can be derived within the preoperative planning phase. The surgical template model (STM) serves as a kind of blueprint for the envisaged surgical result. An STM may also serve as a basis for the design of surgical splints, surgical cutting guides, or any other type of customized instrumentation.[27]

Osteosynthesis and Implant Design

The STM—that is, the altered DPM—represents the planned facial skeleton as it is supposed to be after surgery. That way it is an ideal basis for a preoperative selection and positioning of osteosynthesis plates up to the fully customized design of osteosyntheses[28] [29] or other implants that will secure the desired positions of mobilized bone segments for stable fusion and proper bone healing ([Fig. 11], left). Meanwhile, commercial software and services become available for customized implant design and patient-specific instrumentation that are currently under evaluation by clinical research groups.[30]

Fig. 11 Computerized design of osteosyntheses or augmentation implants on top of an STM.

Besides a design or configuration of osteosyntheses, an STM can also be used for the selection or design of bone augmentations.[31] The optimal shape of an augmentation implant is given by the difference between the targeted and the actual shape of the bone. A planned maxillary advancement, for instance, leads to an STM that differs locally from the PDM. The volumetric difference of the two models in that region yields an estimate for an augmentation implant. Such implants need to have a good fit to the bony support; thus, the DPM can be seen as a local molding or casting form ([Fig. 11], center and right). On the soft tissue side, such implants need to be smooth without introducing unwanted edges. Since the shape of the implant will determine the facial appearance, its shape design is of utmost importance.

An STM can easily be manufactured in 3D by rapid prototyping techniques, serving as a physical template afterward, on which osteosyntheses can be preconfigured prior to surgery.[32] [33] In addition, a physical STM can be sterilized and taken into the operating room as shown in [Fig. 12], or it can be used for a demonstrative communication to the patient. Although a physical representation of an STM is already of practical value for surgeons, a digital representation gives even more flexibility for its subsequent use in CAP systems.

Fig. 12 An individual STM for intraoperative shaping and rearrangement of cranial bone segments.[33] (Top left) preoperative situation, (top right) postoperative situation.

Computational Forecast of a Facial Appearance

Osteotomies and bone segment relocations primarily have a functional rehabilitation goal. In cases where several bone segments are to be arranged in relation to each other or different treatment options are conceivable, the expected aesthetic result becomes an additional important criterion that should be considered in the planning. Since the aim of facial surgery is to unite functional reconstruction with a harmonious, aesthetically pleasing facial profile, the modified facial skeleton of the DPM—that is, the STM—may serve as the basis for further analysis of a resulting facial appearance with respect to facial soft tissues.[34] [35] [36]

In case the facial soft tissue volume can be manufactured out of flexible material where its shape can be derived from the preoperative CT data, then a physical instance of the STM could be coated by this flexible tissue mask. In case the material behaves similar to facial tissue, the STM together with the mask will give an estimate of the resulting facial appearance. A less costly method would be to use a digital STM and to simulate the implications of changes of the facial skeleton in view of the resulting facial appearance. Since the aesthetic outcome also influences the therapeutic concept, a direct simulation of facial soft-tissue deformation within the interactive process of remodeling the facial skeleton would be the most efficient planning approach.

In [Fig. 13], for instance, a correction of dysgnathia with simultaneous maxillary advancement and mandibular setback is shown. Theoretically an arbitrary number of combinations exist, each leading to a proper dental occlusion with neutral bite. A simulation of the resulting arrangement of facial soft tissues and thus the facial appearance allows an assessment of the planning from an aesthetic point of view.

Fig. 13 Computerized planning of a bimaxillary osteotomy with maxillary advancement and mandibular setback including a simulation of the resulting facial appearance.

Basis for a reliable forecast of the soft-tissue arrangement with respect to the modified DPM is an adequate geometric model of the soft-tissue volume with all embedded structures on the one hand, and on the other hand a physical deformation model that describes the mechanical properties of biological soft tissue in close approximation.[37] The latter is based on the theory of elasticity, which is used in many engineering disciplines for stress analysis. The mathematical calculation of deformation is based on the given displacements of bony structures by means of the finite element method.[38] Taking the displacements as boundary conditions, the respective system of PDEs is solved on the entire volumetric mesh that represents the facial soft tissue. In contrast to approaches that take into account heuristically determined, local displacement ratios for soft tissue, a simulation based on continuum mechanics yields a deformation for any point on the facial surface. Such a computation can be performed on conventional computers in a few minutes.

In [Fig. 14], a DPM of a patient with dysgnathia of Angle class III is shown. There are different options of surgical correction varying from maxillary advancement, mandibular setback, or a combination of the two. For the patient on whom bimaxillary osteotomy was planned, the DPM was modified accordingly, and facial outcomes were tested by full advancement of the maxilla, a full setback of the mandibular segment, as well as a gradual advancement and setback of maxilla and mandible, respectively, considering dental occlusion.[21] Each resulting STM serves as input for the soft-tissue simulation. Single or combined bone segment relocations can be continuously adjusted and the resulting effect on the facial appearance can be displayed in three dimensions from any viewing angle. The simulation can even be visualized in photorealistic quality using photographic textures as shown in [Fig. 14]. Animated image sequences of the simulation contribute in a far more intuitive enlightenment of a patient, as it was previously the case in preoperative patient information.

Fig. 14 DPM showing the preoperative situation (left) simulation of maxillary advancement and mandibular setback based on the modified DPM.

Intraoperative Transfer of the Plan

Having planned facial surgery and having derived a therapeutic concept, the main challenge that needs to be addressed is how to surgically achieve the planned result: that is, how to exactly reproduce osteotomies and bone segment relocations on the patient within the operating room?

To reproduce the planned osteotomies, individual or navigated cutting guides of very precise laser or piezoelectric osteotomies are conceivable.[39] [40] For the accurate reproduction of the planned relocation of mobilized bone segments, different approaches have been tested. One is the design and the fabrication of individualized surgical splints.[41] [42] Such splints help in adjusting mobilized bone segments (such as mandible and maxilla) to each other or in adjusting mobilized segments to unaffected regions of the skull base. Another approach is to use the relative transformations of mobilized bone segments resulting from the plan and to reproduce these transformations in a registered coordinate system of the patient using navigation techniques. The latter method leads to a fully computerized planning approach, thus avoiding the production of surgical splints.

Another conceivable approach is to register the coordinate systems of DPM and STM and to generate an artificial tomograph from the STM that contains the modified facial skeleton instead of the original one. This artificial tomograph can be stored in DICOM format and both the original and the artificial tomographs can be visualized as overlay. Planned transformation parameters can be dissolved in particular dimensions, directions, or angles for each mobilized bone segment, serving as input for navigated surgery.[43] Using modern navigation systems, mobilized bone segments can be tracked in real time and continuously updated positions can be assessed via the overlay visualization ([Fig. 15]). Proper relocation is achieved when the planned and the real positions do match exactly. The artificial tomograph thus serves as a target view for positioning mobilized bone segments.

Fig. 15 Combined visualization of DPM and STM (left), discrepancy between actual bony configuration after trauma and the planned one (center), overlay view within navigation system (right).[43]

New Concepts of Facial Surgery Planning—Future Perspectives

As mentioned in the previous section, computer-aided surgery planning is currently based on computerized tools and a rather explorative approach mimicking real surgical procedures, such as osteotomies, segment relocations, virtual articulation, osteosyntheses, and so forth, accompanied or followed by a simulation of the resulting facial appearance to assess the surgical plan from both a functional and an aesthetic point of view. Using mathematical modeling, simulation, and optimization (MSO), as well as population-based knowledge about morphologies and pathologies in combination with advanced statistical methods, surgical objectives could automatically be derived from computational models and applied to a DPM. Based on such a proposal, the resulting STM can still be modified by a surgeon with the help of the aforementioned computerized tools. Thus, planning procedures would change from a manual and explorative approach to a more automated, goal-oriented one. Employing MSO techniques and advanced statistical methods for 3D shape analysis may result in a paradigm shift in facial surgery planning.

Employment of Statistical Shape Analysis

Over a longer period of computational planning in facial surgery, a large variety of anatomical structures have been digitized. The resulting geometric models (DPMs and STMs) can be parametrized in a consistent and corresponding manner. That way, it becomes possible to compute the average of all these shapes as well as to statistically analyze their geometric variation.[44] [45] Via mathematical shape analysis, a high dimensional and thus complex variation in shape can be reduced to a much smaller number of parameters determining the principal modes of variation. Within the resulting shape space (morphospace) of a statistical shape model, an interpolation and thus a morphing between the shapes becomes possible. Assuming a consistent common parametrization of shapes of a particular anatomy, a weighted combination of all shape parameters allows to generate an infinite number of plausible shapes thereof. The more representative the chosen training data are, the more general the resulting statistical 3D shape models will be. Models of this type have been developed for several structures of the bony craniofacial anatomy, as for instance the eye socket,[46] the mandible,[47] [48] the neurocranium,[49] and the midface[43] ([Fig. 16]).

Fig. 16 Statistical shape models of craniofacial anatomy (neurocranium, orbit, midface, mandible).

Based on the data that are captured with our stereo-photogrammetric setup ([Fig. 5]), a statistical 3D shape model of the human face is currently in preparation.[50] Currently, approximately 100 faces have been integrated into our statistical 3D shape model of the face ([Fig. 17]). This model is analyzed in view of population-based facial characteristics that may serve as an objective for a classification of individual types of facial morphology or any individual deviation from normal proportions.

Fig. 17 Statistical shape model of facial skin morphology (Columns: 1st three modes of facial variation showing characteristic types of faces within a set of captured faces. The middle row depicts the average face, top and bottom rows depict the range of variation along the resp. mode.)

A combined statistical 3D shape model of the skull and the face is our envisaged goal. Such a model could be used for correlation analysis between the shape of the skull and the shape of the face to study and classify dysmorphisms or pathologically developed structures. Such a model would also be a very valuable tool in forensic medicine.

A statistical shape model enables us to investigate shape characteristics via multivariate statistical analysis and to determine sets of deformation parameters. With an increasing amount of shape data, population-based studies will become possible and variation in shape can be analyzed with respect to gender, age, body mass, or other demographic information. Owing to a normally distributed variation in shape in a large population, it is not surprising that the averaged shape of a very large set of randomly selected anatomical structures such as skulls or faces represents a kind of normally developed anatomy. Normality relates to the average and to the variation within certain limits around that average. Thus, statistical 3D shape models of anatomical structures may become the foundation for advanced anatomical atlases that not only represent an average shape per anatomical structure but also its normal or even pathological variation in shape. Such atlases are only accessible in their full power in a digital form. However, representatives of anatomical structures can always be manufactured.

Since statistical shape models can be analyzed with respect to demographic attributes, a population-based analysis may be introduced within the planning process. From such an analysis, objectives for the alteration of an individual head or face can be derived, which will become input for a subsequent therapy planning. With the availability of representative, normally pronounced reference anatomies in the form of statistical 3D shape models that can be adapted to individual anatomical landmarks, it becomes possible to generate therapeutic proposals even for highly asymmetric malformations and severe disfigurations,[47] up to the design of missing parts or structures that need to be fully reconstructed, for instance facial epitheses for the nose, eye, or ear.[51] A shape-based classification is also conceivable for an early detection of craniofacial syndromes when applied to regularly acquired stereo-photogrammetric measurements of infant heads, for example, in routinely performed well-child visits.

Instead of testing outcomes in an explorative manner by a modification of a DPM and subsequent assessment of simulations that are driven by such modifications, the planning procedure could start with a facial target that might be derived from anthropometric analysis based on an individualized statistical shape model of the face. Having a desired facial outcome as an objective, surgeon and patient will agree on, the remaining task is to determine the appropriate surgical procedure that leads to the envisaged result. There are already mathematical methods on algorithmic determination of how an underlying skull surface needs to be modified to achieve a desired outcome of a facial appearance.[52] In facial surgery, for instance, it could either be the modification of the facial skeleton or the design of augmentation implants or a combination of the two to alter the appearance of the face. Having a proposal for a modified DPM, a planning approach reduces to the task of deciding how the proposed alteration can be achieved surgically.

Facial Expressions

An important issue that has only marginally been addressed in facial surgery planning so far is the consideration of facial expressions, which are affected by a surgical modification of a face. In surgery planning, it is important to consider and to communicate the postoperative facial appearance of a patient for varying facial expressions. First attempts to simulate facial expressions by applying mechanistic approaches based on the concept of facial action units that has been proposed by Paul Ekman and Wallace Friesen[53] did not lead to convincing results.[54] [55] Using advanced photogrammetric measurement methods in combination with high detail surface reconstruction, as shown in [Fig. 5], faces of different persons with varying facial expressions can be captured and respective statistical 3D shape and appearance models of the human face can be established from such measurements.

A statistical analysis of intra- or inter-individual facial morphology under different facial expressions enables us to identify clusters of similar expressions from a set of measurements of an individual person or a larger cohort and to derive characteristic facial deformation patterns. These deformation patterns can be classified and main modes of facial variation can be statistically extracted and transferred to an individual DPM ([Fig. 18]). That way it becomes possible to assess a facial appearance for a surgically altered DPM in view of different facial expressions. Considering patients with facial palsy, where facial muscles are not properly innervated, simulation and assessment of individual facial expressions may help in establishing a suitable therapeutic concept for muscle re-innervation to achieve, for instance, a natural smile.[56]

Fig. 18 Different facial expressions synthesized via a statistical 3D shape and appearance model.

Craniofacial Growth

An additional and also important issue that needs to be considered in craniofacial surgery is growth. When congenital dysmorphisms of the head and face are to be treated surgically, growth must be taken into account. Growth can be statistically analyzed with respect to its morphometric effects. A statistical analysis of morphometric measurements of skulls and faces in the ages between birth and adolescence yield that the most significant variation in shape is affected by growth. However, growth is not just a linear increase of size ([Fig. 19]). Instead it depends on many factors and occurs in age intervals.[57] [58] Different parts of the human body grow with different velocity starting on different time points. Thus, one cannot simply scale an infant's skull linearly between birth and adolescence to forecast its three-dimensional shape. The final shape of an anatomical structure is genetically predetermined. Thus, a thorough morphological analysis of anatomical growth and the resulting shapes has to be performed. In the following, two examples are given that are influenced by growth: the reshaping of infant skulls in cases of craniosynostosis[33] ([Fig. 12]) as well as the surgical correction of dysgnathia for juvenile patients[22] ([Fig. 1], right).

Fig. 19 Nonlinear morph of aging faces.[58]

For a surgical treatment of craniosynostosis—that is, the reshaping of the neurocranium—it is quite obvious that an ongoing increase in size needs to be taken into account.[59] This also holds—and is even more obvious—in cases of helmet therapies, where helmets are individually designed in such a way that growth is limited in certain directions but desired in others. The success of both therapies will strongly benefit if we know where and when regions of the neurocranium increase in size and to what extent. In addition, it is also beneficial to know how growth locally affects the shape of the neurocranium. A statistical 3D shape analysis gives much more detailed information about cranial growth than just measurements of the circumference of the head. Understanding growth and development in a better way enables us to provide improved therapy plans that, in the case of treating craniosynostosis, take a natural increase in volume and the respective change in shape into account.

For juvenile patients with severe congenital dysmorphisms of the jaws, where chewing, swallowing, and even breathing are negatively affected, a surgical intervention might be indicated. However, mandibular or maxillary osteotomies with pronounced relocation of jaw segments have a negative effect on later growth. Thus, different therapeutic strategies, such as distraction osteogenesis, are better suited.[60] Within the preoperative planning phase, not only suitable osteotomies are to be determined but also positions and orientations of the distraction devices to achieve proper distraction vectors. These vectors (directions) of appropriate lengths (distance) determine the resulting shape of the respective anatomical structure. However, the final shape will also be influenced by ongoing growth that needs to be considered within the therapeutic concept, too.

Hence, results of the analysis of growth patterns need to be taken into account for craniofacial surgery planning. After the respective parameters of a statistical 3D shape model have been identified, they can be isolated and transferred to an individual model. A variation of this parameter will then create characteristic growth effects in combination with an individual patient's craniofacial morphology.

Facial Aging

The facial appearance is also influenced by aging. Especially in cosmetic facial surgery, undesired aging effects such as wrinkles up to deep folds, lacrimalis, loss of lip volume, and sagging, are subject to treatment. The typical objective is to establish a facial appearance with a juvenile look. Therapies range from the application of fillers or botulinum toxin to surgical “face lifts” to strengthen and firm facial soft tissues.

There are mechanistic approaches to simulate skin aging;[61] [62] [63] however, such methods are neither applicable for an accurate forecast of an individual patient's appearance nor are they for any kind of treatment planning. Instead, statistical methods may become important in the future when skin aging effects can be quantified using high detail reconstruction of stereo-photogrammetric measurements ([Fig. 5]). Similar to the aforementioned future perspectives, aging effects can also be statistically analyzed with respect to geometric effects. If the respective parameters of the statistical shape model are identified, they can again be isolated and transferred to an individual face. A variation of this parameter will then create characteristic aging effects, such as wrinkles and folds to communicate a possible aging process. Analogously, such a model can also invert this process, demonstrating how a person might look younger. By quantitative analysis of changes that result from the variation of age-related shape parameters, therapeutic proposals can be derived.

In addition to an age-related alteration of geometric features, skin also exhibits increasing signs of aging due to pigmental moles leading to an inhomogeneous, unevenly toned appearance of the skin. Hence, a combined statistical shape and appearance model is developed that covers the geometry and the image of the facial surface. The averaged appearance of the skin results in a smooth and even-toned image just as the averaged shape does with respect to roughness or geometric texture. Both characteristics can be analyzed independently, although both are correlated with age. A variation of age-related parameters of the individualized statistical shape and appearance model communicates an individual aging or rejuvenation process—a concept that is, although still under research, conceivable with advanced CAP tools.

Conclusion

Computerized facial surgery planning including soft-tissue simulation has been developed at Zuse Institute Berlin (ZIB) since 1999 and was successfully applied in more than 50 cases in cooperations with clinics in Munich, Basel, Leipzig, Erlangen, Vienna, Hannover, Berlin, Stockholm, and Zurich. Predominantly, malformations of the jaws and highly asymmetric malformations of the facial skeleton were treated using our planning approaches.

A comparison of the profile line of simulation results with postoperative photographs already shows a very good correlation between the planning and the simulation results that have been achieved ([Fig. 20]). 3D comparisons of forecasts with postoperative CT data also demonstrate a very good correlation and thus a prediction capability ([Fig. 21]).[37] [64]

Fig. 20 Comparison of the simulated facial profile to photographs.

Fig. 21 Comparison of the simulated facial appearance to postoperative CT data.

Some Final Words on Planning

Model-based planning helps a lot in the preparation of complex surgeries, where insufficient routine would end up in a higher risk of failure. Planning in general assists in structured thinking. Drawing or interactive computer-based practicing—that is, manipulation of computerized models in surgery simulation—teaches much more than just mental preparation. A descriptive visualization helps in memorizing things and also in communicating therapeutic strategies to the patient, which is beneficial for motivation and compliance. Computer-aided design, modeling, simulation, and optimization, all being common practice in mechanical engineering, will become common practice in surgery in the near future.

Acknowledgments

Our research in facial surgery planning was mainly funded by Zuse Institute Berlin. Aspects of our research were funded by DFN CoDisp (grant # TK 602 NT 202.2), Medical University of Vienna, Klinik Professor Sailer, DFG Research Center Matheon, and 1000shapes GmbH. Our research in studying facial morphology is funded by the DFG cluster of excellence “Bild-Wissen-Gestaltung,” Special thanks go to all cooperating surgeons and especially to Prof. Zeilhofer (University Hospital Basel), Prof. Sader (University Hospital Frankfurt), Priv. Doz. Dr. Westermark (formerly at Karolinska Hospital Stockholm), and Priv. Doz. Dr. Haberl (University Hospital Ulm, formerly at Charité Berlin), as well as to all patients who have given their explicit consent to the publication of their images.

• References

• 1 Powell N, Humphreys B. Proportions of the Aesthetic Face. Thieme Medical Publishers; 1984
• 2 Steinhäuser EW. Proportionen des ästhetischen Gesichts im Vergleich zur bildenden Kunst. In: Fortschritte in der Kiefer- und Gesichtschirurgie; Thieme Medical Publishers; 1986: 1-4
• 3 Farkas LG. Anthropometry of the Head and Face. 2nd ed. New York: Raven Press; 1994
• 4 Ricketts RM, Bench RW, Hilgers JJ, Schulhof R. An overview of computerized cephalometrics. Am J Orthod 1972; 61 (1) 1-28
• 5 Landes CA, Zachar R, Diehl T, Kovács AF. Introduction of a three-dimensional anthropometry of the viscerocranium. Part II: evaluating osseous and soft tissue changes following orthognathic surgery. J Craniomaxillofac Surg 2002; 30 (1) 25-34
• 6 Bibb R, Eggbeer D, Paterson A , eds. Physical reproduction. In: Medical Modelling—The Application of Advanced Design and Rapid Prototyping Techniques in Medicine. 2nd ed. Woodhead Publishing Series in Biomaterials. New York: Elsevier Ltd.; 2015: 65-98
• 7 Cutting C, Bookstein FL, Grayson B, Fellingham L, McCarthy JG. Three-dimensional computer-assisted design of craniofacial surgical procedures: optimization and interaction with cephalometric and CT-based models. Plast Reconstr Surg 1986; 77 (6) 877-887
• 8 Yasuda T, Hashimoto Y, Yokoi S, Toriwaki JI. Computer system for craniofacial surgical planning based on CT images. IEEE Trans Med Imaging 1990; 9 (3) 270-280
• 9 Altobelli DE, Kikinis R, Mulliken JB, Cline H, Lorensen W, Jolesz F. Computer-assisted three-dimensional planning in craniofacial surgery. Plast Reconstr Surg 1993; 92 (4) 576-585 , discussion 586–587
• 10 Keeve E. Visualisierungs- und Simulationsverfahren zur interaktiven Planung kraniofazialer Korrekturoperationen. PhD Dissertation, Friedrich-Alexander-Universität Erlangen-Nürnberg; 1996
• 11 Sarti A, Gori R, Lamberti C. A physically based model to simulate maxillofacial surgery from 3D CT images. Future Gener Comput Syst 1999; 15 (2) 217-221
• 12 Koch RM . Methods for Physics Based Facial Surgery Prediction. PhD. Dissertation No. 13912, ETH Zürich; 2000
• 13 Zachow S, Gladilin E, Zeilhofer HF, Sader R. Improved 3D osteotomy planning in cranio-maxillofacial surgery. Medical Image Computing and Computer-Assisted Intervention—MICCAI 2001. Lecture Notes in Computer Science 2001; 2208: 473-481
• 14 Eley KA, Watt-Smith SR, Sheerin F, Golding SJ. “Black Bone” MRI: a potential alternative to CT with three-dimensional reconstruction of the craniofacial skeleton in the diagnosis of craniosynostosis. Eur Radiol 2014; 24 (10) 2417-2426
• 15 Zachow S, Zilske M, Hege HC. 3D reconstruction of individual anatomy from medical image data: segmentation and geometry processing. ZIB Report 07–41, 2007. Available at: https://opus4.kobv.de/opus4-zib/frontdoor/index/index/docId/1044
• 16 Deuflhard P. Differential equations in technology and medicine: computational concepts, adaptive algorithms, and virtual labs. In: Capasso et al., eds. Computational Mathematics Driven by Industrial Problems, Springer, Lecture Notes in Mathematics; 2006. 1739. 69-126 . doi:10.1007/BFb0103918
• 17 Ehlke M, Ramm H, Lamecker H, Hege HC, Zachow S. Fast generation of virtual X-ray images for reconstruction of 3D anatomy. IEEE Trans Vis Comput Graph 2013; 19 (12) 2673-2682
• 18 Swennen GRJ, Schutyser F, Hausamen JE , eds. Three-Dimensional Cephalometry—A Color Atlas and Manual. New York: Springer; 2005. . doi:10.1007/3-540-29011-7
• 19 Zinser MJ, Zachow S, Sailer HF. Bimaxillary ‘rotation advancement’ procedures in patients with obstructive sleep apnea: a 3-dimensional airway analysis of morphological changes. Int J Oral Maxillofac Surg 2013; 42 (5) 569-578
• 20 Zachow S, Gladilin E, Sader R, Zeilhofer HF. Draw & Cut: Intuitive 3D Osteotomy Planning on Polygonal Bone Models. Computer Assisted Radiology and Surgery 2003. Elsevier, Int. Congress Series, 2003;1256:362–69. doi:10.1016/S0531-5131(03)00272-3
• 21 Westermark A, Zachow S, Eppley B. Three-dimensional osteotomy planning in maxillofacial surgery including soft tissue prediction. J Craniofac Surg 2005; 16 (1) 100-104
• 22 Zachow S, Hege HC, Deuflhard P. Computer assisted planning in cranio-maxillofacial surgery. CIT J Comput Inf Technol 2006; 14 (1) 53-64
• 23 Bibb R, Eggbeer D, Paterson A , eds. Surgical applications case study 5. In: Medical Modelling—The Application of Advanced Design and Rapid Prototyping Techniques in Medicine. 2nd ed. Woodhead Publishing Series in Biomaterials, New York: Elsevier Ltd.; 2015: 167-172
• 24 Nadjmi N, Mollemans W, Daelemans A, Van Hemelen G, Schutyser F, Bergé S. Virtual occlusion in planning orthognathic surgical procedures. Int J Oral Maxillofac Surg 2010; 39 (5) 457-462
• 25 Nkenke E, Zachow S, Benz M , et al. Fusion of computed tomography data and optical 3D images of the dentition for streak artefact correction in the simulation of orthognathic surgery. Dentomaxillofac Radiol 2004; 33 (4) 226-232
• 26 Chapuis J, Zachow S, Langlotz F, Schramm A. Device for Planning Orthodontics and/or Orthognathic Surgery. 2008. . Patent WO/2008/080235, PCT/CH2007/00004, A61C 7/00 (2006.01)
• 27 Bibb R, Eggbeer D, Paterson A , eds. Surgical applications case study 3. In: Medical Modelling—The Application of Advanced Design and Rapid Prototyping Techniques in Medicine. 2nd ed. Woodhead Publishing Series in Biomaterials, New York: Elsevier Ltd.; 2015: 145-154
• 28 Pereira C, Ventura F, Gaspar MC, Fontes R, Mateus A. Customized implant development for maxillo-mandibular reconstruction. In: Bártolo et al., eds. Virtual and Rapid Manufacturing. Philadelphia: Taylor & Francis; 2008: 159-166
• 29 Cornelius CP, Smolka W, Giessler GA, Wilde F, Probst FA. Patient-specific reconstruction plates are the missing link in computer-assisted mandibular reconstruction: a showcase for technical description. J Craniomaxillofac Surg 2015; 43 (5) 624-629
• 30 Wilde F, Hanken H, Probst F, Schramm A, Heiland M, Cornelius CP. Multicenter study on the use of patient-specific CAD/CAM reconstruction plates for mandibular reconstruction. Int J CARS 2015;
• 31 Guevara Rojas G, Figl M, Schicho K , et al. Patient specific PEEK facial implants in a computer-aided planning workflow. In preparation for Journal of Oral and Maxillofacial Surgery, available at ResearchGate: http://www.researchgate.net/publication/260217023 , 2015
• 32 Haberl H, Hell B, Zöckler MJ , et al. Technical aspects and results of surgery for craniosynostosis. Zentralbl Neurochir 2004; 65 (2) 65-74
• 33 Zachow S, Lamecker H, Zöckler M, Haberl EJ . Computergestützte Planung zur chirurgischen Korrektur von frühkindlichen Schädelfehlbildungen (Craniosynostosen). Face 02/09, Int. Mag. of Orofacial Esthetics, Oemus Journale Leipzig, 2009:48–53
• 34 Zachow S . Computergestützte 3D Osteotomieplanung unter Berücksichtigung der räumlichen Weichgewebeanordnung (in German). (English translation: Computer assisted osteotomy planning in cranio-maxillofacial surgery under consideration of facial soft tissue changes.) PhD Dissertation, TU Berlin, Germany; 2005
• 35 Mollemans W, Schutyser F, Nadjmi N, Maes F, Suetens P. Predicting soft tissue deformations for a maxillofacial surgery planning system: from computational strategies to a complete clinical validation. Med Image Anal 2007; 11 (3) 282-301
• 36 Swennen GR, Mollemans W, Schutyser F. Three-dimensional treatment planning of orthognathic surgery in the era of virtual imaging. J Oral Maxillofac Surg 2009; 67 (10) 2080-2092
• 37 Zachow S, Weiser M, Hege HC, Deuflhard P. Soft tissue prediction in computer-assisted maxillofacial surgery planning. In: Payan, ed. Biomechanics Applied to Computer-Assisted Surgery. Ontario, Canada: Research Signpost Publisher; 2005: 277-298
• 38 Deuflhard P, Weiser M, Zachow S. Mathematics in facial surgery. AMS Notices 2006; 53 (9) 1012-1016
• 39 Baek KW, Deibel W, Marinov D , et al. A comparative investigation of bone surface after cutting with mechanical tools and Er:YAG laser. Lasers Surg Med 2015; 47 (5) 426-432
• 40 Hennet P. Piezoelectric bone surgery: A review of the literature and potential applications in veterinary oromaxillofacial surgery. Front Vet Sci 2015; . Available at: http://journal.frontiersin.org/article/10.3389/fvets.2015.00008/full
• 41 Hernández-Alfaro F, Guijarro-Martínez R. New protocol for three-dimensional surgical planning and CAD/CAM splint generation in orthognathic surgery: an in vitro and in vivo study. Int J Oral Maxillofac Surg 2013; 42 (12) 1547-1556
• 42 Zinser MJ, Sailer HF, Ritter L, Braumann B, Maegele M, Zöller JE. A paradigm shift in orthognathic surgery? A comparison of navigation, computer-aided designed/computer-aided manufactured splints, and “classic” intermaxillary splints to surgical transfer of virtual orthognathic planning. J Oral Maxillofac Surg 2013; 71 (12) 2151.e1-2151.e21
• 43 Zachow S, Kubiack K, Malinowski J, Lamecker H, Essig H, Gellrich NC. Modellgestützte chirurgische Rekonstruktion komplexer Mittelgesichtsfrakturen. Proc. BMT Biomed Tech 2010; 55 (Suppl. 01) 107-108
• 44 Lamecker H . Variational and Statistical Shape Modeling for 3D Geometry Reconstruction. PhD Dissertation, FU Berlin, Germany; 2008
• 45 Lamecker H, Zachow S, Statistical shape modeling of musculoskeletal structures and its applications, Lecture Notes in Computational Vision and Biomechanics, Vol. 23. In: Zheng G, Li S. eds. Computational Radiology for Orthopaedic Interventions. New York: Springer; 2016: 1-21
• 46 Lamecker H, Kamer L, Wittmers A , et al. A method for the three-dimensional statistical shape analysis of the bony orbit. In: Freysinger et al., eds. Computer Aided Surgery around the Head. Berlin: Pro Business Verlag; 2007: 94-97
• 47 Zachow S, Lamecker H, Elsholtz B, Stiller M. Reconstruction of mandibular dysplasia using a statistical 3D shape model. In: Lemke et al, eds. Computer Assisted Radiology and Surgery, Int. Congress Series (1281), New York: Elsevier; 2005: 1238-1243
• 48 Lamecker H, Zachow S, Wittmers A , et al. Automatic segmentation of mandibles in low-dose CT-data. Int J CARS 2006; 1 (1) 393-395
• 49 Hochfeld M, Lamecker H, Thomale UW, Schulz M, Zachow S, Haberl H. Frame-based cranial reconstruction. J Neurosurg Pediatr 2014; 13 (3) 319-323
• 50 Grewe M, Zachow S . ZIB projects: Camera Facialis ( http://www.zib.de/projects/camera-facialis ) and Facial Morphology ( http://www.zib.de/projects/facial-morphology ), 2015
• 51 Bibb R, Eggbeer D, Paterson A , eds. Maxillofacial rehabilitation case study 6 and 7. In: Medical Modelling—The Application of Advanced Design and Rapid Prototyping Techniques in Medicine. 2nd ed. Woodhead Publishing Series in Biomaterials, New York: Elsevier Ltd.; 2015: 256-277
• 52 Lubkoll L, Schiela A, Weiser M. An optimal control problem in polyconvex hyperelasticity. SIAM J Contr Optim 2014; 52 (3) 1403-1422
• 53 Ekman P, Friesen WV. The Facial Action Coding System: A Technique for the Measurement of Facial Movement. San Francisco, CA: Consulting Psychologists Press; 1978
• 54 Gladilin E, Zachow S, Deuflhard P, Hege HC. Realistic prediction of individual facial emotion expressions for craniofacial surgery simulations. Proc SPIE 2003; 5029: 520-527
• 55 Gladilin E, Zachow S, Deuflhard P, Hege HC. Anatomy- and physics-based facial animation for craniofacial surgery simulations. Med Biol Eng Comput 2004; 42 (2) 167-170
• 56 Zachow S, Gladilin E, Hege HC, Deuflhard P. Towards patient specific, anatomy based simulation of facial mimics for surgical nerve rehabilitation. In: Lemke et al., eds. Computer Assisted Radiology and Surgery 2002. New York: Springer; 2005: 3-6 . doi:10.1007/978-3-642-56168-9_1
• 57 Hermanussen M , Ed. Auxology—Studying Human Growth and Development. Stuttgart, Germany: Schweizerbart Science Publishers; 2013
• 58 Grewe CM, Lamecker H, Zachow S. Landmark-based statistical shape analysis. In: Hermanussen M, ed. Auxology—Studying Human Growth and Development. Stuttgart, Germany: Schweizerbart Science Publishers; 2013: 199-201
• 59 Hayward R, Jones B, Dunaway D, Evans R , eds. The Clinical Management of Craniosynostosis. Vol. 163 of Clinics in Developmental Medicine, London: MacKeith Press; 2004
• 60 Klein C, Howaldt HP. Mandibular distraction osteogenesis as first step in the early treatment of severe dysgnathia in childhood. J Orofac Orthop 1996; 57 (1) 46-54
• 61 Magnenat-Thalmann N, Kalra P, Lévêque JL, Bazin R, Batisse D, Querleux B. A computational skin model: fold and wrinkle formation. IEEE Trans Inf Technol Biomed 2002; 6 (4) 317-323
• 62 Kuwazuru O, Saothong J, Yoshikawa N. Mechanical approach to aging and wrinkling of human facial skin based on the multistage buckling theory. Med Eng Phys 2008; 30 (4) 516-522
• 63 Flynn C, McCormack BAO. Simulating the wrinkling and aging of skin with a multi-layer finite element model. J Biomech 2010; 43 (3) 442-448
• 64 Nadjmi N, Defrancq E, Mollemans W, Hemelen GV, Bergé S. Quantitative validation of a computer-aided maxillofacial planning system, focusing on soft tissue deformations. Ann Maxillofac Surg 2014; 4 (2) 171-175