The need for quantitative evaluation
The tympanic membrane (TM) or eardrum is a conically shaped thin membrane separating the outer ear from the middle ear. It conducts sound vibrations from the external ear canal to the auditory ossicles and protects the middle ear from infections. The three-dimensional shape of the TM plays a crucial role in this process and any structural change to its topography is an important indicator of existing or impending pathology or hearing impairment.
To date, the eardrum is visually examined using a classic otoscope. This device helps doctors to observe pathologies such as perforations and diagnose damage or inflammation of the tympanic membrane.
However, the information provided by the otoscope remains limited to a flat 2D image. This image does not lead to any understanding about the shape of the tympanic membrane, which helps when identifying other pathologies such as a malfunctioning Eustachian tube or weak zones in the eardrum.
To link the pathophysiological mechanisms of chronic middle ear disease (such as retraction pockets and cholesteatoma) to loss of TM stiffness, von Unge et al. have conducted a series of ex vivo tympano-topography experiments on gerbil ears [1]. A correlation between TM volume displacement and the stage of the disease could be made by increasing the level of pressure and measuring the induced shape deformations of TMs having varying degrees of otitis media (inflammation of the middle ear).
Therefore, as aberrant local deformations or ‘weak spots’, due to loss of stiffness of the TM, could indicate structural damage and be an important precursor for cholesteatoma formation and ultimately hearing loss, a need for quantitative evaluation of the TM shape in-vivo has been expressed. However, a technique for in-vivo measurement, as needed in clinical diagnosis, is still missing.
Research prototype
The idea of developing a tool, for quantitative evaluation of the TM shape, came to life. Further analysis of the various techniques that can measure the 3D shape of the tympanic membrane was needed. Inspiration for structured light profilometry, to obtain 3D-shape measurements of the tympanic membrane, was taken from the research of Zhang et al. [2]. They proposed a combination of structured light profilometry and graphics processing unit programming. Although these setups obtain high-resolution (≈ 5-10% of projected fringe period) depth scans of the measurement target surface, these setups are typically designed for macroscopic objects only, and acquiring high-speed and high-resolution depth information of microscopic objects remains challenging.
(I) ‘Zero-generation’ otoscopic unit and (II) schematic overview of its optical layout with enlarged detail of otoscope head. DMD: digital micromirror device, PL: projection lens, SM: semi-transparent mirror, OL: object lens.
In recent work [3], we have developed a custom handheld device based on a modified clinical otoscope that simultaneously projects structured light patterns onto the eardrum and records them with a digital camera, placed at a relative angle to the projection axis (see Figure 1). By employing a high-speed camera and using parallel programming techniques, we have demonstrated that 3D shape data of cadaveric human eardrums can be obtained at real-time video rates, enabling the indirect monitoring of dynamic middle ear cavity pressure fluctuations. However, due to the relatively high transparency of eardrum tissue, the contrast of the reflected fringe patterns is insufficiently high to extract high-resolution depth data from the topograms without first coating the membranes with white cosmetic dye. This currently negates the noninvasive nature of our technique and complicates its application on living patients.
Development of a clinical device
Although the research prototype worked fine on ex-vivo test subjects, several aspects still required a redesign before in-vivo testing could be initiated. This led us to a new prototype, with a novel optical setup featuring a simple projection unit and camera. By using only a single projection grid, the overall weight and size of the device could be reduced.
(I) Second generation of the otoscopic unit and (II) schematic overview of its optical layout.
We called this prototype the 3D otoscope: a compact medical device that ENT physicians will use in the clinical office. By measuring the volumetric displacement of the human tympanic membrane caused by middle ear cavity pressurization, a variety of existing or impending middle ear pathologies can be detected.
The device contains a new single-shot optical engine to observe the surface profile of the eardrum and to display its 3D shape in real time. This non-invasive diagnostic tool will assist the ENT physician in diagnosing specific conditions, such as a malfunctioning Eustachian tube and weak areas of the eardrum, more easily and accurately.
Key features of the 3D otoscope
Patented technology
The device employs a structured light profilometry technique in which a sinusoidal modulated light pattern is projected onto the eardrum and observed by a digital camera, placed under a relative angle to the projection axis. The recorded images are processed in real-time using a novel deep learning-based algorithm to extract the full-field 3D deformation data.
(I) The image captured by the small camera unit. The projected lines are visible on the observed object. (II) A height map is generated based on the projected pattern, which is deformed by the shape of the object.
Mobility
Both the design of the device and the controls are kept simple to ensure an efficient use of the device, regardless of the situation of usage. Since the device is connected to the computer via a simple USB connection and light is passed through an optical fiber, the doctor can use the device at a significant distance from the processing unit (computer).
Flexibility
Since the length of the ear canal can vary between patients and the length of the speculum depends on the manufacturer's characteristics, a simple adjustment system needs to be implemented into the device. This system offers the physician the possibility to reposition the focus point of both the camera module and the sinusoidal grid pattern, which results in better 3D images of the tympanic membrane.
