About the Client

Our client is a large healthcare company with many different divisions; this project began as part of their consumer products division with the objective of understanding how various things like sunlight, cosmetics, aging, or surgical procedures impacted the skin of customers over time. Digital “prosumer”-grade cameras can take very high resolution photographs, but we needed to address a significant challenge around positioning of the patient’s head and face: while a chin rest and directions to the patient could ensure that the patient’s face was in nearly the same position for each image, small variations could make the image data useless. It doesn’t take a shift of very many pixels to turn an eyelid into an eyeball or vice versa.

The Challenge

Facial plastic surgery can produce bruising (ecchymosis), swelling (edema), and redness; surgeons are always interested in ways to reduce those negative outcomes while speeding up healing. Traditionally, a doctor or other clinician would subjectively assess the state of a patient’s skin following surgery, but subjective data is exactly that: subjective. Results are not necessarily consistent between clinicians, and even the same clinician can produce data that would be different on another day. Skin is also a complex and multi-layered organ, and understanding what’s happening beneath the surface helps create a more complete picture of the patient’s state and healing progress. We designed custom software for processing pictures of patients under various frequencies of light, correct for small changes in the position of the patient’s head and face, and analyze the images to understand the state of the patient’s skin and underlying structures. With the consistent, objective data provided by our system our client can now assess which surgical techniques and instruments provide better patient outcomes.

Our Approach

We addressed the positioning challenge in software. We were able to take advantage of the fact that we had multiple images using different wavelengths of light to be able to automatically detect the difference between skin, hair, and clothing. Just the areas of the image consisting of skin are passed to an elastic image registration algorithm which accurately (+/- 1mm) lines up the face images, correcting both for head position and facial expression.


Once we acquired the images, we needed to process them for both registration and analysis. Given that this was a computationally challenging task involving very large image sets, we developed an image processing framework that could execute the analysis algorithms in parallel on large multi-core (32+ ) computers.


Our analysis algorithms assessed apparent concentrations of oxy-hemoglobin (blood flow), deoxy-hemoglobin (ecchymosis), melanin (pigmentation), scattering and water (edema) were calculated for each pixel of a spectral image stack. From the blue channel on cross-polarized images we extracted values for bilirubin, which is produced as red blood cells break down and gives fading bruises their characteristic yellowish-brown tint.


To make the system easy to use for clinicians, we developed a user interface that tied to our image processing framework. The interface enabled browsing images while also displaying image registration, intermediate results, and final results. The interface also enabled the user to select a specific region of the face for further processing. Lastly, we created software that generated web-based reports of the analysis; these reports could then be included in regulatory submission documents.

The Outcome

Originally developed to assess the effectiveness of consumer products at maintaining skin appearance, our client soon realized that being able to objectively assess properties of skin is a powerful tool. At their request we went back and created a validation package for the system, ensuring that it met FDA requirements for a system to be used in medical research. From there our client used the system to assess the effectiveness of one of the instruments they promote for use in plastic surgery. With the objective data gathered from our now-validated system, they were able to substantiate their claim that their product produced less swelling, edema, and redness and provided for an improved patient outcome.