Using Stereo Photogrammetry to Find the Shape of Skulls

The shape and size of a skull can tell wildlife biologists a lot about the animal it came from. Measuring the size of skulls has been done for centuries by hunters and trappers and research biologists to understand more about the overall animal population and the individual animal the skull came from. Sexually dimorphic animals show distinct differences in the size of certain body features and can be especially prominent in skulls. I had the opportunity during my undergraduate career to work with a wildlife biologist to gather data and conduct some analysis on Fisher skulls. This project was my first step into the world of data science and the first time I used a computer to manipulate and organize huge amounts of data. By the end of the project I had a much better understanding of how to apply computers to solve difficult problems, and also gained a passion for data science.

What is a fisher?

Most people probably don’t know what a fisher is unless they spend a lot of time outdoors in the upper Midwest or are regular trappers. In a gist, a fisher is a really big weasel. Some large males can weigh up to 15 lbs and are almost 4 feet long. They occupy many states in upper New England, as well as the northern Great Lakes region, and over to the northern Rockies and even into California. They are a popular target for trappers and really incredible animals. Other than cougars, fishers are the only North America Animals to actively prey on porcupines.

The research biologist I worked with was an expert on mustelids, or members of the weasel family, and had been working with fishers for a few years. He reached out to me and we set up an undergraduate research project trying to study the 3 dimensional shape of a population of fisher skulls. It is a relatively easy process to collect linear size data on skulls with a precise ruler or calipers, but collecting accurate 3d data, especially on a small budget, is a little bit tricky.

Project Overview

This project was quite involved and involved me learning many new techniques and skills. First, we had to acquire and process enough skulls to study. Luckily, my project mentor had already contacted state game wardens to collect fisher heads from trappers willing to donate them for research. Once we got all the biological specimens, we had to clean and prepare them. The skulls had to be absolutely clean and dry after we were done, since they would be entered into the university research collection. Next, we had to collect our data. There are a few ways to collect 3D data on small objects, but with our small budget we decided to use a method called stereo photogrammetry. With this method, a pair of cameras takes photos of an object and a user manually assigns points to certain landmarks on each image. The landmarks are standard, prominent features that appear on a biological specimen. When all the landmarks are assigned some software can correlate the points in 3D space and spit out a point map of all the points. This general process had to be done for each skull and once we had the entire data set of 3d point clouds we were able to run statistical analysis and look for patterns. We were intersted in seeing if males had different skull shape than females. It is well known they are much larger than females, but if size is corrected for, how does the shape differ?

Collecting and Cleaning Skulls

We were fortunate to have a great relationship with many hunters and trappers in the area, and my project mentor personally knew many people who donated their fisher skulls at state game registration stations. I was able to sit with a game warden at a local office at the end of one season and meet a few trappers who came in. They were all very interested in the research and more than happy to donate anything useful. We were able to collect a few hundred fisher skulls, plus there were more in the university collection from previous years.

Once we had all the fresh skulls stored in a freezer at a university lab, we were able to process them in a timely manner. The general flow was to get a few skulls thawed out the day before, then bring them up to the lab and set them on a plastic tray. Some information was recorded from each skull such as where and when it was harvested. Next, a tissue sample was taken from a deep muscle for DNA analysis. After that, linear measurements were taken with calipers and entered into a separate database. We removed as much muscle tissue as possible with scalpels and forceps, along with the eyes and tongue. This was important because the next step involved putting the skulls into large tanks of dermestid beetles. These flesh eating beetles lay eggs in the flesh of animals, and the larvae eat the flesh. The adults breed and continue the cycle. Maintaining a colony of dermestid beetles can be difficult work, and it was important to balance the colonies food supply. If there was not enough food, it could take weeks for the beetle population to grow large enough to fully clean a skull in a reasonable amount of time, but if the population got too large there could be other problems and the colony could collapse if they food supply isn’t gradually reduced.

(Graphic Warning) The images below show a skinned fisher skull and a time lapse of the beetles feeding on the skulls.

After the skulls are cleaned by the beetles, they are softly scrubbed to remove dust and any small particles, then soaked in water to rot the connective tissue holding the teeth in place. Finally, they are let out to slowly dry and gently scrubbed a final time before being cataloged into the university collection.

Collecting Image Data

One of the easiest tasks during this project was actually collecting the images used for analyses. The most important part of taking photos was to be very careful not to move any of the equipment once it was in position. To process the stereo images we used StereoMorph, which is a software package for R specifically developed to collect 3D landark data from biological specimens. To calibrate the cameras, the user places a checkerboard pattern in the image space and takes a few photos before the main shoot. Then, you can begin taking photos of the specimens and as long as the cameras don’t move during the process the calibration is conducted in post processing. This made it extremely fast to process the actual specimens, and helped to reduce overall handling time, which was important because it was difficult and time consuming to arrange the have things taken out of the university collection.

A total of eight pictures were taken of each skull, from each camera. The skull was placed into the image space and rotated around to capture all the landmark features. This had a high degree of success, but in hindsight there were some specimens that had poor photos without visible landmarks which had to be ignored.

Assigning Landmarks to Photos

After all the images were collected, sorted, and organized into folders, we were ready to work with StereoMorph and assign the landmarks. Digitizing landmarks with StereoMorph uses R, but launches an easy to use app in a browser which allows the user to quickly manipulate images and changes settings. The basic workflow is to change the code in R to load the correct specimen folder, then launch the digitizing app, apply the landmarks to each image, close the app, use R to verify the landmarks. Recall that this was done for about 400 specimens, with 8 images per camera, and we used about 20 ventral and 20 lateral landmarks. Also note that in order for the software to detect and unify landmarks into 3D space, there have to be enough which overlap across multiple photos and between cameras. A conservative estimate is about 450 landmarks manually assigned to each specimen, or a total of approximately 180,000 landmarks. It took a few months of late nights working through the images. After all this work, not all of the coordinates were able to be unified and the data was discarded. We were left with 219 usable sets of landmarks.

Extracting 3D Coordinates from Landmarks

Once all the landmarks have been assigned to a specimen with StereoMorph, they are simply a list of 2D coordinates, and since the specimen was rotated in space, we cannot use simple trigonometry to calculate the location of each landmark. Each set of landmarks is positioned in it’s own coordinate space, and must be aligned with the other sets to a common space, a process called unification in StereoMorph. There is a function built into the package to reconstruct and unify the landmarks into a common space and essentially create a point cloud representing the landmarks. Once this has been done, I visualized each specimen to verify that the process was successful.

Visualizing the 3D Coordinates

This graphic shows where the landmarks are located in space after successfully reconstructing and unifying the 2D landmark coordinates gathered from the stereo images. The different colors show left lateral (green), right lateral (blue), and ventral (red) landmarks.

Data Analysis

With the 3D point cloud created for each specimen, we started cleaning and processing the data for analysis. The first step was to align the 3D coordinates to a common space, since StereoMorph just creates the points according to it’s unification algorithm, it doesn’t have any standard orientation along an axis. We used a function called “findOptimalPointAlignment” in the CRAN package for R to align each point set with a standard point set from one good specimen we chose.

Next we used the GeoMorph package developed for performing geometric morphometric analysis on landmark data. We separated the specimen data into a 3D array with each landmark in a row, each coordinate in a column, and specimens separated on different sheets. Using fisher sex and age class as variables we performed a Procrustes ANCOVA (Analysis of Co-variance) to see if there was any statistical differences in skull shape between young to old males and females.

To compare the shapes of two or more objects, the objects must be first optimally "superimposed". Procrustes superimposition (PS) is performed by optimally translating, rotating and uniformly scaling the objects. In other words, both the placement in space and the size of the objects are freely adjusted. The aim is to obtain a similar placement and size, by minimizing a measure of shape difference called the Procrustes distance between the objects.

-Wikipedia

After the point sets were modified with Procrustes transformation we were able to calculate the centroid size of each specimen, which we used for further analysis. In the image below, it is clear that males (red) have significantly larger skulls than females. Also note how centroid size increases as a function of age. The point sets are in groups based on estimated birth year, which was determined using a process called Cementum Annuli Forensic Aging.

Results and Conclusion

This was a fairly intensive undergraduate research project and I learned a lot over the course of the year I worked on this. It was fascinating to dive into the science of wildlife ecology and biology, and I had such a great mentor who was very patient and helped me along the way. This project really jump started my interest in data analysis and computer programming. We were able to present our results at an annual conference held by The Wildlife Society in St. Cloud, MN, where I was fortunate enough to win best student poster based on this research.

Our data showed no effect of age class on shape, only significant effect on sex. Shape changes may still be occurring over the lifetime of a fisher, but our landmarks may not be capturing the locations where this is taking place, or shape changes no longer occur after about 8 months of age. After a juvenile develops their skull shape early in life, any further changes are strictly due to growth. Shape did not change with age, but males and  females have different skull shapes. Males have relatively larger canines, shorter and wider ventral crania, and wider zygomatic attachments. The images below show the change for each landmark between the average male and average female.

Thanks for reading!

Acknowledgements

My research mentor, Dr. Michael Joyce, continues to do great research on mammals in the Upper Midwest: https://www.nrri.umn.edu/faculty-staff/michael-joyce

The creator of the StereoMorph package for R has many other projects in addition to the wonderful software we used, and is well worth a look: https://aaronolsen.github.io/