No Bones About It
Here we talk to Dr. Valerio Rizzo, PhD, Lecturer and Researcher of Neurophysiology at University of Palermo (UNIPA), Photogrammetry Expert at Realities.io.
1) Your work is very unique. Please talk about your background and what you do.
I am a neuroscience researcher, lecturer and contract Professor of Human Physiology for the School of Medicine and Surgery at University of Palermo (Italy) and I have 7 years of experience as a 3D expert and freelance scientific illustrator. My education has always had a predominantly scientific marking and the further specialization in the biomedical field has always gone along with a constant study of the most advanced digital techniques for representing subjects and concepts related to biology and medicine.
“..from Leonardo da Vinci to Jan Steven van Calcar, from Max Brodel to Frank Netter, the art of illustration has been considered essential for biomedical study.”
Our brain is greatly engaged by visual inputs and, in fact, most of learning process mainly relapse on this sense. Visual inputs are fundamental for biomedical subjects learning and / or teaching. That is why, from Leonardo da Vinci to Jan Steven van Calcar, from Max Brodel to Frank Netter, the art of illustration has been considered essential for biomedical study.
2) How do you use reality capture technology in your work?
One of the most fundamental disciplines in medical education is the Human Anatomy. One of the most difficult aspects in the study of this discipline is learning human anatomy solely through books. While 2D medical illustrations are rich in details, an extraordinary effort is required to figure out how these small structures can appear from a slightly different point of view.
Hence, medical students need to explore specimens in the real world. However, to date, several medical schools (in Italy as elsewhere in the world) lack cadavers for research, as well as funding for the costs of storage and maintenance. Although the University of Palermo owns a collection of anatomical specimens, the presence of a large number of students committed in the study of human anatomy made logistically difficult accessing the anatomical pieces included in the collection, and, furthermore, constantly handling those pieces accelerates their process of deterioration, resulting in expenses due to the restoration of the pieces or their replacement.
From all this came the idea to create digital copies of the specimens of human anatomy collection of my university. The goal was to obtain virtual replicates of several specimen with such a quality of detail to be able to substitute the real ones. The idea was greeted with enthusiasm by Prof. Francesco Cappello, teacher of the Human Anatomy course of and therefore he made available some of skeletal specimens of the institute's collection.
Using classic methods of 3D medical imaging, such as MRI and CAT, would have been very costly, as well as involve extremely long acquisition times, not to mention the physical risks associated with the use of dedicated equipment. Then, I thought of resurrecting an older method that is applied extensively in architecture but also in certain areas of medicine: 3D photogrammetry.
“With reality capture, students can have access to these specimens at any time and everywhere they are, in the classroom or from their house without leaving the desk. ”
3) What impact does it make or how does it affect your workflow?
Thanks to 3D reality capture, students have been able to explore anatomical parts as never before. For example, students can easily visualize structures that weren’t easily seen, because they are too small to be seen with the naked eye.
With reality capture, students can have access to these specimens at any time and everywhere they are, in the classroom or from their house without leaving the desk. These 3D reproductions can be used as a digital rendering or animation or a real-world model that students can handle and move.
This technology can also help cut through the language and translate things visually so that a full understanding can be immediately achieved with less efforts.
4) How were you initially introduced to ReMake?
When I started the project to create digital replicas of anatomical pieces, the software available for photo-to-3D reconstructions did not give me the results I expected. Very often the reconstruction process was tedious, and different software was required to process it and eventually the reproductions lacked the equivalent detail of the original specimen.
Being a fan of Autodesk products, I have been actively involved in the beta-testing Autodesk platform for years. That gave me the chance to come across a project named Memento, now known as ReMake. Even in its early stages, its potential and the extent of its possible applications were already appreciable, so I decided to start the beta-testing.
“Bone matrix, very small signs of erosive degeneration, every spur or sulci were perfectly reproduced and distinguishable.”
I remember the first anatomical part I tested with ReMake was a lumbar vertebra. I wanted to try it on a simple and not too small shape. I started the reconstruction on the cloud and after a while I got the result. When I opened the file I was immediately struck by the accuracy in the reproduction of form and texture, but what ReMake was really able to do was beyond my expectations. By disabling texture I was able to appreciate the real detail of geometry. It was amazing, ReMake was able to reconstruct details of the fourth in the order of a millimeter with absolute fidelity. Bone matrix, very small signs of erosive degeneration, every spur or sulci were perfectly reproduced and distinguishable.
Then, I tried again on an unpaired bone of the cranium called Sphenoid, known to be the cradle of the pituitary gland also known as the “Third Eye”. This attempt was actually something really challenging due to the size of the piece and the complexity of its structure. We chose that particular bone for two main reasons:
1) Testing the limits of the software technology
2) Obtain a very high-res 3D reproduction of that bone, at that time not available anywhere, and with a level of detail never achieved even using cutting-edge medical imaging.
Once again the outcome was stunning, the model showed details up to 80 um2.
Fig: The Reconstruction of the Sphenoid Bone and its 3D Print made by Tatjana Dzambazova
“These three characteristics literally made me fall in love.”
5) Why do you like it?
Since its first build ReMake has had a minimalist design and an intuitive interface coupled with a highly advanced software technology. These three characteristics literally made me fall in love. During development, the team has been implementing several tools increasing the possibilities of use nevertheless maintaining an essential design as well as increasing functionality without reducing the ease of use. Last but not least, it allows you to face an entire workflow and without requiring additional software, significantly reducing time and costs for the development of a given project.
Finally, ReMake’s simplicity of use and the intuitiveness of its interface allows an almost immediate approach even for non-professionals, therefore increasing its putative employment in other professional fields where operators could not be expert in reality capturing.
6) How do you use it?
I must say I use ReMake for a variety of things. From strictly professional purposes to personal hobbies. As already discussed above, the first approach was initially professional, but not just for education, as in the case of the study of anatomy. I began to apply it in my research in neurophysiology.
In the laboratory of sensorineural pathophysiology led by Prof. Giuseppe Ferraro and Pierangelo Sardo at the department of Biomedicine and Clinical Neuroscience of the University of Palermo, I used ReMake to create digital copies of real objects that were used for preliminary studies, conducted with the colleague Dr. Girolamo Schiera, about electroencephalographic correlates of virtual haptic exploration.
Furthermore, I found myself using ReMake even for strictly personal or recreational reasons. Often during my leisure travel, I like to store digital 3D memories of statues, façades of historical buildings or parts of environments that have impressed me or where I have spent pleasant moments. I use it to create digital portraits, and thanks to the possibility of optimizing and exporting for 3D printing, has also added the ability to create small souvenirs or gadgets that turn into gifts for friends and relatives.
Fig: Digital 3D Portraits
7) Tell me about some of your favorite or successful projects using reality capture.
One of the projects that has seen me more involved and where I made extensive use of ReMake was the recovery and digital preservation of the Museum of Pathological Anatomy at the Università of Palermo. Since the foundation of the University of Palermo in 1805, the Faculty of Medicine has been one of the most productive centers of anatomical and pathological studies, among those made by Giuseppe Gorgone and his students and successors.
The Department of Pathology today has one of the richest archives of anatomical specimens in the region and perhaps even nation. This archive is not just historical legacy and the memory of so many distinguished scholars, but it represents an important resource for educational purposes and a potential source of wealth if used in the context of new technologies. Unfortunately, independently from the methods used to preserve these specimens, a progressive, even if small, corruption of organic material occurs thus reducing the intrinsic value.
We decided to apply reality capture to a number of rare preparations in the museum Pathological Anatomy of the University of Palermo. Once again the results were amazing and gave us several opportunities:
1) Ensuring digital preservation of some of the rarest and oldest specimens of the archive, such as the 100 year old Acromegalic skeleton (one of the very few exemplar existing in whole world).
2) Starting to create a rare human pathology database, so far not available anywhere else on the Internet and spreading medical science awareness on rare pathological conditions and, for instance, improving pre-surgery training for rare human pathologies.
3) Giving medical students and professionals around the globe the chance to have access to and explore these rare specimens of the museum, which are currently inaccessible to public as it requires funding for restoration.
4) Building up a captivating visual story that could tell the story of the museum, showing the valuable specimens included in the archive in order to attract experts and philanthropists from around the world that could help us to restore the museum so it can be opened back again to public.
Fig: Shooting of the Acromegalic Skeleton (2.5 meters tall) at the Museum of Pathological Anatomy in University of Palermo, Italy
8) What else is coming up or in the pipeline? What should we look forward to from your research?
Nowadays, as everybody knows, the VR / AR field is having huge success and evolving at a breakneck speed. This area has been found to be intimately tied to capture reality techniques.
In this regard, I have just started an extremely interesting professional adventure with the brilliant guys at Realities.io to create large high resolution 3D environments for immersive VR experience by using photogrammetry.
In parallel, my collaborators of the 3DTAM research unit and I are working on other projects concerning the neurophysiological correlates of virtual immersive experiences. I would like to say some more about this latter as it is a very exciting project but at this point of the research I cannot disclose any further detail.
Fig: Acromegalic Skull Reconstruction
Autodesk ReMake is available for both Mac and Windows. Download a free trial at remake.autodesk.com.