The project consisted in the use of 3D markets to serve as real-world representations of digital 3D models. In this case, a cube marker was used to render an interactable digital human heart. Students using the application can visualize a human heart, its parts, and test themselves about their learning through the application. The results of the assessment thought the application are available almost in real time on web-based dashboard. My participation in the project included merging the project from an old Unity  3D version using JavaScript to a more up to date version using C#, enhancing of the user interface to interact with 3D models, build screen capture functionality for recording of interactions, highly efficient parallel processing for multi-threading,  plugin to compress images into  zip files, upload of images to a Linux server to convert the images to video using FFmpeg, and a dashboard using PHP, JavaScript, and MySQL to visualize the results.


VR and AR


unity 3d and vuforia ar







The Challenge

The first challenge was to merge the existing base application from an old version of Unity 3D and JavaScript, to a more recent LTS version of Unity 3D and C#. Once the update of the base app was done, the user interface required improvement. Also, the collection of valuable information from the testing functionality needed to be collected, stored and made available for further processing. The application was required to have a stereoscopic version for the Google Cardboard, and a regular version for other platforms.

The Solution

The user interface was highly improved by the use of the TextMeshPro in Unity, which made the captions, labels,  and system feedback crispier and easier to read. A multi-platform screen recording plugin was developed aiming to cover the stereoscopic and traditional renders. The video was uploaded for further processing on a server with FFmpeg before making it available in the dashboard. The results of the tests were stored in a MySQL database, and the information was displayed almost in real-time in a dashboard build with PHP and JavaScript.

The Results

An mixed reality application which allows a student to visualize anatomic models and test themselves about what they have learn. After they undertake the embedded test, the results are available in a dashboard, including relevant information about the test attempt and a video of the attempt.

Screencap of the results dashboard which can be seen live here: HeartMR Dashboard

Get In Touch

Ready to Chat?