Institute of Fundamental Technological Research

Background: Immersive technologies like Mixed Reality (MR), Virtual Reality (VR) and Augmented Reality (AR) are becoming increasingly popular and gain user trust across various fields, particularly in medicine. In this paper we will use the general term Mixed Reality (MR) to refer to the various virtual reality methods, namely VR and AR. These new immersive technologies require varying degrees of instruction, both in their practice use, as well as in how to adjust to interacting with 3D virtual spaces. This study assesses the pedagogical value of these immersive technologies in medical education. Method: We surveyed a group of 211 students and 47 academic faculty at a medical college regarding potential applications of MR in the medical curriculum by using a questionnaire comprised of eight questions. Results were analyzed accounting for user age and professional position, i.e., student vs faculty. Results: 70% of students and 60% of the academic faculty think that MR-supplemented education is advantageous over a classical instruction. Most highly valued were the 3D visualization capabilities of MR, especially in anatomy classes. There was no significant statistical difference between students and faculty responders. Moreover, screensharing between faculty and students contributed to better, longer lasting absorption of knowledge. Surprisingly, the main issue was related to availability, i.e., only 5% of students had access to MR, while 17% of faculty use MR regularly, and 36% occasionally. Conclusions: MR technology can be a valuable resource that supports traditional medical education, especially via 3D anatomy classes, however MR availability needs to be increased. Moreover, MR expands the capabilities and effectiveness of remote learning, which was normalized during the COVID-19 pandemic, to ensure effective student and patient education. MR-based lessons, or even select modules, provide a unique opportunity to ex-change experiences inside and outside the medical community.

mixed reality, e-learning, remote learning, real-time rendering, 3D visualization, medical education

Background: General data protection regulation (GDPR) provides rules according to which data should be managed and processed in a secure and appropriate way for patient requirements and security. Currently, everyone in Europe is covered by GDPR. Thus, the medical practice also requires access to patient data in a safe and secure way. Methods: Holographic technology allows users to see everything visible ona computer screen in a new and less restricted way, i. e. without the limitations of traditional computers and screens. Results: In this study, a three-dimensional holographic doctors' assistant is designed and implemented in a way that meets the GDPR requirements. The HoloView application, which is tailored to run on Microsoft HoloLens, is proposed toallow display and access to personal data and so-called sensitive information of all individual patients without the risk that it will be presented to unauthorized persons. Conclusions: To enhance the user experience and remain consistent with GSPR, a holographic desk is proposed that allows displaying patient data and sensitive information only in front of the doctor's eyes using mixed reality glasses. Last but not least, it boasts of a reduction in infection risk for the staff during the COVID-19 pandemic, affording medical care to be carried out by as few doctors as possible.

augmented reality, mixed reality, pandemic

Immersive technologies, like Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) have undergone technical evolutions over the last few decades. Their rapid development and dynamic changes enable their effective applications in medicine, in fields like imaging, preprocedural planning, treatment, operations planning, medical students training, and active support during therapeutic and rehabilitation procedures. Within this paper, a comprehensive analysis of VR/AR/MR application in the medical industry and education is presented. We overview and discuss our previous experience with AR/MR and 3D visual environment and MR-based imaging systems in cardiology and interventional cardiology. Our research shows that using immersive technologies users can not only visualize the heart and its structure but also obtain quantitative feedback on their location. The MR-based imaging system proposed offers better visualization to interventionists and potentially helps users understand complex operational cases. The results obtained suggest that technology using VR/AR/MR can be successfully used in the teaching process of future doctors, both in aspects related to anatomy and clinical classes. Moreover, the system proposed provides a unique opportunity to break the boundaries, interact in the learning process, and exchange experiences inside the medical community.

augmented reality, improving the education process, interaction, intraprocedural visualization, mixed reality, preprocedural planning, teaching

Apart fromoperating tables andmodern surgical instruments, themodern operating rooms are equipped with displays and video surveillance systems. The three-dimensional operating room allows users to watch medics perform surgery from different, individually chosen, points of view. For the first time, it is possible to reproduce/repeat the course of the operations and change the perspective or position, fromwhich it is observed. Here, we proposed a solution based on Microsoft HoloLens and Azure Kinect DK devices as remote support to patient management. The operating room is transferred to the digital form in real-time using Augmented Reality based technologies. Users can move around the digital place like a ghost in real space. The approach proposed allows users to see observe surgery from any point of view they want without disturbing the surgeon's workflow. They can change their positions, angle, and place of observation. All environmental restrictions disappear. The presented solution gives trainees a convenient opportunity to learn. It may make a significant contribution to improving the surgeontraining, patients' outcomes, and may allow virtual medical consultations during the surgery between specialists without them leaving their workplace.

3D operating room, augmented reality, HoloLens