3D immersive, interactive and intelligent training for clinicians
Immersive 3D training for clinicians, built with HyperSkill. Learners practice challenging conversations, and a real-time feedback panel shows them the ideal verbal response.

Communication and team training for clinicians
Our aim was to evaluate the usability and performance of the platform as compared to in-person training.
This project addresses communication and team training through a set of virtual simulations built on HyperSkill, the platform developed by SimInsights. HyperSkill replicates the clinical environment in virtual reality without the need to write code. The immersive training then gives users ideal verbal responses to challenging situations through a real-time feedback panel.
This chaperone training is one part of UCLA Health’s wider training program. Read the UCLA Health customer story.
The challenge with conventional methods
Communication and teamwork failures between healthcare team members are responsible for up to 70% of medical errors, because members of healthcare teams come from many different disciplines and isolated education and training programs. Training in communication and teamwork skills is proven to improve teamwork, clinical performance and patient outcomes. Healthcare team training is effective for a variety of outcomes, including trainees’ perceptions of the usefulness of team training, acquisition of knowledge and skills, demonstration of trained knowledge and skills on the job, and patient and organizational outcomes.
Traditional communication and team training sessions have consisted of classroom-based didactic presentations, resource-intense immersive simulator-based programs requiring in-person attendance and facilitated debriefing, or both. These methods have logistic challenges, such as assembling individual team members to train in person. There are also limited opportunities to apply newly acquired skills in relevant contexts, to repeat practice and feedback, and to follow up and assess skills acquisition and retention.
Key features
What each simulation combines, from the 3D clinic to the feedback after every answer.
Virtual reality simulation
The clinic is rebuilt in 3D from 360-degree reference images captured at a UCLA clinic.
Conversational AI
Learners answer out loud, and the simulation understands what they said.
Deep learning
A fine-tuned BERT model classifies each spoken response.
Dialog systems
Storyboards and flowcharts made from existing recorded media drive each conversation.
Asynchronous training
Team members train on their own schedule, with no need to gather everyone in one room.
Communication skill training
A feedback panel shows the ideal response, scores the learner’s answer and measures their confidence.
The solution: immersive 3D simulations with conversational AI, authored in HyperSkill
Three immersive 3D simulations, each with its own speech classifier and a debrief after every response.
Authoring with HyperSkill
SimInsights used existing recorded media to create new storyboards and flowcharts, then implemented them in HyperSkill as three immersive 3D simulations. The clinical environment was replicated in VR by visiting a UCLA clinic and capturing reference images with a 360 camera, which 3D artists worked from. A feedback panel debriefs learners by providing ideal responses, scoring their responses and measuring their confidence.
Natural language processing
For each of the three scenarios, intent detection classifiers based on natural language processing were trained to map user speech to fixed utterance labels. The transfer learning capabilities of the NVIDIA TAO toolkit were used to fine-tune a pre-trained Bidirectional Encoder Representations from Transformers (BERT) model for intent classification.
Measurable benefits
Thousands
of clinicians receive this mandatory training each year, and learners rated it in a feedback survey
Users rated different parameters of the training software, shown in the chart. Based on these performance metrics, the majority of learners felt the training program gave them the tools they need to speak up, and that they will be able to apply the lessons in practice.
The clinical workforce perceives this training as highly innovative and engaging compared with traditional training.
The project shows no-code software can dramatically reduce the cost and time of creating innovative training modules like these.

What learners said
“I now realize why my role as a chaperone matters.”
“…I will be a better manager after attending this training.”
“Being able to [be] active in the scenarios, although I was nervous… allows me to know what to do if this were to happen in a clinical setting.”
Conferences
The case study was accepted at several conferences, where representatives of UCLA Health or SimInsights presented the findings.
7th International XR Conference 2022
3D Immersive, Interactive and Intelligent Training for Clinicians
Rajesh Jha, CEO, SimInsights, Inc.
ICCH 2021: International Conference on Communication in Healthcare
October 17–20, 2021 · Simulation Training to Empower Medical Chaperones in Speaking Up for Patient Safety

Yue Ming Huang, EdD, MHS
UCLA Health

Miguel Drayton, MFA, MS
UCLA Health
IMSH: Society for Simulation in Healthcare Annual Conference
January 18–22, 2020 · 3D Immersive, Interactive and Intelligent Training for Clinicians
Rajesh Jha, CEO, SimInsights, Inc.
Frequently asked questions
What should healthcare organizations look for in a 3D training platform?
Look for a platform your own team can author in without code, with conversational AI that understands spoken answers and a feedback panel that shows the ideal response. In this project HyperSkill also let staff practice asynchronously instead of gathering in person, and HyperSkill runs on headsets, the web, desktop and mobile. See how it is used across healthcare training.
Can we build immersive medical training without game developers?
Yes. HyperSkill is no-code: SimInsights built the three immersive 3D clinical simulations in this case study in HyperSkill without writing code, starting from existing recorded media, storyboards and flowcharts. Get started free.
Which platforms offer immersive training with AI virtual agents?
HyperSkill does: its simulations combine immersive 3D scenes with conversational AI, so learners speak their answers and a trained language model recognizes what they said. Request a demo to see it.
Read the full case study
Download the PDF, or talk to us about building training like this for your own clinical teams.