Virtual Reality Simulation-Based Robotic Technician Assessment

Virtual Reality Simulation

Vr training, VR based assessments, Virtual reality manufacturing training, Conversation AI, Job skill assessment


To demonstrate the potential offered by VR technology for credentialing and training of the robotics workforce.


Siminsights partnered with the ARM Institute to develop a VR based robotics technician assessment. The program focuses on analyzing a potential hire’s expertise in handling the maintenance of a palletizing robot in a virtual environment.

The assessment simulation, authored in HyperSkill, presents the user with a fault and asks the user to troubleshoot. SimInsights created a high-fidelity simulation capable of producing observations of users actions in a manner that supports gathering of detailed evidence for assessment of their proficiency on specific standards.


VR Solutions : Create 6 DOF VR immersive manufacturing environment

Siminsights project team worked with Dr. Ayesha Madni, Professor Mark Steyvers, the SMEs and the ARM Institute team to conduct the Cognitive Task Analysis. Analysis Assessment design was based on inputs from SME, Jonathan Clingan, on tasks related to troubleshooting and maintenance for the machine. Once the standards were set, the Scoring Model scheme was implemented. Based on the Scoring Model scheme, Hyperskill allows the user to proceed with the simulation to demonstrate their competency with other steps, even if they may not have completed a preceding step successfully.

Ultimately, when providing feedback to the user on their completion of the entire simulation, Hyperskill indicates which steps and substeps they completed successfully for partial and total scores. It also provides the requisite remediating information. In other words, a user gets feedback about particular content aspects and steps (e.g., safety procedures) where they may require additional practice, or where they have achieved adequate competence.

AI methods were used to implement conversational capabilities. Users can give voice commands to virtual humans in the simulation to obtain information. Users’ voice commands are logged and can be used in the assessment.


SimInsights’ primary deliverable was a high fidelity simulation of a virtual reality manufacturing training environment. This environment contained highly realistic graphics and interactions, capable of producing observations of users’ knowledge, skills and abilities (KSA). Such information is used to gather evidence for valid and reliable assessment for certification and credentialing.

A tutorial was created and iteratively refined with pilot testing feedback. Another highlight of the simulation was the inclusion of conversational artificial intelligence (AI) to simulate dialogue between the worker and two virtual human assistants who played the role of a supervisor and a work cell technician.


Credentialing for technicians who have years of experience but have not had recognized/formal training and assessment

The net impact of these benefits offered by VR credentialing and assessment is a substantial cost saving to the company, improved assessment of potential hires and a notable increase in the number of potential hires taking this assessment to qualify for open positions.

Conclusion – Constructive Feedback from the Customer

Successful completion of this project demonstrates the potential offered by HyperSkill for credentialing and training the robotics workforce. All users who went through the experience were universally impressed by the environment and the implementation of the robot. It was interesting to note that the robotics expert users quickly made the correct inferences as new information was revealed to them by the simulation.

Participants were also asked about their overall experience with the VR task. All participants indicated that their experience was positive ranging from challenging and educational, to enjoyable and interesting.

The HyperSkill platform and the methodology followed herein are general purpose and applicable to a variety of assessments and VR training simulations. This case study lays out a clear path to scaling up a large collection of VR simulation based assessments for a broad range of careers in robotics and other industries. Such a collection of robotic enabled assessments has the potential to expand access, cut travel costs and promote equity to equip the workforce with in-demand skills today and in the future.

Read the full case study