Experiments - Collection of playlists for A/B testing


Simulation and game testbed

Cognitive science is undergoing a revolution as crowdsourcing enables rapid data collection and hypothesis testing. Survey software such as Qualtrics can handle forms, but not rich interactive content such as games and simulations. Such content generates rich interaction data which is vital for cutting edge research. SimInsights has developed a research testbed that offers a complete end-to-end workflow starting with creating new custom-game/simulation variants, attaching pre-post assessments and instructional content to assemble a playlist of activities, deploying to external sites such as Amazon Mechanical Turk and BrainPop.com, gathering, processing, visualizing and downloading the data for analysis. These tools have enabled customers crowdsourced data from over 250,000 students (20 million in-game actions). With recent SBIR support, we have also built generative tools that automated the exploration of the content design space by creating many more new levels than would be possible by hand.

How it works

Researchers can build multiple playlists of activities. Each playlist can combine arbitrary sequences of games, simulations, quizzes and instructional multimedia content. Multiple playlists can be collected into an experiment which randomizes visitors across the playlists for A/B/n testing. The experiment can be shared via a URL with the target audience or crowd workers (e.g. Amazon Mechanical Turk). When the user opens the URL, a random playlist from the experiment is selected and assigned which helps implement randomized controlled trials (RCT) at the individual or group level.


  • Conduct research with rich interactive game and simulation content.
  • Easy sharing via a URL.
  • Collect rich data and export to desired format.
  • Visualize using interactive dashboard.
  • It’s the simplest way to do rapid A/B/n testing for content optimization and research.



  • Add any number of playlists into an experiment.
  • Equally distributes playlists to students or crowd workers.
  • Can be distributed into Amazon Mechanical Turk.