The rise of Artificial Intelligence is clearing the path to a pilotless future. As such, the risks associated with piloting will shift from human-centric decision making to the inherent quality of the algorithms and sensors within the autonomous system. Currently, pilot readiness is quantified based on simulation training and real-world demonstration of skills. Simulators are currently built to train and test the human. But what happens when the human is taken out of the decision loop? As these systems and decisions are automated, we need simulators and simulation platforms that cater to testing the algorithms as well. Currently, there is no system in place to quantify the efficacy of autonomous systems before they deploy. At best, some manufacturer self-certify their systems based on internal tests, but there is no validation mechanism in place to ensure objectivity.