Justin Krometis

Research Associate Professor, Virginia Tech National Security Institute
“Recent Methodological Advances for Integrated T&E”

Speaker Bio: 

Justin Krometis is a Research Associate Professor with the Virginia Tech National Security Institute and holds an affiliate position in the Math Department at Virginia Tech. He has led the Integrated Testing & Continuous Evaluation line of effort for the Acquisition Innovation Research Consortium’s support of DOT&E for the last three years. His research is largely in the development of theoretical and computational frameworks for Bayesian data analysis. These include approaches to incorporating and balancing data and expert opinion into decision-making, estimating model parameters, including high- or even infinite-dimensional quantities, from noisy data, and designing experiments to maximize the information gained. His research interests include: Parameter Estimation, Uncertainty Quantification, Experimental Design, High-Performance Computing, Artificial Intelligence/Machine Learning (AI/ML), and Reinforcement Learning.

Abstract: 

This session will describe recent work on integrating disparate data sources, such as from developmental and operational testing, to build common understanding of system behavior and to inform subsequent design of tests. The tutorial will compare frequentist and Bayesian approaches for conducting this data synthesis and describe associated pitfalls and best practices. The tutorial will walk through examples on example test data describing reliability, accuracy, and hit/miss or pass/fail. Finally, it will describe efforts to build these methods into hands-on tools to facilitate incorporation into defense and other programs. We will then conclude with a short opportunity for discussion about shortfalls that might be preventing adoption and where research needs to go to translate these methods and tools into practice.