Short Course Instructors

April 1st

8:00 AM – 9:00 AM
Waterford Lobby

Check-In

9:00 AM – 9:40 AM
Singleton-Miller Ballroom

Morning Keynote – Michael Seablom, Science Mission Directorate

9:40 AM – 10:00 AM

Break

10:00 AM – 11:30 AM
Korman Ballroom

Modeling the Human-System Interaction
Session Chair: Heather Wojton
The Challenge of Data for Predicting Human Errors – Thomas Ulrich, Idaho National Laboratory
Improving Cyber Resiliency through the Application of Behavioral Science to . . . Cybersecurity – Poornima Madhavan, MITRE
Development and Analytic Process Used to Develop a 3-Dimensional Graphical User . . . Interface System for Baggage Screening – Charles McKee, DHS
Process/Workflow-Oriented View for Decision-Support Systems – Nicole Murchison, Sandia National Laboratories

10:00 AM – 11:30 AM
Sellier Ballroom

Model & Simulation Validation
A Notional Case Study of Uncertainty Analysis in Live Fire Modeling and . . . Simulation – Thomas Johnson, IDA
A Validation Case Study: The Environment Centric Weapons Analysis Facility . . . (ECWAF) – Elliot Bartis, IDA
Integrating Systems Engineering into Test Strategy Development and Systems . . . Evaluation – Charlie Middleton, OSD Scientific Test and Analysis Techniques Center of Excellence
Building an End-to-End Model of Super Hornet Readiness – Benjamin Ashwell, IDA

10:00 AM – 11:30 AM
Von Sternberg Ballroom

Statistics and Machine Learning for Cybersecurity Threat Detection
Session Chair: Justin Newcomber
Dynamic Model Updating for Streaming Classification and Clustering – Alexander Foss, Sandia National Laboratories
Spectral Embedding and Cyber Networks – David Marchette, Naval Surface Warfare Center, Dahlgren Division
Graph link prediction in computer networks using Poisson Matrix Factorisation – Melissa Turcotte, Los Alamos National Laboratory
AI for Cyber – Adam Cardinal-Stakenas, NSA

10:00 AM – 11:30 AM
Gibson Ballroom

Mini-Tutorial: Design of Experiments
I Have the Power! Power Calculation in Complex (and Not So Complex) Modeling . . . Situations – Caleb King and Ryan Lekivetz, JMP Division, SAS Institute Inc.

11:30 AM – 1:00 PM

Lunch – Yisroel Brumer, CAPE

1:00 PM – 1:15 PM

Break

1:15 PM – 2:45 PM
Korman Ballroom

Statistical Engineering
Session Chair: Martin Bezener
The Role of Statistical Engineering in Creating Solutions for Complex . . . Opportunities – Geoff Vining, Virginia Tech
Statistical Engineering for Service Life Prediction of Polymers – Adam Pintar, National Institute of Standards and Technology
Sequential Testing and Simulation Validation for Autonomous Systems – Jim Simpson, JK Analytics

1:15 PM – 2:45 PM
Sellier Ballroom

Cybersecurity and Software Test & Evaluation
Session Chair: Duane Steffey
Connecting Software Reliability Growth Models to Software Defect Tracking – Lance Fiondella, University of Massachusetts
Title Coming Soon – Kedar Phadke, Phadke Associates, Inc
Natural Language Processing for Safety-Critical Requirements – Ying Shi, NASA GSFC
Can AI Predict Human Behavior? – Dustin Burns, Exponent

1:15 PM – 2:45 PM
Von Sternberg Ballroom

Topic Contributed Session
Design Fractals: Visualizing the Coverage Properties of Covering Arrays – Joseph Morgan, SAS Institute Inc
Optimal Designs for Multiple Response Distributions – Brittany Fischer, Arizona State University
Expanding the Systems Test Engineer’s Toolbox A Multi-Criteria Decision Analysis . . . Technique using the One-Sided Tolerance Interval for Analysis of DOE Planned Tests – Michael Sheehan, MITRE
Aliased Informed Model Selection for Nonregular Designs – Carly Metcalfe, Arizona State University
Combining Physics-Based Simulation & Machine Learning for Fast Uncertainty . . . Quantification – James Warner, NASA Langley Research Center

1:15 PM – 2:45 PM
Gibson Ballroom

Mini-Tutorial: Deep Learning
Deep Learning Models for Image Analysis & Object Recognition – Ridhima Amruthesh, DATA SOCIETY

2:45 PM – 3:00 PM

Break

3:00 PM – 4:30 PM
Korman Ballroom

Autonomous Systems
The Science of Trust of Autonomous Unmanned Systems – Reed Young, Johns Hopkins Applied Physics Laboratory
A Path to Fielding Autonomous Systems – Craig Lennon, CCDC Army Research Lab
Trusted Collaborative Autonomous Systems for Multi Domain Operations – Robert Murphey, Georgia Tech Research Institute : Materials:

3:00 PM – 4:30 PM
Sellier Ballroom

Science Applications of Test and Evaluation
International journal of astrobiology – Kelli Mccoy, JPL
An Approach to Assessing Sterilization Probabilistically – Mike Dinicola, Jet Propulsion Laboratory
Reevaluating Planetary Protection Bioburden Accounting and Verification . . . Reporting – James Benardini, Jet Propulsion Laboratory

3:00 PM – 4:30 PM
Von Sternberg Ballroom

Topic Contributed Session
Introduction to Structural Equation Modeling: Implications for Human-System . . . Interactions – Laura Castro-Schilo, JMP Division, SAS Institute, Inc.
Orbital position in space: What is Truth? A study comparing two astrodynamics . . . systems – Jason Sheldon, IDA
Machine Learning Reveals that Russian IRA’s Twitter Topic Patterns Evolved over . . . Time – Emily Parrish, IDA
Dashboard for Equipment Failure Reports – Cole Molloy, Johns Hopkins Applied Physics Lab
Unlock Trends in Instrument Performance with Recurrence Studies and Text . . . Analyses – Rob Lievense, SAS/JMP

3:00 PM – 4:30 PM
Gibson Ballroom

Mini-Tutorial: Design of Experiments
Strategies for Sequential Experimentation – Martin Bezener, Stat-Ease

3:00 PM – 4:30 PM
Singleton-Miller Ballroom

Mini-Tutorial: Cybersecurity and Software Test & Evaluation
Taking Down a Turret: Introduction to Cyber Operational Test and Evaluation – Peter Mancini, Lee Allison, Mark Herrera, Jason Schlup, Kelly Tran, IDA

4:30 PM – 4:45 PM

Break

4:45 PM – 5:15 PM
Sellier Ballroom

Spotlight Talk
Session Chair: Amy Braverman
Quantifying Science Needs and Mission Requirements through Uncertainty . . . Quantification – Carmen Boening, Jet Propulsion Laboratory/California Institute of Technology

4:45 PM – 5:15 PM
Von Sternberg Ballroom

Spotlight Talk
Session Chair: Doug Ray
Data Science from Scratch in the DoD – Cade Saie, US Army Futures Command

4:45 PM – 5:15 PM
Gibson Ballroom

Spotlight Talk
Session Chair: Reed Young
The Path to Assured Autonomy – Cara LaPointe, Johns Hopkins University Institute for Assured Autonomy

5:30 PM – 7:00 PM

Poster Session and Reception
Characterizing the Orbital Debris Environment Using Satellite Perturbation . . . Anomaly Data – Joel Williamsen, IDA
Analyzing Miss Distance – Kevin Kirshenbaum, IDA
Overarching Tracker: A trend Analysis of System performance data – Caitlan Fealing, IDA
Build Better Graphics – Brian Vickers, IDA
CVN 78 Sortie Generation Rate – Dean Thomas, IDA
Modeling Human-System Interaction in UAM: Design and Application of . . . Human-Autonomy Teaming – Vincent E. Houston, NASA
Assessing the reliability of prediction intervals from Bayesian Neural Networks – Daniel Ries, Sandia National Lab
Creating formal characterizations of routine contingency management in . . . commercial aviation – Jon Holbrook, NASA
Connecting Software Reliability Growth Models to Software Defect Tracking – Melanie Luperon, University of Massachusetts
A Simulation Study of Binary Response Models for the Small-Caliber Primer . . . Sensitivity Test – Zach Krogstad, U.S. Army

April 2nd

8:00 AM – 8:30 AM

Check-In

8:30 AM – 9:10 AM

Morning Keynote – Greg Zacharias, DOT&E

9:10 AM – 9:30 AM

Break

9:30 AM – 11:00 AM
Korman Ballroom

Challenges of Testing in the Evolving Acquisition Environment
Session Chair: Brian Stone
D-Optimally Based Sequential Test Method for Ballistic Limit Testing with a . . . Change in Mech – Leonard Lombardo, U.S. Army Aberdeen Test Center
KC-46A Adaptive Relevant Testing (ART) Strategies to Enable Incremental . . . Evaluation – Joseph Stank, AFOTEC
The OTA Perspective on the Challenges of Testing in the Evolving Acquisition . . . Environment and Proposed Mitigations – Leisha Scheiss, AFOTEC

9:30 AM – 11:00 AM
Sellier Ballroom

Science Applications of Test and Evaluation
Session Chair: Carmen Boening
Post-hoc Uncertainty Quantification for Remote Sensing Observing Systems – Amy Braverman, Jet Propulsion Laboratory, California Institute of Technology
An end-to-end uncertainty quantification framework in predictive ocean data . . . science – Patrick Heimbach, University of Texas at Austin
Calibrate, Emulate, Sample – Alfredo Garbuno-Inigo, Caltech

9:30 AM – 11:00 AM
Von Sternberg Ballroom

Progress and Challenges in Program Evaluation
Session Chair: Jim Simpson
Employing Design of Experiments (DOE) in an Electronic Warfare Test . . . Strategy/Design – Michael Harman, STAT COE
Bayesian Logistic Regression with Separated Data – Jason Martin, U.S. Army CCDC Aviation and Missile Center
Practical Applications for Functional Data Analysis in T&E – James Wisnowski, Adsurgo LLC

9:30 AM – 11:00 AM
Gibson Ballroom

Mini-Tutorial: Machine Learning and Cybersecurity
Machine Learning for cybersecurity – Self Learning Systems for Threat Detection – Nisha Iyer, DATA SOCIETY

11:00 AM – 12:30 PM

Lunch Keynote – Laura McNamara, Sandia National Laboratories

12:45 PM – 2:15 PM
Korman Ballroom

Science Applications of Test and Evaluation
Session Chair: Eric Walker , NASA
Quantifying Computational Uncertainty in Supersonic Flow Predictions without . . . Experimental Measurements – Thomas West, NASA Langley
Physics-Informed Deep Learning for Modeling and Simulation under Uncertainty – Patrick Leser, NASA Langley Research Center
Resilience and Productive Safety Considerations for In-Time System-Wide Safety . . . Assurance – Lawrence J. Prinzel III, NASA Langley Research Center

12:45 PM – 2:15 PM
Sellier Ballroom

Undergraduate Research Session
Session Chair: Nick Clark
Data Analysis for Special Operations Selection Programs – Nicholas Cunningham, United States Military Academy
Predicting System Failures – A Statistical Approach to Reliability Growth . . . Modeling – Kate Sanborn, North Carolina State University
Automated Road Extraction from Satellite Images – Trevor Parker, United States Military Academy
Balancing Human & Virtual Decision Making for Identifying Fraudulent Credit Card . . . Activity – Rao Abdul Hannan, United States Air Force Academy

12:45 PM – 2:15 PM
Von Sternberg Ballroom

Uncertainty Quantification for Machine Learning
The Role of Uncertainty Quantification in Machine Learning – David Stracuzzi, Sandia National Laboratories
Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learning – Jize Zhang, Lawrence Livermore National Laboratory
The Role of Sensitivity Analysis in Evaluating the Credibility of Machine . . . Learning – Kyle Hickmann, Los Alamos National Laboratory
Uncertainty Quantification and Decomposition Methods for Risk-Sensitive Machine . . . Learning – Ahmad Rushdi, Sandia National Laboratories

12:45 PM – 2:15 PM
Gibson Ballroom

Mini-Tutorial: Design of Experiments
21st Century Screening Designs – Thomas Donnelly, SAS Institute Inc.

2:15 PM – 2:30 PM

Break

2:30 PM – 4:00 PM
Korman Ballroom

Machine Learning and Artificial Intelligence
A HellerVVA Problem: The Catch-22 for Simulated Testing of Fully Autonomous . . . Systems – Daniel Porter, IDA
STAT COE Autonomy Test and Evaluation Workshop Highlights – Troy Welker, STAT COE
Finding a Cheater: Machine Learning and the Dice Game “Craps” – Paul Johnson, MCOTEA

2:30 PM – 4:00 PM
Sellier Ballroom

Moedling the Human-System Interaction
Quantifying Uncertainty in Reporting System Usability Scale Results – Nick Clark, West Point – Math Department
Creating Insight into Joint Cognitive System Performance – Taylor Murphy, Air Force Research Laboratory
Generalized Linear Multilevel Model Applied to a Forced-Choice Psychoacoustic . . . Test – Matthew Boucher, NASA Langley Research Center

2:30 PM – 4:00 PM
Von Sternberg Ballroom

Machine Learning and Artificial Intelligence
Bayesian Experimental Design to augment a sensor network – Emily Casleton, Los Alamos National Laboratory
Development of Predictive Models for Brain-Computer Interface Systems: A Case . . . Study – Kevin Eng, US Army CCDC Armaments Center
Designing Competitions that Reward Robust Algorithm Performance – Kary Myers, Los Alamos National Laboratory

2:30 PM – 4:00 PM
Gibson Ballroom

Mini-Tutorial: Uncertainty Quantification
Introduction to Uncertainty Quantification for Practitioners and Engineers – Gavin Jones, SmartUQ