Themes

Breakout sessions and mini-tutorials will be grouped throughout the workshop by conceptual themes. Breakout sessions are intended to promote discussion and knowledge sharing, while mini-tutorials advance the education and refine analytical tools. The breakout sessions and mini-tutorials planned for Wednesday and Thursday (April 1st and 2nd) will focus on the following themes:

1. Progress and Challenges in Test and Evaluation

   Show Theme Description

   The DoD and NASA continue to develop and acquire some of the world’s most advanced and sophisticated systems and technology. Together we face similar challenges in ensuring that these systems undergo adequate and efficient test and evaluation prior to use. Sessions will feature case study examples highlighting advancements in T&E methodologies we have made as a community. Topics covered may include modeling and simulation validation, uncertainty quantification, the use of design of experiments, Bayesian analysis, amongst others. Sessions will also discuss the current challenges and areas where continued research and advancement are needed. 

2. Machine Learning and Artificial Intelligence

   Show Theme Description

   Sessions will discuss how we can quantify confidence that machine learning and/or artificial intelligence algorithms function as intended and are free of vulnerabilities, either intentionally or unintentionally designed or inserted as part of the data/algorithm.  Topics may include test and evaluation metrics and methods for AI algorithms. Threat portrayal for ML/AI algorithms. Designing robust ML/AI algorithms. Test and evaluation metrics and methods for cyber-physical systems that incorporate AI algorithms.

3. Cybersecurity and Software Test and Evaluation

   Show Theme Description

   Sessions will discuss metrics and test methodologies for testing cyber-physical systems with an emphasis on characterizing cyber resiliency. Identification of cybersecurity threats (IP and non-IP) across the critical operational missions.  Methods for developing and testing secure architectures.  Methods for identifying critical system components that enable attack vectors. Development and testing of countermeasures to cyber-attacks. Test methods for cyber-physical systems that include machine learning algorithms.

4. Modeling the Human-System Interaction

   Show Theme Description

   Mission outcomes are determined by how effectively operators interact with systems under operational conditions. Sessions will discuss methods used to collect and model the quality of human-system interaction and its impact on operational performance.

5. Science Applications of Test and Evaluation

   Show Theme Description

   Sessions will cover simulation, prediction, uncertainty quantification, and inference for physical and physical-statistical modeling of geophysical processes. Examples of topics that will be discussed include problems in Earth Science (models of ice sheet evolution, atmospheric and ocean processes, the carbon cycle, land surface processes, natural hazards), astronomy and cosmology (exoplanet detection, galactic formation, cosmic microwave background), and Planetary Science (planetary atmospheres, formation, etc.).