DATAWorks 2023 Agenda Day 2

April 26th Agenda

Audience Legend

  •  1 

    Everyone.

    These talks should be accessible to everyone regardless technical background.

  •  2 

    Practitioners.

    These talks might include case studies with some discussion of methods and coding, but largely accessible to a non-technical audience.

  •  3 

    Technical Experts.

    These talks will likely delve into technical details of methods and analytical computations and are primarily aimed at practitioners advancing the state of the art.

7:30 AM – 8:30 AM

Check-in

8:30 AM – 8:40 AM

Room A+B

Virtual Session

Opening Remarks

The Honorable Nickolas H. Guertin

(OSD/DOT&E)


8:40 AM – 9:20 AM

Room A+B

Virtual Session

Keynote 1

Col Sacha Tomlinson

(United States Space Force)


9:20 AM – 10:00 AM

Room A+B

Virtual Session

Keynote 2

Mr. Peter Coen

(NASA)


10:00 AM – 10:20 AM

Break

10:20 AM – 12:20 PM: Parallel Sessions

Room A

Virtual Session
Session 1A: Cybersecurity in Test & Evaluation
Session Chair: Mark Herrera, IDA
  •  1 

    Planning for Public Sector Test and Evaluation in the Commercial Cloud

    Brian Conway (IDA)

    Materials:

  •  2 

    Cyber Testing Embedded Systems with Digital Twins

    Michael Thompson (Naval Postgraduate School)

    Materials:

  •  2 

    Test and Evaluation of AI Cyber Defense Systems

    Shing-hon Lau (Software Engineering Institute, Carnegie Mellon University)

    Materials:

  •  1 

    Test and Evaluation of Systems with Embedded Artificial Intelligence Components

    Michael Smith (Sandia National Laboratories)

    Materials:

Room B

Virtual Session
Session 1B: Validation of Statistical Software and Self-Validated Ensemble . . . Models
Session Chair: Tom Donnelly, JMP Statistical Discovery
  •  1 

    On the Validation of Statistical Software

    Ryan Lekivetz (JMP Statistical Discovery)

    Materials:

  •  1 

    Validating the Prediction Profiler with Disallowed Combination: A Case Study

    Yeng Saanchi (JMP Statistical Discovery)

    Materials:

  •  2 

    Introducing Self-Validated Ensemble Models (SVEM) – Bringing Machine Learning to DOEs

    Chris Gotwalt (JMP Statistical Discovery)

    Materials:

  •  1 

    Effective Application of Self-Validated Ensemble Models in Challenging Test Scenarios

    James Wisnowski (Adsurgo)

    Materials:

Room C

Session 1C: Statistical and Systems Engineering Applications in Aerospace
Session Chair: Melissa Hooke
  •  2 

    The Containment Assurance Risk Framework of the Mars Sample Return Program

    Giuseppe Cataldo (NASA)

    Materials:

  •  3 

    A Bayesian Optimal Experimental Design for High-dimensional Physics-based Models

    James Oreluk (Sandia National Laboratories)

    Materials:

  •  1 

    Systems Engineering Applications of UQ in Space Mission Formulation

    Kelli McCoy and Roger Ghanem (NASA Jet Propulsion Laboratory, University of Southern California)

    Materials:

  •  3 

    *Virtual Speaker* Epistemic and Aleatoric Uncertainty Quantification for Gaussian Processes

    Pau Batlle (California Institute of Technology)

Room E

Virtual Session
Mini-Tutorial 1
Session Chair: George Khoury, IDA
  •  1 

    Introduction to Design of Experiments in R: Generating and Evaluating Designs with skpr

    Tyler Morgan-Wall (IDA)

    Materials:

12:20 PM – 1:30 PM

Lunch

1:30 PM – 3:00 PM: Parallel Sessions

Room A

Virtual Session
Session 2A: Statistical Engineering
Session Chair: Peter Parker, NASA Langley Research Center
  •  1 

    An Overview of the NASA Quesst Community Test Campaign with the X-59 Aircraft

    Jonathan Rathsam (NASA Langley Research Center)

    Materials:

  •  1 

    Dose-Response Data Considerations for the NASA Quesst Community Test Campaign

    Aaron Vaughn (NASA Langley Research Center)

    Materials:

  •  2 

    Infusing Statistical Thinking into the NASA Quesst Community Test Campaign

    Nathan Cruze (NASA Langley Research Center)

    Materials:

Room B

Virtual Session
Session 2B: Situation Awareness, Autonomous Systems, and Digital Engineering in . . . T&E
Session Chair: Elizabeth Green, IDA
  •  1 

    Towards Scientific Practices for Situation Awareness Evaluation in Operational Testing

    Miriam Armstrong (IDA)

    Materials:

  •  1 

    T&E Landscape for Advanced Autonomy

    Kathryn Lahman (Johns Hopkins University Applied Physics Laboratory)

    Materials:

  •  1 

    Digital Transformation Enabled by Enterprise Automation

    Nathan Pond (Edaptive Computing, Inc.)

    Materials:

Room C

Students & Fellows Speed Session 1

Please note many of the Speed Sessions will also include a poster during the Poster Session

Session Chair: Alyson Wilson, NC State
  •  1 

    Comparing Normal and Binary D-optimal Design of Experiments by Statistical Power

    Addison Adams (IDA / Colorado State University)

    Materials:

  •  2 

    Uncertain Text Classification for Proliferation Detection

    Andrew Hollis (North Carolina State University)

    Materials:

  •  1 

    A Data-Driven Approach of Uncertainty Quantification on Reynolds Stress Based on DNS Turbulence Data

    Zheming Gou (University of Southern California)

    Materials:

  •  1 

    I-TREE: a Tool for Characterizing Research Using Taxonomies

    Aayushi Verma (IDA)

    Materials:

  •  2 

    An Evaluation Of Periodic Developmental Reviews Using Natural Language Processing

    Dominic Rudakevych (United States Military Academy)

    Materials:

  •  2 

    Optimal Release Policy for Covariate Software Reliability Models.

    Ebenezer Yawlui (University of Massachusetts Dartmouth)

    Materials:

  •  2 

    A Stochastic Petri Net Model of Continuous Integration and Continuous Delivery

    Sushovan Bhadra (University of Massachusetts Dartmouth)

    Materials:

  •  2 

    Neural Networks for Quantitative Resilience Prediction

    Karen Alves da Mata (University of Massachusetts Dartmouth)

    Materials:

  •  1 

    A generalized influence maximization problem

    Sumit Kumar Kar (University of North Carolina at Chapel Hill)

Room D

Contributed Session 1
Session Chair: Elizabeth Gregory, NASA Langley Research Center
  •  1 

    Avoiding Pitfalls in AI/ML Packages

    Justin Krometis (Virginia Tech National Security Institute)

    Materials:

  •  3 

    Reinforcement Learning Approaches to the T&E of AI/ML-based Systems Under Test

    Karen O’Brien (Modern Technology Solutions, Inc)

    Materials:

  •  2 

    Analysis Opportunities for Missile Trajectories

    Kelsey Cannon (Lockheed Martin Space)

    Materials:

  •  1 

    *Virtual Speaker* Using Changepoint Detection and Artificial Intelligence to Classify Fuel Pressure Behaviors in Aerial Refueling

    Nelson Walker and Michelle Ouellette (United States Air Force)

    Materials:

Room E

Virtual Session
Mini-Tutorial 2
Session Chair: Caleb King, JMP Statistical Discovery
  •  2 

    A Tour of JMP Reliability Platforms and Bayesian Methods for Reliability Data

    Peng Liu (JMP Statistical Discovery)

    Materials:

    Materials URL:

3:00 PM – 3:20 PM

Break

3:20 PM – 4:50 PM: Parallel Sessions

Room A

Virtual Session
Session 3A: Methods & Applications of M&S Validation
Session Chair: Benjamin Ashwell, IDA
  •  1 

    Model Verification in a Digital Engineering Environment: An Operational Test Perspective

    Jo Anna Capp (IDA)

    Materials:

  •  2 

    Recommendations for Statistical Analysis of Modeling and Simulation Environment Outputs

    Curtis Miller (IDA)

    Materials:

  •  2 

    Back to the Future: Implementing a Time Machine to Improve and Validate Model Predictions

    Olivia Gozdz and Kyle Remley (IDA)

    Materials:

Room B

Virtual Session
Session 3B: Software Quality
Session Chair: Sushovan Bhadra, UMASS Dartmouth
  •  2 

    Assessing Predictive Capability and Contribution for Binary Classification Models

    Mindy Hotchkiss (Aerojet Rocketdyne)

    Materials:

  •  2 

    STAR: A Cloud-based Innovative Tool for Software Quality Analysis

    Kazu Okumoto (Sakura Software Solutions (3S) LLC)

    Materials:

  •  2 

    Covariate Software Vulnerability Discovery Model to Support Cybersecurity T&E

    Lance Fiondella (University of Massachusetts)

    Materials:

Room C

Students & Fellows Speed Session 2

Please note many of the Speed Sessions will also include a poster during the Poster Session

Session Chair: Jim Starling, United States Military Academy at West Point
  •  3 

    A Bayesian Approach for Nonparametric Multivariate Process Monitoring using Universal Residuals

    Daniel Timme (Florida State University)

    Materials:

  •  1 

    Implementing Fast Flexible Space Filling Designs In R

    Christopher Dimapasok (IDA / Johns Hopkins University)

    Materials:

  •  3 

    Development of a Wald-Type Statistical Test to Compare Live Test Data and M&S Predictions

    Carrington Metts (IDA)

    Materials:

  •  1 

    Energetic Defect Characterizations

    Naomi Edegbe (United States Military Academy)

    Materials:

  •  2 

    Covariate Resilience Modeling

    Priscila Silva (additional authors: Andrew Bajumpaa, Drew Borden, and Christian Taylor) (University of Massachusetts Dartmouth)

    Materials:

  •  2 

    Application of Software Reliability and Resilience Models to Machine Learning

    Zakaria Faddi (University of Massachusetts Dartmouth)

    Materials:

  •  2 

    Application of Recurrent Neural Network for Software Defect Prediction

    Fatemeh Salboukh (University of Massachusetts Dartmouth)

    Materials:

  •  2 

    Topological Data Analysis’ involvement in Cyber Security

    Anthony Cappetta and Elie Alhajjar (United States Military Academy)

    Materials:

Room D

Contributed Session 2
Session Chair: Jim Warner, NASA Langley Research Center
  •  3 

    Empirical Calibration for a Linearly Extrapolated Lower Tolerance Bound

    Caleb King (JMP Statistical Discovery)

    Materials:

  •  2 

    Analysis of Surrogate Strategies and Regularization with Application to High-Speed Flows

    Gregory Hunt (William & Mary)

    Materials:

  •  2 

    Case Study on Test Planning and Data Analysis for Comparing Time Series

    Phillip Koshute (Johns Hopkins University Applied Physics Laboratory)

    Materials:

  •  1 

    Model Validation Levels for Model Authority Quantification

    Kyle Provost (STAT COE)

    Materials:

Room E

Virtual Session
Mini-Tutorial 3
Session Chair: Jason Sheldon, IDA
  •  1 

    Data Management for Research, Development, Test, and Evaluation

    Heather Wojton (IDA)

  • Matthew Avery (IDA)

    Materials:

5:00 PM – 7:00 PM: Parallel Sessions

Café

Poster Session and Reception
  •  1 

    Comparison of Magnetic Field Line Tracing Methods

    Dean Thomas (George Mason University)

  •  1 

    Framework for Operational Test Design: An Example Application of Design Thinking

    Miriam Armstrong (IDA)

  •  2 

    The Component Damage Vector Method: A Statistically Rigorous Method for Validating AJEM

    Tom Johnson (IDA)

  •  1 

    Fully Bayesian Data Imputation using Stan Hamiltonian Monte Carlo

    Melissa Hooke (NASA Jet Propulsion Laboratory)

  •  1 

    Introducing TestScience.org

    Sean Fiorito (IDA / V-Strat, LLC)

  •  1 

    Developing a Domain-Specific NLP Topic Modeling Process for Army Experimental Data

    Anders Grau (United States Military Academy)

  •  2 

    The Application of Semi-Supervised Learning in Image Classification

    Elijah Dabkowski (United States Military Academy)

  •  1 

    Best Practices for Using Bayesian Reliability Analysis in Developmental Testing

    Paul Fanto (IDA)

  •  2 

    Test and Evaluation Tool for Stealthy Communication

    Olga Chen (U.S. Naval Research Laboratory)

  •  2 

    Comparison of Bayesian and Frequentist Methods for Regression

    James P Theimer (Homeland Security Community of Best Practices)

  •  2 

    Post-hoc UQ of Deep Learning Models Applied to Remote Sensing Image Scene Classification

    Alexei Skurikhin (Los Alamos National Laboratory)

  •  2 

    Multimodal Data Fusion: Enhancing Image Classification with Text

    Jack Perreault (United States Military Academy)

  •  1 

    Predicting Success and Identifying Key Characteristics in Special Forces Selection

    Mark Bobinski (United States Military Academy)

  •  2 

    The Calculus of Mixed Meal Tolerance Test Trajectories

    Skyler Chauff (United States Military Academy)

  •  1 

    Using Multi-Linear Regression to Understand Cloud Properties’ Impact on Solar Radiance

    Grant Parker (United States Military Academy)

  •  2 

    Data Fusion: Using Data Science to Facilitate the Fusion of Multiple Streams of Data

    Madison McGovern (United States Military Academy)

  •  1 

    Assessing Risk with Cadet Candidates and USMA Admissions

    Daniel Lee (United States Military Academy)

  •  1 

    Overarching Tracker of DOT&E Actions

    Buck Thome (IDA)

PosterMap