DATAWorks Agenda
DATAWorks Map
April 18th, 2024
7:30 AM – 8:30 AM:
Check-In / Light Breakfast
8:30 AM – 8:45 AM: Room A+B
Opening Remarks
Dr. Bram Lillard, Director, IDA
8:45 AM – 9:25 AM: Room A+B
Keynote 3
Dr. Missy Cummings, Professor, George Mason University
9:25 AM – 9:50 AM:
Morning Break
9:50 AM – 11:50 AM: Room A
Session 4A: Methods, Applications, and Tools for AI
Session Chair: Brian Vickers
Data VV&A for AI Enabled Capabilities
John Dennis, Research Staff Member (Economist), IDA
Theme: Advancing Test & Evaluation of Emerging and Prevalent Technologies
Audience: 2 – Practitioners.
Development, Test, and Evaluation of Small-Scale Artificial Intelligence Models
David Niblick, AI Evaluator, Army Test and Evaluation Command
Audience: 1 – Everyone.
MLTEing Models: Negotiating, Evaluating, and Documenting Model and System Qualities
Audience: 2 – Practitioners.
Bringing No-Code Machine Learning to the average user
Alex Margolis, Subject Matter Expert, Edaptive Computing, Inc
Theme: Sharing Analysis Tools, Methods, and Collaboration Strategies
Audience: 2 – Practitioners.
9:50 AM – 11:50 AM: Room B
Session 4B: Statistical Methods for T&E
Session Chair: Corinne Stafford
Failure Distributions for Parallel Dependent Identical Weibull Components
Gina Sigler, STAT COE contractor, HII/STAT COE
Theme: Solving Program Evaluation Challenges
Assessing the Calibration and Performance of Attention-based Spatiotemporal Neural Network
Nathan Gaw, Assistant Professor of Data Science, Air Force Institute of Technology
Theme: Advancing Test & Evaluation of Emerging and Prevalent Technologies
Case Study: State Transition Maps for Mission Model Development and Test Objective Identifiers
Nicholas Jones, STAT Expert, STAT COE
Theme: Sharing Analysis Tools, Methods, and Collaboration Strategies
Audience: 1 – Everyone.
Bayesian Reliability Growth Planning for Discrete Systems
Harris Bernstein, Data Scientist, Johns Hopkins University Applied Physics Lab
Theme: Improving the Quality of Test & Evaluation
Audience: 1 – Everyone.
9:50 AM – 11:50 AM: Room C
Session 4C: Cost Estimation
Session Chair: Kelli McCoy
Cost Considerations for Estimating Small Satellite Integration & Test
Audience: 1 – Everyone.
Examining the Effects of Implementing Data-Driven Uncertainty in Cost Estimating Models
Victoria Nilsen, Operations Research Analyst, NASA HQ
Theme: Solving Program Evaluation Challenges
Audience: 2 – Practitioners.
Best of Both Worlds: Combining Parametric Cost Risk Analysis with Earned Value Management
Christian Smart and Murray Cantor
Theme: Solving Program Evaluation Challenges
Audience: 2 – Practitioners.
Beyond the Matrix: The Quantitative Cost and Schedule Risk Management Imperative
Christian Smart, Senior Programmatic Risk Analyst, JPL
Theme: Solving Program Evaluation Challenges
9:50 AM – 11:50 AM: Room D
Session 4D/Mini-Tutorial 4: Bayesian Methods
Session Chair: John Haman
Leveraging Bayesian Methods to support Integrated Testing
Justin Krometis and Adam Ahmed
Audience: 2 – Practitioners.
Dynamo: Adaptive T&E via Bayesian Decision Theory
James Ferry, Principal Research Scientist, Metron, Inc.
Theme: Improving the Quality of Test & Evaluation
11:50 AM – 1:00 PM:
Lunch
1:00 PM – 3:00 PM: Room A
Contributed Session 1 – AI & Large Language Models
Session Chair: Jay Dennis
Demystifying Deep Learning – Aircraft Identification from Satellite Images
Cameron Liang, Research Staff Member, IDA
Theme: Sharing Analysis Tools, Methods, and Collaboration Strategies
Audience: 2 – Practitioners.
Unlocking our Collective Knowledge: LLMs for Data Extraction from Long-Form Documents
Patrick Bjornstad, Systems Engineer I, Jet Propulsion Laboratory
Theme: Advancing Test & Evaluation of Emerging and Prevalent Technologies
Audience: 2 – Practitioners.
Creating Workflows for Synthetic Data Generation and Advanced Military Image Classificatio
James Starling, Associate Professor, United States Military Academy
Theme: Sharing Analysis Tools, Methods, and Collaboration Strategies
Audience: 2 – Practitioners.
Hypersonic Glide Vehicle Trajectories: A conversation about synthetic data in T&E
Karen O’Brien, Senior Principal Data Scientist, Modern Technology Solutions, Inc
Theme: Advancing Test & Evaluation of Emerging and Prevalent Technologies
Audience: 2 – Practitioners.
Sensor Fusion for Automated Gathering of Labeled Data in Edge Settings
Robert Edman, Machine Learning Research Scientist, Software Engineering Institute
Audience: 2 – Practitioners.
Experimental Design for Usability Testing of LLMs
Jasmine Ratchford, Research Scientist, CMU Software Engineering Institute
Theme: Improving the Quality of Test & Evaluation
Audience: 1 – Everyone.
1:00 PM – 3:00 PM: Room B
Contributed Session 2 – Cyber, UQ & Simulations
Session Chair: Pete Mancini
Design of In-Flight Cyber Experimentation for Spacecraft
Meghan Sahakian, Principal Member of Technical Staff, Sandia National Laboratories
Theme: Improving the Quality of Test & Evaluation
Lessons Learned for Study of Uncertainty Quantification in Cyber-Physical System Emulation
Jamie Thorpe, Cybersecurity R&D, Sandia National Laboratories
Theme: Solving Program Evaluation Challenges
Audience: 2 – Practitioners.
Cobalt Strike: A Cyber Tooling T&E Challenge
Nathan Wray, Senior Operator, DOT&E ACO
Theme: Improving the Quality of Test & Evaluation
Audience: 2 – Practitioners.
Global Sensitivity Analyses for Test Planning under Constraints with Black-box Models
Giuseppe Cataldo, Planetary Protection Lead, NASA
Theme: Sharing Analysis Tools, Methods, and Collaboration Strategies
Audience: 2 – Practitioners.
Simulated Multipath Using Software Generated GPS Signals
Russell Gilabert, Researcher/Engineer, NASA Langley Research Center
Theme: Advancing Test & Evaluation of Emerging and Prevalent Technologies
Audience: 2 – Practitioners.
1:00 PM – 3:00 PM: Room C
Contributed Session 3 – Stats and Survey Methods
Session Chair: Elise Roberts
A Bayesian Approach for Credible Modeling, Simulation, and Analysis
Terril Hurst, Senior Engineering Fellow, Raytheon
Audience: 1 – Everyone.
The Role of Bayesian Multilevel Models in Performance Measurement and Prediction
Austin Amaya and Sean Dougherty
Theme: Advancing Test & Evaluation of Emerging and Prevalent Technologies
Audience: 2 – Practitioners.
Overview of a survey methods test for the NASA Quesst community survey campaign
Jonathan Rathsam, Senior Research Engineer, NASA Langley Research Center
Audience: 1 – Everyone.
Bayesian Projection Pursuit Regression
Gavin Collins, R&D Statistician, Sandia National Laboratories
Theme: Sharing Analysis Tools, Methods, and Collaboration Strategies
Statistical Validation of Fuel Savings from In-Flight Data Recordings
Keltin Grimes, Assistant Machine Learning Research Scientist, Software Engineering Institute
Theme: Solving Program Evaluation Challenges
Audience: 2 – Practitioners.
Rocket Motor Design Qualification Through Enhanced Reliability Assurance Testing
Todd Remund, Staff Data Scientist, Northrop Grumman
Theme: Improving the Quality of Test & Evaluation
1:00 PM – 3:00 PM: Room D
Mini-Tutorial 5
Session Chair: Maura McCoy Klink
Rethinking Defense Planning: Are We Buying Weapons and Forces? Or Security?
Theme: Sharing Analysis Tools, Methods, and Collaboration Strategies
Audience: 2 – Practitioners.
3:00 PM – 3:15 PM:
Afternoon Break
3:15 PM – 4:00 PM:
Awards & Closing Remarks
Dr. Pete Parker, Statistician and Team Lead, NASA