The agency’s proposed four-stage process would let organizations build tests around the risks, goals, and setting of a particular AI system.

The National Institute of Standards and Technology (NIST) asked companies and other stakeholders to help shape the proposed TEVV-Athlon Framework before it is finalized.

NIST released the initial public draft on August 7 and will accept comments through October 6. The TEVV-Athlon Framework is guidance, not a new requirement.

The TEVV-Athlon Framework would give AI developers, testing organizations, and companies deciding whether to buy or deploy AI a four-stage process for testing whether a specific system can meet its intended purpose in the real-world setting.

NIST calls the process Test, Evaluation, Verification, and Validation, or TEVV. In plain terms, it is a way to test whether an AI system meets an organization’s goals while limiting negative effects.

Draft process starts with business goals

The process would first ask organizations to define what they want to test and why. They would then decide what to measure, run the tests, and review the results.

That structure could let companies tailor their testing to a system’s use, rather than relying on one standard test for every type of AI.

NIST said the approach is designed for AI systems that analyze text, images, or other data, as well as systems that can perform tasks with limited human direction.

NIST wants business input before finalizing the draft

NIST specifically invited comments from organizations that conduct AI evaluations and from organizations that use evaluation reports to make decisions about AI systems, including business leaders, procurement specialists, researchers, and technical staff.

Comments are due October 6. NIST said comments may be made public and should not include proprietary information.