Governance of Model and Knowledge Base Changes

Governance of model and knowledge base changes controls behavior changes that may bypass ordinary application-code review. A model swap, prompt edit, threshold move, retrieval-index refresh, tool schema change, or safety policy update can change user outcomes as much as a code deployment, so it needs the same level of evidence as model versioning and auditability.

Change classes and evidence depth

Change classes should determine evidence depth:

change_id: ai_release_2026_07_11_004
class: medium_risk_behavior_change
artifact_changes:
  model: support_llm_v4.1 -> support_llm_v4.2
  retrieval_index: refund_policy_2026_06_01 -> refund_policy_2026_07_01
  prompt_policy: safety_policy_v18 -> safety_policy_v19
required_evidence:
  - golden_dataset_regression
  - adversarial_prompt_injection_suite
  - hallucination_and_grounding_review
  - privacy_and_pii_log_review
approvals:
  - product_owner
  - risk_owner
rollback:
  restore_model: support_llm_v4.1
  restore_index: refund_policy_2026_06_01
  restore_policy: safety_policy_v18

The rollback plan must include non-code artifacts. Restoring the old application container does not restore the old embedding model, retrieved corpus, or policy enforcement rule if those live in separate control planes.

Sourced artifact

NIST AI RMF separates mapping, measuring, and managing risks across the AI lifecycle. A release record can mirror that:

RMF function	Release evidence
Map	Intended use, affected users, data sources, deployment context
Measure	Adversarial evaluation, factual correctness, fairness, privacy, security tests
Manage	Go/no-go decision, risk treatment, monitoring owner, rollback plan
Govern	Approval policy, roles, versioned documentation, escalation path

Caveats

Change governance fails when “content-only” updates are treated as harmless. A new benefits document can change eligibility answers; a new tool schema can expand agency; a prompt tweak can alter refusal behavior. Reclassify changes when the user population, jurisdiction, tool access, or decision impact changes.

References

• NIST AI RMF 1.0
• NIST AI 600-1: Generative AI Profile

Nav

Section — Responsible AI, Safety, and Governance

← Compliance