Model risk plugin

What it does #

The model-risk plugin classifies and gates each model invocation by tier. Every model used by the agent is registered in the ModelRegistry with an ID, a tier (0–3), and a validation evidence pointer.

The tier model #

Four tiers, aligned with the typical UK MRM (PRA SS1/23) classification plus the EU AI Act risk taxonomy:

• Tier 0 — informational. No regulated decision. No HITL needed.
• Tier 1 — internal decision affecting non-customer flows. Auditable but no HITL by default.
• Tier 2 — customer-affecting decision. HITL required above a configurable threshold.
• Tier 3 — material customer impact (credit, fraud, KYC, AI Act high-risk). HITL required for every invocation; dual-control authorisation for the model itself.

Each tier maps to specific evidence requirements; the plugin emits the tier into every audit event so the second-line dashboard can filter by it.

Model registration #

regulus:
  model-risk:
    registry:
      - id: gemini-2.5-pro
        provider: vertex-ai
        tier: 2
        validation-evidence: https://validation.example.com/gemini-2.5-pro-v1
        approved-by: model-risk-committee
        approved-at: 2026-04-15
        review-due: 2026-10-15
      - id: gemini-2.5-flash
        provider: vertex-ai
        tier: 1
        validation-evidence: https://validation.example.com/gemini-2.5-flash-v1

Unregistered model invocations are denied by default (DENY_UNKNOWN_MODEL). This is a deliberate fail-closed posture — if MRM hasn’t approved the model, the agent can’t call it.

HITL on tier-3 invocations #

For tier-3 model invocations, the plugin emits an ADK ToolConfirmation request before the model is called. The HITL surface captures the human reviewer’s Principal and structured justification; both land in the audit chain alongside the model invocation event.

Ongoing monitoring (SS1/23 Principle 5) #

The audit chain is the monitoring substrate. Filter by model_id × outcome × tier; export to ServiceNow IRM for the second- line dashboard. The ModelRegistry tracks review-due dates and the plugin emits a MODEL_REVIEW_OVERDUE event when the date passes without a re-approval.