Troubleshooting AI Control Checks

Troubleshooting AI Control Check Results

Common Control Check Results: Questions & Triage Steps

Understanding What Each AI Control Check Uses

Trustero AI performs targeted control checks designed to specifically assess a specific aspect of your compliance program. While Trustero may consider policy language, control objectives, evidence, and testing procedures, no single check relies on all of these inputs. Instead, the system selects relevant components based on the specific check being performed.

For example:

  • A material difference check compares control objective language to policy documentation.
  • A completeness check focuses on whether all required evidence for a control is present.
  • An operating effectiveness check evaluates whether evidence supports that the control is functioning as intended.

Here are the key elements the system may draw from:

  • Policy: The documented policies linked to the control, validating intent and framework alignment.
  • Objective: The control’s stated purpose and intended outcome.
  • Required Evidence: The specific artifacts used to prove the control is in place and functioning.
  • Test Procedures: The logical rules and steps used to evaluate evidence and implementation.

Note: The type of check being run determines which of these elements are used.

For more on foundational concepts, see:

Troubleshooting Common Check Results

What Does “Out of Date” Mean?

This result appears when something has changed, such as a new version of a policy or updated evidence, since the last check ran. The current result may no longer reflect your latest data.

How to fix it: Re-run the control check. Trustero will generate a fresh result using the most current inputs.

Resolving a Control-Policy Material Difference

This check evaluates whether the control language and linked policy are aligned. A material difference means there may be conflicting intent or insufficient clarity.

How to resolve it:

  • Your own policies: Ensure the language clearly supports the control objective and uses consistent terminology.

Addressing a Completeness Check Failure

The completeness check is designed to verify whether all required evidence artifacts have been attached to the control in the Trustero platform. It does not assess the quality or accuracy of the evidence, only that the expected files are present and properly mapped.

How to resolve it:

  • Review the control’s Required Evidence section to identify which items are expected.
  • Label each file clearly so it’s easy to understand what the artifact is and how it supports the control.
  • Attach the missing evidence directly to the control.

Once all required evidence is attached, re-run the check to confirm completeness.

Interpreting an Operating Effectiveness Check

This check assesses whether provided evidence proves that a control is functioning as described.

How to resolve it:

  • Review the test procedure logic to understand what the AI is evaluating.
  • Confirm that your evidence supports actual performance, not just intent.
  • When possible, provide evidence in structured formats (such as CSV files, plaintext logs, JSON outputs, spreadsheets). These formats allow the AI to more accurately interpret and validate the evidence, especially when evaluating tabular data. 
  • If there’s a mismatch between evidence and control expectations, Trustero will flag it for review.

Why can’t the AI read this PDF?

Trustero AI cannot interpret scanned PDFs or documents saved as images. These files lack embedded, readable text.

Supported formats include:

  • Machine-readable PDFs
  • Word documents (.docx)
  • Excel files (.xlsx)
  • Text files (.txt)
  • JSON, CSV, and YAML (for structured evidence)

For more details, refer to: Manual Evidence Best Practices

What Does “Passing Evidence” Look Like?

To see examples of controls with passing results, explore our customer-facing demo environments:

These examples show how well-structured policies, objectives, and evidence can drive successful check results.

Final Tip

By understanding how Trustero AI checks operate and following these troubleshooting steps, you can confidently resolve issues, improve test accuracy, and maintain a defensible, up-to-date compliance posture.