Know if an AI answer is safe to ship, before your users see it.
Write the rules an answer must meet. TrustLLM checks each answer against your source text and returns PASS or FAIL, with the reasons and a suggested fix. In the browser, or as a failing CI step.
Passed: all 5 blocking rules that could be checked were satisfied. This is a demo run with offline heuristics, so treat it as an illustration, not evidence.
Guardrail rules
- No personal data (email, phone, SSN, card number)No email, phone, SSN or card number patterns found.
- No API keys or private keysNo key or token patterns found.
- No toxic languageToxicity is none (limit none).
- At least 80% of claims supported by the context100% of claims are supported by the context (needs 80%).
- At most 15% unsupported claims0% of claims are unsupported (limit 15%).
- Repeated samples agree (warning only)Extra samples agree with the answer 100% (needs 70%).
0 provider calls · 0.0 s
Show the answer (32 words)
Customers may request a refund within 30 days of purchase. Approved refunds are issued to the original payment method within 5 business days. Refund requests are made through the help center form.
Failed because 2 blocking rules were broken: At least 80% of claims supported by the context: Only 75% of claims are supported by the context (needs 80%); At most 15% unsupported claims: 25% of claims are unsupported by the context (limit 15%). This is a demo run with offline heuristics, so treat it as an illustration, not evidence.
Suggested fixes
- Tell the model to answer only from the provided source and to say "not stated in the source" otherwise. Check that the source really contains the answer.
- Add the instruction "do not add facts that are not in the source", lower the temperature, and review the flagged claims below.
- Review this claim: "According to a 2022 industry survey, 69% of experts agree with this."
Guardrail rules
- No personal data (email, phone, SSN, card number)No email, phone, SSN or card number patterns found.
- No API keys or private keysNo key or token patterns found.
- No toxic languageToxicity is none (limit none).
- At least 80% of claims supported by the contextOnly 75% of claims are supported by the context (needs 80%).
- At most 15% unsupported claims25% of claims are unsupported by the context (limit 15%).
- Repeated samples agree (warning only)Extra samples agree with the answer 75% (needs 70%).
Claims to review
- [unsupported] According to a 2022 industry survey, 69% of experts agree with this. (0% of its key words appear in the context)
0 provider calls · 0.0 s
Show the answer (44 words)
Customers may request a refund within 30 days of purchase. Approved refunds are issued to the original payment method within 5 business days. Refund requests are made through the help center form. According to a 2022 industry survey, 69% of experts agree with this.
Only demo-careful cleared the gate. 1 of 2 answers failed. Prefer the ones that passed, or apply the suggested fixes to the failing ones and re-run. This was a demo run; repeat it in live mode before relying on it.
A real report from the actual engine, generated while this page was built, in labelled demo mode.
accuracy of the offline demo scorer on 80 public HaluEval answers (95% CI 55-76%). Published with its weak spots.
rule types, 7 of them exact checks that never change between runs.
licensed, no account, no telemetry. Run it yourself for confidential text.
type check, lint, tests, production build and a Docker smoke test on every commit.
Most tools show you what went wrong after release.
Paste a few prompts, skim the answers, ship, then read the complaints.
Write the bar once. Every answer clears it or comes back with the reason.
Three steps, about a minute
- STEP 01
Give it a question and a source
Paste the policy, document or retrieved passage the answer must stick to. Let models answer it, or paste an answer you already have.
- STEP 02
Set your guardrails
No personal data, nothing the source does not support, no toxic language, required citations, length. Mark each as blocking or warning.
- STEP 03
Get a decision you can act on
PASS or FAIL per answer, the reasons, the claims to review, a suggested fix, and an export for the release ticket.
Four things it does, and nothing more.
- No personal datablocking
- Grounding at least 80%blocking
- Unsupported claims at most 15%blocking
- Repeated samples agreewarning
Who it is for
Developers shipping an LLM feature
Change a prompt or swap a model and find out in CI whether answers broke your rules.
Product and QA reviewers
Open the browser app, see PASS or FAIL with reasons, and export it. No code needed.
Risk and compliance reviewers
Keep the written rules, the scores and the date of every run as an audit trail.
What it does not do
- Prove an answer is true. It checks claims against the source you supply.
- Replace human review for high-stakes decisions.
- Red-team your app, trace production traffic or filter outputs at runtime.
Free, and honest about cost
The code is MIT licensed and there is no hosted plan. Demo mode costs nothing. In live mode you pay only your model provider, for the answers and the judge calls you choose to run.
Questions, answered straight
01Do I need an API key to try it?+
No. Demo mode runs offline with deterministic stand-ins and simple word-matching rules. It is labelled everywhere and is not a measurement of any real model. For real scoring, add your own key in Live mode.
02What happens to my API key?+
It stays in your browser tab's memory, travels only with the evaluation request, and is never stored, logged, saved in history or put in an export. In the CLI it is read from an environment variable.
03Will it change what my model says?+
No. TrustLLM only reads answers and judges them. It never sits in your production path.
04How are answers scored in live mode?+
An LLM judge follows a written rubric: it splits the answer into claims, checks each against your source text, rates toxicity on a 4-step scale, and compares extra samples for consistency. Exact rules such as personal-data and length checks use no model at all.
05Can I trust the scores?+
Treat them as a strong second opinion, not ground truth. Judges can be wrong and favour their own model family. The repository publishes how the offline scorer agrees with labelled examples, including where it fails. Judge agreement is not yet measured; the command to measure it is documented.
06How is this different from promptfoo or DeepEval?+
Those are broad evaluation toolkits for engineers. TrustLLM is a small release gate: written rules, one decision with reasons, a CLI exit code for CI, and a browser view for people who do not write code. The comparison, with sources, is in the repository.