Verify and evaluate

An agent that answers confidently is easy to build. An agent whose answers you would forward to a client without checking is the actual goal. This page covers the four tools that get you there: chat with provenance, run audit, the eval set, and the readiness checklist.

Chat, but read the receipts

Open the agent and ask a question you already know the answer to. Then expand How I worked this out under the answer.

The panel is the audit trail for that specific answer:

Which metrics and data steps ran, in order.
Which results the answer was built from.
Whether the agent fell back to something less precise.
The adjacent verification entry lists the sources and metric citations actually used, and lets you inspect metric definitions.

Grounded answers carry citations for the metrics they used; other governed data sources also appear in the source information. An answer produced without any data step says so — no citation is invented to make it look sourced. If you see a number with no provenance, treat it as untrusted.

Follow-up questions keep the previous answer's provenance in context, so "and the month before?" stays anchored to the same metric instead of drifting to a similar one.

Query-construction errors in tool steps can be sent back to the model and retried. Terminal failures, such as unavailable data, are reported directly. An intermediate tool failure remains in the steps but is collapsed and de-emphasized by default so a recoverable event is not mistaken for a failed run; expand it or retry when needed.

Run audit

Platform operators can open Run log beneath an answer to inspect that run's status, phases and durations, tool calls and their inputs and outputs, execution traces, and errors. It is for reviewing what actually ran; it does not expose model prompts or the full response text.

Fix meaning, not wording

When an answer is wrong, the fix usually belongs upstream:

Pinned facts record durable business truths for this agent — "the Nordic region includes Iceland", "we exclude internal test accounts". They apply to every future answer. Facts can live at workspace, agent or personal level; a useful personal fact can be promoted so the whole team gets it.
The glossary maps the words your team uses to the metrics that implement them, so "churn" resolves to your definition of churn.
Metric descriptions are the single most under-used fix. One clarifying sentence on the metric beats ten corrections in chat.

Anyone using the agent — including a Portal client — can report an answer with This looks wrong, pick a reason (wrong number, misunderstood, out of scope, other) and add a note. The report captures the question and the answer as they were, and lands in the builder's Answer feedback tab of the inbox.

The eval set

Chatting proves an agent worked once. Evals prove it still works after you change a metric.

Open the Evals tab on the agent. The case set is seeded from the scope you wrote — your in-scope examples plus curated starter prompts — and can be expanded automatically with:

Paraphrases of the same question, so it is not memorising one phrasing.
Edge cases, especially ambiguous time wording.
Out-of-scope probes it is supposed to refuse.

Add or remove cases by hand, then Run. Each result shows the question, the answer produced, and the expected ("gold") answer where you provided one, and you mark it pass, needs work, or fail. Owners and collaborators can review and approve a completed run; members can read it but not change it.

The card summarises total cases, how many have gold answers, coverage and the pass rate against the required threshold.

Re-run evals after changing a metric definition, swapping a dataset, or editing scope. That is when regressions actually happen.

Readiness

The Readiness tab turns "is this ready for other people?" into a checklist with a score, and links straight to whatever is missing. Pass rate, coverage, and the overall score are quality signals that help you judge risk; the eval pass rate is not, by itself, a publication gate. The actual publication checks are:

Scope is written — a purpose plus at least one in-scope example question.
Metrics are documented — every metric the agent's pipeline uses has a name and a non-empty description.
Refusals are reviewed — at least one out-of-scope sample exists and every refusal route has been reviewed.

The agent must also be in a publishable lifecycle state (at least one successful run), and the publication must be performed by an owner or collaborator. A low eval score neither bypasses these checks nor independently blocks publication; it is a quality warning to address.

What agents can produce

Verification is not only about numbers in chat. Depending on the skills you gave it, an agent can deliver:

Written reports, PDF, and DOCX documents support complete, long, multi-chapter generation. If a safety resource boundary is reached, generation fails clearly rather than producing a partial document.
Editable PPTX slide decks
Infographics, social cards, and downloadable record exports
Forecasts with scenario ranges
Briefing videos and short social clips
Audio briefings and podcast-style summaries

Deliverables are grounded the same way answers are — the numbers in a report or slide deck come from data the run actually pulled, not from a plausible-sounding draft. For reports and slide decks, the system checks before rendering that charts and other numbers can be traced to run data. The source/verification panel behind the answer shows chart-check results for the answer and for each file. A deliverable with no verifiable chart cannot be described as verified.