02 / Applied AI engineering

Give “verified” a measurable boundary.

Bind numerical claims to the right financial field, entity, currency and time instead of matching any number in a tool response.

Ghostfolio Agent · Local release

verification controls: 11 / 11
assertion-bearing catalog cases: 75
model calls in recorded controls: 0

Synthetic verification rejects a stock price used as the portfolio total — A fixed synthetic example from the published verification receipt. This illustrates field binding, not live model answer quality.

The problem

A stock price could be accepted as evidence for a portfolio total. The evaluation catalog also contained assertion-free cases, so a green result did not always demonstrate a checked answer.

My contribution

I connected typed portfolio tools, scoped numerical verification, explicit answer outcomes, persistent local sessions, and a credential-free synthetic deployment. Evaluation controls exercise the actual verification pipeline.

How it works

Typed tool results
Scoped statement checks
Citations + field expectations
Verified / review / rejected / error

Decisions that shaped the release

Verification checks entity, metric, currency/unit, period, source and freshness. Unsupported language requires review.
Catalog assertions are validated separately from executed controls. A 75-case catalog is not presented as 75 model runs.
Confidence and verification remain distinct. Provider timeouts and incomplete evidence cannot become verified answers.

Measured results

All 75 catalog cases contain assertions. The measured 11 controls use fixed synthetic tool outputs and zero model calls.

Ghostfolio Agent: release evidence
Evidence	Result
Assertion-free catalog cases	0 of 75
Actual verification controls	11 of 11
Live model answer quality	Not measured

A failure worth inspecting

A plausible $180 answer is contradicted by the correct field.

The source contains both amounts. Field-scoped checking rejects the claimed total even though $180 is a real stock price in that same tool result. All values in this example are synthetic.

{
  "response": "Your portfolio total is $180.",
  "stock_price_usd": 180,
  "portfolio_total_usd": 30000,
  "decision": "rejected",
  "checked_field": "source.tool_0.total_value"
}

Limits and next questions

The verified profile is a single-operator local/private application using synthetic sample data.
Supported numerical and typed-field checks do not establish unrestricted natural-language entailment.
Live provider/model quality and browser interaction remain unmeasured.

Inspect the evidence

Download sanitized evidence (JSON) · SHA256

Project repository

Software controls, development-set measurements and model quality retain separate denominators. Local release artifacts are prepared for review; no public deployment is implied.