Skip to content
Battery Arbitrage Lab (home)

Methods

How the battery decides, and how the numbers are checked

The mandatory task is a textbook optimisation: a linear program with one twist. Below is the exact model, the assumptions it rests on, how two independent solvers and the statistics are checked, the evaluation design, the decisions behind it all, a model card, what the optional AI feature does, and the answers to Deliverable 1.

On this page12 sections

01

The model

Split the horizon into half-hours t = 1…T of length Δ = 0.5 h, with spot price pt in $/MWh. For every half-hour the optimiser chooses:

  • ct: MW drawn from the grid to charge (ηc of it is stored)
  • rt: MW drawn from storage to discharge (ηd of it reaches the grid)
  • st: MWh stored at the end of the half-hour
  • ut ∈ {0, 1}: whether the battery is in charging mode
maximiseΣt MLF · pt · Δ · (ηdrt − ct) − v · ηdrtΔ
subject tost = st−1 + ηcctΔ − rtΔ, s0 = 0
0 ≤ st ≤ E
0 ≤ ct ≤ P · ut, 0 ≤ rt ≤ P · (1 − ut)
optionalΣt rtΔ ≤ cycles × E

With P = 300 MW, E = 580 MWh, ηc = ηd = 0.9, MLF = 0.991 and variable cost v = $0/MWh. Fixed O&M ($8.10/kW/year) does not change the schedule and is reported separately.

Why the binary ut? When prices are negative, a pure LP discovers it can be paid to charge and discharge in the same half-hour, burning energy through the 19% round-trip loss. A real battery does one or the other, so the model forbids it.

02

Assumptions

Price taker
The battery's own bids never move the price.
Raw power
300 MW caps the grid draw when charging and the storage draw when discharging, so at most 270 MW reaches the grid.
Loss factor
MLF scales the price for both buying and selling, as in NEM settlement.
Market day
Calendar days in market time (AEST, UTC+10). AEMO labels a half-hour by its end, so its “15:00” is the 14:30–15:00 period here.
Start and end
Starts empty. Leftover energy is worth nothing, so the optimiser ends empty unless prices are negative at the very end.
Ties
When prices repeat exactly, several schedules earn the same. A tiny time-weighted preference (≤ $0.0002/MWh) picks one deterministically, identical in Python and TypeScript.

03

Two solvers, checked against each other

Reference (Python). scripts/battery_model.py builds the model above for SciPy's milp, which calls the HiGHS solver. It solves the LP relaxation first and only adds the binaries when the relaxation overlaps charging and discharging. It produced every precomputed number on this site.

Browser (TypeScript). web/src/lib/battery re-implements the model with no dependencies. Without a cycle cap it uses an exact dynamic programme: the value of stored energy is a piecewise-linear function, so each half-hour's best move sits on a breakpoint and the whole day solves in about a millisecond. With a cap it falls back to a bounded-variable simplex, a Lagrangian bound and a small branch-and-bound.

Parity tests. Vitest replays 10 days (Deliverable 1, test-window highlights, the wildest and most negative days, and others from 2019 to 2025) on 3 battery designs. Revenue matches HiGHS to the cent for perfect foresight, a part-charged start, a one-cycle cap and all four practical policies, and every perfect-foresight schedule matches half-hour by half-hour to within 0.001 MW. The six-week replays of the rule-based policies match too. One caveat: with a cycle cap on days with many negative prices (Christmas 2025 is the extreme case), the browser's branch-and-bound can hit its search budget and return a schedule a fraction of a percent short of HiGHS (under 0.5% in every case found). The Lab flags it on screen whenever that happens.

04

No-foresight policies (bonus task)

Each policy only uses prices up to and including the current half-hour, as the brief allows. Parameters are tuned on the training years (1 Jan 2018 to 31 Dec 2020), checked on cross-validation (1 Jan 2021 to 30 Jun 2021) and reported on the test window.

Price thresholds (illustrative example 1)
Charge at full power when the price is at or below $88.81, discharge at or above $116.33. Both levels come from a grid search over training-price percentiles.
Time of day (illustrative example 2)
Optimise the average 2018–2020 day once, then replay that schedule daily: charge 02:00–04:30 and 11:30–13:30, discharge 06:30–08:30, 16:30–17:00 and 17:30–19:00.
Day-ahead persistence
At midnight, assume today will repeat yesterday, optimise that forecast, then follow the plan.
Rolling persistence (model-predictive control)
Every half-hour, take the current price as known, assume the next 47 half-hours repeat the same times yesterday, re-optimise from the current charge and act only on the first step. In the daily-reset evaluation and, by default, in the Dispatch Lab, which also scores each day on its own, its plan stops at midnight instead, because energy left over then is worth nothing there (DR-005).

05

Data provenance

Prices are AEMO's public aggregated price and demand files for region VIC1, 1 Jan 2018 to 31 Dec 2025 (2,922 days). scripts/fetch_aemo.py downloads them and scripts/build_analytics.py writes a 2 MB read-only SQLite database of daily prices, optimal daily revenue, policy runs and sweeps, which the site queries on the server.

The course supplied its own spreadsheet extract, which is not redistributed here. Checked against it, all 63,456 overlapping half-hours from 2018 to mid-August 2021 match the public AEMO price exactly (largest difference $0.00).

From 1 Oct 2021 the NEM settles every five minutes. Those prices are averaged into half-hours, which is how the old 30-minute trading price was defined, so the whole series stays comparable with the brief.

  • web/data/analytics.db

    Prices, daily optimum, yearly summaries, policy runs and sweeps: the original results.

    $ uv run scripts/build_analytics.py

  • web/data/evaluation.json

    Per-day revenues for the sensitivity and regret analyses (scipy.optimize.milp (HiGHS), SciPy 1.18.1).

    $ uv run scripts/build_evaluation.py

  • web/src/lib/battery/__fixtures__/parity.json

    HiGHS results the TypeScript solver is tested against.

    $ uv run scripts/build_analytics.py

  • web/src/lib/stats/__fixtures__/reference*.json

    SciPy, statsmodels and R values the statistics helpers are tested against.

    $ uv run scripts/stats_reference.py && Rscript scripts/stats_reference.R

Why public data rather than the course file: DR-002.

06

Evaluation design

Every kind of claim on the site has a check, and every check runs in CI on each push. The 2026 evaluation adds uncertainty: the market day is the unit, every day starts empty (daily reset), and every estimate on /evaluation has a 95% interval from a circular block bootstrap over days (blocks of 7 days, B = 4,000, seed 20210701). Shares of days use Wilson intervals; comparisons are paired by day and Holm-adjusted (DR-005). The Wilson intervals and the Wilcoxon p-values treat days as independent, so they are optimistic; the block-bootstrap intervals are the primary evidence. Every policy that plans ahead plans only to midnight, the horizon the protocol scores.

Claims and how they are checked
ClaimEvidenceWhere
The browser solver is the brief's modelRevenue matches HiGHS to the cent on 10 days × 3 designs; schedules half-hour by half-hourbattery/*.test.ts
The evaluation numbers are what the solver producesThe TypeScript optimiser reproduces every design on every day of both sensitivity samples (to $0.02). The policies reproduce 2,350 of 2,352 strategy-days across all three samples (to $0.05); the two that differ are pinned ties, where the forecast has two equally good plans and the solvers pick different onesevaluation.test.ts
The statistics helpers are rightNormal functions, Wilson intervals, quantiles, Wilcoxon, exact McNemar, Newcombe's interval for paired proportions, Spearman, Holm and the bootstrap against SciPy, statsmodels and R; the block bootstrap against its exact variancestats*.test.ts
The prose quotes the computed intervalsEvery interval in the decision records and the model card is recomputed and matched as textclaims.test.ts
The AI client is safe with a keyAdapters, errors, key storage, redaction, audit log, snapshot and grounding check, network mockedai.test.ts
The explanation scorer catches what it claims toA template reply passes every check on all 24 schedules, with times on either clock; 7 planted faults are each caught wherever they apply, by their own check and by no other; a whole run against a mocked provider, including a key forgotten mid-runexplain-eval.test.ts
Samples
Brief test window (n = 42); Cross-validation window (n = 181); Calendar 2025 (n = 365). The policies were calibrated on 2018 to 2020, so all three are out of sample.
What is compared
Sensitivity: perfect-foresight revenue with one battery setting changed at a time. Regret: perfect foresight minus each no-foresight policy, day by day.

The optional AI explanations have an evaluation of their own on /evaluation/explanations: 24 fixed schedules, the Lab's prompt, six automatic checks against the optimiser's output (the scorer is tested first on planted faults), Wilson intervals, and paired comparisons of two runs with Newcombe's interval, decided by an exact McNemar test (DR-006). It runs with the visitor's key, so the site quotes no AI results.

07

Deliverable 1, answered

  1. How much energy is used in 10 minutes at 90 MW?

    15 MWh

    Energy = power × time = 90 MW × 1/6 h.

  2. What was the VIC spot price on 1 Jul 2020 at 15:00?

    $20.70

    The half-hour 14:30–15:00 (AEMO's 15:00 label), per MWh.

  3. What is the most a battery could earn on 17 Jul 2020, starting empty, with one 580 MWh discharge?

    $43,360

    Charge 03:00–04:00, 05:30–06:00 and 13:30–15:00; discharge 08:00–08:30, 10:00–10:30 and 17:30–18:30. The limit is read, as the brief words it, as at most 580 MWh discharged in the day, so the optimum splits that energy over two charge-then-sell episodes. Under the stricter reading of one charging phase followed by one selling phase, the best is $34,902 (charge before 06:30, sell after). Without any limit the same day is worth $51,350. Open Fri 17 Jul 2020 in the Lab with the one-cycle limit.

  4. Proposed approach

    This site

    Treat perfect foresight as a mixed-integer linear program and solve it exactly; use it as the ceiling. For the practical battery, feed the same optimiser forecasts built only from past prices, re-plan every half-hour, and compare each policy with the ceiling on the held-out test window.

08

Limitations

  • Energy arbitrage only: no frequency-control (FCAS) income, which real batteries rely on.
  • A 300 MW battery would move Victorian prices in reality; the model assumes it cannot.
  • Half-hour resolution throughout, even after five-minute settlement began.
  • No degradation unless you set one, constant efficiency and loss factor, perfect availability (DR-003).
  • The intervals on /evaluation describe day-to-day sampling variability, not the model being wrong. Six weeks is little data: most test-window intervals are wide.
  • The daily-reset protocol does not credit a policy for carrying energy past midnight, so the policies there plan only to midnight; the six-week benchmark carries energy over, and remains the brief's comparison.
  • The AI explanation has only been tested with the network mocked: this project has no API key of its own.
  • Practice project, not an assessed submission. The source is in a private repository for now; the original 2022 scaffold is preserved in it unchanged under coursework/original.

09

What I'd change

  • Co-optimise FCAS income and model five-minute settlement from October 2021, the two largest gaps between this model and a real Victorian battery.
  • Replace the persistence forecast with a proper probabilistic price forecast and evaluate the policies on it with the same regret analysis.
  • Run HiGHS as WebAssembly for capped problems and delete the branch-and-bound caveat.
  • Choose the bootstrap block length from the data and show how the intervals move with it.
  • Add a helpfulness rubric scored by people to the AI explanation evaluation, beside its automatic checks of facts and instructions, and let reviewers work through the replies in a seeded random order so their agreement covers the whole run.

10

Decision records

Each record states the decision first, then the context, the options, the reasons, what actually happened (weak numbers included) and what I would change. Records are never edited after the fact; a new record supersedes an old one. The sources are in docs/decisions.

  1. DR-001A mixed-integer LP for the perfect battery, with an exact solver in the browser
  2. DR-002Public AEMO prices instead of the course's staff spreadsheet
  3. DR-003Degradation as an optional linear cost per MWh delivered
  4. DR-004“Explain this schedule”: optional, bring your own key, grounded and audited
  5. DR-005Evaluate per market day, daily reset, with a seeded block bootstrap
  6. DR-006Evaluate the AI explanations against the solver on a fixed set of schedules

DR-001 · Accepted · 2026-10-09

A mixed-integer LP for the perfect battery, with an exact solver in the browser

Applies to: scripts/battery_model.py, web/src/lib/battery, /lab, /benchmark, /sizing, /methods#model

Decision

Model the brief's "perfect but impractical" battery as a linear program with one binary per half-hour that forbids charging and discharging at once, solve it with HiGHS in Python as the reference, and solve the same model exactly in the browser with a dynamic programme (plus simplex and branch-and-bound when a cycle cap applies), checked against HiGHS to the cent.

Context

The brief's mandatory task is to find the revenue-maximising charge and discharge schedule for a 300 MW / 580 MWh battery that knows every future Victorian spot price, under power, capacity, efficiency and loss-factor limits. It hints at a linear program. The revived site also needed the same optimiser to run in a visitor's browser (the Dispatch Lab re-solves on every slider move) and to answer Deliverable 1, which caps the day at a single cycle.

Decision

  • Variables per half-hour: grid-side charging power c_t, storage-side discharging power r_t, state of charge s_t, and a binary u_t for "charging mode". Revenue is MLF × price × (η_d r_t − c_t) × 0.5 h, less an optional degradation cost (DR-003).
  • Constraints: the storage balance s_t = s_{t−1} + η_c c_t Δ − r_t Δ, 0 ≤ s_t ≤ E, c_t ≤ P u_t and r_t ≤ P (1 − u_t), the battery starts empty, and an optional cap on energy discharged for the cycle limit.
  • Reference solver: SciPy's milp (HiGHS). It solves the LP relaxation first and adds the binaries only when the relaxation overlaps charging and discharging.
  • Browser solver: an exact dynamic programme over a piecewise-linear value of stored energy when there is no cap; a bounded-variable simplex, a Lagrangian bound and a small branch-and-bound when there is one.
  • Ties: a tiny time-weighted preference (at most $0.0002/MWh) makes the optimum unique, identically in both languages, so forecast-driven policies cannot depend on which solver happened to break a tie.
  • "Single cycle" is read as the brief words it: at most 580 MWh discharged in the day.

Options considered

  1. A plain LP. Exact and fast whenever prices are positive, but at negative prices it charges and discharges in the same half-hour to burn energy through the round-trip loss and get paid for it. A real battery cannot do that.
  2. An LP with a post-hoc fix (zero out the overlap). Simple, but the fixed schedule is no longer optimal and the size of the error is unknown.
  3. A mixed-integer LP (chosen for the reference). Exact, and HiGHS solves a 48-period day in milliseconds.
  4. Dynamic programming only. Exact and very fast, but a cycle cap couples every period, so it needs a Lagrangian outer loop that does not always close the gap on its own.
  5. HiGHS compiled to WebAssembly in the browser. Exact everywhere, but a large download and a build step for a page that otherwise has no dependencies.

Why

The binary is the smallest change that makes the model physically honest, and solving the LP first keeps it as fast as a plain LP on ordinary days. Two independent implementations checked against each other are stronger evidence than either alone: a bug would have to appear in both, in the same way.

What happened

  • The TypeScript port matches HiGHS to the cent on 10 days × 3 battery designs for perfect foresight, a part-charged start, a one-cycle cap and all four practical policies, and every perfect-foresight schedule matches half-hour by half-hour to within 0.001 MW (web/src/lib/battery/*.test.ts).
  • In 2026 the same check was extended to the evaluation artefact: the TypeScript optimiser reproduces every one of the 42 test-window days at all 25 sensitivity designs, and the policies on a spread of 2025 days, to within a few cents (web/src/lib/evaluation/evaluation.test.ts).
  • The weak spot is branch-and-bound with a cycle cap on days with many negative prices. It can stop at its node budget a fraction of a percent short of HiGHS (under 0.5% in every case found, Christmas 2025 being the worst). The Lab says so on screen when it happens.
  • During the 2026 evaluation build HiGHS printed an internal diagnostic (transformNewIntegerFeasibleSolution) on a handful of 2025 rolling-persistence solves. The solves completed, no policy beat perfect foresight on any day, and the TypeScript solver reproduces the sampled days, so I kept the results, but I have not traced the message to its cause.
  • Under the brief's reading the best single-cycle revenue on 17 Jul 2020 is $43,360. Under the stricter reading (one charging phase, then one selling phase) it is $34,902. Both are on the site; I chose the literal one.

What I'd change

  • Ship HiGHS as WebAssembly for the capped case and delete the hand-written branch-and-bound and its caveat.
  • Model five-minute settlement from October 2021 instead of averaging to half-hours.
  • Co-optimise frequency-control (FCAS) income, which real batteries depend on and this model ignores.

DR-002 · Accepted · 2026-10-09

Public AEMO prices instead of the course's staff spreadsheet

Applies to: scripts/fetch_aemo.py, scripts/aemo_data.py, scripts/check_staff_parity.py, web/data/analytics.db, /methods#data

Decision

Build every number on the site from AEMO's public Aggregated Price and Demand files for region VIC1 (2018 to 2025), check them against the course's staff-supplied spreadsheet where the two overlap, and never redistribute the spreadsheet.

Context

In 2021 the subject supplied students with a spreadsheet extract of Victorian prices. It is course material: I may use it to check my work but not publish it, and it stops in August 2021. The revival is public, needs to be reproducible by anyone, and is more interesting with the years after the brief (negative midday prices, five-minute settlement).

Decision

  • Download AEMO's monthly VIC1 "price and demand" CSVs with scripts/fetch_aemo.py into an ignored cache.
  • Map each interval to the half-hour that contains it (AEMO labels an interval by its end), and from October 2021 average the six five-minute prices in each half-hour, which is how the old 30-minute trading price was defined.
  • Keep only complete days (48 half-hours) and store the prices, rounded to four decimals, in the read-only web/data/analytics.db. Every later script, including the 2026 evaluation, reads its prices from that file, so all pages use identical numbers.
  • Compare against the staff extract with scripts/check_staff_parity.py and commit only the result (scripts/staff_parity.json), never the data.

Options considered

  1. The staff spreadsheet. The brief's own data, but not mine to redistribute, and it ends in August 2021.
  2. AEMO's dispatch-level archives (MMSDM / NEMweb). The most detailed source, but large, awkward to parse and more than a half-hourly model needs.
  3. AEMO's public aggregated price and demand files (chosen). Small, stable, public, and the same series the course extract was drawn from.

Why

Anyone can rebuild the site from public data with two commands, the extra years make the analysis current, and the parity check means nothing is lost by not using the course file.

What happened

  • All 63,456 overlapping half-hours from 1 Jan 2018 to 14 Aug 2021 match the staff extract exactly (largest difference $0.00).
  • Deliverable 1's price question checks out independently: $20.70/MWh for the half-hour ending 15:00 on 1 Jul 2020.
  • The dataset covers 2,922 complete market days. The five-minute averaging is an assumption I have not validated against a settlement-weighted alternative; it matters only after October 2021, so it affects the 2025 results on /evaluation but not the brief's windows.

What I'd change

  • Record a checksum of every downloaded file, so a later AEMO revision of a month is detected rather than silently absorbed.
  • Use dispatch-level five-minute prices from October 2021 and let the optimiser act every five minutes.

DR-003 · Accepted · 2026-10-09

Degradation as an optional linear cost per MWh delivered

Applies to: web/src/lib/battery/params.ts, scripts/battery_model.py, /lab, /sizing, /evaluation#sensitivity

Decision

Keep the brief's $0/MWh variable cost for every brief result, and model battery wear as an optional linear cost per MWh delivered to the grid, so it can be swept and set in the Lab without leaving the linear program.

Context

The brief sets variable O&M to $0/MWh, so the optimiser will happily cycle for a spread of a few dollars. Real lithium-ion batteries wear with every cycle, and that wear is the main reason operators skip marginal trades. A revival that only ever showed the $0 case would overstate how much a battery should cycle; one that changed the default would no longer answer the brief.

Decision

  • Add a variable cost v ($ per MWh delivered to the grid) to the objective: revenue falls by v × η_d × r_t × Δ in every discharging half-hour.
  • Default v = $0 everywhere the brief's numbers are reported (Overview, Benchmark, Deliverable 1).
  • Expose v as a slider in the Dispatch Lab ($0 to $100/MWh), sweep it on /sizing (six-week joint horizon) and include it in the sensitivity analysis on /evaluation, with intervals over days.

Options considered

  1. Ignore wear (the brief). Faithful, but it hides the most important practical trade-off.
  2. A linear throughput cost (chosen). One extra coefficient; the model stays a linear program and the solvers stay exact.
  3. A cycle limit only, like Deliverable 1's cap. Easy to explain, but it treats a shallow and a deep cycle the same and needs integer reasoning in the browser.
  4. A depth-of-discharge model (rainflow counting, a non-linear wear curve). Closest to the chemistry, but non-linear and non-convex, and the brief gives no data to calibrate it.

Why

It keeps the exactness of DR-001, it answers the practical question (how much revenue and cycling does wear take away), and it leaves every brief number untouched.

What happened

  • On the brief's six-week test window (joint horizon), a $10/MWh cost trims revenue by 9.9% but cuts cycling by 18%: the optimiser drops its marginal round trips first.
  • With intervals over days (daily reset, /evaluation), +$10/MWh changes revenue by −9.8% (95% CI −11.3% to −8.3%) in the test window and by −8.0% (95% CI −11.2% to −5.6%) in 2025. The 2025 interval is wider because a few extreme days dominate that year's revenue.
  • The weak part is the number itself: I have no calibrated cost for a specific cell chemistry, so the slider shows a range rather than an answer.

What I'd change

  • Calibrate v to a published degradation curve and report revenue net of wear alongside gross revenue.
  • Approximate depth-dependent wear with a piecewise-linear cost on state-of-charge bands, which keeps the model linear.

DR-004 · Accepted · 2026-10-09

“Explain this schedule”: optional, bring your own key, grounded and audited

Applies to: /lab, /ai-log, /methods#ai-use, web/src/lib/ai, web/src/components/ai

Decision

An optional AI explanation of the schedule on screen in the Dispatch Lab, called from the visitor's browser with their own key, fed only that schedule's numbers from the optimiser, labelled AI-generated, checked for numbers and times it did not receive, and recorded in a local audit log with the visitor's accept, edit or reject decision.

Context

A schedule chart answers "what" (charge here, sell there) but not "why", and the why is the interesting part: which price gaps beat the round-trip loss, why the optimiser charges twice, why a no-foresight policy missed the evening spike. A short plain-language reading is useful, and it is also a small, realistic test of governing generative AI, because the explanation is about numbers and language models are prone to inventing numbers. The project has no budget for an API key, no server for AI, and nothing on the site may depend on AI.

Decision

  • Bring your own key, browser only. The visitor pastes an Anthropic (default Claude Haiku 4.5, the cheapest tier; Claude Sonnet 5.5 optional) or OpenAI key (model id editable, default gpt-5-mini). It stays in sessionStorage unless they tick "remember on this device" (localStorage); "Forget key" removes it. Calls go straight from the browser to the provider (anthropic-dangerous-direct-browser-access: true for Anthropic). The key never reaches this site, a log or the repository.
  • Grounded input. The request holds only a JSON snapshot built from the solver's output: the day's price summary, the battery settings and cycle limit, the selected strategy's schedule as charging and selling blocks (times, average prices, energy, money), its revenue arithmetic, and every strategy's revenue and share of perfect foresight that day. No other data, and nothing about the visitor.
  • Structured output. The reply must follow a JSON schema (headline, charging, selling, economics, comparison, caveats) through each provider's structured-output mode, and is validated again with zod against the same contract. Long replies are shortened on screen only.
  • Refusals. Requests to Claude Sonnet 5.5 opt into Anthropic's server-side refusal fallback (fallbacks: "default", beta server-side-fallback-2026-07-01): if Sonnet declines, Anthropic may re-run the request on another Claude model, billed to the visitor's key at that model's rates. The settings dialog says so when Sonnet is chosen, and the audit log records the requested model, the model that answered and whether a fallback ran (a fallback content block or a fallback_message entry in usage.iterations). Haiku 4.5 does not take the fallback. A refusal that still comes back is shown as a plain message and logged.
  • Transparency. Every explanation is labelled "AI-generated" and shows the model, latency and token usage. A grounding check lists every clock time and every number in the reply that is not in the snapshot (dollar amounts with thousands separators, "k" and "M" suffixes and percentages of shares are understood; whole numbers from 0 to 10 are skipped as counts).
  • Human in the loop and audit. The visitor accepts, edits or rejects each explanation. Every call, failures included, is written without the key to an IndexedDB audit log, viewable and exportable (JSON, CSV) at /ai-log. A reply that arrived but was discarded (refused, cut off or off-schema) is kept in the log and labelled AI-generated too.
  • Scope of the AI use statement. This feature is the only AI that runs on the site. The site's code, analysis scripts and text were developed with an AI coding assistant (Claude Code) under my direction; I reviewed the results, and every number is computed by the deterministic solvers and statistics code and checked by the test suite, not by an AI. The statement on /methods says both.

Options considered

  1. No AI. Safe, but misses a useful explanation and a governance showcase.
  2. A server proxy with my key. Costs money, invites abuse, and puts a secret on a server.
  3. The Anthropic and OpenAI SDKs in the browser. Typed and maintained, but two extra client bundles for one request shape each, and harder to mock uniformly in tests. Plain fetch against the documented HTTP APIs keeps the bundle small and every request visible in one place.
  4. Bring your own key, plain fetch from the browser (chosen).

Why

The site stays free, static and fully functional without AI, and the governance controls become concrete: what is sent, what comes back, who decided what, all visible to the visitor and exportable. The design is informed by the Australian Government's policy for the responsible use of AI in government, the EU AI Act's transparency principles and the NIST AI Risk Management Framework; it is not a claim of compliance with any of them.

What happened

  • The adapters, error mapping, key storage, redaction, audit log, snapshot and grounding check are unit-tested with the network mocked (web/src/lib/ai/ai.test.ts): the key only ever travels in a request header, a key echoed in a provider error is redacted before it is stored or shown, and the snapshot never contains anything key-like.
  • No live call was made during development or CI, because the project has no key. The request shapes follow the providers' documented APIs; a real call is the first thing to check after deployment.
  • A review before publication found that the first draft of the AI use statement said the site's text was produced without AI, which the commit history contradicts: every commit is co-authored by an AI coding assistant. The statement now says how AI was used to build the site as well as what runs on it.
  • The grounding check is lexical. It catches an invented price or time, but not a correct number in a wrong sentence ("charged at the dearest price"), so it prompts the reviewer rather than deciding.
  • The snapshot only describes the schedule; it does not include the optimiser's dual values, so the model cannot say what one more MWh of storage would have been worth that day. It is told to say a claim is not available rather than guess.

What I'd change

  • Build a small evaluation set of days with known facts (which blocks charge, which policy wins, whether the cycle cap binds) and score explanations from each model against it, with intervals, as the regret analysis on /evaluation does for the policies.
  • Add the marginal value of stored energy from the dynamic programme to the snapshot, so explanations can say why a block was worth trading.

DR-005 · Accepted · 2026-10-09

Evaluate per market day, daily reset, with a seeded block bootstrap

Applies to: scripts/build_evaluation.py, web/data/evaluation.json, web/src/lib/stats, web/src/lib/evaluation, /evaluation

Decision

Add uncertainty to the results by treating the market day as the unit of analysis, solving every day from empty with every plan ending at midnight (daily reset), and putting 95% intervals on every estimate with a seeded circular block bootstrap over days (blocks of seven days), with paired comparisons whenever two methods or two designs are compared; the original six-week totals stay as they were.

Context

The original revival reports single numbers: the six-week revenue of each strategy and a sizing sweep on one window. A single six-week total cannot say whether time of day beat rolling persistence because it is better or because of a few lucky days, or whether a 10% larger battery reliably earns more. The brief's test window is only 42 days, and days are not independent: weather systems and outages run for several days.

Decision

  • Unit: the market day. scripts/build_evaluation.py writes per-day revenues for three samples: the brief's test window (42 days), its cross-validation half-year (181 days) and calendar 2025 (365 days).
  • Protocol: daily reset. Every strategy and the optimiser start each day empty and are solved on that day alone, so a day's regret (perfect foresight minus policy) is never negative and days are comparable units. Prices and calibrated policy parameters are read from analytics.db, so the numbers agree with every other page.
  • Horizon: energy still stored at midnight is worth nothing under daily reset, so every policy that plans ahead plans only to midnight: perfect foresight and day-ahead persistence by construction, rolling persistence by planning 48 − k half-hours ahead at half-hour k. The original rolling policy, which plans 24 hours ahead, is kept beside it as a variant, and the energy each strategy still holds at midnight is reported.
  • Intervals: circular block bootstrap (Politis and Romano) with blocks of 7 consecutive days, B = 4,000 resamples, seed 20210701, 95% percentile intervals. Shares of days use Wilson intervals. The Wilson intervals and the Wilcoxon tests below treat days as independent, so they are optimistic; the block-bootstrap intervals are the primary evidence.
  • Comparisons: paired by day. Each is reported as a mean daily difference with its interval first, then d_z, the share of days won (Wilson), and a Wilcoxon signed-rank p-value, Holm-adjusted across the three questions asked of each sample.
  • Sensitivity: one parameter at a time around the brief's design, with changes as paired ratios of totals and arc elasticities from the designs 10% either side.
  • Code: helpers in web/src/lib/stats, each checked against SciPy, statsmodels and base R (scripts/stats_reference.py, scripts/stats_reference.R).

Options considered

  1. Keep the six-week totals only. Faithful, but no uncertainty at all.
  2. Carry charge across days, as the benchmark does. Faithful to the benchmark, but a policy can buy on one day and sell on the next, so a day's result is no longer a property of that day and regret can be negative.
  3. An ordinary (iid) bootstrap over days. Simple, but too narrow when neighbouring days are correlated; on an AR(1) test series with φ = 0.7 the block interval is more than 1.3 times wider (stats.test.ts).
  4. Daily reset with a block bootstrap (chosen).
  5. A parametric model of daily revenue (for example a gamma regression with autocorrelated errors). More efficient if right, but daily revenue is heavy-tailed and spiky, and the model would be another assumption to defend.

Why

Daily reset makes the day a clean experimental unit, which turns every comparison into a paired one on the same days. The block bootstrap respects the dependence between days without a parametric model. Printing the seed, B and block length makes every interval reproducible.

What happened

  • The daily-reset perfect-foresight total for the test window is $5,921,103, the same as the original "perfect foresight, daily reset" row; time of day is identical under both protocols ($3,748,713) because its plan ends each day empty. The threshold rule earns less under daily reset than with charge carried over; rolling persistence, planning only to midnight, earns a little more ($2,564,692 against the benchmark's $2,443,858).
  • Time of day captured 63.3% (95% CI 52.7% to 73.0%) of perfect foresight in the test window and beat rolling persistence by $28,191 (95% CI $13,441 to $46,854) a day, winning 31 of 42 days.
  • Re-planning every half-hour was not clearly different from planning once at midnight in the test window, $7,212 (95% CI −$4,517 to $20,118) a day. In 2025 it was worse: −$26,116 (95% CI −$59,224 to −$7,323) a day, and rolling persistence lost money on 39 of 365 days. Why is not established. Its regret tracks the persistence forecast's error more closely than day-ahead's (Spearman ρ 0.69 (95% CI 0.62 to 0.74) against 0.46 (95% CI 0.36 to 0.56)), which fits re-planning on a noisy forecast amplifying its errors, but I have not tested that mechanism directly.
  • A review before publication found that the first version of this evaluation let rolling persistence plan 24 hours ahead, past midnight, while daily reset values energy left at midnight at nothing. It ended 21 of 42 test-window days and 176 of 365 days in 2025 holding energy it had paid for. That version made re-planning look worse than planning at midnight in the test window, −$4,136 (95% CI −$15,813 to $8,150) a day, and inflated time of day's lead over rolling persistence to $39,539 (95% CI $22,197 to $64,514) a day; I had attributed the 2025 result to the forecast chasing spikes without testing it. Planning to midnight is worth $11,535 (95% CI $9,478 to $13,750) a day in 2025. Both versions are on /evaluation.
  • The threshold rule does not plan at all and still ends 34 of 42 test-window days holding energy, so daily reset is harsh on it; the carried-charge totals on Benchmark are kinder (the reconciliation is on /evaluation).
  • The intervals are wide. Six weeks is not much data, and in the test window the share captured by rolling persistence, 43.3% (95% CI 33.1% to 54.7%), is compatible with anything from a poor to a middling policy.
  • The block length of seven days is a judgement, not an estimate; I did not tune it.

What I'd change

  • Choose the block length from the data (for example Politis and White's automatic selection) and show how the intervals move with it.
  • Add a carried-charge version of the regret analysis with weekly units, so the benchmark's own protocol gets intervals too.
  • Report the false discovery rate across the whole page, not just Holm within each sample's three questions.
  • Put shares of days and the paired tests on the same footing as the rest: block-bootstrap intervals for shares and a block sign-flip test in place of the Wilcoxon test, so no number assumes independent days.
  • Test the noisy-forecast explanation directly, for example by re-running both persistence policies on the real prices plus noise of known size.

DR-006 · Accepted · 2026-10-10

Evaluate the AI explanations against the solver on a fixed set of schedules

Applies to: /evaluation/explanations, /api/explain-eval, /ai-log, web/src/lib/ai/explain-eval*.ts, web/src/lib/ai/grounding.ts, web/src/lib/stats/paired.ts

Decision

Add an evaluation harness for "Explain this schedule": 24 fixed schedules (six edge cases chosen by rule, 18 days drawn with a printed seed), the Lab's prompt unchanged, six automatic checks against the optimiser's own output, a scorer that is itself tested on a template reply and on planted faults, Wilson intervals, paired comparisons between runs decided by an exact test, the visitor's own key, every call in the audit log, and no published results.

Context

DR-004 made the explanation optional, grounded and audited, and its grounding check flags any number the model was not sent. That check is lexical and works one reply at a time: it cannot say how often a model gets the facts right, and it cannot catch a true number or time in the wrong sentence ("it buys at 18:00" when 18:00 is a selling block). DR-004 named the fix under "What I'd change": a small evaluation set of days with known facts, scored with intervals. The project still has no budget for an API key, so whatever is built has to run with a visitor's key, and the site cannot quote results it never measured. This record extends DR-004 without replacing it: the feature, its prompt and its safeguards are unchanged.

Decision

  • Item set: 24 schedules on the brief's battery. Six edge cases are chosen by rule: Deliverable 1's day with and without the one-cycle limit, the 2025 day with the most negative half-hours, the 2025 day with the highest price, the test window's most valuable day (explained for rolling persistence) and the least valuable day of 2025 (for the time-of-day schedule). Then six days are drawn at random, without replacement, from each sample on /evaluation (seed 20261010), with perfect foresight, time of day and rolling persistence taking turns. Every schedule is solved exactly as in the Lab, and the 24 snapshots are public at /api/explain-eval.
  • Prompt: the Lab's system prompt, snapshot and JSON schema, unchanged, so the evaluation measures what ships.
  • Checks: a valid reply; grounded (the Lab's check); every clock time in the charging section inside a charging block and every time in the selling section inside a selling block, with at least one cited in each (times are read on either clock, "18:00" or "6 pm", since the prompt asks for neither); the day's revenue stated in the headline or the economics (gross or net of fixed O&M, within 1%, or to the precision it is written with up to 5%) with the right sign, so a loss must read as a loss and no revenue figure may be reported with the wrong sign; 90 to 270 words. A schedule passes if no check fails, and a check that cannot apply (no selling block, revenue under $1) is skipped. Refusals and cut-off or off-schema replies fail "valid"; network, key and rate-limit failures are not scored.
  • Testing the scorer: a template reply written from each snapshot by rule, with no AI, must pass every check, with its times on the 24-hour clock and again on the 12-hour clock. Seven planted faults (an off-schema reply, an invented price, buying and selling swapped, one wrong time, the revenue left out, a loss reported as earnings, padding past the limit) must each be caught by the check aimed at it and by no other, on every schedule where the fault can apply. This runs when the site is built and is shown on the page.
  • Statistics: shares of schedules with 95% Wilson intervals. The 18 random days are reported on their own, as the estimate for a typical day in the Lab, and the six edge cases as counts: they are hand-picked to be hard, so a share pooled over all 24 estimates no population and is kept only as the headline for comparing two runs, where the pairing makes the mix irrelevant. The mean number of ungrounded numbers per reply has a 95% bootstrap percentile interval over schedules (B = 4,000, seed 20261010). Two runs are compared by the paired difference in pass rates, with Newcombe's hybrid score interval, and an exact McNemar test on the schedules where they disagree; the exact test decides which run is ahead, at the 5% level. Both are checked against R (base R and the contingencytables package) and against statsmodels and SciPy.
  • Human review: the visitor can mark each automatic verdict right or wrong. Agreement with the scorer is reported separately for automatic passes and automatic fails, and the pass rate after review, all with Wilson intervals. The visitor chooses what to review, so these figures describe the reviewed replies only, and the page says so. The audit entry's decision stays "not applicable", because the review judges the scorer, not the reply.
  • Records: every call, retries and failures included, goes to the AI audit log with the feature explain-schedule-eval, the run, the schedule, its score and the set's hash (SHA-256 of the snapshots, the prompt and the settings, computed when the site is built). Runs are read back from the log, so they survive a reload and export as CSV or JSON, and runs are only compared within one version and one hash, so a rebuilt set is never paired with the old one. A run sends 24 requests, two at a time, retries a call once after a rate limit, overload, server or network error (so up to 48 requests), and stops on a bad key or an unknown model, or as soon as the visitor forgets the key.

Options considered

  1. Keep only the per-reply grounding check (DR-004). No measure of how often a model is right, and blind to a true time in the wrong sentence.
  2. An LLM judge that grades each reply. It could catch subtler errors, but it needs a second paid call per reply, has an error rate of its own to measure, and grades with the same kind of system it is judging.
  3. Publish results from my own runs. The page would have numbers, but that needs a key and a budget I do not have, and a published score goes stale as providers update models behind the same name.
  4. Fixed schedules, rule-based checks against the solver, a tested scorer and the visitor's key (chosen).

Why

The facts that matter most in an explanation of a schedule (when it buys, when it sells, what it earned, which numbers it quotes) are already known exactly from the optimiser, so they can be checked without another model. Testing the scorer on planted faults shows what each check can catch before it is trusted with a model's replies. Pairing two runs on the same schedules makes "is model B better?" and "how much do replies vary between runs?" the same paired question the regret analysis asks of the policies.

What happened

  • The template reply passes every check on all 24 schedules, with its times on either clock, and each of the 7 planted faults is caught wherever it applies by the check aimed at it, with no other check firing: on all 24 schedules for six of them, and for a loss reported as earnings, on the 2 schedules that lose money (explain-eval.test.ts, and on the page).
  • A whole run was tested against a mocked provider: replies scored, a refusal scored as not valid, a rate-limited call retried once, a bad key stopping the run, the key forgotten mid-run (no further request sent), every call in the log and the key in none of them.
  • A review before release found four flaws in the first version, all fixed before it shipped. The comparison called a winner whenever a percentile bootstrap interval excluded 0, which is too narrow with few disagreements: a 4 to 0 split of 24 schedules gave an interval above 0 while the exact McNemar p was 0.125. The revenue check ignored the sign, so "earned $52,360" on a day that lost $52,360 passed. Times written as "1:00 pm" were read as 01:00, so a correct reply on the 12-hour clock failed two checks. And a rounded revenue such as "about $0.1 million" passed at 33% off, against the stated 1%.
  • With a handful of disagreements, Newcombe's interval can just exclude 0 while the exact test does not reach 5% (a 5 to 0 split: interval above 0, p = 0.0625). The page then says the runs cannot be told apart, and why.
  • No live run was made, because the project has no key, so the page shows no results until a visitor runs it. Whether the checks agree with a careful reader on real replies is unknown until someone reviews a run; the planted faults are cruder than a model's mistakes.
  • With 24 schedules the intervals are wide: a run that passes all 24 still has a Wilson interval reaching down to 86%, and one that passes all 18 random days down to 82%. The set can tell a model that often gets the facts wrong from one that rarely does, not 90% from 95%.
  • The time checks will fail some fair replies, such as "it waits past the 04:00 dip" in the charging section. That is why the reviewer can overrule a verdict, and why agreement is reported.

What I'd change

  • Grow the set, or draw a fresh one for each run, once real runs show how wide the intervals are in practice.
  • Check claims beyond times, numbers and revenue: which strategy did best, whether a no-foresight policy is said to have known a price, whether the cycle limit binds.
  • Add a helpfulness rubric scored by people, so the evaluation covers whether an explanation is useful and not only whether it is right.
  • Compare effort settings and prompt variants on the same schedules, paired, before changing the Lab's defaults.
  • Offer a review mode that presents replies in a seeded random order, so agreement with the scorer estimates the whole run rather than the replies a visitor chose to open.

11

Model card

A model card for the decision models at the centre of this project: the perfect-foresight optimiser for a grid battery on Victoria's spot market (DR-001) and the four no-foresight policies from the brief's bonus task. Original results come from web/data/analytics.db; the intervals come from the 2026 evaluation (DR-005), with every interval a 95% block-bootstrap interval over market days (7-day blocks, B = 4,000, seed 20210701) unless it says otherwise.

Model details

  • Optimiser: a mixed-integer linear program per market day or window. It chooses charging and discharging power and the state of charge for every half-hour to maximise MLF × price × energy traded, less an optional degradation cost (DR-003), subject to power, capacity and efficiency limits, starting empty, and never charging and discharging at once.
  • Solvers: SciPy milp (HiGHS) in Python as the reference; an exact dynamic programme, a bounded-variable simplex and a small branch-and-bound in TypeScript for the browser, tested against HiGHS to the cent.
  • Policies without foresight: fixed price thresholds and a fixed time-of-day schedule (both calibrated on 2018 to 2020), and day-ahead and rolling persistence (the optimiser run on the forecast "today repeats yesterday", once at midnight or every half-hour). In the daily-reset evaluation rolling persistence plans only to midnight; on Benchmark and in the Lab it plans 24 hours ahead, as originally built.
  • Battery (the brief's): 300 MW, 580 MWh, 90% charge and 90% discharge efficiency (81% round trip), marginal loss factor 0.991, variable cost $0/MWh, fixed O&M $8.10/kW/year.
  • Author and dates: Sunchuangyu (Rin) Huang. Brief 2021 S2, practice scaffold May 2022, models and site 2026, evaluation and model card October 2026.
  • How it was built: the code, analysis scripts and text were developed with an AI coding assistant (Claude Code) under my direction, and I reviewed the results. Every number comes from the deterministic solvers and statistics code and is checked by the test suite, not by an AI.
  • Licence: MIT.

Intended use

  • Learning and teaching: how an arbitrage schedule is found, what perfect foresight is worth, and how much of it simple policies keep.
  • Comparing battery designs and policies on historical Victorian prices, with honest uncertainty.
  • Not intended for bidding a real battery into the NEM, for investment or financing decisions, or as financial advice. It has no FCAS income, no network constraints and no market power.

Data

  • AEMO Aggregated Price and Demand data, region VIC1, 1 Jan 2018 to 31 Dec 2025: 2,922 complete market days of half-hourly prices, with five-minute prices averaged into half-hours from October 2021 (DR-002).
  • Public market data only: no personal information. The course's staff extract was used only to check the public series (all 63,456 overlapping half-hours match exactly) and is not redistributed.
  • Calibration (policies): 2018 to 2020. Cross-validation: Jan to Jun 2021. Test: 1 Jul to 11 Aug 2021 (the brief's windows). Calendar 2025 is an additional out-of-sample year.

Evaluation

Original results (six-week test window, charge carried between days, single totals): perfect foresight $5,926,534; time of day $3,748,713 (63%); rolling persistence $2,443,858 (41%); day-ahead persistence $2,254,116 (38%); fixed thresholds $1,594,771 (27%).

With intervals (daily reset with every plan ending at midnight, share of perfect-foresight revenue captured):

PolicyTest window, n = 42Cross-validation, n = 1812025, n = 365
Time of day63.3% (95% CI 52.7% to 73.0%)63.4% (95% CI 50.9% to 72.5%)51.6% (95% CI 40.9% to 58.7%)
Day-ahead persistence38.2% (95% CI 23.0% to 55.3%)50.2% (95% CI 39.8% to 59.0%)55.2% (95% CI 48.3% to 60.7%)
Rolling persistence43.3% (95% CI 33.1% to 54.7%)57.7% (95% CI 49.0% to 64.4%)38.4% (95% CI 27.3% to 48.1%)
Fixed thresholds19.8% (95% CI 12.9% to 26.9%)27.6% (95% CI 3.9% to 41.4%)18.0% (95% CI 12.0% to 26.2%)
  • Paired comparisons (test window): time of day beat rolling persistence by $28,191 (95% CI $13,441 to $46,854) a day and on 31 of 42 days; re-planning every half-hour was not clearly different from planning at midnight: $7,212 (95% CI −$4,517 to $20,118) a day. Shares of days and the Wilcoxon tests treat days as independent, so they are optimistic; the block-bootstrap intervals are the primary evidence.
  • Sensitivity (perfect foresight, elasticity at the brief's design): in the test window, round-trip efficiency 0.99 (95% CI 0.63 to 1.30), energy capacity 0.53 (95% CI 0.44 to 0.61), power 0.47 (95% CI 0.40 to 0.56). In 2025, round-trip efficiency 0.71 (95% CI 0.64 to 0.79), energy capacity 0.67 (95% CI 0.64 to 0.70) and power 0.33 (95% CI 0.30 to 0.36): longer storage became worth more than more power. On 35 of 365 days in 2025 the battery 10% more efficient earned less than the brief's: at negative prices the battery is paid to take energy, and conversion losses let a less efficient battery take more of it.
  • Solver accuracy: the TypeScript solver matches HiGHS to the cent on the parity fixtures and on every sensitivity design of the evaluation artefact, and the TypeScript policies reproduce 2,350 of 2,352 strategy-days (the other two are ties between equally good plans, pinned in the tests). With a cycle cap on days with many negative prices the browser solver can stop up to 0.5% short (flagged in the Lab).

All of these, with their seeds and settings, are on /evaluation.

Known failure modes

  • Perfect foresight is a ceiling, not a forecast. Its revenue is unattainable; only shares of it are meaningful for real policies.
  • Re-planning on a persistence forecast can hurt. In 2025, rolling persistence earned −$26,116 (95% CI −$59,224 to −$7,323) a day against planning once at midnight, and lost money on 39 of 365 days. Its regret tracks the forecast's error more closely than day-ahead's, which fits re-planning amplifying a noisy forecast's errors; that mechanism has not been tested directly.
  • A plan that outlives the scoring window. A policy that plans past the point where it is scored buys energy it is never paid for. Planning 24 hours ahead under daily reset cost rolling persistence $11,535 (95% CI $9,478 to $13,750) a day in 2025, and the threshold rule, which never plans, ends 208 of 365 days in 2025 still holding energy. Match the planning horizon to the evaluation protocol.
  • Regime change. Policies calibrated on 2018 to 2020 meet a different market in 2025 (negative midday prices on most spring days); the time-of-day schedule falls from about 63% to 52% of perfect foresight.
  • Fat tails. A handful of extreme-price days carry a large share of revenue, so totals and their intervals are dominated by a few days; the intervals are wide for that reason.
  • Branch-and-bound budget. With a cycle cap on many negative-price half-hours, the browser solver may return a near-optimal schedule (flagged).

Ethical considerations

  • A battery that really is as large as 300 MW moves Victorian prices; a price-taker model overstates what it could earn and ignores its effect on other consumers.
  • Arbitrage revenue rewards buying when power is cheap and plentiful (often solar-heavy middays) and selling into evening peaks; the model optimises money, not emissions or reliability, and should not be read as a statement about either.
  • The optional AI explanation feature sends only a schedule's numbers to a provider of the visitor's choosing, with the visitor's own key, and records every call locally; see the AI use statement on /methods. Its replies can be scored against the optimiser's output on a fixed set of schedules at /evaluation/explanations (DR-006).

Caveats and recommendations

Report battery arbitrage results with the sample they come from, the protocol (daily reset or carried charge), and intervals over days. Compare policies on the same days. Treat a single six-week total as one draw, not a property of the policy.

12

AI use statement

What the AI does

One optional feature, “Explain this schedule” in the Dispatch Lab, writes a short plain-language reading of the schedule on screen: when the battery charges and sells, why, and how it compares with the other strategies. It is off until a visitor adds their own API key, and it is the only AI that runs on this site. The explanation evaluation runs the same feature, with the same prompt, on 24 fixed schedules and scores each reply against the optimiser's numbers.

The site's code, analysis scripts and text were developed with an AI coding assistant (Claude Code) under my direction. I reviewed the results, and every number is computed by the deterministic solvers and statistics code and checked by the test suite, not by an AI.

What it never does

  • It never computes, changes or ranks any number shown on the site.
  • It never sees anything except the schedule snapshot described below.
  • It never runs without a click, and never with a key belonging to this site (there is none). One click on the evaluation page sends 24 requests (up to 48 if calls are retried once after a rate limit, overload, server or network error), and the page says so; forgetting the key stops a run before its next request.
  • Its output is never shown without the “AI-generated” label.
  • It gives no trading or investment advice.

What is sent, and where

A fixed system prompt and a JSON snapshot built from the solver's output: the day's price summary, the battery settings and cycle limit, the selected strategy's charging and selling blocks (times, average prices, energy, money), its revenue arithmetic, and every strategy's revenue that day. It goes straight from the browser to the provider the visitor chose: Anthropic (default Claude Haiku 4.5, or Claude Sonnet 5.5) or OpenAI (default gpt-5-mini, editable). If Claude Sonnet 5.5 declines a request, Anthropic may re-run it on another Claude model, billed to the visitor's key; the log records which model answered. The key stays in the browser's session storage, or local storage if the visitor asks, and travels only in the request header.

Human in the loop, and the record

The reply must match a JSON schema and is validated again in the browser. A grounding check lists anything in it that was not in the snapshot; it checks every clock time (24-hour or am/pm), every number with decimals, every number above 10 (including $ amounts, thousands and millions) and anything in scientific notation; whole numbers from 0 to 10 are not checked. The visitor accepts, edits or rejects each explanation, and every call, failed ones included, is written without the key to an audit log in the browser's IndexedDB: time, feature, provider, the model asked and the model that answered, input, output (a discarded reply too, labelled AI-generated), latency, token usage and the decision (for an evaluation call, its automatic score and the visitor's review of that score). View or export it on the AI audit log.

The design is informed by the Australian Government's policy for the responsible use of AI in government (transparency, human oversight), the EU AI Act's transparency principles (people should know when content is AI-generated) and the NIST AI Risk Management Framework (map, measure and manage, with records). It is not a claim of compliance with any of them. Trade-offs: DR-004 and, for the evaluation, DR-006.