PreFlight — the pre-installation evidence scan for Agentforce
An in-org, LLM-free Salesforce package that answers the question every Agentforce project skips: would an agent have worked on this customer's real history? It scans 12–24 months of closed cases at zero credits, scores the data on a six-component Agent Readiness Index, and produces a board-ready German evidence report in which every number is tagged Measured, Observed, Derived or Projected. Would your agent have worked last year? Prove it before you build it.
- Year
- 2026
- Role
- Salesforce Developer — Pre-Installation Evidence
- Tech stack
- Apex (with sharing) + Queueable orchestrationAgent Readiness Index — 6 components, frozen weights 25/20/15/15/15/10Phase 0 is LLM-free — zero credits, read-onlyEvidence tiers: Measured / Observed / Derived / Projected (ADR-013)Q-tags — every metric traceable to a documented query (Q1–Q15)Seeded-LCG demo corpus with built-in answer keyModels API behind a gateway seam + mock (runs without Einstein)Live replay via in-org invocable action — no callout, no secretGerman Nachweisbericht (Visualforce + LWC)79/79 tests · 93% coverage · 137 components (measured Jul 2026)
Highlights
- The question it answers before anyone builds: PreFlight scans 12–24 months of the org's real closed cases and closing emails — LLM-free, zero Flex Credits, read-only toward standard objects — and asks whether an agent would have worked on this history. Demand mining (German-stopword keyword clustering), a Flow/Apex capability inventory, and the headline: a 0–100 Agent Readiness Index
- The ARI is engineered, not vibes: six components with frozen weights (25/20/15/15/15/10), saturating curves, an inverted-U for topic concentration, medians instead of means, four readiness bands — and it is honestly reframed (ADR-013) as a diagnostic of the data substrate, not a prediction of agent success
- Every number in the report carries an evidence tier — Measured (reproducible via a query), Observed (raw system output), Derived (heuristic interpretation) or Projected (extrapolation) — and a Q-tag pointing to the exact documented query it came from. The savings figure is labelled a projection, on purpose
- The classifier is scored against an answer key, never against itself: the seeded-LCG demo generator (same seed, byte-identical corpus — the Prüfstand philosophy) writes each case's true intent into the record, so LLM classification quality is measured with Cohen's kappa against ground truth, and a model-judged answer-key check is kept as a separate axis so a confidently-wrong 'Answered' can never hide as a success
- Live replay with honest limits: a case's real first message is replayed against the deployed draft agent through an in-org invocable action — no external callout, no secret — and the result is framed as what it is: the measured floor of a starter draft, not a ceiling; the outcome classifier openly states it is a deterministic keyword heuristic judging one first reply, not the full conversation
- Positioning against the platform's own tooling: Testing Center tests an agent with synthetic questions; PreFlight tests the agent idea against the customer's real history — before the agent is even built. It's the pre-sales evidence a delivery team needs to escape pilot purgatory before entering it
- Deploys clean to a scratch org (137 components across three packages) — 79/79 tests green, 93% test-run coverage, measured July 2026. Live LLM classification requires an Einstein-enabled org; everything else runs without it by design (gateway seam + mock)
The problem
Agentforce projects start with a demo and a promise, and die in pilot purgatory — because nobody measured, before building, whether the org's own history could carry an agent. The best predictor of an agent's success is sitting in the customer's closed cases: what people actually asked, how cases were actually closed, whether the closing texts have any substance at all. But that evidence is never mined; scoping is done on gut feeling, the business case on a slide. And the platform's own tooling can't help here — Testing Center tests an agent that already exists, with synthetic questions. What's missing is the step before: evidence, from real history, before a single build hour is spent.
The approach
Make the org's history the witness, and keep the model out of the courtroom. Phase 0 is a Lite Scan — pure Apex, LLM-free, zero credits, read-only: SOQL aggregates over closed cases, German-stopword keyword clustering for demand mining, closing-text substance analysis, and a Flow/Apex capability inventory, all feeding a deterministic Agent Readiness Index — six components, frozen weights (25/20/15/15/15/10), medians not means, four readiness bands. Every metric carries a Q-tag traceable to a documented query, and every reported number an evidence tier: Measured, Observed, Derived or Projected — a discipline codified in its ADRs. Only in Phase 1 does a model enter, strictly behind a gateway seam with a mock (the package deploys and runs in orgs without Einstein), with mandatory PII masking, stratified sampling, and Cohen's-kappa calibration against a built-in answer key — the seeded demo generator writes each case's true intent into the record, so the classifier is scored against ground truth, never against itself. Phase 1.5 replays real first messages against the deployed draft agent through an in-org invocable action and honestly frames the result as the floor of a starter draft, with a model-judged answer-key check kept as a separate axis so a confidently-wrong answer can't count as a success.
The outcome
Deploys clean to a scratch org — 137 components across three packages, 79/79 tests green, 93% test-run coverage, measured July 2026. The deliverable is a bilingual evidence report (the German Nachweisbericht, demonstrated on the fictional HanseWatt Energie GmbH — the same German energy-utility scenario the rest of the portfolio builds on) plus a partner delivery layer: board-ready HTML, CSV/JSON exports, a Testing Center dataset, a Jira import, a consultant playbook. Honestly bounded, by design: Phase 0 makes no prediction — the ARI is a diagnostic of the data substrate; the savings figure is labelled a projection; the replay measures a starter draft, not a finished agent; and live LLM classification requires an Einstein-enabled org while everything else runs without it. In the portfolio's lifecycle it is the first gate: PreFlight proves the ground before HanseWatt builds on it — and Blast Radius, Prüfstand, Nabız and Aktenlage take over once the agent exists.