KPI Recipe Library¶

The KPI recipe library gives analysts reusable business definitions without making hidden runtime decisions. A recipe describes a business question, calculation, compatible aggregate inputs, presentation defaults, and method caveats. Installing one materializes normal catalog YAML for review; the packaged recipe is inert until that explicit action.

Contract¶

The built-in library is stored in src/valuestream/recipes/kpis.yaml, validates against schemas/kpi-recipes.json, and is loaded through a typed Pydantic model. The same browser and instantiation logic is used by Configuration Builder and AI Configuration Studio.

A recipe contains:

Field	Meaning
`id`, `version`	Stable recipe identity and immutable version number
`title`, `domain`, `summary`	Business-facing discovery metadata
`business_questions`, `tags`	Search and interpretation aids
`maturity`	`draft`, `reviewed`, or `certified` governance state
`processor_kinds`	Processor families that may satisfy the recipe
`parameters`	Optional bounded install-time numbers, including percentages displayed in business units
`inputs`	Required business roles, field/algorithm selection mode, accepted state types, metadata and filtered-state requirements, recipe-authored state templates, pairing/exclusion rules, and preferences
`default_metric_id`	Proposed ID; the installer adds a stable numeric suffix on collision
`metric`	A normal metric definition with exact-value placeholders such as `${processor_id}`
`method`	Calculation, accuracy class, algorithm, and caveat
`report`	Recommended chart, placement, optional KPI comparison defaults, and optional required `x` axis for non-scalar charts

Template substitution is deliberately closed: a placeholder must occupy the whole YAML scalar. Metric-template placeholders must name a declared binding or built-in installer value; state-template placeholders must name a declared, bounded recipe parameter. Recipes cannot inject Python, SQL, or expression strings. Formula recipes and recipe-authored filtered states materialize the same closed expression AST used by hand-authored catalog objects.

Readiness¶

The compatibility resolver evaluates a recipe against one Processor and returns one of four states:

State	Meaning	Install behavior
`ready`	Every required state/stage maps unambiguously	Installer preselects all bindings
`mapping_required`	Compatible candidates exist, but business intent is ambiguous	User chooses each unresolved input
`backfill_required`	A required aggregate state or stage is absent from the current processor contract	Configurable sketch inputs and closed recipe-authored state templates can propose a processor state; other non-configurable inputs remain blocked
`incompatible`	Processor kind cannot execute the recipe	Processor is excluded from the selector

Matching is deterministic. The resolver filters by source (state or stage), state type, required/absent state metadata, required where predicates or the exact parameter-resolved state template, and strict semantic roles, then applies ordered name/algorithm preferences. A sole remaining candidate is safe to map automatically; multiple business fields, algorithms, stages, or populations require a user choice. Paired score digests require matching score metadata and funnel endpoints must be different.

Readiness never examines raw event rows. It reads only processor configuration and its effective aggregate-state contract.

For a field_algorithm input, the browser augments that static readiness with safe configuration choices. It lists every processor-owned candidate field — including every group_by field and configured identity/property field — and every algorithm declared compatible by the recipe. A field/algorithm pair that is not yet a processor state is shown as a proposal, not as unavailable. The browser never invents a business-specific where predicate. It may propose one only when the recipe contains the complete closed AST and every substituted value is a declared, bounded parameter.

Install Workflow¶

Both authoring surfaces provide the same steps:

Search by KPI, business question, tag, calculation, or domain.
Read the business definition, accuracy class, algorithm, and caveat.
Edit any bounded recipe parameters. Percentages are displayed as percentages even though the generated catalog expression stores their decimal value.
Select a compatible Processor and review readiness.
Select business fields and any recipe-compatible algorithm for sketch-backed metrics, stages or populations for funnels, and any other ambiguous roles. Proposed states are identified as requiring a first run or backfill.
Choose a unique metric ID.
Optionally add the recommended tile to an existing dashboard page.
Select Review changes and inspect the exact generated processors.yaml, metrics.yaml, and dashboards.yaml patches.
When a processor state is proposed, review the source, source fields, affected states, and current/proposed processor computation hashes.
Apply explicitly. If materialization is required, follow the named source handoff to Data Load and start the run there.

When the recipe's default metric ID is retained, installation also retains the recipe-authored display label. Choosing a custom metric ID derives the initial display label from that ID; the label can then be edited independently without renaming the metric.

Configuration Builder writes any proposed processor state first, followed by the materialized metric and optional tile. All writes and post-write catalog validation run inside one rollback boundary: a write or validation failure restores every catalog file. After success, the Builder reloads the catalog, switches to Edit Existing Metric, opens the new metric, and presents a direct Data Load handoff when the processor contract changed.

AI Configuration Studio adds the same processor/metric/tile artifacts to its session-local draft. The workspace remains unchanged until Apply to workspace writes the reviewed revision. Applying uses one rollback boundary for all four catalog files plus ai.yaml, including post-write validation. Recipe confirmation and apply never start ingestion; the outcome links to Data Load when the processor contract requires materialization.

For recipe-created metrics, display.label is derived from the chosen Metric ID. The recipe still supplies units, formatting, direction, and calculation metadata, but it cannot replace the metric's identity with a generic recipe name.

An installed metric carries recipe provenance in the strict typed metric contract:

metrics:
  VS_Unique_Entities:
    processor: ih_engagement
    kind: approx_distinct_count
    state: UniqueCustomers_cpc
    description: Approximate distinct entity count from a persisted mergeable sketch.
    display:
      label: Unique entities
      unit: entities
      value_format: integer
      direction: higher_is_better
    recipe:
      id: audience.unique_entities
      version: 1

When a recipe has editable parameters, its installed metric also records the resolved values under recipe.parameters. This makes the chosen definition reviewable without making the packaged recipe part of runtime execution.

If Channel and CPC were selected before that state existed, the same action also adds the ordinary processor configuration:

processors:
  - id: ih_engagement
    # existing processor fields remain unchanged
    states:
      Channel_cpc:
        type: cpc
        source_column: Channel
        lg_k: 11

After installation, metrics.yaml and dashboards.yaml remain the sole source of runtime behavior. Editing or removing the packaged recipe does not silently change an installed metric.

The authoring UI does not use state IDs as business choices. For example, UniqueSubjects_hll is presented as field SubjectID, algorithm HLL; SubjectID_theta is presented as field SubjectID, algorithm Theta. The state ID and parameters remain available under Technical aggregate bindings. If SubjectID_cpc is not configured, CPC still appears as a recipe-compatible algorithm and is labelled as a proposed state. Exact engagement roles are locked to the processor's Positives/Negatives states, ROC AUC selects one score field and pairs its positive/negative digests automatically, distribution quantiles exclude outcome-conditioned digests, and funnel recipes expose stages/populations. If two states have identical business field/algorithm metadata and no distinguishing population, installation is blocked instead of exposing an internal state-ID choice.

Built-in Recipes¶

Recipe ID	Business KPI	Required capability	Accuracy	Default report
`engagement.engagement_rate`	Engagement rate	Binary positive/negative counts	Exact	KPI card
`engagement.positive_outcomes`	Positive outcomes	Binary positive count	Exact	KPI card
`audience.unique_entities`	Distinct audience/reach	CPC, HLL, or Theta state; CPC preferred	Approximate	KPI card
`distribution.median`	Median numeric/score value	t-digest or KLL state	Approximate	KPI card
`distribution.p95`	High-tail numeric/score value	t-digest or KLL state	Approximate	KPI card
`distribution.boxplot`	Distribution boxplot	t-digest or KLL state	Approximate	Boxplot
`engagement.decision_to_outcome_latency_p95`	Decision-to-outcome latency (P95)	Unconditioned duration t-digest or KLL state	Approximate	KPI card with previous-period comparison
`model_quality.roc_auc`	Ranking discrimination	Matched positive/negative t-digests	Approximate	KPI card
`model_quality.score_calibration`	Observed-to-predicted score calibration	Positive count, score sum, and observation count	Exact	KPI card
`decisioning.material_upward_exploration_rate`	Material upward score-revision rate	Purpose-built relatively filtered upward-revision and total counts on a numeric-distribution processor	Exact	KPI card
`model_quality.implied_evidence_index`	Heuristic implied evidence index	Pooled score-variance and squared-delta means	Statistical	KPI card
`model_quality.relative_exploration_variance`	Relative exploration variance	Pooled squared-delta and score-variance means	Statistical	KPI card
`funnel.conversion_rate`	Funnel completion rate	Start/completion count states	Exact	KPI card
`funnel.dropoff_rate`	Funnel stage loss	Ordered funnel stages	Exact	KPI card
`lifecycle.summary`	Entity lifecycle measures	Entity lifecycle processor	Exact	Table
`category.top_items`	Frequent categories	Frequent-items/Top-K state	Approximate	Table
`contact_policy.frequency_marginal_ctr`	Selected rank-1 action CTR by number of impressions	Exact `Clicks` and `Contacts` states	Approximate fixed-window interpretation	Line
`contact_policy.frequency_comparable_ctr`	Selected rank-1 action CTR on contacts with a selected rank-2 action	Exact `ComparableClicks` and `ComparableContacts` states	Approximate fixed-window interpretation	Line
`contact_policy.runner_up_expected_ctr`	Mean selected rank-2 action raw Propensity	Exact `RunnerPropensitySum` and `ComparableContacts` states	Approximate fixed-window interpretation	Line
`contact_policy.runner_up_coverage`	Selected rank-2 action coverage of selected rank-1 action contacts	Exact `ComparableContacts` and `Contacts` states	Approximate fixed-window interpretation	KPI card
`contact_policy.response_opportunity_margin`	Selected rank-1 action response minus selected rank-2 action expectation	Exact `ComparableClicks`, `RunnerPropensitySum`, and shared `ComparableContacts` states	Approximate fixed-window interpretation	Bar
`contact_policy.priority_opportunity_gap`	Selected rank-1 action minus selected rank-2 action Priority index	Exact comparable Priority sums and `PriorityComparableContacts`	Approximate arbitration diagnostic	Bar

The six Contact policy recipes target only the frequency_response processor and require the exact state names shown above. They never propose a generic count or sum as a substitute for those population contracts. Their line and bar recommendations use ExposureBucket, whose value is an upstream fixed-window approximation; changing a report date filter does not recompute customer contact history.

Contacts are derived from the processor's configured exposure outcomes. The user-facing number of impressions is therefore based on Impression proxies rather than measured viewability; dismiss telemetry is never inferred when the source does not provide it. Within the processor's implicit customer + interaction keys plus the physical fields configured in alternative_group_by, “selected rank-2 action” means exact rank 2 when present, otherwise the next recorded rank greater than 1; its response uses raw Propensity as the probability. Placement is the default additional field, so comparison stays inside one decision and placement. Comparable selected rank-1 action CTR, selected rank-2 action expected CTR, and response opportunity margin use the same ComparableContacts denominator. Priority is retained only for the separate, neutral arbitration index; it is never treated as CTR or probability.

The unique-entity recipe prefers CPC states created by current processor defaults while accepting HLL and Theta states. Theta is useful when the same persisted set also supports intersections or differences. The recipe does not convert or merge different sketch families together.

The decision-to-outcome recipe is unit-neutral: it inherits the selected duration field's unit and must not be labelled as seconds unless the source contract establishes seconds. It measures the time until an outcome is recorded, not request-serving latency.

The three exploration recipes are draft diagnostics. Upward exploration is an adaptive-decisioning policy KPI, not a predictive-model-quality metric. It uses a dedicated numeric-distribution processor so outcome filtering cannot change the population. Material upward exploration rate counts a decision only when the raw score is positive and FinalPropensity - Propensity > Propensity × threshold. The threshold defaults to 10% and is editable during installation; equality at the boundary and zero/negative raw scores are excluded. The implied evidence value is a heuristic index, not a response count or fitted posterior parameter. Relative exploration variance is an unbounded ratio, not an uncertainty probability. Randomised control arms must be excluded from all three.

Versioning and Governance¶

A recipe version is immutable once published. Calculation, input semantics, direction, or accuracy changes require a new version.
Copy edits that do not alter interpretation may remain in the same version, but installed metrics are never rewritten automatically.
certified means the calculation and business interpretation have named owners and reference tests; it does not make an approximate sketch exact.
An installed metric records the recipe ID/version so future upgrade tooling can show a diff instead of silently migrating it.
Workspace-owned recipes, approval owners, deprecation, upgrade assistance, and report packs are planned future work, not current behavior.

Aggregate and Backfill Rules¶

Installing a recipe that uses an already configured state does not change the processor. When a selected field/algorithm pair is absent, the installer adds a deterministic state definition to processors.yaml (or the AI draft) and binds the metric to that state. Default parameters are CPC lg_k=11, HLL/Theta lg_k=12, t-digest k=500, KLL k=200, and Top-K lg_max_map_size=10; they remain inspectable in the technical binding.

A parameter that changes aggregate-state semantics is part of that state definition. The resolver reuses a state only when its normalized definition, including the resolved predicate value, matches exactly. Choosing a different material-upward threshold therefore proposes a distinct filtered count state, changes the processor computation hash, and requires materialization rather than silently reinterpreting existing counts.

This configuration is intentionally allowed before any data is loaded. The preview names the changed processor states, source fields, source, and the processor computation-hash transition. The next normal source run materializes the new state for a fresh workspace and reprocesses discovered chunks whose computation contract changed. Operators who need a narrower historical window can use the normal backfill workflow. Until matching-hash aggregates exist, reports continue to show Backfill required.

Adding the metric does not convert an HLL or Theta blob to CPC, reconstruct a digest, read persisted raw rows, or start a run. Replay and computation-hash behavior follow the compatibility rules in the domain model.

All recipe metrics execute through the normal query layer. Recipe browsing, mapping, and report placement never persist or query raw event rows.