Compliance Metrics¶

Compliance measures whether restaurant staff responded to Emilia's real-time AI suggestions (drink opportunities, dessert opportunities).

The Problem Clustering Solves¶

Emilia lacks context about what's happening at the table. When no one responds to a suggestion, she re-sends the same alert every ~5 minutes. Without clustering:

5 events for the same situation + waiter goes once = ⅕ = 20% compliance

But the waiter DID attend the need → should be ~100%

The hybrid clustering algorithm groups rapid-fire alerts into opportunity moments, so one waiter visit covers the whole burst. But if events happen at genuinely different moments (separated by a waiter visit), they correctly form separate clusters.

Algorithm¶

flowchart TD
    A["EchoBase Events<br/>from metadata.tables[].echobase_events"] --> B["Filter by alertType<br/>second-drink-opportunity<br/>dessert-opportunity"]
    B --> C["Sort by time per type"]
    C --> D{"Gap > 600s?"}
    D -->|Yes| E["New Cluster"]
    D -->|No| F{"Annotation between<br/>consecutive events?"}
    F -->|Yes| E
    F -->|No| G["Same Cluster"]
    E --> H["Evaluate Cluster"]
    G --> H
    H --> I{"First annotation<br/>in cluster window?"}
    I -->|waiter-touch| J["COMPLIANT"]
    I -->|no-waiter-touch| K["NOT COMPLIANT"]
    I -->|none found| L["EXCLUDED<br/>events not counted"]

Cluster Splitting Rules¶

A cluster boundary is created when either condition is met:

1. Time gap > 600 seconds between consecutive events of the same type
2. Annotation between events — a waiter-touch or no-waiter-touch annotation falls between two consecutive events, meaning the situation was acknowledged

Cluster Evaluation: First Annotation Wins¶

For each cluster, the algorithm searches its time window [window_start, window_end] for annotations:

• waiter-touch found first → cluster is COMPLIANT
• no-waiter-touch found first → cluster is NOT COMPLIANT
• Neither found → cluster is EXCLUDED (all events omitted from counts)

The result is propagated to every event in the cluster.

Real-World Example¶

sequenceDiagram
    participant E as Emilia AI
    participant W as Waiter
    participant A as Annotator

    Note over E: Cluster 1 starts
    E->>E: second-drink-opportunity at 11980s
    E->>E: second-drink-opportunity at 12526s
    W->>A: waiter-touch at 12913s
    Note over E: Cluster 1 = COMPLIANT

    Note over E: Cluster 2 starts
    E->>E: second-drink-opportunity at 13795s
    E->>E: second-drink-opportunity at 14116s
    W->>A: waiter-touch at 14128s
    Note over E: Cluster 2 = COMPLIANT

Both clusters are compliant. Result: drink_suggestion_count = 4, drink_suggestion_compliance_pct = 100.0.

Output Fields¶

Percentages are rounded to 1 decimal place.

real_time_suggestions Detail¶

When compliance events are found, a real_time_suggestions array is added to the session (alongside metrics):

{
  "alert_type": "second-drink-opportunity",
  "time_seconds": 11980.0,
  "alert_timestamp_utc": "2026-03-08T18:05:04Z",
  "operator_id": "valeria",
  "compliant": true,
  "cluster_id": 0
}

This array is included in the fingerprint calculation — changes to suggestion matching trigger a new version even if metrics values are identical.

Dessert Compliance Conversion¶

Conversion measures whether a compliant dessert suggestion actually resulted in a dessert sale in the POS system. It extends the compliance chain one step further:

Alert → Waiter Visit (compliance) → POS Dessert Order (conversion)

Gate Conditions¶

All 4 conditions must be met for conversion to be calculated:

1. At least one dessert-opportunity event in real_time_suggestions with compliant=True
2. pos_correlation.status == "confirmed"
3. POS items with valid UTC timestamps available
4. video_start_utc available in shift metadata

When any condition is not met, dessert_compliance_conversion is null.

Causal Alert Selection¶

When multiple compliant dessert clusters exist, the algorithm selects the one with the tightest causal chain:

flowchart TD
    A["Compliant dessert-opportunity clusters"] --> B["For each cluster find<br/>waiter-touch that made it compliant"]
    B --> C["Find first POS dessert<br/>item offset from video start"]
    C --> D{"Any WT before<br/>dessert order?"}
    D -->|Yes| E["Pick cluster with WT<br/>closest to dessert order"]
    D -->|No| F["Fallback: absolute<br/>closest WT"]
    E --> G["delta = dessert_offset - alert_time"]
    F --> G
    G --> H{"0 <= delta <= window?"}
    H -->|Yes| I["conversion = true"]
    H -->|No| J["conversion = false"]

This ensures the selected alert has the most defensible link to the dessert sale.

Delta Interpretation¶

The raw delta in seconds (dessert_compliance_conversion_delta_seconds) is always stored regardless of the conversion boolean. This allows reporting layers to apply any threshold:

delta = first_POS_dessert_offset - causal_alert_time

The default conversion window is 1800 seconds (30 minutes), configurable via SESSION_METRICS_DESSERT_CONVERSION_WINDOW_SECONDS.

Items and Revenue¶

Items and revenue count only POS dessert items ordered after the causal cluster's waiter-touch time, not the alert time. This ensures we count desserts that could have been influenced by the waiter's visit.

Output Fields¶

Drink Compliance Conversion¶

Drink conversion measures whether compliant second-drink-opportunity alerts resulted in additional drink sales in POS. Unlike dessert conversion (a single event), drink conversion evaluates each compliant cluster independently — each is a separate "did the customer order another round?" moment.

Why Per-Cluster?¶

Drink opportunities repeat throughout a meal. A typical session has 3-5 drink alert clusters as the AI detects empty glasses at different moments. Each cluster is an independent upsell opportunity:

sequenceDiagram
    participant AI as Emilia AI
    participant W as Waiter
    participant POS as POS System

    Note over AI: Cluster 1 - empty glasses
    AI->>W: second-drink-opportunity
    W->>W: waiter-touch
    POS->>POS: BEER $21 ordered
    Note over AI: Cluster 1 = CONVERTED

    Note over AI: Cluster 2 - empty glasses again
    AI->>W: second-drink-opportunity
    W->>W: waiter-touch
    Note over POS: No new order
    Note over AI: Cluster 2 = NOT CONVERTED

    Note over AI: Cluster 3 - empty glasses
    AI->>W: second-drink-opportunity
    W->>W: waiter-touch
    POS->>POS: BEER $21 ordered
    Note over AI: Cluster 3 = CONVERTED

Result: conversion_count = 2, compliant_clusters = 3, conversion rate = 67%.

Gate Conditions¶

Same as dessert conversion:

1. At least one second-drink-opportunity in real_time_suggestions with compliant=True
2. pos_correlation.status == "confirmed"
3. POS drink items with valid UTC timestamps available
4. video_start_utc available

When gate not met → drink_compliance_conversion = null.

Cluster Boundary¶

Each cluster's conversion window naturally ends at the next cluster's first alert time (or session end for the last cluster). This prevents attributing a drink order to the wrong alert cycle.

Cluster 0 alert ──── WT ──── [drinks here count for cluster 0] ──── Cluster 1 alert ────

Delta Array for Re-Windowing¶

The drink_compliance_conversion_deltas array stores raw delta seconds for each converted cluster:

delta = first_POS_drink_offset - cluster_alert_time

This enables flexible time window analysis in reporting without re-enrichment:

-- Conversions within 5 minutes
arrayCount(x -> x >= 0 AND x <= 300, drink_compliance_conversion_deltas)

-- Conversions within 10 minutes
arrayCount(x -> x >= 0 AND x <= 600, drink_compliance_conversion_deltas)

-- Fastest conversion delta
arrayMin(drink_compliance_conversion_deltas)

The default conversion window for the boolean drink_compliance_conversion_count is 900 seconds (15 minutes), configurable via SESSION_METRICS_DRINK_CONVERSION_WINDOW_SECONDS.

Output Fields¶