One engine · every workload · everywhere

Spark Rust Mission Control

Global commerce digital twin · SQL + Streaming + ML + Spatial in one browser tab

LOADING WASM…
Starting the engine…Loading Spark Rust into this browser tab.
Orders / batch—LIVE STREAM
Revenue / batch—SPARK SQL
Active vehicles—SPATIAL
Mean late risk—SPARK ML
Fraud alert rate—CLASSIFICATION
Pipeline latency—WASM

Global operations map

ST_Contains · ST_DistanceSphere
0 assets in storm · 0 deliveries likely late · batch —

Execution pipeline

real local stages
wall —input —coefficients —
Architecture preview — connect Spark Connect for live executor telemetry. Local mode does not claim remote execution.
DriverExecutor 1Executor 2Executor 3

Highest-risk deliveries

LogisticRegression
DeliveryRegionRiskAction

Same workload, every API

copy-ready


LIVE
Local WASM mode executes real Spark SQL, streaming micro-batches, Spark ML, and geospatial UDFs. Cluster/AQE/lakehouse/GPU views require an attached backend and are labeled previews.


Guided system tour
What is happening in Mission Control?
This page is a live global-commerce digital twin. Every second, a Web Worker runs one coordinated Spark Rust pipeline and sends only the scored result back to the dashboard.
01
The live processing loop
The simulator emits 30 fleet positions, 60 delivery orders, and warehouse capacity every tick. The Worker registers three in-memory Arrow-backed tables, then executes four real stages:
  1. JSON batch ingestion creates the current micro-batch.
  2. Spatial scoring calls ST_Contains, ST_DistanceSphere, and ST_AsText for every vehicle.
  3. Spark ML scoring fits spark_ml::LogisticRegression and calculates late-delivery probabilities.
  4. Spark SQL analytics joins deliveries to warehouses and groups revenue, risk, fraud, and capacity by region.
The pipeline latency card measures this complete Worker-side cycle.
02
What “Launch incident” changes
The incident is one coordinated scenario—not a cosmetic animation:
  • Demand triples from 60 to 180 orders per batch.
  • A storm polygon expands over the fleet, changing real ST_Contains results.
  • Warehouse H-01 loses up to 70% of capacity.
  • Storm and load features raise ML late-delivery risk.
  • Fraud intensity increases, producing payment alerts.
  • The map, KPIs, regional aggregates, risk queue, model coefficients, and event feed all update from the new Spark results.
Press Reset world to return future batches to normal conditions.
03
How to read the dashboard
Operations map
Cyan dots are vehicles, trails show recent movement, green hubs are warehouses, and red pulses mark vehicles inside a storm or restricted geofence.
Regional command
Each bar is mean late risk calculated after the live delivery-to-warehouse join.
Event ribbon
Records storm entry, demand escalation, fraud blocks, reset actions, and clearly labeled AQE architecture previews.
Risk queue
The highest logistic-regression probabilities are sorted to the top with WATCH or INTERVENE recommendations.
Execution pipeline
Shows real local stages and their row counts. The coefficient strip is the current fitted ML model.
API tabs
Shows equivalent SQL, PySpark, Rust, and TypeScript for the same workload.
04
Pause, inspect, and rewind
Pause stops new micro-batches. The snapshot slider stores the last 30 processed browser states. Drag it left to compare the map and KPIs before the incident. Selecting a historical batch pauses live processing; press Resume to continue.
A useful walkthrough is: wait for two normal batches → launch the incident → watch storm/risk/fraud rise → pause → drag to batch 1 → compare before and after → resume.
05
What is real—and what is a preview?
Executing for real in this tab
  • Spark SQL parsing, planning, joins, and aggregates
  • Worker-owned streaming micro-batches
  • Rust Spark ML logistic regression
  • Geospatial UDF evaluation
  • Arrow/JSON in-memory table registration
  • Browser snapshot rewind
Requires an attached backend
  • Remote executor topology and task rescheduling
  • Actual AQE plan replacement
  • Cloud lakehouse writes and time travel
  • GPU execution
  • Real Kafka/Event Hubs ingestion
The Cluster preview button only shows the intended connected architecture. It is deliberately labeled a preview and never claims that browser-local work ran remotely.
Try the story now
Close this guide, watch normal operations establish a baseline, then press the red incident button.