How I Ingested Deck Occupancy Telemetry and Queried Near-Full Alerts in KQL
TL;DR
I built DeckPulse as a laptop experiment for one narrow garage question. If four invented decks fill unevenly during a busy window, can I land those gate readings in a Microsoft Fabric Eventhouse, prove the rows arrived, and get a watch order out of KQL without inventing a second definition of "near full" in a notebook?
The sample is 6,000 rows. Seed 19. Twenty gates. Readings every 30 seconds from 2026-11-01T00:00:00Z. Status flips to ALERT when OccupancyPct crosses 88. On this seed that happened 229 times, 3.82 percent of the file. Midtown leads with 83 alerts. Plaza has 72. River 52. Campus 22. The peak is gate-006 at 100.00 percent on Midtown. Campus leads average occupancy. Midtown leads alerts. The watch order follows the spikes. The same numbers come out of python main.py and out of kql/01_explore.kql after a local-file ingest. The code is at https://github.com/aniket-work/DeckPulse.
This is a PoC. Midtown is not a garage I operate. The Eventhouse path is the create-and-ingest lesson I wanted to rehearse on a parking story instead of another generic sensor dump.
Introduction
The first version of this experiment lived in a spreadsheet I opened after a Saturday rush. I had pasted a few gate IDs, typed an occupancy percent, and decided "too full" in the header while the coffee was still hot. The next morning I opened a second sheet for wait times and the cutoff drifted by five points because I had forgotten the first sheet. Both sheets still looked official. They were two moods wearing column names.
Parking decks gave me a clean place to stop doing that. I did not want hospital beds, warehouse racks, or a ledger. I wanted a gate loop, an occupancy number, a wait estimate, a throughput tip, and a status that means one thing. Each row has a timestamp, a gate id, a deck, three measurements, and a status. That is a small world. I can hold it in one CSV and one KQL table. I named the project DeckPulse. It is my PoC. The four decks are invented so the rows have a story. It is not a garage system I run, and it is not work I did for an employer.
I started from the question I actually wanted the brief to answer. Which deck in this sample throws the most near-full alerts? "Near full" had to become a number or the question was a shrug. I set the line at 88 percent. Then I had to say that out loud in one place, or the chart would invent a second line later. The generator writes ALERT when occupancy crosses that line. The KQL filter uses the same status. The laptop brief counts the same rows. From my experience, if the threshold lives only in a chart title, I will quietly move it when the picture looks kinder.
gate-006 is the boundary I care about for the peak. Midtown, 100.00 percent, timestamp 2026-11-01T01:11:30Z. If I had written the alert rule as "occupancy at or above 100," that row would have sat on a polite edge and the story would have flattened. I wanted the packed gate inside the queue.
Campus is the holdout I care about for averages. Highest average occupancy in the sample. Only 22 alerts. A deck that runs busy-but-stable still has to show up in the average table, or I start pretending the map only contains spike decks. I kept Campus in the sample on purpose so the watch order would disagree with the average table. That disagreement is the point.
What's This Article About?
I walk through how I built the ingest path, in the order I wish I had used the first time. The question comes first. Then the CSV grain. Then the Eventhouse container. Then the local-file wizard. Then the queryset. Then the laptop mirror that refuses to disagree with KQL. The code under Let's Get Cooking is the same code the tests run.
You will see why I pinned seed 19, why status is a column instead of a live calculation in every query, and why ranking by alerts beats ranking by average occupancy on this sample. Setup is the clone path on GitHub. The run section is the 6,000-row table, the Midtown line, and the chart. If you only want the repo, start at https://github.com/aniket-work/DeckPulse. README.md has the same images and the same counts.
Tech Stack
I kept the list short. An Eventhouse experiment that needs a streaming mesh before the first table is an experiment I will not rerun on a weeknight.
- Python 3.11+ is the runtime I used for the generator, the brief, and the tests.
- The CSV under
data/gate_telemetry.csvis the sample that lands in Fabric and on the laptop. - Microsoft Fabric Eventhouse is the container. The default KQL database shares the Eventhouse name.
- KQL in the default queryset is how I prove the rows arrived and how I rank decks by ALERT.
- The analyzer in
deckpulse/analyze.pymirrors those queries locally so a failed ingest cannot hide behind a pretty chart. - FastAPI exposes
GET /api/briefand servesstatic/dashboard.html. - Matplotlib writes the ALERT-by-deck bar chart.
- pytest locks the 6,000-row count, the four decks, and the alert threshold.
I skipped Eventstream for this PoC. Continuous ingest is a real next step. It is also a different failure mode, and I wanted the create-and-ingest path alone first. I skipped OneLake availability too. Turning that on later can expose the same table as Delta for other Fabric engines. I left it off so the lesson stayed on the Eventhouse and the queryset. I also skipped a hosted model. The brief I wanted is a table of counts, and a model that paraphrases "near full" would be a second definition.
The picture matches the runtime. main.py and the dashboard both call build_report. That function asks analyze.py to load the CSV and score the same rules the KQL script uses. Sample rows stay in data/gate_telemetry.csv so the article, the tests, and the GIF describe one run. The Eventhouse path is documented in docs/eventhouse-ingest.md and exercised with kql/01_explore.kql.
Why Read It?
Read it if you have a folder of garage notes that each redefine the same operational word. This experiment fails in an obvious way. If the CSV says ALERT and a query invents OccupancyPct > 92, the laptop brief and the queryset stop agreeing. I left that disagreement visible on purpose. The useful claim is not that Midtown is chaos. The useful claim is that I can point at one threshold and one status column and say "this is the near-full I meant."
A few choices are the ones I would repeat.
- Pin the sample. Seed 19, 6,000 rows, 30-second interval. If the file regenerates into a new story every run, the article becomes a moving target.
- Store status on the row. Computing the alert only in a query invites a second cutoff in a chart. The generator writes the status. The query filters it. The laptop counts it.
- Keep a laptop mirror. Fabric is where I want the long-lived table. The laptop brief is how I notice an ingest that quietly dropped rows.
- Rank decks by ALERT, not by average occupancy alone. Campus leads the average. Midtown leads the spikes. The watch order follows the spikes.
- Leave a quiet-alert deck with a loud average. Campus exists so the brief cannot pretend average and alert are the same question.
I do not think this replaces a parking management system. I think it is a decent skeleton for a local Eventhouse rehearsal when the raw record is already a flat file. The GitHub project includes the tests and the GIF so you can see the same 229 alerts I am quoting here.
Let's Design
Before tables, I wrote the question and the cutoff on paper.
- Question. Which decks throw the most near-full alerts in this busy window?
- Grain. One reading, meaning one gate at one timestamp, not a daily rollup and not a ticket.
- Near full.
OccupancyPct > 88, stored asStatus = "ALERT". - Also show. Average occupancy by deck, first alert rows, and the single peak reading.
That list kept me from adding a column every time a noun sounded important. Attendant, plate, and reservation are real words around garages. They are not in the file, because none of my questions used them. I would rather add a join later than decorate the CSV now.
One fact, four decks
A telemetry model gets muddy when two things both claim to be the row. I made the reading the only fact. Deck is a label on the gate, assigned once when the sample is generated. Gate ids are gate-001 through gate-020. Each gate keeps one deck for the whole file. That is a limit. A gate that redirects mid-day would need a different grain, and then the rate would have to say whether the reading counts twice. I kept one deck per gate so the watch order stayed honest.
- Plaza is the street-facing entrance in the story. Plenty of readings. Seventy-two alerts.
- River sits on the waterfront side. Fifty-two alerts. Enough signal to matter, not enough to win.
- Campus is the stable-busy edge. Highest average. Twenty-two alerts. I almost cut it, then kept it so average and alert would disagree.
- Midtown is the dense pocket that wins the watch order on this seed. Eighty-three alerts. Peak reading lives here.
I am using "semantic" here only in the plain sense: names that mean the same thing in the CSV, the KQL table, and the laptop brief. I did not build an ontology service. I built a file and a scorer. An Eventhouse, in Fabric terms, is the container for event-like, time-based data. Inside it you keep one or more KQL databases, and each database holds tables. When you create an Eventhouse, Fabric also creates a default child KQL database with the same name, plus a default queryset so you can start querying immediately. That is the shape I leaned on.
I almost put the sample in a Lakehouse first "because everything else lives there." That would have been the wrong rehearsal. A Lakehouse is fine for batch tables and notebook work. This file is a time-ordered event feed with a status that flips on spikes. I wanted KQL over that feed with almost no ceremony. Eventhouse is the item type Fabric gives me for that shape. If the work had been a monthly rollup of cleaned tickets, I would have started somewhere else. It was not.
Compute on an Eventhouse can pause when idle, and wake-up can take a few seconds. Storage sits in two tiers I care about only enough to name: a cache tier for recent query speed and a standard retention tier for durable data. I left Capacity Planner and custom schedules alone for this PoC. A 6,000-row file does not need me to pretend I am tuning an always-on hot path. Knowing that pause exists stopped me from declaring the ingest broken the first time a query waited on a cold start.
Why a parking-garage story, not another generic sensor dump
I have written enough "DeviceId / Temperature / Humidity" samples that my eyes skip them. Garage gates forced better column names. Occupancy is not temperature. Wait seconds are not humidity. Deck is not a cloud region. When the alert fires, the sentence I say out loud is "that entrance is near full," not "the sensor is warm." That sentence is the whole reason I picked this domain for the experiment. From my experience, a sample that sounds like a tutorial also gets treated like a tutorial: nobody argues with the threshold, nobody notices when Campus leads the average and loses the watch order, nobody asks whether average occupancy is the wrong ranking key.
The business problem I am rehearsing is small and practical. During a busy window, a parking desk wants a watch order. Not a digital twin. Not a pricing engine. A ranked list of decks where the near-full status already flipped, plus proof that the underlying rows actually landed. That is a real sentence I have heard in other domains, translated here into garage language so the Eventhouse practice would stick to something I would bother to re-open.
Columns are what the gate controller could report
On each reading the columns are the things I could imagine coming off a cheap logger.
-
Timestampis UTC. The generator starts at2026-11-01T00:00:00Zand steps thirty seconds. I refused local time because busy windows cross midnight and I did not want a DST argument in the first query. -
GateIdis the sensor identity. Twenty gates. The sample picks one at random per row so the file looks like a multiplexed feed, not a round-robin that is too tidy. -
Deckis the ops grouping I want the rate over. It is not a GPS point. I have no basemap in this PoC, so I refused to invent coordinates. -
OccupancyPctis the fill level. Base process is a Gaussian around 62 with a 14 point spread, clipped to 5..100. About 1.3 percent of rows get a +22 point spike so the alert set is real without becoming a constant emergency. -
WaitSecondsandThroughputPerMinare companions. They are not in the alert rule. I kept them so the table looks like a gate packet, not a single-number toy. Wait rises a bit as occupancy climbs. Throughput softens a bit when the deck is packed. Those are soft couplings, not a physics model. -
StatusisALERTorOK. The generator writes it. Queries filter it. The brief counts it.
Each of those names appears the same way in the CSV header, the Eventhouse mapping, and the Python dataclass. When I came back the next day, "88 percent" stopped me from flipping Campus into the watch list just because the average chart looked loud. The picture got less flattering. The metric got clearer.
Why status is stored, not recomputed everywhere
I almost left status out and wrote where OccupancyPct > 88 in every query. That works until a chart uses 85 and a notebook uses 92. I have done that to myself. Storing status does not remove the need for a threshold. It forces the threshold to live in one writer. If I change the line later, I regenerate the CSV and re-ingest. The queryset does not silently disagree with the laptop brief because somebody edited one filter.
The 88 percent line lives in data/generate_gate_telemetry.py as ALERT_OCCUPANCY_PCT and again in deckpulse/analyze.py as a constant used for assertions. The KQL script filters on Status. That is intentional. The explore path should read the column the ingest landed, not re-litigate the policy in every cell.
Constraints the tests actually check
The tests are short on purpose.
- The sample has exactly 6,000 rows.
- Every ALERT row has
OccupancyPctabove 88. - The four deck names are Plaza, River, Midtown, and Campus.
- Deck reading counts sum to 6,000.
- Alert ranking is sorted descending.
I do not parse the README into the tests. The README quotes the same numbers so a future edit that changes the seed without changing the prose fails my eyes before it fails CI. From my experience, a test that only checks "alerts > 0" will green-light a broken story.
Let's Get Cooking
This section is the code I actually run. Each block is followed by why I wrote it that way.
Generate the sample once, then stop touching it
SEED = 19 ROW_COUNT = 6_000 INTERVAL = timedelta(seconds=30) START_TIME = datetime(2026, 11, 1, tzinfo=timezone.utc) GATES = [f"gate-{i:03d}" for i in range(1, 21)] DECKS = ["Plaza", "River", "Midtown", "Campus"] SPIKE_PROBABILITY = 0.013 SPIKE_OFFSET_PCT = 22.0 ALERT_OCCUPANCY_PCT = 88.0 I pinned the constants at the top of data/generate_gate_telemetry.py so the article and the file share one dial board. Seed 19 is arbitrary. What matters is that it is fixed. If I change the seed, I change the story, and then I have to rewrite every number in this post. I would rather regenerate on purpose than drift by accident.
def build_rows(seed: int = SEED, row_count: int = ROW_COUNT) -> list[tuple]: rng = random.Random(seed) deck_of = {gate: rng.choice(DECKS) for gate in GATES} rows: list[tuple] = [] timestamp = START_TIME for _ in range(row_count): gate = rng.choice(GATES) occupancy = round(min(100.0, max(5.0, rng.gauss(62, 14))), 2) if rng.random() < SPIKE_PROBABILITY: occupancy = round(min(100.0, occupancy + SPIKE_OFFSET_PCT), 2) wait_seconds = round( max(0.0, rng.gauss(45, 25) + max(0.0, occupancy - 70) * 1.8), 1 ) throughput = round( max(0.5, rng.gauss(12, 3) - max(0.0, occupancy - 80) * 0.15), 2 ) status = "ALERT" if occupancy > ALERT_OCCUPANCY_PCT else "OK" rows.append( ( timestamp.strftime("%Y-%m-%dT%H:%M:%SZ"), gate, deck_of[gate], occupancy, wait_seconds, throughput, status, ) ) timestamp += INTERVAL return rows The deck map is built once per gate. That keeps a gate from teleporting between entrances mid-file. The spike is a rare additive bump, clipped at 100. I tried a second process first and the alert rate jumped into a range that made the brief look like a constant emergency. 1.3 percent with a +22 point bump gave me 229 alerts on seed 19, which is enough to rank and still readable.
I write the CSV with CRLF on purpose. Fabric's local-file wizard is fine with either line ending in practice, but a stable byte layout makes "did I regenerate?" a diff instead of a feeling.
Load the same columns the Eventhouse will see
@dataclass(frozen=True) class Reading: timestamp: str gate_id: str deck: str occupancy_pct: float wait_seconds: float throughput_per_min: float status: str def load_readings(path: Path) -> list[Reading]: rows: list[Reading] = [] with path.open("r", encoding="utf-8", newline="") as handle: reader = csv.DictReader(handle) for raw in reader: rows.append( Reading( timestamp=raw["Timestamp"], gate_id=raw["GateId"], deck=raw["Deck"], occupancy_pct=float(raw["OccupancyPct"]), wait_seconds=float(raw["WaitSeconds"]), throughput_per_min=float(raw["ThroughputPerMin"]), status=raw["Status"], ) ) return rows The dataclass field names are snake_case because I am in Python. The CSV headers stay PascalCase because that is what I want in KQL. Mapping at the edge is better than renaming the Fabric table to please the laptop. I have renamed columns "for convenience" before and then spent an evening chasing a mapping that no longer matched the wizard preview.
Mirror the KQL summarize and alert filters
def summarize_by_deck(rows: list[Reading]) -> list[dict[str, Any]]: totals: dict[str, list[float]] = defaultdict(list) for row in rows: totals[row.deck].append(row.occupancy_pct) summary = [ { "deck": deck, "avg_occupancy_pct": round(sum(levels) / len(levels), 2), "readings": len(levels), } for deck, levels in totals.items() ] summary.sort(key=lambda item: item["avg_occupancy_pct"], reverse=True) return summary def alert_rows(rows: list[Reading]) -> list[Reading]: return sorted( [row for row in rows if row.status == "ALERT"], key=lambda row: row.timestamp, ) def alerts_by_deck(rows: list[Reading]) -> list[dict[str, Any]]: counts: dict[str, int] = defaultdict(int) for row in alert_rows(rows): counts[row.deck] += 1 ranked = [{"deck": deck, "alerts": count} for deck, count in counts.items()] ranked.sort(key=lambda item: item["alerts"], reverse=True) return ranked These three functions are the laptop twins of the explore queries. Average by deck. Filter alerts. Count alerts by deck. I deliberately sort averages by occupancy and alerts by count, because those are different questions. On this seed Campus wins averages and Midtown wins alerts. If I had sorted only by average, I would have told the wrong story.
def build_report(rows: list[Reading]) -> dict[str, Any]: alerts = alert_rows(rows) by_deck = summarize_by_deck(rows) alert_rank = alerts_by_deck(rows) peak = max(rows, key=lambda row: row.occupancy_pct) return { "row_count": len(rows), "alert_count": len(alerts), "ok_count": len(rows) - len(alerts), "alert_rate": round(100.0 * len(alerts) / len(rows), 2) if rows else 0.0, "deck_summary": by_deck, "alerts_by_deck": alert_rank, "peak_gate": peak.gate_id, "peak_occupancy_pct": peak.occupancy_pct, "peak_deck": peak.deck, "peak_timestamp": peak.timestamp, "first_alerts": alerts[:8], "alert_threshold_pct": ALERT_OCCUPANCY_PCT, "preview": rows[:10], } The report is a plain dict so FastAPI can return it without a second schema layer. first_alerts is capped at eight because the ASCII table is for a terminal, not for a dump. The peak is computed across all rows, not only alerts, which is a small honesty check. On this seed the peak is an alert anyway. If a future seed produced a peak under 88, I would want to know my spike logic broke.
Print a brief a human can argue with
def format_ascii_table(report: dict[str, Any]) -> str: lines: list[str] = [] lines.append("DeckPulse parking-garage gate telemetry brief") lines.append( f"rows={report['row_count']} alerts={report['alert_count']} " f"ok={report['ok_count']} alert_rate={report['alert_rate']:.2f}%" ) lines.append(f"threshold OccupancyPct > {report['alert_threshold_pct']}") # ... deck averages, alert ranks, peak, first ALERT rows ... return "\n".join(lines) I care about the first lines more than the chart. If the terminal says 6,000 rows and 229 alerts, I can trust the Eventhouse count query when it says the same thing. If the Eventhouse says 5,980, the wizard did not finish, and no amount of dashboard polish fixes that. In my opinion, the ASCII table is the acceptance test that survives a bad wifi day.
The KQL side of the same brief
GateReadings | count GateReadings | summarize AvgOccupancyPct = avg(OccupancyPct), Readings = count() by Deck | order by AvgOccupancyPct desc GateReadings | where Status == "ALERT" | project Timestamp, GateId, Deck, OccupancyPct | order by Timestamp asc GateReadings | where Status == "ALERT" | summarize Alerts = count() by Deck | order by Alerts desc Run those one at a time in the default queryset. Clear placeholder sample text first. KQL will throw a syntax error if a comment and a query share a line in a messy paste. I learned that the hard way and left a note in the .kql file for future me.
.show table GateReadings schema as json is the last check. I want Timestamp as datetime, occupancy as a numeric type, and status as string. If the wizard treated the header row as data, the schema looks wrong immediately.
The Fabric create-and-ingest path, in the order I actually click
I keep this sequence next to the code because the code alone does not land the table.
- Open a Fabric workspace that already sits on a Fabric-enabled capacity.
- Select New item, then Eventhouse, name it
DeckPulseEventhouse, and wait out the short provision. - Confirm the default KQL database and the default queryset exist. Same name family. Different jobs.
- Open the database, choose Get data, then Local file.
- Create table
GateReadings, uploadgate_telemetry.csv, turn on first-row header, keep the auto mapping, finish. - Wait for create table, create mapping, and data queuing to show success. Then look for about 6,000 rows in the activity tracker.
- Open the queryset, clear placeholders, run the explore file one statement at a time.
I wrote those steps into docs/eventhouse-ingest.md so I would not reconstruct them from memory next month. Memory is where my thresholds go to mutate. The doc is deliberately boring. Boring is the point.
The wizard creates an ingestion mapping for me. On this sample it is effectively GateReadings_mapping. I do not hand-author a mapping for a flat CSV with a header row. I do glance at the preview. If Status shows up as a number, something is wrong. If Timestamp looks like a string of gibberish, the header toggle is off. Those two glances catch most of the pain I used to discover only after writing a wrong where clause.
Why the laptop mirror is not optional in this PoC
People sometimes ask why I bother with main.py if Fabric already...




