When building FocusFlow — a spatial storytelling canvas that turns architecture diagrams into 60fps cinematic presentations — we faced an intriguing computer vision problem: Click-to-Snap.

When a user imports a rasterized architecture diagram (PNG/JPEG/SVG) and clicks anywhere inside an element (like an ECS task card, a VPC boundary, or a database cylinder), the engine needs to extract its precise bounding box in < 5ms to smoothly anchor camera viewport frustums.

1. The Initial Mirage: WebGPU + MobileSAM

In 2026, the knee-jerk reaction for image segmentation in the browser is to reach for a machine learning model:

  • Run a quantized Segment Anything (MobileSAM) model via ONNX Runtime Web.
  • Offload inference to WebGPU.

The Real-World Costs:

  • Payload Penalty: An initial ~30MB model download.
  • VRAM Overhead: 250MB–350MB GPU buffer allocation.
  • Latency Friction: 120ms–180ms inference per click.
  • Failure Modes: Over-segmenting internal text lines or bleeding into complex canvas backgrounds.

For a lightweight web application that users open in their desktop browser, forcing a 30MB weight download just to detect rectangles felt like engineering malpractice.

We asked ourselves: Can classical computer vision, implemented in pure TypeScript and tuned for CSS geometry, beat neural nets in speed, accuracy, and footprint?

2. Five Real-World CSS Edge Cases That Break Naive Sobel

Traditional Sobel gradient filters assume simple solid shapes on clean contrast. But modern UI diagrams look nothing like that:

  1. Text Hole Perforation: Cards are packed with bold headers, monospace code snippets, and badge pills. High-frequency font glyph edges easily trick naive ray-casters into snapping to inner text.
  2. CSS Linear Gradients: Backgrounds subtly drift across 400px. Standard global thresholding either cuts off mid-card or bleeds across boundaries.
  3. Emoji & Colorful Badges: Clicking directly on an inline icon (e.g. 🛡️ or 🐘) corrupts single-point background color sampling.
  4. Long Title Blockades: A 24-character monospace title forms a contiguous horizontal boundary that cuts the card's flood-fill front in half.
  5. Dark-on-Dark Contrast: Dark-mode cards (#0f172a) against canvas backgrounds (#0c1220) yield miniscule perceptual color differences ($\Delta E < 25$) surrounded by diffuse drop shadows.

3. The Pure Math Architecture: A 3-Tier Pipeline

To solve this without machine learning, we built a 3-tier deterministic pipeline:

3.1 Compact 3-Ring Profiler (Phase 1.6)

Instead of single-point sampling, we sample 33 concentric points across 4 rings ($r \in [1, 8, 16, 24]\text{px}$). Each sample is weighted inversely by its local Sobel gradient flatness:
$$W_i = \max(1, 35 - G_i)$$
High-gradient text pixels are suppressed, while flat card pixels dominate. If a cluster fails validation, a Top-3 candidate fallback loop recovers the true background.

3.2 Floating Gradient BFS

Rather than comparing against an absolute seed color, our BFS tracks local step-to-step color deviation ($\Delta E_{\text{step}} \le 8$).

  • On flat background pixels, expensive $3 \times 3$ Sobel convolution is bypassed entirely.
  • This optimization dropped BFS runtime from 106ms down to 1.8ms (a 58x speedup).

3.3 Spatially Separable Anisotropic Morphological Closing

To seal text perforations without blurring vertical card edges, we employ an anisotropic separable kernel:

  • Horizontal kernel ($r_x = 6\text{px}$): Locks tight around card borders.
  • Vertical kernel ($r_y = 14\text{px}$): Bridges across multi-line text lines and headers. Because the 2D kernel is mathematically separable into 1D passes, algorithmic complexity drops from $O(K^2)$ to $O(2K)$, saving 90% of active pixel computations.

4. Memory & Zero-GC Guarantees

JavaScript garbage collection pauses are the enemy of smooth 60fps canvas interaction.

  • All intermediate masks and flood-fill queues utilize pre-allocated Uint8Array and Float32Array pools.
  • Total heap footprint is strictly capped under 450 KB.
  • Result: 0 GC collections during click-and-drag interactions.

5. Benchmarks & Industrial Validation

Across our 20-suite automated E2E test matrix (including nested cloud diagrams, dark-mode wireframes, and accent-bar cards):

  • End-to-End Latency: < 2.5ms
  • TypedArray Memory: < 450 KB
  • External Dependencies: 0 KB
  • Pass Rate: 20 / 20 PASS (100%)

Conclusion

Sometimes the most scalable engineering decision in 2026 is not to deploy a transformer model. For structured UI domains, classical computer vision combined with modern TypedArrays delivers 60x faster performance with zero bundle bloat.

Our full architecture and implementation details are open-sourced: