This is a submission for the Presage SmartSpectra SDK Challenge
What I Built
It's late, I'm watching a recorded lecture, my eyes close "for a second"… and when I open them the lecturer is three slides ahead. Or I get up for water and come back to five minutes I never heard. Then I scrub back and forth trying to find where I stopped following.
Drift is a desktop lecture player that notices those moments through the webcam and handles them for you:
- Eyes closed for 3 s → the lecture pauses and a marker drops on the progress bar where your eyes closed.
- You leave your seat → it pauses and waits: "Paused: you stepped away · Waiting for you…"
- When you're back → one key (R) rewinds to 30 seconds before you drifted.
- Looking down at your notes never pauses it. Only a long look-away (20 s, adjustable) does.
- At the end → a list of every moment you missed, each one click away.
It's a study aid for anyone learning from recorded video: university lectures, online courses, exam prep. It makes no health claims and doesn't judge whether you're tired; it only notices closed eyes and an empty seat.
Demo
Code
Drift
A lecture player that pauses when you drift off, and rewinds to where you stopped following.
You're studying from a recorded lecture late at night. Your eyes close for a few seconds, or you get up for a drink, and when you're back you've lost your place. Drift watches for that through your webcam, using Presage SmartSpectra face signals:
- It pauses the lecture when your eyes stay closed for 3 s, you leave your seat, or you look away for a long time.
- It marks the moment on the progress bar.
- It offers to rewind when you're back: "Rewind to 14:02?", 30 s before you drifted.
- It lists the moments you missed at the end, so you can rewatch just those parts.
Drift is built for the DEV Presage SmartSpectra SDK Challenge.
Drift is a study aid. It is not a medical, fatigue or safety device, does not…
Windows 11 / Apple Silicon Mac / Linux x64 · Node 20+ · npm install && npm start · bring your own Presage key and any MP4 lecture. A 3-minute test script for judges is in the README.
How It Responds to Human Signals
Drift requests face metrics only from SmartSpectra (faceMetrics) and reacts in real time at 30 fps:
| What happens | Signal | Pause after |
|---|---|---|
| Eyes closed | Eye openness computed from the 478 face landmarks, below 60% of your own open-eye level (measured in a 5-second calibration) | 3 s |
| You left | Landmarks stop moving (the SDK repeats the last ones when the face is gone) | 4 s |
| You stood up (backup) | No landmarks + validation code NoFaceFound / FaceTooHigh | 4 s |
| Long look-away | No face tracked | 20 s |
| Glance at notes | No face tracked briefly, code FaceTooLow | never |
I tested the SDK before building on it. My first plan was the obvious one: use face.blinking. So before writing the app I built a small signal recorder and ran a 12-minute guided test (eyes closed 3 s × 10, notes, phone, leaving the seat) on my Dell Latitude's webcam. What I found changed the design:
-
blinkingreports blinks, not closed eyes: it fired for only 0.14–0.21 s even during a 6-second closure, so it caught 0/10 three-second closures. - Eye openness from the landmarks caught 10/10, with 0 false alarms in 5 minutes of normal watching, notes or phone use.
- After I left, validation still said
Ok, but the landmarks froze. That freeze is now the "away" signal. -
FaceTooLowappeared both when I looked down and when I got up, so it never triggers a pause on its own. - Lit from behind, my webcam fell to 15 fps and the SDK stopped (
kSustainedLowFrameRate); with a lamp in front it held 30 fps. - Clear glasses work. Tinted or mirrored lenses hide the eyelids from the camera, so the camera check now tells you to take those off.
The recordings (eye-openness ratios and timings only, no images) are replayed by the unit tests, so every rule is checked against real data on every build.
How I integrated the SDK, step by step:
- Electron, following Presage's official quickstart.
bindSmartSpectraIpc(win)in the main process, the SDK's preload bundled with my own narrow bridge (encrypted key + settings), context isolation and a sandboxed renderer. - Drift owns the camera. It opens the webcam at 640×480 / 30 fps, asks the camera to hold at least 24 fps, and hands the stream to the SDK with
useMediaStream. A camera check measures the real frame rate first, so the SDK never starts in light where it would stop. - Face metrics only, telemetry off:
new SmartSpectraSDK({ apiKey, requestedMetrics: [...faceMetrics], enableTelemetry: false }). - Every
metricsevent is decoded withdecodeMetrics; fromface.landmarksDrift computes eye openness (the eye-aspect ratio on the MediaPipe eye points) and how much the face moved since the last frame.validationStatuscodes feed the position checks and the on-screen hints. - A presence tracker turns those into watching / eyes closed / away / looking away, and a drift engine maps the start of each episode onto the video timeline, pauses, and works out the rewind point. Only time while the lecture plays counts.
- Failure handling: if the SDK stops (low frame rate, network) or Windows cuts the camera, the lecture pauses with an explanation instead of silently no longer watching, and Drift reconnects once by itself.
Prize Categories
Best Thoughtful Real-World Use
The need: recorded lectures are now how many students learn, and they are often watched late and tired. Losing your place costs time and leaves gaps you don't notice until the exam.
Who it's for: students and self-learners watching long recorded lectures and courses on a laptop.
How I designed it to be usable for them:
- Calibrated to you: a 20-second camera check measures your own normal eye openness and tells you in plain words if the light or framing is wrong.
- Never in the way: glancing at notes doesn't pause; only clear drifts do, and only while the lecture is playing. Three sensitivity presets and an adjustable look-away pause.
- Recovery in one key: R rewinds to just before you drifted; Space continues. Markers and an end-of-lecture list show exactly what to rewatch.
- Keyboard and screen-reader friendly: every action has a shortcut; the pause card is announced; reduced motion is respected; status never relies on colour alone.
- Honest when it can't see you: "Signal poor" and "Camera off" banners, and a pause card if the camera drops, so it never pretends to be watching.
Consent and Data Handling
- Consent first: a plain-language screen explains what the camera looks at, what Drift never does and what leaves the computer. The camera stays off until you agree and click "Turn on camera".
- Always visible, always stoppable: a "Camera on" pill shows whenever it's on; Esc or Stop camera turns it off from any screen, and you can withdraw consent in Settings. No covert or background monitoring: the camera only runs while Drift is open and you've turned it on.
- No video stored or sent by Drift: frames go only to the SmartSpectra SDK running inside the app. Drift never records, saves or uploads video or images.
- Only what's needed: face metrics only; no pulse, breathing or other vitals requested.
- SDK telemetry: turned off (
enableTelemetry: false). The SDK still contacts Presage with your key to check the key and credits. - Storage: your API key, encrypted with Windows DPAPI via Electron
safeStorage(never plain text, never in the repo), and a small settings file (consent, preferences, your open-eye level). The list of drift moments lives in memory and is gone when you close the app. - Deletion: Settings → Remove key; delete Drift's user-data folder (
%APPDATA%\Drift) to remove everything else. - Sharing and retention: nothing is shared and nothing else is retained.
- AI services: none. Drift uses no AI service at run time.
- Not a health tool: no diagnosis, no fatigue scores, no medical decisions, not for driving.
Agent Session
I built Drift with Claude as a coding assistant, milestone by milestone, testing each one on my laptop before moving on.



