This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass
Built with my guy @hsmuyu ._.
What I Built
I love exploring. Trying new food, finding streets I've never walked, going out just to enjoy the day. But most of my days happen in one room, staring at a screen.
Part of it is being a student. Most fun things cost money: arcades, skiing, trips. And living on campus, even simple hobbies like gardening aren't really possible. So most of the time, the only option left is walking out the door.
But honestly, I barely walked anywhere for fun. I only go out when I have to: grab food, to buy groceries, to get through the week. Same route there & back. I'd lived here for months and still only knew the way to the canteen and the supermarket :')
When I tried to explain this to my teammate, it came down to games. I can spend hours wandering a game map for no real reason, because games reward you. You take a quest, you go somewhere, you get something back: a sound, a badge, a number going up. Every small step feels like an achievement. My walks had none of that. I'd go out, get what I needed, and come back. Nothing found, and nothing to show or brag about.
So we built footfolio. It turns the real world around you into a map, wherever you are: your campus, your hometown, a city you're visiting. And it rewards you for walking it.
The fog clears only where you actually walk, and the real cafes, parks or shops underneath appear as you uncover them. Gemma, running on your phone, gives you quests: hints about real places still hidden in the fog. You read one, put the phone away, and go look. If you're stuck for a while, a small light shows up to help, but only after you've tried.
When you find the place, the app notices. The quest completes, the place joins your map, and you earn your way toward badges: Explorer for the places you've uncovered, Local legend for a famous spot to eat, Landmark for a famous place to see.
It's small, but it's the exact same feeling that keeps me playing a game: I did something, and I have something to show for it.
Over time, the cleared map becomes a record of where your feet have been. That's the name: a folio of your footsteps. And the fog that's left tells you where to go discover next :D
Demo
Live: https://footfolio.onrender.com/
footfolio is made for phone. Open the link, allow location, and it starts wherever you are. Then go for a walk. Keep the screen on while you walk, since phones pause GPS when the screen locks.
Not outside right now? Add ?debug to the link and tap the map to walk. Debug mode drops you on our campus in Singapore. ?debug=10 walks ten times faster.
Gemma is optional. Turn it on from the quest card or Settings. It downloads once (about 800 MB, or 240 MB on phones without WebGPU), so Wi-Fi is a good idea. Without it, quests still work with plain clues.
Code
footfolio
Clear the fog, one walk at a time.
footfolio covers your neighbourhood map in fog. Walk outside and the fog clears where you go; the cafes, parks and shops underneath appear as you uncover them. A small open AI model, Gemma, runs on your phone and writes riddles about places still hidden in the fog, so you end up walking streets you have never tried. Over time the map becomes a record of everywhere you have been.
The screen is meant to be the shortest part: read a riddle, put the phone away, look around.
Built for the DEV "Hacktoberfest Open-Source AI Challenge: Week 1" (Touch Grass).
How it plays
| Fog | Each good GPS fix clears about 40 m around you. Your uncovered map is saved on the phone. |
| Places | Real places from OpenStreetMap, within 2 km of you. They appear as gold pins when your walk uncovers them. |
How I Built It
footfolio is a phone web app: Vite and plain TypeScript, Leaflet for the map, and OpenStreetMap for the tiles and the places. Almost everything runs on the phone. The fog is a grid of ~11 m squares drawn on a canvas; every good GPS reading clears the squares around you, and what you've cleared is saved in the browser.
Render: hosting the app, and feeding Gemma real places
The app is a Render Static Site. That matters more than it sounds: phones only share GPS with HTTPS pages, so it needs real hosting to work at all. The build command runs the test suite before building, so a broken build never goes live.
The places come from OpenStreetMap through the Overpass API, and that's where I needed a backend. Browsers can't call Overpass reliably: the public servers are slow, rate-limited, and want a header browsers aren't allowed to set. So there's one Render Web Service, server/places.ts, about 85 lines with one job: GET /places?lat&lng.
The heart of it is a few lines:
// server/places.ts (trimmed) const c = roundCentre({ lat, lng }); // round to ~550 m again: never an exact position const entry = await region(c.lat, c.lng); // cached for a week; one Overpass call per region send(200, entry.gz, { 'content-encoding': 'gzip', 'cache-control': 'public, max-age=86400' }); // ... console.warn('upstream failed:', String(e)); // the error only: no coordinates are ever logged The app sends a position already rounded to ~550 m, and the server rounds it again. It asks Overpass for the places within 2 km, filters them down to the ones worth finding, gzips the answer (a dense city goes from about 1 MB to 100 KB) and caches it for a week. No database, no accounts. When the official Overpass server went down on the day I built this, I added two mirrors it falls back to, and if they all fail, it serves the last copy it has.
That one service is why footfolio works wherever you are. The app loads a 2 km circle around you, and when you've walked 1.5 km away it quietly loads the next one.
Every riddle Gemma writes is about a place that came through Render. Render doesn't run the model, on purpose (more on that below), but it's what gives the model something real to write about.
Gemma: on the phone, writing riddles
The rule I set early: the code decides, Gemma writes. The code picks the hidden place, how far it is and when to give more help, and your GPS decides when you've found it. Gemma only writes the clues. A small model can't be trusted with game rules or map data, but it's good at turning "a cafe, 300 m away, near the park you found yesterday" into a riddle.
It runs in the browser, two ways:
- With WebGPU: Gemma 3 1B (4-bit ONNX, ~800 MB) through Transformers.js.
- Without WebGPU (older iPhones, many Android phones, in-app browsers): Gemma 3 270M (4-bit GGUF, ~240 MB) on the CPU through wllama, which is llama.cpp compiled to WebAssembly.
The second path exists because of a real moment: I turned Gemma on and my phone said "needs WebGPU", while my teammate's worked fine. Plenty of phones are like mine, and telling those users "no" felt wrong, so they get a smaller model on the processor instead. It's slower, so their clues are written ahead in the background.
I picked the models by testing them side by side. The 270M model on its own mostly copied my examples back or wrote as if it were the player. The 1B wrote 16 usable clues out of 16, at about a second each on a laptop GPU. Nothing downloads until you switch Gemma on, and the switch shows the size first.
Making a small model sound human (no fine-tuning)
My first riddles were robotic: "Head north-east to a cafe concealed by the fog." Correct, but it read like a GPS, and honestly, most people can't use "north-east" on a walk. I didn't fine-tune anything. Four things in src/questText.ts did the work.
1. Give it a character and a tiny job. Gemma doesn't play an assistant here; it plays the fog itself:
const RULES = 'You are the fog in a walking game. Speak to the player (you) in one short, playful sentence of at most 12 words. ' + 'Say what the hidden place is like, never its name. Do not give a direction, a distance or a number. Do not invent other places, people or stories.'; The direction moved out of the words entirely. The wisp on the map shows where to go, so Gemma only has to describe what a place feels like: something a small model is actually good at.
2. Show, don't tell. Small models follow examples far better than rules. So every prompt opens with two hand-written examples, sent as real back-and-forth chat turns, picked for the stage of the quest:
return [ { role: 'user', content: `${RULES}\n\n${ask(a[0])}` }, // "Hidden place: Swee Choon Dim Sum (restaurant). Give the first riddle." { role: 'assistant', content: a[1] }, // "Somewhere ahead, little baskets of dumplings are steaming all day." { role: 'user', content: ask(b[0]) }, { role: 'assistant', content: b[1] }, { role: 'user', content: ask(f) }, // the real quest ]; Sending them as separate turns mattered. When I put both examples in one message, both models just kept continuing the pattern instead of answering.
3. Feed it facts, one stage at a time. A quest has three clues. The first is a riddle. The second comes when the wisp appears, and it gets one more fact: the nearest place you've already found, so the clue can start from somewhere you know ("Past the library, follow the smell of steamed buns."). The third comes when you're close: "it's right around you, one last detail."
4. Never trust the answer. Everything Gemma writes goes through clean() before you see it. It keeps the first line, strips emoji, markdown and labels like "Clue:", and cuts to whole sentences. Then it throws the answer away if it:
- uses a number we didn't give it
- uses a compass word
- talks as the player ("I'm ready…")
- copies one of the examples
- gives the place away
That last check is the subtle one. Banning the whole name would be too strict ("Bedok Reservoir Park" would ban the word park), so only the distinctive words are banned:
// "Bedok Reservoir Park" → bans "bedok" and "reservoir", not "park" (the kind already says that) export const giveaways = (f) => { const said = `${f.what} ${f.near ?? ''}`.toLowerCase(); return f.hidden.toLowerCase().split(/[^\p{L}\p{N}]+/u).filter((w) => w.length >= 4 && !said.includes(w)); }; Every rule in that list came from a real bad answer. Gemma gets up to five tries (with some randomness, so each try is different), and every rejected answer is logged so I could keep tuning the checks. A plain line shows instantly while Gemma writes, and its clue replaces it a second or two later. If all five fail, the plain line simply stays. The game never waits on the model.
Why Does Open Innovation Matter?
For footfolio, open isn't a nice extra. It's the reason the idea can exist at all.
A walking game is, by nature, a stream of where you are. Building the riddles on a closed API would mean sending each player's position, and the places around them, to someone else's server on every quest. With Gemma, that never happens:
- Your location stays private. Gemma runs in the browser on your phone, so your path, your fog and your finds never leave it. The only thing that goes out is a point rounded to ~550 m, to the places service we run.
- No third party. No AI company sees where anyone walks, what they found, or what they were looking for.
- Nothing to pay per quest. No API key to hide, no charge per riddle. That matters for who this is for: walking is the free option, and an app that costs money every time it speaks wouldn't fit that. The only running cost is one small places service on Render.
- No lock-in. The model is one line of config. I swapped the 270M for the 1B when the riddles weren't good enough, then brought a 270M build back alongside it for phones without WebGPU. With a closed API, I'd get whichever models the vendor decides to offer.
- It keeps going when the signal drops. Once the page is open, GPS, the fog, the quests and Gemma all run on the phone, and the places for your area are already saved. Lose signal halfway down a back street and the game carries on; only new map tiles wait for the connection.
Open wasn't the easy path, though. Google's own web-ready Gemma 3 models are gated behind a Hugging Face login, and a public website has nowhere to keep that key, so I used the openly published ONNX and GGUF builds instead. A small model also needs guidance a big API wouldn't: the narrator prompt, the examples, the checks on every answer. And the download, 240 to 800 MB, is a real ask, which is why it's opt-in and shows its size first.
A hosted model would have written better riddles with no download. But it would have meant a server watching everyone's walk, and for a game that is literally a map of where you go, that's the wrong trade.
The data is open too. Every place in footfolio comes from OpenStreetMap, mapped by volunteers, and when the official Overpass server went down mid-build, the open ecosystem had mirrors to fall back on.
My Agent Session
This is the session where I worked on the Gemma part. First checking if it could run on a phone at all, then getting it to write clues without saying the place's name :)

