This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
What I Built
What I Built
Ghost Gallery is a haunted castle you scroll through. You start outside in the fog with one window lit. Scroll, and the camera pushes through the glass into a midnight banquet where Einstein, Cleopatra, Shakespeare, Marie Curie and Tesla sit at a long table. Scroll again and a portal opens onto a graveyard with six headstones for people history forgot. You can leave each one a white rose.
That’s the Halloween half. The half I care about is that you can add someone of your own. Give a name, a memory and a photo. The photo is cut out, washed pale and made see-through, and your person drifts into the room with the others. Then you can talk to them.
They only know what you told them. Ask something outside it and they say, gently, that the memory has faded, and ask you to tell them more. I didn’t want a grandmother ghost that invents things.
That’s how I read the theme. Día de los Muertos is about telling the stories of the dead so they stay. The castle is the costume. The “tell me about her” box is the ofrenda.
TL;DR. The thinking (reading the photo, playing the ghost in chat) runs on Gemma 3 through Ollama on your own machine. The see-through cutout runs in your browser. There are no accounts and no API fees. A cloud model is only used if you add a Gemini key yourself.
Demo
Code
How I Built It
The open pieces:
Gemma 3 through Ollama does two jobs. It looks at the uploaded photo, picks what the ghost wears on their head and holds in their hand, picks a glow colour, and writes one warm line about them. It also plays the ghost in chat. Both calls go to Ollama on localhost, and the model is one environment variable.
@imgly/background-removal (an ISNet model on ONNX Runtime Web) cuts the person out inside the browser. Canvas code then dims them, tints them pale blue, keeps a faint trace of the real clothing colour, and fades the lower half into mist.
Wikipedia supplies portraits for the famous spirits.
The rule I followed was that the model proposes and the server decides. For the photo, the model must choose from fixed lists (ten head accessories, eight held items). Anything off the list becomes “none”, and a colour that isn’t valid hex falls back to a default. If Ollama is down or the call times out, the ghost is still created with gentle default text instead of an error. Chat then answers with a “voice is faint tonight” message that tells you to start Ollama.
The chat prompt is built only from facts you gave it: the name, your story and the epitaph. It keeps the last 14 turns, caps replies at two to four sentences, and forbids inventing painful facts, diagnoses, secrets or promises.
The original photo is never saved. The server keeps the see-through cutout and the text.
Not everything is a model. The banquet whispers and headstone epitaphs are hand-written. The wind and the low drone are synthesised with the Web Audio API, with no audio files. The whole journey is one animation loop driven by scroll position.
Summon Spirit (type the name of someone who has passed) [IS THE ONE FEATURE THAT STILL CALLS GEMINI / NOW ALSO GOES THROUGH OLLAMA. PICK ONE.]
[HOW YOU BUILT THE FRONT END, e.g. “I built the front end with Google AI Studio’s agent and edited by hand.”] [HOW THE SCENE ART WAS MADE.]
What didn’t go to plan. My first test ghosts came back with the default text, which means the model never answered in time. [WHY: Ollama not running, or the vision call timing out?] Last week Gemma 12B restarted my laptop three times, so I [SWITCHED TO / TESTED] the 4B, which also reads images. [WHAT HAPPENED.] The fallback did its job, but a ghost with default text isn’t a ghost, so I [WHAT YOU CHANGED].
Why Does Open Innovation Matter?
A photo of someone who has died, and the story you type under it, is about as personal as data gets. I don’t want it on a server I don’t control.
Private by design. Photo and story go to Ollama on localhost. There are no accounts. Ghosts are private by default, tied to an anonymous id in your browser, and sharing to the gallery is a checkbox you have to tick. That’s a convenience, not a security boundary.
A memorial shouldn’t have an expiry date. An ofrenda is kept for generations. A model I’ve downloaded still runs if an API is repriced or retired.
It works with the internet off, mostly. Once the models are downloaded, reading a photo and talking to a ghost need no connection. What still does:
The first run downloads the cutout model (about 80 MB from IMG.LY’s CDN, unless you self-host it). Offline, it falls back to a soft oval.
The page fonts come from Google Fonts.
Summoning a famous person looks them up on Wikipedia.
I could control it. JSON mode, reply length, temperature and the model itself are all mine to set.
The Spirit of It
[YOUR REAL MOMENT: who you tried first, what you typed, what the ghost said back. Paste the exchange.]
[SAMPLE, KEEP ONLY IF TRUE: “I tested it on a photo of someone who wore a red sweater all the time. The ghost came out deep red. That part isn’t even the model. A few lines of code pick the most common saturated colour from the middle of the photo. But it was the first thing that looked like her.”]
