This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Trail Companion is an offline-first Progressive Web App (PWA) built to do something unusual for an AI app: get people off the screen and into the real world.

Most nature identification tools force you to be tethered to a cellular network. You hike into a national park, spot a rare bird or autumn leaf, pull out your phone—and stare at a loading spinner because there's no signal. Even worse, your location coordinates and photos get uploaded to centralized servers.

Trail Companion turns your phone into an autonomous on-device field scanner:

🌲 Zero-Signal Nature Identification: Point your camera at any tree leaf, songbird, wildflower, mushroom, or insect. The open-weight vision model identifies the species in ~140ms right inside your browser—no internet connection, no servers, and zero API keys.
🥾 Screen-Free Walk Mode: Using the Page Visibility API, the app tracks how much of your hike you spent off-screen. It calculates a real-time "Touch Grass" percentage, rewarding you for keeping your phone in your pocket.
📵 Anti-Screen Negative Detector: Point the camera at a laptop, monitor, or office desk, and the model flags it immediately: "That's not grass 📵. Put the phone down, open a door, and find the nearest tree."
🎶 Sound of the Trail: An offline Web Audio synthesizer that generates authentic bird whistling calls and autumn wind rustles, helping hikers learn species by ear without downloading heavy audio files.
🏅 Trail Badges & Daily Bingo: 8 offline achievements (e.g. Foliage Scout, Songbird Seeker, True Grass Toucher) and a 3×3 bingo card that update dynamically as you explore.
🎨 Nature Passport Canvas Export: Generates a shareable high-resolution field passport graphic with your photos and walk statistics directly to an HTML5 Canvas.
🔬 Zero-Shot "Teach a Species": Encounter a rare local plant? Type its common name in the Studio, and the open-weight text encoder computes new embeddings on-device in seconds. No model retraining required.

Demo

Live Deployed App: https://trail-companion.onrender.com
PWA Capabilities: Installable directly to your home screen on iOS and Android. Open it once on Wi-Fi to cache the weights (~87 MB), and it works in Airplane Mode anywhere on Earth.
Instant Sample Deck: Includes 6 built-in field test photographs (Maple Leaf, Northern Cardinal, Fly Agaric, Monarch Butterfly, Sunflower, and an Indoor Desk) so you can test the classification and anti-screen rejection right from your desk before heading outside.

Code

Master66999 / Trail-Companion-

🌲 Trail Companion

Offline Nature Identification & Exploration PWA with Open-Source AI
Built for Hacktoberfest 2026: Open-Source AI Challenge Week 1 ("Touch Grass")

Trail Companion is an offline-first Progressive Web App (PWA) that identifies leaves, birds, flowers, fungi, and insects entirely in your browser with zero cellular signal. It runs an open-weight foundation model directly on device, tracks screen-free time on your walks, and reminds you to pocket your phone and look at the trees.

🌐 Live Deployed App: https://trail-companion.onrender.com

✨ Features

⚡ 100% On-Device AI Inference: Powered by an int8-quantized CLIP ViT-B/16 vision model via Transformers.js and ONNX Runtime Web. Zero API calls, zero server bills, and absolute privacy.
📵 Anti-Screen Detector: Snap an indoor room, laptop screen, or car, and the app detects non-nature scenes to gently encourage you to go outside.
🥾 Screen-Free Walk Mode: Clocks your total walk time vs…

View on GitHub

GitHub Repository: https://github.com/Master66999/Trail-Companion-

The entire codebase is open-source under the MIT License.

Key Architecture:

index.html: Semantic PWA with clean white & emerald green design system across 6 dedicated views (Home, Field Scanner, Native Guide, Walk Mode, Field Journal, and Studio).
css/style.css: Professional high-contrast white & forest green aesthetic with bento grids, glassmorphism cards, and fluid layouts.
js/worker.js: Dedicated Web Worker running Transformers.js on WebAssembly SIMD without locking the UI thread.
js/scoring.js: Shared zero-shot cosine similarity and softmax calculation.
js/species.js: Curated taxonomy of 76 native flora and fauna species.
scripts/build-embeddings.mjs: Build-time prompt ensembling that precomputes text embeddings into a compact 208 KB JSON file.
sw.js: Service worker providing cache-first offline capability.

How I Built It

Trail Companion is architected entirely around open-weight foundation models and open-source local inference:

1. The Open-Weight Core: CLIP ViT-B/16

We utilized OpenAI's open-weight CLIP (Contrastive Language-Image Pretraining) ViT-B/16, converted to ONNX by the open-source community (Xenova/clip-vit-base-patch16).

CLIP projects both images and natural language text into a shared 512-dimensional vector space. Classification is performed via vector cosine similarity:
$$\text{similarity} = \frac{\mathbf{v}{\text{image}} \cdot \mathbf{v}{\text{text}}}{|\mathbf{v}{\text{image}}| |\mathbf{v}{\text{text}}|}$$

2. Local Inference via Transformers.js & ONNX Runtime Web

We execute inference inside the browser using Transformers.js v4 on top of ONNX Runtime Web:

int8 Quantization: Using the quantized vision encoder (q8) cut the download size down from 345 MB to just 87 MB.
WASM Acceleration: Runs on WebAssembly SIMD in a background worker, delivering blazing inference times (130–160ms per image) on consumer laptops and mobile devices.
Cache Storage: The model weights persist in the browser's Cache Storage. Once cached, the app operates with complete autonomy in airplane mode.

3. Build-Time Prompt Ensembling

To avoid forcing mobile users to download both the text encoder (64 MB) and the vision encoder (87 MB), we decoupled the two:

At build time, scripts/build-embeddings.mjs runs the open-weight text encoder across our 76 species using prompt templates (e.g. "a photo of a Northern Cardinal, a type of bird").
The resulting vectors are shipped in a tiny 208 KB file (data/text-embeddings.json).
Mobile devices only download the vision encoder, saving bandwidth and battery.

Why Does Open Innovation Matter?

This week's challenge asks: Where does an open-based approach work better than a closed one? For Trail Companion, closed APIs simply could not have powered this app:

True Backcountry Reliability: Proprietary APIs (OpenAI, Gemini, Anthropic) require a live internet connection. In state parks, forests, and remote valleys, cell towers don't exist. An open-weight model running locally is the only architecture that actually works when you're touching grass.
Absolute Privacy of Location & Imagery: Nature enthusiasts and families frequently log GPS-tagged trail photos. By running inference on-device, 0 kb of personal image or location data leaves the device. There are no accounts, no server databases, and no behavioral tracking.
Zero Marginal Infrastructure Cost: Calling a cloud multimodal vision API for every camera snap incurs ongoing API bills and rate limits. With open-weight models on client hardware, hosting costs are $0.00—the app can live on static hosting forever without maintenance fees.
Instant Zero-Shot Extensibility: Closed vision classification services restrict you to predetermined taxonomy IDs. Because open-weight CLIP is a dual-encoder, anyone can teach the app an endemic plant from their region in seconds without retraining or fine-tuning.

My Agent Session

This project was built collaboratively with an AI coding agent through pair-programming: from architecting the build-time embedding pipeline and writing headless accuracy checks in Node.js, to engineering the white & emerald green design system, synthesis of Web Audio bird whistles, and browser subagent end-to-end verification.

Trail Companion — Touch Grass Open-Source AI Agent Session

You

Touch Grass
Build something with open-source AI at its core.
That can mean running an open-weight model, building on an open-source agent harness or framework, running inference locally, or all three. Whatever you pick, the open pieces should be what makes your project work.
In your post, tell us why open innovation matters for what you built. Does it run on a laptop with no internet? Keep someone's data off a server they don't control? Let you fine-tune, swap models, or change how your agent behaves? Cost nothing to run? Tell us where your open-based approach worked better than a closed one.
This Week's Theme: Touch Grass
Build something with open-weight models or open-source AI that gets people off the screen and into the world.
That can mean running an open-weight model, building on an open-source agent harness or framework, running inference locally, or all three. Whatever you pick, the open pieces should be what makes your project work.
Hiking, gardening, birding, run clubs, fall foliage: if it gets someone outside, it counts. The best builds here should make the screen the shortest part of the experience. A few ideas to get you going:
A bird call identifier that works on the trail with no signal
A garden planner that tells you what to plant this week based on your local frost dates
A run club route builder that finds the best fall foliage near you
Bonus points if you take it outside, use it, and tell us how it went.
Submission TemplateS and embed it in your post, or link to it. It's optional, but it helps the judges understand your process.

You

Touch Grass
Build something with open-source AI at its core.
That can mean running an open-weight model, building on an open-source agent harness or framework, running inference locally, or all three. Whatever you pick, the open pieces should be what makes your project work.
In your post, tell us why open innovation matters for what you built. Does it run on a laptop with no internet? Keep someone's data off a server they don't control? Let you fine-tune, swap models, or change how your agent behaves? Cost nothing to run? Tell us where your open-based approach worked better than a closed one.
This Week's Theme: Touch Grass
Build something with open-weight models or open-source AI that gets people off the screen and into the world.
That can mean running an open-weight model, building on an open-source agent harness or framework, running inference locally, or all three. Whatever you pick, the open pieces should be what makes your project work.
Hiking, gardening, birding, run clubs, fall foliage: if it gets someone outside, it counts. The best builds here should make the screen the shortest part of the experience. A few ideas to get you going:
A bird call identifier that works on the trail with no signal
A garden planner that tells you what to plant this week based on your local frost dates
A run club route builder that finds the best fall foliage near you
Bonus points if you take it outside, use it, and tell us how it went.
Show your work. We'd love to see how you built it. Save your agent session with DevRelay and embed it in your post, or link to it. It's optional, but it helps the judges understand your process.

Agent

Executed actions: list_dir (Workspace contents), view_file (Challenges skill)

Agent

The workspace is empty, so I'll check a few choices with you before I start building.

Actions executed: ask_question (Project direction)

Agent

Executed actions: generate_image (Maple leaf sample), generate_image (Cardinal bird sample), generate_image (Mushroom sample), run_command (Tool versions)

Agent

Sample photos are done. Next I'll check the current Transformers.js version and confirm the CLIP model files exist before writing any code.

Actions executed: run_command (Transformers.js version), read_url_content (CLIP ONNX files)

Agent

Executed actions: grep_search (ONNX file list)

Agent

Executed actions: view_file (ONNX file list)

Agent

Executed actions: write_to_file (Species field guide)

Agent

Next, the build script. It runs CLIP's text encoder once in Node and ships the label embeddings, so phones only need to download the 87 MB vision model.

Actions executed: write_to_file (Package manifest), write_to_file (Embedding build script), run_command (npm install)

Agent

While npm installs, I'll write the shared scoring code and the inference worker.

Actions executed: write_to_file (Zero-shot scoring), write_to_file (CLIP inference worker)

Agent

On second thought, WebGPU would need the 172 MB fp16 weights, double the 87 MB q8 download. For a trail app, a smaller download matters more, so I'll use quantized WASM everywhere.

Actions executed: replace_file_content (Worker init), manage_task (npm install status)

Agent

Dependencies are installed. Next I'll copy the sample photos into the project and build the embeddings.

Actions executed: run_command (Text embeddings build)

Agent

While that runs, I'll write a Node test that sends the sample photos through the same scoring code.

Actions executed: write_to_file (Classification test)

Agent

Now the app shell, starting with the HTML.

Actions executed: write_to_file (App HTML)

Agent

The 76 label embeddings are built. Next I'll run the accuracy test on the sample photos.

Actions executed: run_command (Accuracy test)

Agent

The test printed nothing. Either the sample photos are missing or the file-path handling is wrong, so I'll check the folder.

Actions executed: list_dir (Samples folder)

Agent

The photos are there, so I'll run the script directly with Node to see what's failing.

Actions executed: run_command (Accuracy test)

Agent

While that runs, I'll write the stylesheet.

Actions executed: write_to_file (App styles), manage_task (Test status)

Agent

Executed actions: multi_replace_file_content (Test script fix), replace_file_content (Ring stroke), write_to_file (IndexedDB helper)

Agent

Next, the main app logic.

Actions executed: write_to_file (Main app logic)

Agent

Next: the service worker, manifest, and icon that make it an installable offline PWA.

Actions executed: write_to_file (Offline service worker), write_to_file (PWA manifest), write_to_file (SVG icon), run_command (Dev server)

Agent

The local server is up. Next I'll test the full flow in a browser.

Actions executed: browser_subagent (End-to-end app test)

You

read thisnfolder and which feacture and what new thing can we want to add in it give me and start implementing all one by one

You

continue

Agent

Executed actions: list_dir (List directory)

Agent

Executed actions: list_dir (List js files)

Agent

Executed actions: list_dir (List samples), list_dir (List data)

Agent

Executed actions: view_file (View HTML)

Agent

Executed actions: list_dir (Root files)

Agent

Executed actions: run_command (Start server)

Agent

Executed actions: generate_image (Monarch sample photo), generate_image (Sunflower sample photo), generate_image (Indoor desk sample photo)

Agent

Executed actions: run_command (Copy samples)

Agent

Executed actions: run_command (Test all samples)

Agent

Executed actions: view_file (View service worker)

Agent

Executed actions: replace_file_content (Update cache in sw.js)

Agent

Executed actions: view_file (View styles)

Agent

Executed actions: view_file (View CSS end)

Agent

Executed actions: view_file (View Hero HTML)

Agent

Executed actions: view_file (View Result Body HTML)

Agent

Executed actions: view_file (View Journal HTML)

Agent

Executed actions: write_to_file (Update HTML layout)

Agent

Executed actions: view_file (View tabbar styles)

Agent

Executed actions: view_file (View samples CSS)

Agent

Executed actions: write_to_file (Update styles)

Agent

Executed actions: view_file (View app.js)

Agent

Executed actions: view_file (View app.js end)

Agent

Executed actions: write_to_file (Update app.js logic)

Agent

Executed actions: run_command (Curl localhost:5173)

Agent

Executed actions: browser_subagent (Browser E2E test)

Agent

Executed actions: write_to_file (Write SUBMISSION.md)

Agent

Executed actions: write_to_file (Write README.md)

Agent

Executed actions: run_command (Git status)

Agent

Executed actions: view_file (View package.json)

Agent

🌿 Project Overview & Implemented Features

Your project in Week2/ is Trail Companion — an offline-first Progressive Web App (PWA) built for the Hacktoberfest Open-Source AI Challenge: Week 1 ("Touch Grass").

🚀 What We Added & Implemented

We expanded the app from an initial concept into a full, verified offline field scanner:

#	Feature	Details & Implementation
1	6 Built-in Field Test Samples	Added realistic test photographs (`cardinal.jpg`, `maple-leaf.jpg`, `fly-agaric.jpg`, `monarch.jpg`, `sunflower.jpg`, and `laptop-desk.jpg`) for instant testing without needing an immediate camera feed.
2	"That's Not Grass 📵" Anti-Screen Detection	Zero-shot negative classification that spots laptop screens, indoor desks, and keyboards. The app responds with: "That's not grass 📵. Put the phone down, open a door, and find the nearest tree."
3	Interactive Live Viewfinder (Camera HUD)	Added an in-app viewfinder with camera stream (`getUserMedia`), reticle crosshairs, camera flip toggle, and live capture button that automatically powers off the stream after scanning to save battery on trails.
4	Sound of the Trail (Web Audio Synthesizer)	Synthesizes realistic birdsong chirps (frequency-swept sine oscillators), autumn leaf wind rustles (modulated bandpass noise), and nature chimes completely offline with zero audio file downloads.
5	Offline Field Guide (76 Species)	A dedicated tab (`#view-guide`) with real-time text search and category filters (🐦 Birds, 🍁 Trees, 🌼 Flowers, 🍄 Fungi, 🐞 Bugs) detailing scientific names, facts, and outdoor tips.
6	Trail Badges & Outdoor Quests	8 offline achievements (🌱 First Step, 🍁 Foliage Scout, 🐦 Songbird Seeker, 🍄 Mushroom Hunter, 🦋 Pollinator Pal, 🥾 True Grass Toucher, 🧭 Trail Mapper, 🔬 Field Naturalist) stored in IndexedDB.
7	Nature Passport Canvas Export	A visual field card rendered dynamically to an HTML5 `<canvas>` that exports a high-resolution PNG showing your walk stats, off-screen percentage, recent nature photographs, and badges.
8	Zero-Shot "Teach a Species"	Add any local plant or animal on-device. The open-weight text encoder generates new embeddings on the fly without retraining.
9	Full Challenge Submission Article	Created SUBMISSION.md formatted to the official DEV challenge template, covering why open-weight AI is necessary for offline trails.
10	Comprehensive Documentation	Created README.md detailing architecture, setup, and testing.

🧪 Verification & Test Results

1. Accuracy Test (int8 Quantized CLIP ViT-B/16)

Running npm run test:classify across the sample suite in Node:
- cardinal.jpg: Northern Cardinal (97.5% confidence) — 166 ms
- monarch.jpg: Monarch Butterfly (87.2% confidence) — 148 ms
- sunflower.jpg: Sunflower (94.9% confidence) — 137 ms
- maple-leaf.jpg: Red Maple (80.5% confidence) — 135 ms
- fly-agaric.jpg: Fly Agaric (59.8% confidence) — 148 ms
- laptop-desk.jpg: Category None (93.6% Indoors / Screen) — 132 ms (Anti-screen rejection triggered)

2. End-to-End Browser Subagent Testing

Using a mobile viewport simulation (430×900):
- Model Status: Cached in Cache Storage; top pill shows Offline-ready.
- Anti-Screen Test: Desk screenshot triggered the "That's not grass" screen-free reminder.
- Monarch Test: Identified at 82%, played audio cue, and saved to IndexedDB.
- Guide Tab: Search for "oak" returned Northern Red Oak & English Oak; category filters showed 8 fungi species.
- Walk Badges: Unlocked First Step and Pollinator Pal dynamically.
- Nature Passport: Generated the 800×1000 PNG card with observation photos and stats.

🧭 How to Run & Submit

Start the local server: powershell npm run dev Open http://localhost:5173/ in your browser.
Review your DEV submission post: The draft is saved at SUBMISSION.md. You can copy its contents straight into the DEV Hacktoberfest Submission Form.

You

if i want to do this project at vaset level can we take datafrom kaggle

Agent

Yes, absolutely! Kaggle has some of the largest and highest-quality biodiversity datasets in the world.

Here is how you can use Kaggle datasets to scale this project to thousands of species while keeping it fast and running offline in the browser.

1. Top Kaggle Datasets to Use

Dataset on Kaggle	What It Contains	Best For
iNaturalist 2021 / 2024 Challenge	10,000+ species, 2.7M annotated images covering plants, insects, birds, reptiles, fungi.	The gold standard for global biodiversity.
PlantNet-300K	1,081 plant species, 300,000 high-resolution photos.	Flora, leaves, trees, and wildflowers.
NABirds / CUB-200-2011	200–555 North American bird species with precise trait annotations.	Fine-grained bird classification.
FungiCLEF / Danish Fungi	1,600+ wild mushroom species with toxicity labels and habitat metadata.	Foraging safety and mycology.

2. How to Scale This Architecture to 5,000+ Species

Because our project uses dual-encoder CLIP, you don't need to retrain a heavy neural network from scratch to support 5,000 species. There are two practical ways to do it:

Method A: Expand the Precomputed Label Bank (Easiest & Fastest)

In our current pipeline, the browser only downloads the vision encoder (~87 MB). The text embeddings for all species are precomputed and stored in data/text-embeddings.json.

Extract Taxonomy from Kaggle: Download the metadata CSV (e.g. categories.json or train_mini.json from the iNaturalist Kaggle competition). Extract:
- Common name
- Scientific binomial
- Kingdom / Class (Aves, Plantae, Fungi, Insecta)
Batch Embed with Node.js / Python: Run scripts/build-embeddings.mjs on the Kaggle species list.
Memory Footprint:
- Each 512-dimension vector takes only 2 KB.
- 1,000 species = ~2 MB file.
- 5,000 species = ~10 MB file.
Speed in Browser: Computing dot products between 1 image vector and 5,000 label vectors takes less than 10 milliseconds in JavaScript using a typed Float32Array.

Method B: Benchmark Accuracy Using Kaggle Test Sets

You can download 1,000 Kaggle test images across various categories and run our headless test script:
powershell node scripts/test-classify.mjs path/to/kaggle_test_images/*.jpg
This lets you calculate your top-1 and top-5 accuracy percentages and tune your text prompts (e.g. adding specific cues like "bark of an oak", "underside gills of a mushroom").

Method C: Use BioCLIP (Open-Weight Model Trained on iNaturalist)

If you want expert-level biological accuracy, there is an open-weight model called BioCLIP (imageomics/bioclip), which was specifically pretrained on the iNaturalist dataset. You can convert BioCLIP to ONNX format with:
bash optimum-cli export onnx --model imageomics/bioclip bioclip_onnx/
Then plug it directly into js/worker.js with zero architectural changes.

3. Pro Architecture: GPS-Aware Regional Filtering

If you scale to 20,000 species worldwide, asking the model to distinguish between an Amazonian parrot and an Arctic gull on every snap can introduce false positives.

A high-performance pattern is offline geo-filtering:
1. Store a lightweight mapping of species_id -> region_code (e.g., NA-EAST, EU-CENTRAL, IN-SOUTH).
2. When the user opens the app or turns on GPS, the app filters the 20,000 candidates down to the ~600 species actually found in their local ecosystem.
3. This boosts accuracy significantly and keeps inference instantaneous.

Summary

You can download the iNaturalist 2021 metadata CSV from Kaggle, script the generation of several thousand species vectors into data/text-embeddings.json, and the existing app will scale to thousands of species with zero changes to the client runtime.

You

before that change all ui then chnage all structure workflow change it add laning page then after create differnt page and try fill fil empty space use professinalcolor like white and green only

Agent

Executed actions: write_to_file (White & Green CSS)

Agent

Executed actions: write_to_file (White & Green HTML)

Agent

Executed actions: view_file (View route in app.js)

Agent

Executed actions: view_file (View route logic in app.js)

Agent

Executed actions: view_file (View app.js wire-up)

Agent

Executed actions: replace_file_content (Update app.js router)