The pivot
I spent the last months building a local multi-agent CLI code generator - planner → coder → tester → executor → reviewer, running on open models via Ollama. It closed the loop: it wrote code, ran it, tested it, and fixed it.
It worked. And that's exactly the problem: it solved a problem I don't have. Writing code for files on disk is a solved niche - I was competing with every other code-gen tool for a user who already has one.
What I actually do all day is 3D. So the project became something else: an AI assistant that lives inside Blender and holds real tools for working with models, not just a text box that emits code.
What works right now
The add-on is called Game Asset Assistant, targets Blender 4.2+, and ships as a proper extension zip.
Chat where you work
Docked in the 3D Viewport sidebar (N → AI tab)
Also opens as a standalone floating window with Ctrl+Alt+A
Streamed responses, message history, stop/clear buttons
It knows your scene
Every request carries a compact snapshot injected into the system prompt:
Blender 4.2.0
Scene: Scene | fps=24 | units=METRIC | render=CYCLES
Objects: 12
- Cube [MESH] loc=(0.00, 0.00, 0.00) rot_deg=(0.0, 0.0, 0.0) scale=(1.00, 1.00, 1.00) verts=8 edges=12 faces=6 | uv=UVMap | materials=Steel | modifiers=SUBSURF Selected: Cube Active: Cube | mode=OBJECT So when you ask "how many polygons does the active object have?", the model answers about your object - by name.
One click from answer to action
When a reply contains a
python block, it appears in a box with two buttons: Run code - executes it in the current scene on the main thread, captures print() output and tracebacks, and pushes both back into the chat. That closes the loop: the model writes, Blender executes, the model sees the error and fixes it. To Text Editor - saves it as a text datablock for manual review. Backends Local (Ollama, e.g. qwen2.5-coder:7b) or any OpenAI-compatible API - switchable in preferences, no code changes. The part that broke my brain: Blender forbids threads Here's the non-obvious bit worth the whole post. Blender's API docs say it plainly: Python threads that outlive the script cause hard-to-diagnose crashes in Blender's own drawing code. So the obvious approach - "spawn a thread, call the LLM API, update the UI when it returns" - is off the table. The add-on therefore never blocks and never spawns a thread. Instead: Blender (main thread) worker process network ┌────────────────────────┐ ┌──────────────┐ │ operator appends a │commands │ │ │ JSON line ─────────────┼────────→│ worker.py │──HTTP──→ Ollama / API │ │ │ │ │ bpy.app.timers, │events │ │ │ every 0.1s, reads new │←────────┤ │ │ lines ─────────────────┤ └──────────────┘ └────────────────────────┘ Commands are just JSON objects appended to commands.jsonl. Events (chunk, done, error) are appended to events.jsonl. A bpy.app.timers callback pumps events every 0.1 s - and a timer is not a thread, it's a callback Blender invokes on the main thread between frames. That's the only legal way to do background work. Side effect: the whole add-on ships without any third-party dependency, and the protocol already has room for tool-call requests in a later phase. What's not there yet Being honest about the gaps: No structured tool-calling - the model emits a code block and we exec() it No confirmation for destructive operations (the prompt asks the model to warn; that's a request, not a guarantee) No undo grouping - one Ctrl+Z won't roll back a whole script UV, rigging and texturing exist only as chat advice, not as one-click tools Scene snapshot is blind to UV islands, texel density, weights and armature pose Roadmap Phase What Why first 0 Richer snapshot, undo-push, destructive-op confirmation Cheap, unlocks everything after 1 Tool-calling over the JSONL channel, executed on the main thread with a registry + JSON schemas The reliability jump: from "80 lines of text that may fail" to named calls 2 Texturing: PBR material presets, texture assignment, packing, bake helpers Stable API, high daily value 3 UV: quick unwrap/pack/texel-density, seam-by-angle, and a UV diagnostics report (overlaps, density) Most repetitive work; the report lets the model read the result and propose fixes 4 Auto-rig: metarig → Rigify, auto-weights, weight validation, export checks Hardest - errors are silent, so diagnostics matter more than generation 5 One-click pipelines: GBC Prep, Char Prep, PBR Bake, Game Export Combines 2–4 into buttons 6 Teach the model: tool catalog in the prompt, domain recipes, few-shot examples Ongoing, parallel Integration with other Blender add-ons (delegating to UV packers, for example) is on the list - but the final roadmap is still open. I'd rather ship phases 0–3 and let real usage decide what phase 4 looks like. The plan from here Post progress as it lands. If you're working on AI × 3D pipelines, or you've wrapped a DCC tool for an LLM - I'd love to compare notes. Repo link in the comments. #blender #ai #python #gamedev #3d #showdev 

