There's a particular kind of bug you don't expect when building an AI-powered math tutor.

You expect incorrect calculations. You expect malformed LaTeX. You expect the occasional hallucinated explanation.

You don't necessarily expect your LLM to start inventing images.

Apparently, when asked to provide visual content, my LLM decided that actually using the application's image-generation or graphing capabilities was optional.

Instead, it started generating Markdown image references pointing to external image hosts.

The images weren't real. The links weren't useful. The AI was essentially telling students, "Here's a picture!" and then pointing at something that didn't exist.

Fantastic. Just fantastic. 😂

The AI equivalent of drawing something and pretending it's a photograph

Here's the problem: Pythos already has an actual visualization system.

It can generate interactive mathematical graphs. It has a dedicated visualization engine. It knows how to turn equations into something students can explore.

But the LLM occasionally decided to skip all that and fabricate image references.

This is a good example of why simply telling an LLM what tools are available isn't enough. You need to enforce the boundaries between what the model can describe and what the application can actually render.

So we gave it a cleanup crew

I implemented a new Hallucinated Image Scrubber and Auto-Graph Recovery system.

The changes touch three areas:

  • server/vizEngine/visualFidelity.js
  • server/server.js
  • app.js

Here's what changed:

1. Stop the hallucinations at the source

The system prompt now explicitly prohibits fabricated Markdown image tags and external image-host URLs.

2. Scrub fabricated image references

Post-processing filters catch and remove hallucinated image links before they become part of the user-facing response.

3. Recover the actual mathematics

Rather than simply deleting the bad image references, Pythos attempts to extract the equation being described and convert it into a live [GRAPH: ...] interactive canvas.

In other words, if the AI tries to pretend it has drawn something, we try to make the application actually draw it.

The bigger lesson

This is another reminder that building dependable AI applications isn't just about improving prompts.

LLMs can generate convincing-looking content that has no relationship to anything the application can actually deliver.

The solution isn't to blindly trust the model. It's to build systems around it that validate, constrain, and recover from its output.

Pythos already uses deterministic mathematical engines to verify calculations. Now we're applying that same philosophy to visual output.

Because apparently, I also need to teach my AI the difference between drawing a graph and making one up.

The AI is supposed to help students learn mathematics, not become an abstract artist.

You can try Pythos here:

🌐 https://pythos.lanzar.me

Source code:

💻 https://github.com/JonScott79/Pythos

And yes, the images were fake. The bug was real.