Introducing Starlight Forge: Evidence-Backed Documentation for Real Codebases

Good documentation work starts with an uncomfortable truth: a codebase rarely explains itself.

A package.json tells only part of the story. A directory name can be misleading. A framework convention may not actually be used. The most valuable details are often spread across source files, configuration, tests, CI, deployment files, and existing documentation.

That is why I built Starlight Forge: an open-source Agent Skill that guides coding agents through repository discovery, evidence-backed documentation planning, and safe documentation generation.

It can produce either portable Markdown documentation or an Astro + Starlight documentation site. It is deliberately not an npm package, hosted service, custom CLI, or documentation bot. It is a set of instructions, references, templates, and small local validation helpers that work with coding agents that support the Agent Skills format.

The problem: documentation generators can sound confident while being wrong

Documentation generated from a shallow repository scan often looks polished but has a trust problem. It may:

describe routes that do not exist;
suggest commands that are not in the project;
infer architecture from a framework convention rather than source evidence;
miss an existing contributor guide or deployment workflow;
overwrite documentation written by the team; or
include sensitive information copied from configuration files.

Those are not cosmetic mistakes. They create friction for maintainers and can send new contributors in the wrong direction.

Starlight Forge treats documentation as a repository-analysis task first and a writing task second. The agent should inspect before it plans, plan before it writes, and validate before it reports success.

What Starlight Forge does

The skill provides a canonical workflow in one SKILL.md, with focused reference guides for deeper work such as monorepos, safe repository discovery, evidence quality, incremental updates, Markdown output, and Astro + Starlight generation.

At a high level, the workflow is:

Discover repository scope ↓ Build an evidence map ↓ Propose a documentation plan ↓ Generate in an agreed destination ↓ Validate links, claims, commands, and navigation ↓ Report what actually passed, failed, or was not checked

The purpose is simple: documentation should make clear what was observed, what was inferred, and what remains unknown.

1. Start with repository discovery, not a generic template

Before creating a page, Starlight Forge asks the agent to establish the actual repository root and inspect the files that reveal how the project works:

repository guidance such as AGENTS.md and contribution instructions;
manifests, workspace definitions, and declared scripts;
entry points, framework configuration, tests, CI, and deployment files;
existing documentation that should be preserved; and
representative packages when the repository is a monorepo.

The discovery process is bounded. It skips dependency directories, build output, large binaries, generated artifacts, symlinks during broad scans, and likely secret files unless they are explicitly relevant. Repository text is treated as data, never as instructions that can redirect the agent.

For large repositories, this matters. Reading every file is usually expensive and does not make documentation more accurate. A targeted inventory and focused source inspection produce a better starting point.

Starlight Forge includes an optional deterministic inventory helper for this purpose. It lists paths and useful manifests while avoiding common dependency trees, sensitive-looking file names, and symlinks. It does not generate prose or make network requests.

2. Tie important claims to evidence

The core idea behind the skill is a small evidence map.

For each meaningful claim, the agent should have a source path, configuration file, test, or clearly labelled reasoning behind it. For example:

Documentation claim	Supporting evidence	How to describe it
A package is an application entry point	Manifest script and source entry	Observed fact
Two packages form a shared boundary	Imports and workspace configuration	Inference, with rationale
A deployment command is available	CI workflow or checked-in script	Observed fact
A service has a certain runtime behavior	No inspected source or test	Unknown until verified

This changes the tone of generated documentation in a useful way. Instead of inventing a default command because a project resembles a familiar framework, the agent can say that the command is unknown or point to the script that actually exists.

It also keeps pages reviewable. A maintainer can trace a setup instruction, architecture explanation, or environment-variable name back to the repository instead of accepting a vague summary on faith.

3. Plan pages before modifying files

The skill separates planning from generation.

Once it understands the repository, it proposes the documentation pages, their purpose, the output location, and the existing files that may be affected. If the requested output directory already contains authored documentation, it preserves those files. The agent should ask only when an overwrite is genuinely necessary.

This makes the skill suitable for documentation work that evolves over time. A later refresh can target pages affected by source changes instead of treating the whole docs/ directory as generated content.

4. Choose Markdown or Astro + Starlight

Starlight Forge supports two output modes because documentation needs vary.

Portable Markdown

Markdown is the default choice when a repository needs lightweight, reviewable pages that can live alongside the source. The skill creates linked pages in the destination you approve and adapts the page set to the project. It avoids producing empty boilerplate simply to fill a standard template.

Example prompt:

Use Starlight Forge to inspect this repository and create a concise Markdown developer guide in docs/developer/. Show me the proposed pages before writing, preserve existing files, and cite source files for important claims.

Astro + Starlight

For a navigable documentation website, the skill can generate or integrate an Astro + Starlight site. It inspects the repository first, then uses current Starlight conventions for content, navigation, and built-in search. It does not modify application code just to make documentation generation easier.

Example prompt:

Use Starlight Forge to document this project as an Astro + Starlight site in website-docs/. Inspect existing files first, show the plan, then build the site if dependencies are available. Do not deploy it.

In either mode, the agent should validate local links, source references, and navigation. In Starlight mode, it should run an available build only when the environment and project dependencies make that safe and practical. A skipped build must be reported as skipped, not passed.

5. Build documentation with safety boundaries

Documentation work often has broader access to a codebase than a narrow feature task. That makes safe defaults essential.

Starlight Forge directs the agent to:

keep reads and writes within the agreed repository and documentation destination;
avoid likely credential files and never reproduce secret values;
respect ignore rules and avoid unnecessary dependency or build trees;
avoid executing project code just to learn what it does;
inspect scripts before running them;
preserve existing user-authored documentation;
avoid installing packages or contacting external services unless requested; and
distinguish passed, failed, skipped, and not-checked validation results.

These measures reduce risk; they are not a claim that prompt injection or incorrect documentation can be eliminated. Repository files, generated content, comments, issue text, and tool output are still untrusted data. The real security boundary remains the permissions and tooling provided by the agent host.

Small helpers, not another product to maintain

The repository includes two dependency-free Node.js helpers:

inventory-project.mjs produces a bounded, filename-only project inventory.
validate-docs.mjs checks local Markdown links and verifies the basic Agent Skill frontmatter.

They are intentionally narrow. They do not analyze architecture, write documentation, call an AI service, or install dependencies. The coding agent remains responsible for source inspection, judgement, and writing.

That separation is deliberate. I wanted the project to stay portable and easy to audit instead of turning it into another toolchain that needs a service, account, or build pipeline before it is useful.

Installation and use

The canonical skill is in skills/starlight-forge/. Copy that directory into a supported skills location for your agent, or use the repository plugin packaging where your agent supports it.

For example, after installing the skill, invoke it in a coding-agent session:

# Codex $starlight-forge # Claude Code /starlight-forge:starlight-forge # GitHub Copilot CLI /starlight-forge

The repository also contains plugin manifests for Codex, Claude Code, and GitHub Copilot CLI. For Claude Code, the documented repository installation flow is:

/plugin marketplace add ShubhamTiwari909/starlight-forge /plugin install starlight-forge@starlight-forge

For Codex and GitHub Copilot CLI, see the repository README for the current plugin commands and manual installation paths. Agent installation behavior changes over time, so it is worth checking the linked official documentation and the agent's own skill or plugin listing after installation.

What has been evaluated so far

The project includes small fixture repositories covering TypeScript utilities, a React UI package, a Next.js app, a Node API-shaped project, a monorepo, malformed project metadata, and an adversarial repository. The fixtures are designed to test specific failure modes, such as invented commands, unsafe file reads, unsupported architecture claims, broken links, and unexpected overwrites.

The local checks validate the skill metadata, local Markdown links, inventory behavior around sensitive-looking files and symlinks, and generated fixture output. They do not simulate a full coding agent or prove documentation quality for every repository.

There is also early real-world feedback: I used the skill on an Astro project, where it generated accurate documentation; I also installed it from the repository in Claude Code and used it to create a thorough Astro documentation site. Those results are encouraging, but they are experience reports rather than a controlled compatibility benchmark.

Where I want to take it next

The most useful next step is feedback from real repositories.

I am especially interested in cases where the skill:

missed an important source of truth;
made a claim that needed stronger evidence;
proposed an awkward documentation structure;
needed better monorepo guidance;
found an existing documentation folder that required a more careful update strategy; or
produced Starlight content that needed a framework-specific adjustment.

If you try it, please open an issue with the repository shape, the prompt you used, what it generated, and what you expected. Redact private code and credentials. Concrete feedback will make the reference guides and fixtures much more valuable than generic feature requests.

Try Starlight Forge

You can find the project, installation notes, references, fixtures, and contribution guidance here:

github.com/ShubhamTiwari909/starlight-forge

If your project has documentation that is technically correct but hard to trust, start with a small target: a setup guide, package overview, or contributor page. Ask the agent to show its evidence-backed plan before it writes. That one habit makes generated documentation much easier to review and maintain.

Instagram - https://www.instagram.com/supremacism__shubh/
LinkedIn - https://www.linkedin.com/in/shubham-tiwari-b7544b193/
Email - shubhmtiwri00@gmail.com
Portfolio - https://astro-space-portfolio.vercel.app

You can help me with some donation at the link below Thank you👇👇
https://www.buymeacoffee.com/waaduheck

Also check these posts as well

Shubham Tiwari

Aug 20 '25