Say you're building a TypeScript app that transcribes audio, classifies images, or learns from your own data. How much of the machine learning work can stay in TypeScript?

That depends on the job. Calling a model service keeps the app in TypeScript, but the model runs elsewhere. Running an existing model locally brings its inputs, weights, memory use, and device support into your app. Training your own model adds data preparation, gradients, an optimizer, evaluation, and checkpoints.

I wanted to explore the last two paths in TypeScript, so I've been building Affon, a runtime for numerical computing and machine learning.

What can it handle beyond a demo?

Affon's decoder language model app defines attention and embeddings, prepares text batches, trains and evaluates a model, saves checkpoints, and generates text. A saved local WikiText-2 run summary records 12 epochs and 52,128 steps, ending at 4.55 training loss and 5.09 validation loss. These are results from that run, not a comparison with another framework. The app source shows the full workflow.

For existing models, the inference app runs text, image, and audio examples. Apertus 8B generates text on CUDA from a BF16 checkpoint; Whisper transcribes speech on Metal. This clip shows those two separate runs on separate machines:

Closer view of Apertus 8B generating text, then Whisper transcribing speech in separate Affon runs

What can you inspect?

In Affon, a model is a Program: a description of its inputs, parameters, and operations. Given a loss and optimizer, Affon derives a training Program with parameter updates. You can inspect its structure, including the model's dependency graph, before compiling and running it. That matters when you need to understand what a larger model asks the runtime and the device to do.

Readable detail of the Affon decoder model dependency graph

Open the full resolution Inspector view

This is the Program Inspector showing a decoder model.

Try a small training run

The housing example is a quick entry point. It reads a bundled CSV, trains a small network, and checks predictions on rows held out from training. It runs on a CPU without downloading a model. Install Affon, then run:

git clone https://github.com/neucine/affon.git cd affon affon examples/housing-program.ts 

Affon is experimental. CPU has the broadest compatibility; Metal and CUDA support depends on the operations, types, and shapes in a workload. See the backend support guide for those limits.

If you build ML features in TypeScript, which part would you want to keep local: inference, training, or both?