Disclosure: I built the Apify Actor used in this tutorial. This post was written with AI assistance.

A talk, a lecture series or a product channel is a knowledge base that happens to be stored as video. To search it or feed it to a language model you need the captions as text, split into pieces, each piece turned into a vector. This tutorial does that with three pieces: an Apify Actor that returns timestamped transcripts, a small chunker that keeps the start time of every chunk so results link back to the exact second in the video, and a local open-source embedding model so nothing except the transcript download leaves your machine.

What you need:

  • Python 3.11 or newer
  • An Apify API token
  • A few hundred megabytes of disk for the embedding model

The Actor is YouTube Transcript Scraper. It costs $0.001 per transcript delivered, so $1.00 per 1,000 videos. Videos with no captions, private or unavailable videos, bad links, and the listing of a playlist or channel are all free: you pay only for transcripts that arrive in your dataset. The free Apify plan's monthly usage credit covers the 20-video example below many times over.

pip install apify-client sentence-transformers numpy export APIFY_TOKEN=apify_api_xxxxxxxx

pip install sentence-transformers downloads PyTorch if you do not have it, which is the bulk of that disk space.

1. What a transcript record looks like

The Actor's only required input is videos: a list of video links, youtu.be links, Shorts, playlist links, channel links (@handle or /channel/…) or bare video IDs, mixed freely. For a playlist or channel, maxVideosPerSource (default 50, up to 1,000) says how many of its videos to take; a channel yields its newest uploads first. language (default en) picks the caption track, preferring a human-written track over an auto-generated one in that language.

Each video comes back as one record:

{ "videoId": "aircAruvnKk", "url": "https://www.youtube.com/watch?v=aircAruvnKk", "title": "But what is a neural network? | Deep learning chapter 1", "channel": "3Blue1Brown", "status": "OK", "language": "en", "isAutoGenerated": false, "languageMatched": true, "lengthSeconds": 1132, "segmentCount": 286, "segments": [ { "start": 4.22, "duration": 1.18, "text": "This is a 3." }, { "start": 6.06, "duration": 4.653, "text": "It's sloppily written and rendered at an extremely low resolution of 28x28 pixels," } ], "text": "This is a 3. It's sloppily written and rendered ..." }

Three fields matter for RAG:

  • segments: every caption line with start and duration in seconds. This is what lets a search result point at a timestamp.
  • text: the whole transcript joined into one string, for when you do not need timestamps.
  • status: OK, NO_CAPTIONS or FAILED, with error explaining the last two. Nothing comes back silently empty, and the two failure cases cost nothing.

isAutoGenerated: true means YouTube's automatic captions: fine for search, but expect missing punctuation and the occasional misheard term. languageMatched: false means the video had no track in the language you asked for and you got its best available one instead; availableLanguages lists what the video has.

2. Fetch, chunk, embed

Here is the whole pipeline. Save it as index.py.

import os, json import numpy as np from apify_client import ApifyClient from sentence_transformers import SentenceTransformer client = ApifyClient(os.environ["APIFY_TOKEN"]) run = client.actor("highbrow_fame/youtube-transcript-fast").call(run_input={ "videos": ["https://www.youtube.com/@3blue1brown"], "language": "en", "maxVideosPerSource": 20, }) WINDOW = 60 # seconds of speech per chunk chunks = [] for v in client.dataset(run.default_dataset_id).iterate_items(): if v.get("status") != "OK": print("skipped", v.get("url"), v.get("status"), v.get("error")) continue buf, start = [], None for seg in v["segments"]: start = seg["start"] if start is None else start buf.append(seg["text"]) if seg["start"] + seg["duration"] - start >= WINDOW: chunks.append({"video": v["videoId"], "title": v["title"], "start": int(start), "text": " ".join(buf)}) buf, start = [], None if buf: chunks.append({"video": v["videoId"], "title": v["title"], "start": int(start), "text": " ".join(buf)}) model = SentenceTransformer("all-MiniLM-L6-v2") vectors = model.encode([c["text"] for c in chunks], normalize_embeddings=True, show_progress_bar=True) np.save("chunks.npy", vectors) json.dump(chunks, open("chunks.json", "w")) print(len(chunks), "chunks from", len({c["video"] for c in chunks}), "videos")

That is the 30 lines. What each part does:

Fetch. call() starts the Actor run and waits for it. The Actor page reports fifteen videos processed in nine seconds in its own test run. Twenty videos cost at most $0.02 (fewer if some videos have no captions).

Chunk by time, not by characters. The chunker accumulates caption lines until the window covers 60 seconds of video, then emits a chunk carrying the start of its first line. Sixty seconds of speech is roughly 120 to 180 words, a comfortable size for a small embedding model and a readable size for a human looking at a result. The last partial window of each video is kept too. Time-based chunking has a property character-based chunking lacks: every chunk maps to a deep link, https://www.youtube.com/watch?v=<id>&t=<start>s.

Embed locally. all-MiniLM-L6-v2 is a 384-dimension sentence-transformers model, under 100 MB, fast on a CPU, and good enough for "find the part of the talk about X". normalize_embeddings=True makes every vector unit length, so cosine similarity is a plain dot product later. If you want better retrieval quality and can spare the time, swap in BAAI/bge-small-en-v1.5 or intfloat/e5-small-v2; the code does not change (the e5 family expects a query: / passage: prefix, check its model card).

Store. A NumPy array and a JSON file are enough for a few thousand chunks. Replace the two saves with a call to Chroma, FAISS, pgvector or whatever you already run once you outgrow it.

3. Search it 

import sys, json import numpy as np from sentence_transformers import SentenceTransformer chunks = json.load(open("chunks.json")) vectors = np.load("chunks.npy") model = SentenceTransformer("all-MiniLM-L6-v2") q = model.encode([sys.argv[1]], normalize_embeddings=True)[0] scores = vectors @ q for i in np.argsort(-scores)[:5]: c = chunks[i] print(f"{scores[i]:.2f} {c['title']}") print(f" https://www.youtube.com/watch?v={c['video']}&t={c['start']}s") print(f" {c['text'][:160]}...\n") 
python search.py "how does backpropagation update the weights"

Each hit prints its similarity score, the video title, a link that opens the video at the right second, and the first line of the chunk. That is already a usable "search inside this channel" tool.

4. Handing the chunks to a language model

Retrieval-augmented generation is this search plus a prompt. Take the top few chunks, put them in the prompt with their source, and ask the model to answer only from them:

top = [chunks[i] for i in np.argsort(-scores)[:5]] context = "\n\n".join( f"[{c['title']} @ {c['start']}s] {c['text']}" for c in top ) prompt = ( "Answer the question using only the transcript excerpts below. " "Cite the video title and timestamp you used. If the excerpts do not " "contain the answer, say so.\n\n" f"{context}\n\nQuestion: {sys.argv[1]}" )

Send prompt to whichever model and SDK you use. Because every chunk carries a title and a start time, the model can cite [title @ 412s] and you can turn that back into a link.

Choices you may want to make differently

  • Window size and overlap. 60 seconds with no overlap is simple. If answers often straddle a boundary, add a 10-second overlap by starting the next chunk from the last few segments of the previous one, or shrink the window to 30 seconds and retrieve more hits.
  • Use text instead of segments when you do not care about timestamps; then any text splitter will do.
  • Re-indexing. Keep the run's dataset ID. Transcripts of published videos rarely change, so for a growing channel fetch only new videos (a channel link with a small maxVideosPerSource returns the newest uploads) and append to your store instead of re-embedding everything.
  • Subtitles for a player. The Actor can add srt and vtt fields (includeSrt, includeVtt) if you also want to show captions alongside the search result.

Limits

  • Videos without captions return NO_CAPTIONS and are free, but they are also absent from your index. There is no speech-to-text step here; add one yourself if you need those videos.
  • Machine translation to your language (translate: true) is best effort and YouTube restricts it heavily; in my own test it came through for 1 of 8 videos. The original transcript is returned either way.
  • Up to 1,000 videos per playlist or channel per run.
  • Transcripts are the creator's content. Indexing them for your own search or research is a different thing from republishing them; respect YouTube's terms and the creators' rights.

The complete input schema, output fields and pricing are on the Actor page.