Quick answer: When an API key lands in Git, revoke or rotate it first, within minutes, because public repositories are scanned by bots almost immediately. Then check the provider's logs for use since the commit time, find every place the key was copied, and only after that clean history. Rewriting history never un-leaks a secret; rotation does.

This week one of the most-discussed #security posts on DEV was titled "My Agent Scrubbed a Leaked API Key by Force-Pushing main. It Made Everything Worse." The instinct it describes is very common: you see the key in a commit and your first move is to make the commit disappear. That feels like fixing it. It is not, and a rushed force-push can break every teammate's clone and CI while the key is still valid.

I'm Muhammad Ali Mughal, a Principal Software Engineer in Karachi. I build the free developer tools at PixelValut and have handled several of these incidents for clients. This post starts a new series on leaked secrets: detecting them, rotating them and recovering cleanly. I wrote it with help from AI tools and checked the commands against the Git, GitHub and git-filter-repo documentation. Everything here is about your own repositories and keys.

📚 Series Navigation — Leaked Secrets: Detect, Rotate, Recover

  • Part 1 (This Post): You Leaked an API Key in Git: The First 60 Minutes
  • Part 2: Scanning Repos and CI Logs for Secrets With Regex and Entropy (coming next)
  • Part 3: Rotating Secrets Without Downtime: Dual Keys and Overlap Windows
  • Part 4: Secrets in Container Images and Build Logs
  • Part 5: Pre-Commit and CI Guards That Stop the Next Leak
  • Part 6: Post-Incident Review: Proving a Leaked Key Was Never Used

Related earlier posts: Leaked TLS Private Keys: Proving Compromise & Rotating Safely · Break-Glass Accounts & SSO Audit Logs · Webhook Security in Production: HMAC-SHA256 Signatures

Row of padlocks on a metal rack, AI-generated

Why history rewriting is step five, not step one

Once a commit is pushed, assume the secret has been copied. Public GitHub repositories are watched by automated scanners, both GitHub's own secret scanning partners and people looking for keys to abuse. Forks, local clones, CI caches, pull request refs and mirror services all keep copies that your rewrite will not touch. Even on private repos, everyone with read access and every CI runner has seen it.

So the order matters:

Minute Step Goal Common mistake
0–10 Revoke or rotate the key Make the leaked value useless Rewriting history first while the key still works
10–25 Check provider logs for use Know if this is a leak or a breach Assuming "nobody noticed"
25–40 Find every copy and dependent system Avoid breaking prod during rotation Missing a cron job or second service using the same key
40–50 Deploy the new key everywhere Restore service Pasting the new key into the same file
50–60 Decide whether to clean history, and how Reduce future exposure Force-pushing main without telling anyone

Minutes 0–10: kill the value

Go to the provider's console and revoke the key, or create a new one and disable the old one. If revoking immediately would take production down and you have evidence the key has not been used yet, you can create the new key first and swap within minutes, but do not leave the old key alive for hours "until the deploy window".

Note the exact time and the commit in an incident note. You will need both:

# When did the secret first land on the remote? git log --all --format='%H %aI %an %s' -S 'sk_live_' -- . # Which branches and tags contain that commit? git branch -a --contains <commit-sha> git tag --contains <commit-sha> ~~~{% endraw %} The {% raw %}`-S`{% endraw %} "pickaxe" search finds commits where the number of occurrences of the string changed, which shows you both when it was added and whether it was ever removed. Search for a distinctive prefix of the key, not the whole value, so you do not paste the full secret into your shell history. ## Minutes 10–25: was it used? Most serious providers expose access logs or "last used" timestamps for keys. Check activity between the commit time and the revocation time: - Requests from IP ranges or regions you do not use. - Unusual endpoints: listing users, creating new keys, changing webhooks, high-volume reads. - Billing spikes (cloud and AI API keys are often abused for compute or model calls). If you see any use you cannot explain, this is now a security incident with possible data exposure. Escalate according to your process and keep the logs; do not just rotate and move on. If the leak was a token in a JWT-shaped format, decode it locally to see its scopes and expiry before you assess the damage. The [PixelValut JWT Decoder](https://pixelvalut.com/tools/jwt-decoder?utm_source=devto&utm_medium=series&utm_campaign=leaked-api-key-first-60-minutes) does this in the browser, but treat even a revoked token as sensitive and prefer a local tool for live secrets. ## Minutes 25–40: find every copy A key in one file is rarely only in one file. Search the working tree, all history, and the places around the repo:{% raw %} ~~~bash # All commits, all branches, by prefix git grep -n 'sk_live_' $(git rev-list --all) | cut -c1-120 | sort -u | head # Other files that might reference it grep -rn --exclude-dir=.git -E 'STRIPE_(SECRET|API)_KEY' . ~~~{% endraw %} Then check outside Git: CI variables, deployment platform secrets, other services that were configured by copying the same key, team password managers, and shared docs. If two services share one key, rotating it for one breaks the other; this is the moment to give each its own. For building a reliable search pattern across many key formats, I test the expression against sample strings first in the [Regex Tester](https://pixelvalut.com/tools/regex-tester?utm_source=devto&utm_medium=series&utm_campaign=leaked-api-key-first-60-minutes). Part 2 goes deep on patterns and entropy checks. ## Minutes 40–50: deploy the new key properly Put the new key in a secret store or your platform's environment variables, not back in a committed config file. For a Laravel or Node app, that means {% raw %}`.env` on the server or the host's secret manager, with {% raw %}`.env`{% endraw %} in {% raw %}`.gitignore`{% endraw %}:{% raw %} ~~~bash # Confirm .env is ignored and never tracked git check-ignore -v .env git ls-files --error-unmatch .env 2>/dev/null && echo "TRACKED - remove with: git rm --cached .env" ~~~{% endraw %} Restart or redeploy every service that used the old key, then confirm in the provider's dashboard that the old key shows no new activity and the new one does. ## Minutes 50–60: should you rewrite history? Rewriting is about reducing future exposure and keeping scanners quiet. It is optional for a key that is already revoked, and it is disruptive. Decide calmly: - **Private repo, key revoked:** often fine to leave history and add the commit to your scanner's allowlist with a note. - **Public repo, or policy requires removal:** rewrite, but coordinate. If you do rewrite, use {% raw %}`git filter-repo`{% endraw %} rather than the old {% raw %}`filter-branch`{% endraw %}, on a fresh mirror clone:{% raw %} ~~~bash git clone --mirror git@github.com:org/app.git app-mirror && cd app-mirror printf 'sk_live_REDACTED_PREFIX==>REMOVED_SECRET\n' > ../replacements.txt git filter-repo --replace-text ../replacements.txt git push --force --mirror # only after telling the team and pausing merges ~~~{% endraw %} Then everyone must re-clone or hard-reset, open pull requests need rebasing, and on GitHub you may need to contact support to purge cached views of the old commits. This is exactly why it comes last: done in a panic, it creates an outage for your team while protecting nothing. ## First-hour checklist - [ ] Key revoked or rotated, with the time written down. - [ ] Commit SHA, author time and push time recorded. - [ ] Provider logs checked from commit time to revocation time. - [ ] Any unexplained use escalated as a security incident. - [ ] Every service, CI variable and doc using the key found. - [ ] Shared keys split into one key per service. - [ ] New key stored in a secret manager or ignored {% raw %}`.env`{% endraw %}, never committed. - [ ] History rewrite decided deliberately, coordinated, and done on a mirror clone. ## FAQ ### Is deleting the file in a new commit enough? No. The secret is still in history and in every clone. Deleting it only stops new readers of the latest version. Rotation is the fix. ### Should I make the repo private immediately? It does not hurt, but it does not undo exposure either. Bots may have read it within minutes. Rotate first, then change visibility if it makes sense. ### What if the key cannot be rotated quickly? Some legacy integrations make rotation painful. Restrict the key as much as the provider allows (IP allowlist, reduced scopes) right away, and treat building a rotation path as an urgent follow-up. Part 3 covers dual-key rotation. ### Do I need to tell anyone? Your team, so nobody pushes on top of a planned rewrite, and your security contact if the key had access to user data. If logs show misuse involving personal data, follow your legal and contractual notification duties. ### Will GitHub catch it for me? GitHub secret scanning and push protection catch many well-known formats and may notify the provider, which sometimes revokes the key automatically. Do not rely on it for custom or less common tokens. ### My AI coding agent committed the key. Anything different? The response is the same, but also check the agent's configuration and logs: what files it can read, whether it echoed the secret elsewhere, and whether it has permission to force-push. It should not. ## What's next Part 2 builds a practical scanner for repos and CI logs using regex and entropy checks. Until then, browse the [PixelValut tools](https://pixelvalut.com/tools/?utm_source=devto&utm_medium=series&utm_campaign=leaked-api-key-first-60-minutes) and use the [Hash Generator](https://pixelvalut.com/tools/hash-generator?utm_source=devto&utm_medium=series&utm_campaign=leaked-api-key-first-60-minutes) when you need to record a fingerprint of a leaked value in an incident note without storing the value itself. --- ### Follow & Connect - **Website:** [PixelValut Tools](https://pixelvalut.com/tools/?utm_source=devto&utm_medium=series&utm_campaign=leaked-api-key-first-60-minutes) - **Watch on YouTube:** [Resume Boost AI](https://www.youtube.com/@resume-boost-ai) · [PixelValut](https://www.youtube.com/@pixel-valut) · [Daily AI News By Ali](https://www.youtube.com/@AIDailyNewsByAli) - **TikTok:** [PixelValut (@pixelvalut007)](https://www.tiktok.com/@pixelvalut007) - **X:** [Muhammad Ali Mughal (@AliMughal887)](https://x.com/AliMughal887) - **GitHub:** [Muhammad Ali Mughal (@ali111887)](https://github.com/ali111887) - **LinkedIn:** [Muhammad Ali Mughal](https://www.linkedin.com/in/muhammad-ali-m-03434a120/)