On November 27, 2021, GitHub ran a routine schema migration on a large MySQL table. The last step was a rename to swap the updated table into place, and it triggered a failure nobody had seen before: a large share of MySQL read replicas got stuck in a semaphore deadlock. According to GitHub’s own incident report, the outage lasted two hours and fifty minutes and degraded core services, including GitHub Actions, API requests, Codespaces, Git operations, Issues, Packages, Pages, Pull Requests, and Webhooks. Writes stayed healthy the whole time, and GitHub later confirmed that no data was corrupted. Still, it was serious enough that the company paused schema migrations everywhere while it investigated.
That’s the uncomfortable thing about database migrations. A statement can be valid and perfectly safe in a test environment and still take down production, because production has things your laptop doesn’t: open transactions, background workers reading the same tables, and several versions of your application code running at once in the middle of a deploy. ITIC’s 2024 downtime survey, which polled more than a thousand firms worldwide, found that over 90 percent of mid-size and large enterprises now put the cost of a single hour of downtime above $300,000. It’s worth getting the sequence right before you find the gaps live.
This piece walks through that sequence for a PHP application running on MySQL. Assess before you touch anything. Pick a strategy that fits what the operation really needs. Keep old and new code compatible during the rollout. Backfill without losing writes, validate before you commit to the change, and know exactly what you’ll do if something breaks halfway through.

Why a Live Database Doesn’t Behave Like Your Local One
A lot of the risk isn’t in the SQL itself. It comes from transactional side effects in MySQL’s DDL statements that most developers don’t expect. According to MySQL’s documentation on implicit commits, running an ALTER TABLE, CREATE INDEX, or similar DDL statement implicitly commits any open transaction, both before and after the statement runs. So if your migration code assumes it can roll back a schema change together with a data change in the same transaction, it can’t. The schema change is permanent the moment it runs.
The second surprise is that “online” doesn’t mean “lock-free.” MySQL’s own documentation on online DDL limitations notes that even an in-place online DDL operation may need a brief exclusive metadata lock while it runs, and it always needs one in its final phase. A long-running or idle transaction holding a lock on that table can make the migration stall or time out. The same page points out that MySQL has no built-in way to pause or throttle an online DDL operation once it’s started, and a long one can cause replication lag. You won’t see any of this until there’s real concurrent traffic on the table.
Build a Real Picture Before You Touch Anything
Don’t guess. Find out exactly who reads and writes the table: web requests, queue workers, scheduled jobs, reporting queries, and any older API or mobile client that might still use a column your current code no longer touches. Then check what the operation needs on your specific MySQL version, how much room you have on disk and in replica lag, and who can call a pause if something looks wrong mid-run.
Before you pick an approach, confirm:
The exact MySQL version and which algorithm the planned operation supports on it
Current disk headroom, binlog growth rate, and replica lag
That your backup actually restores, not just that it exists
Who makes the go or no-go call during the run
Matching the Strategy to What the Operation Actually Requires
When native DDL is enough
Since the online DDL feature matured, MySQL has run most ALTER TABLE operations with minimal locking by default. It uses the INSTANT, INPLACE, or COPY algorithm depending on what each operation allows, and MySQL’s documentation confirms that INSTANT became the default for supported operations in MySQL 8.0.12. One detail catches people out. Before MySQL 8.0.29, INSTANT could only add a column at the end of the table, and MySQL didn’t check whether the new row size would exceed InnoDB’s limits until that same release. So if you’re adding a column anywhere but the end on an older 8.0 build, you won’t get INSTANT, even though the syntax looks exactly the same.
When expand and contract is worth the extra work
Renaming a column or changing its type breaks any code that still expects the old shape. That’s exactly what happens during a rolling deploy, when two versions of the app are live at once. A SaaS company called Autumn ran into this in March 2026. According to their published post-mortem, a collation change on two production tables spiked database CPU and started failing queries. When the team tried to reverse it with another ALTER TABLE, the statement couldn’t get the exclusive lock it needed because active queries kept holding it. The result was repeated deadlocks, and they had to cut application traffic completely to recover. Their fix going forward was to stop altering columns in place: add the new column, write to both, backfill in batches, then cut over. That’s the expand-and-contract pattern in practice.
When an online schema-change tool is the right call
For large tables where even INPLACE would lock too long, two tools dominate. Percona’s documentation for pt-online-schema-change explains that it works by building an empty copy of the table with the new structure, then using triggers to replicate ongoing writes while it copies existing rows in the background, and it will refuse to run if foreign keys reference the table unless you explicitly tell it how to handle them. GitHub’s gh-ost requirements and limitations documentation describes its binlog-based approach, which avoids using triggers to capture ongoing changes. However, newer releases add support for existing triggers through the --include-triggers flag, which recreates them on the ghost table at cutover. Foreign keys remain unsupported. Neither tool is a drop-in replacement for understanding your schema’s actual dependencies first.
When a maintenance window is the honest answer
Not every migration needs a zero-downtime engineering effort. If the table is small, traffic is low at 3 a.m., and the operation needs COPY whatever tool you use, a short planned window is often less risky than adding an online schema-change tool nobody on the team has run before.
A Worked Example: Replacing One Column With Another
Here’s a concrete case: replacing customers.full_name with customers.display_name. First, add display_name as a nullable column. That costs nothing and breaks nothing. Next, deploy code that writes to both columns but still reads from full_name, inside the same transaction as any other customer update. Keep in mind that the schema change already committed on its own, so it can’t be rolled back with that transaction.
Then backfill the older rows with a dedicated PHP command. It should page through the table by primary key in stable batches, not by offset, save its progress so a crash doesn’t send it back to the start, and throttle itself based on current database load. Check the backfilled values against live writes before you switch which column the app treats as the source of truth. Add the NOT NULL constraint only after you’ve checked for nulls, duplicates, and orphaned references. Dropping full_name and the dual-write code comes last, once nothing in your logs still touches it.
What Actually Goes Wrong Halfway Through
GitHub’s 2021 incident is a good example because the failure wasn’t in the ALTER statement. It happened in the rename step after the data copy finished, which is the part most teams assume is safe. GitHub’s report notes that migrations on its largest tables often take weeks to complete, and it was that final rename that set off the replica deadlock. The takeaway isn’t to avoid renaming tables. Every phase of a migration needs its own rollback or repair plan, not just the main ALTER.
When something fails partway, what you do depends on which layer broke. A clean, reversible schema step can be rolled back directly. A broken backfill usually needs a roll-forward repair from its last saved checkpoint, not a restart. Reach for a full restore only when neither of those is safe, and only once you know exactly which writes happened after your last backup. Don’t assume there weren’t any.
Laravel gives you some useful visibility here, although it doesn’t make the DDL itself any safer. According to Laravel’s migrations documentation, migrate:status shows which migrations have actually run against the database, and --pretend prints the SQL a migration would run without running it. The --isolated option makes Laravel take an atomic lock through your cache driver before migrating, so a second deploy server trying the same migration at the same time won’t run it too. That helps you avoid duplicate runs on a multi-server deploy. It tells you nothing about whether the ALTER TABLE will lock acceptably at your table size. Only your preflight assessment can answer that.
Where a Second Set of Eyes Helps
Most migrations don’t need outside review. The ones worth pausing on involve large, busy tables where the team hasn’t run a backfill at this scale before, or where nobody is quite sure what depends on the table. In those cases, a focused review of the migration sequence and rollback plan before anything touches production usually costs less than finding the gaps during an incident.
This is the kind of work Techosquare does every day: PHP and MySQL backend development for database-driven systems. If your team is planning a schema change on a table this important and wants a second set of eyes on the sequence before anything runs in production, it’s worth having that conversation early rather than mid-incident.
GitHub and Autumn both learned the same lesson the hard way. The risk in a schema migration is rarely in the statement itself. It’s in whatever is still running against the table while that statement executes, and in whether you’ve actually tested the step you’re counting on to recover.
About the Author
Vineet Arora covers backend engineering, database architecture, and PHP application development. His articles explore schema design, migration strategy, query performance, and the operational practices that keep live systems stable as they scale.
His content aims to simplify technical concepts for developers, business owners, and technology decision-makers. By explaining engineering trade-offs and common technical challenges, he helps readers better understand the considerations involved in building, maintaining, and scaling backend systems.
Areas of Focus
PHP and Laravel backend development
MySQL database design and migrations
Application performance and reliability
API development and integration
Legacy system modernization