There is one moment in day-to-day deployments that anyone who has worked with CloudFormation for a while knows: the update fails, the rollback starts, and then the stack just sits there with UPDATE_ROLLBACK_FAILED. No new deployment, no deletion from the pipeline, just a red status and the question of who changed “just a little thing in the console” yesterday. Whether you use CDK, the Serverless Framework or SAM, there is always a CloudFormation stack underneath, so this affects everyone.
On October 22, 2026, AWS introduced a new option for exactly this situation: Force Rollback. It gets a stack out of UPDATE_ROLLBACK_FAILED with a single call, even when several resources in nested stacks are stuck. That is convenient. It is also a tool that quietly lets drift creep into your infrastructure. In this article I show you step by step how I would handle a stuck stack, when Force Rollback is the right choice and what you absolutely should check afterwards.
In short
With
continue-update-rollback --force-rollback, CloudFormation skips every resource whose rollback failed and brings the stack toUPDATE_ROLLBACK_COMPLETEin one step. The stack can be deployed again afterwards, but at the skipped spots it no longer matches the template. That is why a drift check with a drift-aware change set belongs before the next deployment.
What you will learn in this article
- What
UPDATE_ROLLBACK_FAILEDmeans and which causes are typical - When to fix, skip selectively or use Force Rollback
- How to run Force Rollback safely with the AWS CLI and a TypeScript script
- What
cdk rollback --forcedoes differently from Force Rollback today - How to clean up drift with a drift-aware change set before the next deployment
What UPDATE_ROLLBACK_FAILED means
CloudFormation treats an update like a transaction: either all changes go through, or the stack is reset to the last working state. This reset is called a rollback. Most of the time it runs quietly, and at the end you see UPDATE_ROLLBACK_COMPLETE.
It gets problematic when CloudFormation cannot bring a resource back to its old state. The classic example: someone deleted a table or a database instance outside of CloudFormation. CloudFormation still believes it exists, tries to reconfigure it and fails. The stack ends up in UPDATE_ROLLBACK_FAILED. In this state it accepts no more updates. You have to finish the rollback before you can deploy again.
🎬 Status flow of a CloudFormation stack during a failed update – this animated diagram only runs on my site: watch it on sascha-ms.de
One thing matters for understanding this: UPDATE_ROLLBACK_COMPLETE does not mean “everything is as before”. It only means that CloudFormation considers the rollback finished. How well the state of the real resources matches the template depends on how you got there.
Typical causes of a stuck rollback
Before you skip anything, take a look at the cause. The CloudFormation documentation lists a handful of classics, and for most of them there is a clean fix that works without skipping:
| Cause | What you can do |
|---|---|
| Resource was changed or deleted outside of CloudFormation | Restore the resource with the same name and properties, then continue the rollback |
| Missing IAM permissions for the role CloudFormation uses | Add the permissions, then continue the rollback |
| Service quota reached (limit) | Clean up resources you no longer need or request an increase |
| Expired security token | Simply continue the rollback, the credentials are refreshed in the process |
| Resource does not stabilize, e.g. during a service disruption | Wait, then continue the rollback |
| Not enough signals (e.g. with a CreationPolicy) | Send the missing signals with signal-resource |
Only when none of these fixes works does skipping come into play. Until now, the only way was --resources-to-skip, where you had to list every affected resource by its logical ID. The logical ID is the name of a resource in the template; with CDK it is the generated identifier such as OrdersTable1A2B3C4D. Nested stacks added the format NestedStackName.LogicalId on top. Anyone who has had to piece that together for three levels of nested stacks under time pressure knows why AWS has followed up here.
What Force Rollback in CloudFormation does
Force Rollback is a new option of the existing ContinueUpdateRollback API call. In the SDK the parameter is called ForceRollback, in the AWS CLI --force-rollback. According to the AWS documentation it behaves like this:
- Every resource stuck in
UPDATE_FAILEDdue to a failed rollback gets skipped, and that also applies to nested stacks. - Resources that already failed during the actual update (before the rollback) are not skipped.
- In the stack, skipped resources then show
UPDATE_COMPLETE, with “Resource skipped during UpdateRollback” as the status reason. - Drift detection afterwards reports these resources as
NOT_CHECKED. -
--force-rollbackand--resources-to-skipare mutually exclusive. If you pass both, you get a validation error.
The feature costs nothing extra and, according to the announcement, is available in all regions. What it does not do: repair your resources. It only makes CloudFormation stop clinging to them. The gap between template and reality remains, and you take care of it in the second part of this article.
My decision path: fix the cause first, then skip selectively or with Force Rollback, then check drift.
Step by step: getting a stack out of UPDATE_ROLLBACK_FAILED
Prerequisites
- A current AWS CLI v2. If your CLI does not know the option, it reports
Unknown options: --force-rollback. In my test, AWS CLI v1 version 1.46.2 did not know the parameter yet, while the JavaScript SDK@aws-sdk/client-cloudformation3.1156.0 already did. - Permissions for
cloudformation:DescribeStacks,DescribeStackEvents,ContinueUpdateRollbackand, for the drift check,DetectStackResourceDriftplus read permissions on the affected resources. - For the script further down: Node.js 22 or newer.
- Calm. Under pressure it is easy to copy the first command you find in a forum, and that is exactly what you want to avoid here.
I call the example stack OrdersStack in all commands. Replace the name with yours.
1. Check status and reason
aws cloudformation describe-stacks \ --stack-name OrdersStack \ --query "Stacks[0].[StackStatus,StackStatusReason]" \ --output text Expected result: UPDATE_ROLLBACK_FAILED and a reason such as “The following resource(s) failed to update: [OrdersTable]”. If you see a different status, you are dealing with a different problem, and this article will only help you to a limited extent.
2. Read the cause from the stack events
aws cloudformation describe-stack-events \ --stack-name OrdersStack \ --max-items 50 \ --query "StackEvents[?ResourceStatus=='UPDATE_FAILED'].[Timestamp,LogicalResourceId,ResourceType,ResourceStatusReason]" \ --output table Events come newest first. The interesting ones are the UPDATE_FAILED entries after the rollback started, i.e. above the stack’s UPDATE_ROLLBACK_IN_PROGRESS event. The status reason column almost always tells you what is going on: “Resource not found”, “AccessDenied”, “LimitExceeded”.
3. Fix first, then continue normally
If you can fix the cause, for example by adding missing permissions or restoring a deleted resource with the same name, continue the rollback without options:
aws cloudformation continue-update-rollback --stack-name OrdersStack aws cloudformation wait stack-rollback-complete --stack-name OrdersStack This is the cleanest path, because template and reality match afterwards. The wait command blocks until the stack reaches UPDATE_ROLLBACK_COMPLETE and exits with an error if the rollback fails again.
4. Skip a single resource selectively
If exactly one resource is affected and you know you are deliberately giving it up, --resources-to-skip is still the more precise choice. You decide per resource instead of letting CloudFormation skip everything:
aws cloudformation continue-update-rollback \ --stack-name OrdersStack \ --resources-to-skip OrdersTable 5. Force Rollback when several resources are stuck
If several resources are stuck, possibly spread across several levels of nested stacks, Force Rollback saves you the search:
aws cloudformation continue-update-rollback \ --stack-name OrdersStack \ --force-rollback aws cloudformation wait stack-rollback-complete --stack-name OrdersStack Expected result: the stack is in UPDATE_ROLLBACK_COMPLETE, and the status reason points out that resources were skipped. You can deploy again. But you should not do it just yet.
6. Find the skipped resources
aws cloudformation describe-stack-events \ --stack-name OrdersStack \ --max-items 50 \ --query "StackEvents[?contains(ResourceStatusReason || '', 'skipped during UpdateRollback')].LogicalResourceId" \ --output text With nested stacks, run the same command for the child stacks as well. You can find their names in the “Resources” tab of the root stack or in the stack ARN.
7. Check the drift of every skipped resource
Since drift detection lists skipped resources as NOT_CHECKED, you check them one by one. Drift means that the real configuration of a resource differs from what is in the template.
aws cloudformation detect-stack-resource-drift \ --stack-name OrdersStack \ --logical-resource-id OrdersTable \ --query "StackResourceDrift.[StackResourceDriftStatus,PropertyDifferences]" The result is IN_SYNC, MODIFIED or DELETED, and for MODIFIED it includes the differing properties. Now you know what needs to be sorted out before the next deployment.
The whole thing as a TypeScript script with the AWS SDK
If you need this more often or want it as a runbook tool for your team, a small script pays off. Mine has two modes: without a flag it is a dry run that only shows the status and the causes of the failure. Only with --force does it trigger the Force Rollback, wait for it to finish and check the drift of the skipped resources. This two-step approach is intentional. Whoever runs the script should have seen the causes before anything gets skipped.
Set up the project
mkdir stack-rescue && cd stack-rescue npm init -y npm pkg set type=module npm install @aws-sdk/client-cloudformation npm install -D typescript tsx @types/node // tsconfig.json { "compilerOptions": { "target": "ES2022", "module": "NodeNext", "moduleResolution": "NodeNext", "strict": true, "noEmit": true, "skipLibCheck": true }, "include": ["scripts", "test"] } The logic for evaluating the events
I put the evaluation of the stack events into a separate file with pure functions. That way you can test it without AWS access.
// scripts/rollback-events.ts import type { StackEvent } from "@aws-sdk/client-cloudformation"; export const SKIP_REASON = "Resource skipped during UpdateRollback"; export interface FailedResource { logicalId: string; type: string; reason: string; } /** * Returns all resources whose rollback failed. * CloudFormation returns events newest first. We read up to the * latest UPDATE_ROLLBACK_IN_PROGRESS of the stack itself and collect * all UPDATE_FAILED events of resources along the way. */ export function rollbackFailures(events: StackEvent[], stackName: string): FailedResource[] { const result = new Map<string, FailedResource>(); for (const e of events) { const isStackItself = e.LogicalResourceId === stackName && e.ResourceType === "AWS::CloudFormation::Stack"; if (isStackItself && e.ResourceStatus === "UPDATE_ROLLBACK_IN_PROGRESS") break; if (isStackItself || e.ResourceStatus !== "UPDATE_FAILED" || !e.LogicalResourceId) continue; if (!result.has(e.LogicalResourceId)) { result.set(e.LogicalResourceId, { logicalId: e.LogicalResourceId, type: e.ResourceType ?? "unknown", reason: e.ResourceStatusReason ?? "", }); } } return [...result.values()]; } /** Resources that CloudFormation skipped during Force Rollback. */ export function skippedResources(events: StackEvent[], stackName: string): string[] { const ids = new Set<string>(); for (const e of events) { const isStackItself = e.LogicalResourceId === stackName && e.ResourceType === "AWS::CloudFormation::Stack"; if (isStackItself && e.ResourceStatus === "UPDATE_ROLLBACK_IN_PROGRESS") break; if (e.ResourceStatus === "UPDATE_COMPLETE" && e.ResourceStatusReason?.includes(SKIP_REASON) && e.LogicalResourceId) { ids.add(e.LogicalResourceId); } } return [...ids]; } The actual script
// scripts/rescue-stack.ts import { randomUUID } from "node:crypto"; import { CloudFormationClient, ContinueUpdateRollbackCommand, DescribeStacksCommand, DetectStackResourceDriftCommand, paginateDescribeStackEvents, waitUntilStackRollbackComplete, type StackEvent, } from "@aws-sdk/client-cloudformation"; import { rollbackFailures, skippedResources } from "./rollback-events.js"; const [stackName, flag] = process.argv.slice(2); if (!stackName) { console.error("Usage: npx tsx scripts/rescue-stack.ts <StackName> [--force]"); process.exit(1); } const force = flag === "--force"; const cfn = new CloudFormationClient({}); async function loadEvents(maxPages = 5): Promise<StackEvent[]> { const events: StackEvent[] = []; let pages = 0; for await (const page of paginateDescribeStackEvents({ client: cfn }, { StackName: stackName })) { events.push(...(page.StackEvents ?? [])); if (++pages >= maxPages) break; } return events; } const { Stacks } = await cfn.send(new DescribeStacksCommand({ StackName: stackName })); const status = Stacks?.[0]?.StackStatus; console.log(`Stack ${stackName}: ${status}`); if (status !== "UPDATE_ROLLBACK_FAILED") { console.log("No Force Rollback needed."); process.exit(0); } const failures = rollbackFailures(await loadEvents(), stackName); console.log("Rollback failed for:"); for (const f of failures) console.log(`- ${f.logicalId} (${f.type}): ${f.reason}`); if (!force) { console.log("Dry run. Check the causes, then run again with --force."); process.exit(0); } await cfn.send( new ContinueUpdateRollbackCommand({ StackName: stackName, ForceRollback: true, ClientRequestToken: randomUUID(), }), ); console.log("Force Rollback started, waiting for UPDATE_ROLLBACK_COMPLETE ..."); await waitUntilStackRollbackComplete({ client: cfn, maxWaitTime: 1800 }, { StackName: stackName }); const skipped = skippedResources(await loadEvents(), stackName); console.log(`Skipped: ${skipped.length ? skipped.join(", ") : "none (in the root stack)"}`); for (const logicalId of skipped) { try { const drift = await cfn.send( new DetectStackResourceDriftCommand({ StackName: stackName, LogicalResourceId: logicalId }), ); console.log(`Drift ${logicalId}: ${drift.StackResourceDrift?.StackResourceDriftStatus}`); } catch (err) { console.log(`Drift ${logicalId}: could not be checked (${(err as Error).name})`); } } Tests for the evaluation
// test/rollback-events.test.ts import { test } from "node:test"; import assert from "node:assert/strict"; import type { StackEvent } from "@aws-sdk/client-cloudformation"; import { rollbackFailures, skippedResources, SKIP_REASON } from "../scripts/rollback-events.js"; const stack = "OrdersStack"; const ev = (id: string, type: string, status: string, reason?: string): StackEvent => ({ LogicalResourceId: id, ResourceType: type, ResourceStatus: status, ResourceStatusReason: reason }) as StackEvent; // Newest events first, as returned by DescribeStackEvents const failedEvents: StackEvent[] = [ ev(stack, "AWS::CloudFormation::Stack", "UPDATE_ROLLBACK_FAILED", "The following resource(s) failed to update: [OrdersTable]."), ev("OrdersTable", "AWS::DynamoDB::Table", "UPDATE_FAILED", "Resource not found"), ev("OrdersFn", "AWS::Lambda::Function", "UPDATE_COMPLETE"), ev(stack, "AWS::CloudFormation::Stack", "UPDATE_ROLLBACK_IN_PROGRESS"), ev("OrdersFn", "AWS::Lambda::Function", "UPDATE_FAILED", "forward update failed"), ]; test("finds only failures from the rollback phase", () => { const result = rollbackFailures(failedEvents, stack); assert.deepEqual(result.map((r) => r.logicalId), ["OrdersTable"]); assert.equal(result[0].reason, "Resource not found"); }); test("detects skipped resources by status reason", () => { const after: StackEvent[] = [ ev(stack, "AWS::CloudFormation::Stack", "UPDATE_ROLLBACK_COMPLETE"), ev("OrdersTable", "AWS::DynamoDB::Table", "UPDATE_COMPLETE", SKIP_REASON), ev(stack, "AWS::CloudFormation::Stack", "UPDATE_ROLLBACK_IN_PROGRESS"), ...failedEvents, ]; assert.deepEqual(skippedResources(after, stack), ["OrdersTable"]); }); test("returns an empty list without rollback failures", () => { assert.deepEqual(rollbackFailures([ev(stack, "AWS::CloudFormation::Stack", "UPDATE_ROLLBACK_IN_PROGRESS")], stack), []); }); Run it
# Type check and tests npx tsc --noEmit npx tsx --test test/*.test.ts # Dry run: only shows status and failure causes npx tsx scripts/rescue-stack.ts OrdersStack # The real thing: Force Rollback, wait, check drift of skipped resources npx tsx scripts/rescue-stack.ts OrdersStack --force This is what the output might look like in a real case (example values):
Stack OrdersStack: UPDATE_ROLLBACK_FAILED Rollback failed for: - OrdersTable (AWS::DynamoDB::Table): Resource not found Force Rollback started, waiting for UPDATE_ROLLBACK_COMPLETE ... Skipped: OrdersTable Drift OrdersTable: DELETED The script deliberately looks only at the events of the root stack. For nested stacks, run it again with the name of the child stack to see the skipped resources there. I ran the type check and the three tests with Node.js 22.22.0, TypeScript 5.9.3 and SDK version 3.1156.0. I could not test it against a real stuck stack, simply because I did not have one at hand at that moment.
If you use the script in your team, give it its own narrowly scoped role. This policy covers the CloudFormation side; read permissions on the checked resources are needed in addition for the drift check:
{ "Version": "2012-10-17", "Statement":Source：https://dev.to/sascha-ms/updaterollbackfailed-rescue-stacks-with-force-rollback-4n84Related posts
Convert YAML to JSON in the Browser (and JSON to YAML) Without Uploading the File
Vercel agent-desktop: computer use for coding agents through the accessibility tree
WARN Act and Job Search — How to Use the 60-Day Notice Period to Find Your Next Job
Node.js SMS Verification Code: Generating and Sending One-Time Passwords via API Integration
Introducing Takeout Mender – A Free Tool to Recover Metadata from Google Takeout
