Upgrade a framework across a repo with a coding agent and the test suite as the gate
Run the official codemod first, then let a coding agent work through the failing tests one at a time with small commits. Turn deprecation warnings into errors so the agent gets a concrete list. Go one major version at a time. Forbid it to change settings defaults, migrations or tests without asking, because the reported failures come from quiet config changes, outdated API knowledge and broken behaviour that tests do not cover. You push and deploy, not the agent.
The repo runs on the new version on a branch of small commits, the full test suite passes with no tests removed, and you have a list of behaviour changes to check in staging.
- Difficulty
- Advanced
- Time
- Days for a large app, a few hours for a small one
- Sources
- 4 · 4 independent
- Updated
- 2026-10-13
- Cost:
- Included in your coding agent plan. Long sessions use a lot of tokens.
The prompt
Fill the blanks here, or leave them for your agent to ask. Pick the tab for your agent. The hand-off button gives any agent this page as markdown.
We are upgrading [framework] from [current version] to [target version] in this repo. You are the hands; I make the risky decisions. Before you edit anything: 1. Read the official upgrade guide and changelog for each version between [current] and [target]: [link to upgrade guide]. Do not rely on memory for APIs that changed after your training data. If you cannot open the docs, tell me. 2. List the breaking changes that affect this repo, with file references, and the third-party packages that need a new version. Mark any package compatibility you could not confirm from its own changelog. 3. Propose the order of steps. Go one major version at a time. Wait for my approval. Then, for each step: 4. Create a branch [upgrade/framework-N]. Run the official codemod: [command, for example `npx @next/codemod upgrade 16` or `git ls-files -z -- "*.py" | xargs -0 django-upgrade --target-version 5.2`]. Commit its output alone. 5. Run [test command] with deprecation warnings as errors: [for example `python -W error::DeprecationWarning -m pytest`]. 6. Loop: take the first failing test, make the smallest fix, run that test and the related tests, commit with a message that names the test. Then the next one. Rules: - Never delete, skip or loosen a test or an assertion to make it pass. If a test looks wrong, stop and ask. - Ask before you change a settings default, a database migration, a lockfile entry outside this upgrade, or CI config. - Do not refactor or modernise code that is not broken. Note it in a follow-up list instead. - Do not push, merge or deploy. At the end, give me: the commits, the full test result, the list of behaviour changes I must check in staging, and the follow-up list.
Human checkpoints: the agent stops and asks you
- Approve the upgrade plan and the order of version steps before any code changes.
- Approve every change to settings defaults, database migrations, dependency versions or existing tests.
- Review the branch and deploy it to staging yourself before production.
Passing tests do not prove the upgrade is safe. Settings defaults and untested paths can change behaviour silently. Deploy to staging first and keep rollback ready.
What you need
- A test suite with decent coveragedata
The tests are the gate. Thin coverage means silent breakage.
- The official upgrade guide and codemod for your frameworkdata
- A coding agent that can run your tests locallyapp
- A staging environmentdata
Some breakage only shows in real config, for example settings read from environment variables.
Steps
Get the current version green
Fix or quarantine flaky tests before you start. Record the test count and runtime.
Check: The suite passes twice in a row on the current version, and you know the test count.
Plan from the official docs
Ask the agent to read the upgrade guides and list breaking changes with file references and packages to bump. Spot-check the package list against each package's changelog.
Check: You have a step plan, one major version at a time, and you checked a sample of the package claims.
Run the codemod and commit it alone
Run the framework's codemod, for example `npx @next/codemod upgrade [version]` for Next.js or django-upgrade for Django. Use a dry run first where it exists (`--dry` for Next.js codemods).
Check: One commit holds only codemod output, and the build still starts.
Make deprecations fail loudly
Run the tests with deprecation warnings as errors so the agent works from a list instead of guesses.
Check: You see a concrete list of failing tests.
Fix one failing test per commit
Let the agent take the first failing test, make the smallest fix and commit. Small commits make review and rollback easy. Watch for edits to test files.
Check: Each commit is small, names the test it fixes, and touches no test assertions.
Stop at every settings, migration or dependency decision
When the new version changes a default, keep the old value explicitly unless you decide otherwise. Check settings that come from environment variables, not just the base file.
Check: Every changed default is a decision you made and wrote down.
Compare the test count and run the full suite
Confirm the number of tests is the same or higher than at the start, and that nothing is newly skipped.
Check: Same or more tests, all passing, no new skips.
Deploy to staging and check behaviour
You push the branch and deploy it to staging. Click through login, forms and pages that tests do not cover. Fix what breaks with the same one-fix-per-commit loop.
Check: Staging works on the flows in your behaviour checklist.
What goes wrong
The agent changes a settings default to match the new version, which silently changes behaviour.
Fix: Forbid default changes without approval. In one Django upgrade, the agent proposed turning on time zone support, which the author says would have reinterpreted stored times, so he pinned the old value.
Tests pass but staging breaks, because the fix only covered the base config.
Fix: Ask the agent to search every settings file and environment variable for the changed setting. Add a boot-time check. The same Django author hit this with a login setting.
The agent writes code for the old API or invents a migration path.
Fix: Give it the upgrade guide and changelog, or a docs MCP server, and run the type checker after each change.
Read the lesson →The agent skips a required upgrade command, such as a framework's config update step.
Fix: Put each command as an explicit step in the prompt or skill. One user of a Rails upgrade skill reports the agent skipped bin/rails app:update several times because it was only mentioned in report text.
The agent deletes or loosens tests to get to green.
Fix: Compare the test count before and after, review every test file diff, and forbid test edits without approval.
Read the lesson →Each change is fine on its own, but the code becomes a mess across many sessions.
Fix: Set the target structure up front and write the decisions into CLAUDE.md or AGENTS.md. One team reports a template that grew past 1,100 lines during a migration before they reset the design.
Package compatibility claims are wrong.
Fix: Check each package's own changelog. The Django author found two errors in a third of the agent's package table that he checked, caused by stale metadata.
Evidence
Two or more public accounts from different people or sites describe the same job. We summarise each source in our own words. Open the original before you rely on it.
- Blog2026-10-07· Claude CodeHow I upgraded a Django 3.2 app to Django 5.2 with Claude Code in 6 days
The author upgraded a 70,000-line app one LTS at a time with the django-upgrade codemod and a one-failing-test-per-commit loop. He reports 187 agent commits, one staging incident and no production rollback, and lists wrong settings and package advice he caught.
yureki_lab · dev.to - GitHub2026-09-30· Claude CodeConfusion when to actually run app:update (Rails upgrade skill)
Upgrading a production app from Rails 6.1 to 8, the user says the agent skipped or ignored running rails app:update several times, because the skill mentioned it only in report text.
emcoding · github.com - Blog2026-05-15· Claude CodeFrom WordPress to Next.js in 5 weeks with Claude Code
The team says each page migration was reasonable alone, but a shared template grew past 1,100 lines and long sessions forgot earlier decisions. A redesign, CLAUDE.md rules and CI checks fixed it.
Rory Miller (Behaim IT Solutions) · behaimits.com - Web2025-04-13· Custom agentMigrating code at scale with LLMs at Google
Google reports 39 migrations by three developers over twelve months. The authors say 74.45% of the 595 submitted code changes were generated by their LLM pipeline, and developers estimated total migration time fell by about half.
Celal Ziftci and others (Google) · arxiv.org
- Next.js codemods and upgrade commandchecked 2026-10-13
- Best practices for Claude Code (/batch, plan mode, verification)checked 2026-10-13
- Codex non-interactive modechecked 2026-10-13
- since-cutoff, find dependency APIs that changed after a model's training cutoffchecked 2026-10-13
Report how it went
Every report makes this page better for the next person and the next agent. Reports are reviewed before they count. Agents can report with the report_run tool.
Loading reports…