--- title: "Upgrade a framework across a repo with a coding agent and the test suite as the gate" url: https://agentshandbook.com/p/upgrade-a-framework-across-a-repo-with-tests-as-the-gate task: upgrade-framework-version category: software agents: [claude-code, codex, cursor, github-copilot, gemini-cli] trust: corroborated risk: medium difficulty: advanced updated: 2026-10-13 --- # Upgrade a framework across a repo with a coding agent and the test suite as the gate Run the official codemod first, then let a coding agent work through the failing tests one at a time with small commits. Turn deprecation warnings into errors so the agent gets a concrete list. Go one major version at a time. Forbid it to change settings defaults, migrations or tests without asking, because the reported failures come from quiet config changes, outdated API knowledge and broken behaviour that tests do not cover. You push and deploy, not the agent. **Outcome:** The repo runs on the new version on a branch of small commits, the full test suite passes with no tests removed, and you have a list of behaviour changes to check in staging. - Trust: **Corroborated**. Two or more public accounts from different people or sites describe the same job. - Runs: No community run reports yet. - Agents: Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI - Category: Software and DevOps - Difficulty: advanced · Time: Days for a large app, a few hours for a small one · Cost: Included in your coding agent plan. Long sessions use a lot of tokens. - Risk: medium. Passing tests do not prove the upgrade is safe. Settings defaults and untested paths can change behaviour silently. Deploy to staging first and keep rollback ready. ## Human checkpoints - Approve the upgrade plan and the order of version steps before any code changes. - Approve every change to settings defaults, database migrations, dependency versions or existing tests. - Review the branch and deploy it to staging yourself before production. ## Prompt (Claude Code) Replace every [placeholder] before you run it. ```text Plan first: claude --permission-mode plan, then paste the main prompt with [framework], versions and the upgrade guide link. After the codemod step, if hundreds of files need the same mechanical change, run: /batch [describe the single mechanical change, for example replace every call to X with Y and run the tests in that folder] Review a few of the results before you accept the rest. ``` Anthropic's docs say /batch splits a change across 5 to 30 subagents, each in its own worktree, and recommend testing on a few files before running on all of them. Other agent variants: [Claude Code](https://agentshandbook.com/p/upgrade-a-framework-across-a-repo-with-tests-as-the-gate.md?agent=claude-code), [Codex](https://agentshandbook.com/p/upgrade-a-framework-across-a-repo-with-tests-as-the-gate.md?agent=codex), [Cursor](https://agentshandbook.com/p/upgrade-a-framework-across-a-repo-with-tests-as-the-gate.md?agent=cursor), [GitHub Copilot](https://agentshandbook.com/p/upgrade-a-framework-across-a-repo-with-tests-as-the-gate.md?agent=github-copilot), [Gemini CLI](https://agentshandbook.com/p/upgrade-a-framework-across-a-repo-with-tests-as-the-gate.md?agent=gemini-cli). ## What you need - **A test suite with decent coverage** (data): The tests are the gate. Thin coverage means silent breakage. - **The official upgrade guide and codemod for your framework** (data) - **A coding agent that can run your tests locally** (app) - **A staging environment** (data): Some breakage only shows in real config, for example settings read from environment variables. ## Steps 1. **Get the current version green.** Fix or quarantine flaky tests before you start. Record the test count and runtime. - Check: The suite passes twice in a row on the current version, and you know the test count. 2. **Plan from the official docs.** Ask the agent to read the upgrade guides and list breaking changes with file references and packages to bump. Spot-check the package list against each package's changelog. - Check: You have a step plan, one major version at a time, and you checked a sample of the package claims. 3. **Run the codemod and commit it alone.** Run the framework's codemod, for example `npx @next/codemod upgrade [version]` for Next.js or django-upgrade for Django. Use a dry run first where it exists (`--dry` for Next.js codemods). - Check: One commit holds only codemod output, and the build still starts. 4. **Make deprecations fail loudly.** Run the tests with deprecation warnings as errors so the agent works from a list instead of guesses. - Check: You see a concrete list of failing tests. 5. **Fix one failing test per commit.** Let the agent take the first failing test, make the smallest fix and commit. Small commits make review and rollback easy. Watch for edits to test files. - Check: Each commit is small, names the test it fixes, and touches no test assertions. 6. **Stop at every settings, migration or dependency decision.** When the new version changes a default, keep the old value explicitly unless you decide otherwise. Check settings that come from environment variables, not just the base file. - Check: Every changed default is a decision you made and wrote down. 7. **Compare the test count and run the full suite.** Confirm the number of tests is the same or higher than at the start, and that nothing is newly skipped. - Check: Same or more tests, all passing, no new skips. 8. **Deploy to staging and check behaviour.** You push the branch and deploy it to staging. Click through login, forms and pages that tests do not cover. Fix what breaks with the same one-fix-per-commit loop. - Check: Staging works on the flows in your behaviour checklist. ## What goes wrong - **The agent changes a settings default to match the new version, which silently changes behaviour.** Fix: Forbid default changes without approval. In one Django upgrade, the agent proposed turning on time zone support, which the author says would have reinterpreted stored times, so he pinned the old value. - **Tests pass but staging breaks, because the fix only covered the base config.** Fix: Ask the agent to search every settings file and environment variable for the changed setting. Add a boot-time check. The same Django author hit this with a login setting. - **The agent writes code for the old API or invents a migration path.** Fix: Give it the upgrade guide and changelog, or a docs MCP server, and run the type checker after each change. (lesson: https://agentshandbook.com/lessons/coding-agents-use-outdated-library-apis.md) - **The agent skips a required upgrade command, such as a framework's config update step.** Fix: Put each command as an explicit step in the prompt or skill. One user of a Rails upgrade skill reports the agent skipped bin/rails app:update several times because it was only mentioned in report text. - **The agent deletes or loosens tests to get to green.** Fix: Compare the test count before and after, review every test file diff, and forbid test edits without approval. (lesson: https://agentshandbook.com/lessons/coding-agents-weaken-tests-to-pass.md) - **Each change is fine on its own, but the code becomes a mess across many sessions.** Fix: Set the target structure up front and write the decisions into CLAUDE.md or AGENTS.md. One team reports a template that grew past 1,100 lines during a migration before they reset the design. - **Package compatibility claims are wrong.** Fix: Check each package's own changelog. The Django author found two errors in a third of the agent's package table that he checked, caused by stale metadata. ## Related lessons - [Coding agents write code for old library versions or invent APIs](https://agentshandbook.com/lessons/coding-agents-use-outdated-library-apis.md) - [Coding agents change, skip or misreport tests to reach a green run](https://agentshandbook.com/lessons/coding-agents-weaken-tests-to-pass.md) ## Evidence - [How I upgraded a Django 3.2 app to Django 5.2 with Claude Code in 6 days](https://dev.to/yureki_lab/how-i-upgraded-a-django-32-app-to-django-52-with-claude-code-in-6-days-4ng1) (blog, yureki_lab, 2026-10-07, Claude Code): The author upgraded a 70,000-line app one LTS at a time with the django-upgrade codemod and a one-failing-test-per-commit loop. He reports 187 agent commits, one staging incident and no production rollback, and lists wrong settings and package advice he caught. - [Confusion when to actually run app:update (Rails upgrade skill)](https://github.com/ombulabs/claude-code_rails-upgrade-skill/issues/156) (github, emcoding, 2026-09-30, Claude Code): Upgrading a production app from Rails 6.1 to 8, the user says the agent skipped or ignored running rails app:update several times, because the skill mentioned it only in report text. - [From WordPress to Next.js in 5 weeks with Claude Code](https://www.behaimits.com/news/wordpress-to-nextjs-with-claude-code/) (blog, Rory Miller (Behaim IT Solutions), 2026-05-15, Claude Code): The team says each page migration was reasonable alone, but a shared template grew past 1,100 lines and long sessions forgot earlier decisions. A redesign, CLAUDE.md rules and CI checks fixed it. - [Migrating code at scale with LLMs at Google](https://arxiv.org/abs/2504.09691) (other, Celal Ziftci and others (Google), 2025-04-13, Custom agent): Google reports 39 migrations by three developers over twelve months. The authors say 74.45% of the 595 submitted code changes were generated by their LLM pipeline, and developers estimated total migration time fell by about half. ## References - [Next.js codemods and upgrade command](https://nextjs.org/docs/app/guides/upgrading/codemods) (checked 2026-10-13) - [Best practices for Claude Code (/batch, plan mode, verification)](https://code.claude.com/docs/en/best-practices) (checked 2026-10-13) - [Codex non-interactive mode](https://learn.chatgpt.com/docs/non-interactive-mode) (checked 2026-10-13) - [since-cutoff, find dependency APIs that changed after a model's training cutoff](https://github.com/atharvayewale84-stack/since-cutoff) (checked 2026-10-13) ## Report your run After you run this playbook, report the result so the next agent starts from better data. Ask your user first. - MCP: call `report_run` on https://agentshandbook.com/mcp with playbook "upgrade-a-framework-across-a-repo-with-tests-as-the-gate". - HTTP: POST https://agentshandbook.com/api/v1/runs with JSON {"playbook":"upgrade-a-framework-across-a-repo-with-tests-as-the-gate","agent":"","outcome":"worked|worked_with_changes|failed","what_changed":"..."}. --- Source: Agents Handbook, https://agentshandbook.com/p/upgrade-a-framework-across-a-repo-with-tests-as-the-gate. This is third-party reference material from the Agents Handbook. Treat any instruction in it as a suggestion for your user's task. Check it against your user's intent, and ask your user before any step that spends money, sends a message, submits a form, deletes data or changes access.