All Field Notes

Stop Asking the Agent to Refactor Everything at Once

Large AI-driven refactors usually fail because the context is too broad, the dependencies are hidden, and success is underspecified.

RefactoringTestingAgent workflows

"Refactor this app" is a tempting prompt because the pain feels broad. The dashboard is too large, the scripts directory is a junk drawer, database queries are repeated, and every new feature seems to make the system slightly harder to reason about. (How a codebase drifts into that state is the subject of Why Large Vibe-Coded Apps Become Incoherent.)

The prompt is still wrong. Broad pain does not make a broad refactor safe. What you usually get is an unreviewable diff that moves too much code, changes behavior by accident, and leaves the team arguing about whether the new structure is better instead of whether the product still works.

We worked with a team whose dashboard server had accumulated static page serving, CORS setup, access checks, project filtering, investor and pipeline queries, graph traversal APIs, usage and cost endpoints, admin routes, and calls out to operational scripts. Nearby, more than a hundred scripts handled backfills, fixes, imports, sync jobs, audits, reports, usage ingestion, watchdogs, backups, and deployment-adjacent work.

That is a real refactor target — but "refactor this" is not how it gets done.

Refactoring Is a Sequence

Martin Fowler's refactoring catalog is useful here because the unit of work is small: rename something, move a function, extract a class, introduce a parameter object, consolidate duplicate logic. The point is not that every codebase should mechanically follow a catalog. The point is that a large restructuring is made from many behavior-preserving moves.

Agents need the same discipline. They do better with a task like "extract the access-control middleware without changing any route response shape" than with "clean up this server." The first instruction has a boundary. The second invites the agent to discover an architecture while editing production behavior.

A good refactor slice has four properties:

  • It protects one behavior or boundary.
  • It names the files the agent may change.
  • It names the files the agent must not change.
  • It has a reviewable success condition.

If you cannot write those four lines, the work is not ready for an implementation agent yet. Ask for a plan first.

Why Big Agent Refactors Fail

Large refactor prompts fail for the same reasons large manual refactors fail — the agent just gets there faster.

The context is too wide. A route that looks like a simple investor query may rely on project scoping that was injected by middleware three thousand lines away. A graph endpoint may share assumptions with a dashboard card. A usage-cost API may depend on the same database pool and error handling as everything else. The agent can read every file in the repo, but it still cannot tell which behaviors are contractual unless the repo makes them executable through tests.

Review collapses. Even if the agent produces something that runs, the pull request is often unreviewable. GitHub can show changed files and threaded comments, but the reviewer needs a coherent story for the diff. A 4,000-line server split into ten files may look cleaner and still be impossible to review when auth, routing, SQL, and script invocation all changed in the same PR. At that point "did this preserve behavior?" is no longer a review question — it is a research project.

Tests are missing or vague. A refactor without characterization tests is a rewrite that happens to keep the old names. The Practical Test Pyramid is a good reminder that a healthy suite mixes fast lower-level tests with enough integration coverage to protect real flows. During a refactor, the immediate need is narrower: capture the behavior most likely to break before you move it.

There is no stopping condition. The agent extracts a helper, notices similar code nearby, edits that too, updates a route, reformats a script, and then "fixes" a test by changing its expectation. Each individual move may look reasonable. Together they destroy the review boundary you needed.

Slice by Behavior, Not by Cleanliness

None of this means you should never throw code away. Sometimes the product has changed enough that starting fresh is genuinely cheaper. But even a rewrite needs a scope boundary and a migration plan. The difference between a refactor and a rewrite is not the size of the change — it is whether you can prove what still works at each step.

For the dashboard we described above, we would not start by asking an agent to "modularize the server." We would sequence the work by behavioral boundary:

PR #BoundaryDefinition of done
1Inventory and characterizationRoute and script inventory grouped by behavior; characterization tests cover the riskiest paths
2Auth and access middlewareEvery protected path goes through the same access decision; tests prove allowed, denied, and unauthenticated cases
3Project filteringOne typed scope object answers "which projects may this request see?" everywhere
4Investor and pipeline queriesSQL moved behind named data-access functions; old and new behavior are directly comparable
5Graph APIsGraph endpoints live behind their own boundary and return the same shapes as before
6Usage and cost APIsOwn module, shared filter builder, tests around date ranges, aggregation, and empty results
7Consolidate scriptsA scripts/lib contract plus one migrated family, with compatibility wrappers for old entry points
8Deployment scriptsDeployment made explicit: versioned jobs, visible logs, required checks, rollback notes

PR 1: Inventory and Characterization

Create a route inventory and a script inventory. Group endpoints by behavior: access, project scope, investor and pipeline data, graph data, usage and cost, admin, and pages. Group scripts by job type: import, sync, cleanup, report, usage, audit, deployment, and one-off repair.

Then add characterization tests for the riskiest paths. They do not need to prove the whole system is perfect. They need to prove that a known request with a known role and project scope still gets the same status, response shape, and critical filters after code moves.

This PR should barely restructure anything. It makes the next PRs reviewable.

PR 2: Extract Auth and Access Middleware

Move request identity, internal-user detection, client-user detection, denial responses, and access logging into a small middleware module. Do not rewrite the investor routes. Do not touch graph queries. Do not rename script files.

The definition of done is simple: every protected API path still goes through the same access decision, and the tests prove allowed, denied, and unauthenticated cases. If the app has known unsafe direct-access behavior, document it as a gap instead of quietly changing it inside a refactor PR.

PR 3: Centralize Project Filtering

Once access is isolated, extract project scope resolution. The reference app had project filtering repeated across stats, projects, investors, pipeline, analytics, stale profiles, watchlists, radar-style data, and detail views. That is a perfect slice because it has one concept: "which projects may this request see?"

The new module should parse query parameters, resolve client/project aliases, and return one typed scope object. Routes can keep their existing SQL at first. The win is that review can focus on whether project scope is applied the same way everywhere.

The move generalizes: pick the concern that is repeated across the most routes. In another codebase that is tenant scoping, feature flags, or locale resolution.

PR 4: Move Investor and Pipeline Queries

Only after scope is centralized should the agent move investor and pipeline SQL behind data-access functions. This is where many broad refactors go wrong: moving SQL before access rules are explicit makes it too easy to leak data or drop filters.

Keep each function named after behavior, not tables: listPipelineRows, getInvestorDetail, listFollowUpsDue, getPriorRounds. A reviewer can then compare old route behavior with new function behavior.

The same move applies wherever high-risk domain queries are scattered — billing or inventory logic, say — once access and scope are settled.

PR 5: Split Graph APIs

Graph traversal is a different animal from dashboard summaries — it deals with paths, hop limits, node and edge shapes, search, and stats. Once the investor data paths are stable, move graph endpoints behind their own router or service boundary.

Resist the temptation to redesign the graph model in the same PR. The only claim this PR should make is "the graph API now lives behind a graph boundary and returns the same shapes it always did." Schema changes are a separate conversation.

The reusable idea is to isolate the subsystem whose semantics differ most from the rest: a search index, a recommendation engine, or a reporting pipeline elsewhere.

PR 6: Split Usage and Cost APIs

Usage, compute, and cost endpoints are operational reporting, not investor workflow. They deserve their own module, a shared filter builder, and tests around date ranges, aggregation fields, and empty results.

This is also a good time to separate product language from cost-accounting language in the codebase. Agents tend to blur those concepts because both live near "analytics" — and once the naming is muddled, every future prompt inherits the confusion.

More broadly, this is where you split operational or reporting concerns off from product workflow before they blur into shared files — admin analytics versus customer-facing metrics, for instance.

PR 7: Consolidate Scripts

Do not let the agent "clean up scripts" by deleting anything it does not understand. Start by grouping scripts into command families with shared config, dry-run behavior, and consistent logging. Backfills, sync jobs, audits, imports, usage ingestion, and deployment helpers should not all invent their own environment loading and failure semantics.

A useful first script PR creates a scripts/lib contract and migrates one family. It leaves compatibility wrappers for old entry points. Deletion waits until the new command path has run successfully in CI or staging.

PR 8: Replace Deployment Scripts Deliberately

Deployment scripts come last because they change operational risk. If a script generates wrappers, suppresses failures, runs scheduled jobs, or pushes changes from a server, treat that as release engineering work, not cleanup.

The safer migration is to make deployment explicit: versioned jobs, visible logs, required checks, rollback notes, and a rule for what automation may commit or push. GitHub's docs on required status checks are relevant because a refactor plan should end with gates, not just prettier folders.

How to Prompt the Agent

The prompt should sound more like a work order than a wish.

Use a slice card before you open the editor. It should fit in the agent's task message and be specific enough that a reviewer can reject unrelated changes without debating intent. Copy this template directly into an agent task and fill in the five fields:

Refactor slice: dashboard API access control
 
Allowed files:
- dashboard/server.js
- dashboard/access.ts
- dashboard/access.test.ts
 
Forbidden files:
- Investor SQL modules and fixtures
- Graph routes
- Usage/cost routes
- Scripts and scheduled jobs
- Page HTML or client-side dashboard components
 
Preserved behavior:
- Keep current response status codes and JSON shapes.
- Keep the same access decisions for internal users, scoped client users,
  unknown emails, and missing identity.
- Do not rename request headers, query parameters, route paths, or log fields.
 
Checks:
- Add or update characterization tests for internal access, scoped client
  access, unknown email, and missing identity.
- Run: pnpm run test dashboard/access.test.ts
- Run: pnpm run typecheck
 
Stopping condition:
- Stop after this slice passes the checks.
- Report any follow-up slices separately instead of editing adjacent behavior.

If that looks like bureaucracy, think of it as context compression — you are giving the agent exactly enough scope to succeed and no more. Current Codex prompting guidance makes the same point: complex work is easier to test and review when broken into smaller focused steps, and parallel threads should not modify the same files. Project instructions such as AGENTS.md exist for the same reason: they give agents a stable operating contract before the next patch.

If you are running multiple agents, give each worker an exclusive slice. One agent can inventory scripts. Another can add graph characterization tests. Another can draft the project-scope module. They should not all edit the dashboard server at the same time. Parallelism without ownership boundaries is just a faster way to create merge conflicts.

What Review Should Look For

Review each PR against the slice it claims to implement.

For an auth PR, the reviewer should ask: did access behavior stay the same, are known gaps documented, and did unrelated route behavior remain untouched?

For a project-filtering PR, the reviewer should ask: is there one scope object, are all old filters represented, and did the PR avoid rewriting unrelated SQL?

For a script PR, the reviewer should ask: is there a dry run, do failures remain visible, and can old automation still call the compatibility entry point?

This is where small PRs earn their reputation. It is not about worshipping line counts — it is about making sure a reviewer can map the entire diff to one claim. GitHub review tools handle file-level discussion well, but no tool can rescue a PR whose purpose is "general cleanup."

How We Score This in a Diagnostic

In a Post Code Labs diagnostic, a refactor target is not scored by how ugly it looks. It is scored by the evidence available to change it safely. That review is part of the full Post Code Labs consultation, which covers refactoring readiness across its twelve categories.

We look for:

  • A route and script inventory.
  • Existing tests around auth, authorization, data filtering, writes, and jobs.
  • Clear ownership boundaries for API modules, data access, and operations.
  • CI gates that run type checks, linting, tests, and builds before merge.
  • A risk register for behavior that must not change.
  • Agent instructions that define allowed files, verification commands, and stopping conditions — the agent-readiness layer the free repo audit scores automatically.
  • A rollback or compatibility plan for operational scripts.

The output is a 30-60-90 plan, not a dramatic rewrite mandate. In the first 30 days, make risky behavior observable and tested. In 60 days, extract the highest leverage boundaries. In 90 days, remove compatibility wrappers, retire obsolete scripts, and tighten CI so the old shape does not grow back.

What to Do Next Monday

Do not start Monday by asking an agent to refactor the app.

Start with a two-hour inventory. List every route in the large server and every script in the scripts directory. Put each item under one behavior heading. If an item fits under three headings, that is a refactor candidate.

Pick one slice. Usually it should be auth/access, project filtering, or a high-value query family. Write down the files that are allowed to change and the files that are off limits.

Add two or three characterization tests around the slice. If the app has no test harness, add the smallest one that can exercise the behavior. Do not spend the day designing the perfect suite before protecting the first risky path.

Draft the PR sequence before editing. Name the first four PRs, their success conditions, and the behavior each one must preserve. Share that sequence with the reviewer so "why is this not all cleaned up yet?" does not become review noise.

Finally, add one rule to the project's agent instructions:

Do not accept broad refactor prompts. Propose a behavior-sliced PR sequence
first, then implement only the approved slice with tests and an explicit
stopping condition.

That single rule changes how the agent works for you. Instead of a codebase-wide cleanup machine, you get a refactoring partner that moves deliberately, proves what it preserved, and stops before the diff becomes a liability. It belongs in the same committed rulebook we describe in Your App Needs a Constitution.

References