The practice of refactoring has long been accepted as an effective tool to maintain code quality. The argument has always been that higher code quality allows an application to be edited and maintained more effectively for longer. Refactoring is an investment that extends the life of the asset.
However, historically there have been no direct, objective measures of this promise. The best measure available has been developer productivity over the long term. There are a number of problems with this as a measure.
-
It’s indirect. Developer productivity is affected by many things: distractions, the developers involved changing, the surrounding environment changing. Individual contributing factors are hard to identify.
-
It’s subjective. It requires surveys to measure, is very dependent on the specific developers and is hard to compare across projects. It also requires a large number of samples, therefore compounding indirect effects and differences across developer environment.
-
It lags. Productivity takes a long time to measure and can only be measured in aggregate. This makes it hard to use it to decide how to work or what to work on.
-
It’s expensive. Measuring productivity takes time and is therefore a significant investment. This means that companies can rarely measure their own productivity, and instead must rely on attempting to infer from wider studies.
These challenges have left refactoring in a difficult place. Refactoring is often left until some unspecified later, especially when deadlines get tight. Developers frequently don’t agree which refactorings to apply. Even whether code should be refactored at all is dismissed as a matter of ‘taste.’
With agentic engineering this has changed. There are now direct, objective measures of the value of refactoring. These measures are cheap and easy to obtain. Any engineer can measure this on their own code base, right now. The ‘productivity’ of agentic engineering tools can be measured directly in the consumption and production of tokens. As tokens are the charging metric for these tools, improvements in token use have a direct economic benefit. As these tools report token use, we can start to make changes and measure the impact of these changes by observing how token use changes.
We have reached a world where, perhaps, software engineering can finally earn that label.
An experiment to measure the benefit of refactoring
Agentic engineering is a massive change. I wanted to learn about what it is actually like. I discovered that I could build a sophisticated, useful, production grade application. Without writing or even reading any of the code. But, I could see the agents making choices that were at best flawed and, at worst, actual mistakes. I decided there was a great opportunity for an experiment.
This was an experiment to show that refactoring can reduce the token consumption of an agent when making a change. This has now been written up on Martin Fowler’s bliki. And I’m very excited to present this now.
The Economic Benefit of Refactoring
My hope, and belief, is that there is now a world of experiments that can be run. I’m looking forward to discovering and running more.