Anyone who has done client work or worked inside an organization where the priority is on "new" has faced this dilemma: how do you "sell" housekeeping/maintenance and refactoring tasks up the chain of command to your boss or your client? Often times developer claims about wanting or needing to do refactoring work, maintenance tasks, "small tweaks", etc - are dismissed because they don't demonstrate immediate progress to the application, or don't deliver on any new business requirements.
What is refactoring?
By continuously improving the design of code, we make it easier and easier to work with. This is in sharp contrast to what typically happens: little refactoring and a great deal of attention paid to expediently adding new features. If you get into the hygienic habit of refactoring continuously, you'll find that it is easier to extend and maintain code.
-Joshua Kerievsky, Refactoring to Patterns
That definition is fine for my purposes here, let's use that.
I think most of the objection to "allowing" developers to do refactoring tasks come from a misunderstanding, but it's important as a developer to make sure you're on the right side of the fence on this matter. There are certain tasks which really are a huge waste of time, there are certain tasks which are almost critical to stop what you're doing and deal with, but the majority are probably somewhere in the middle.
It's about the context
I think things are somewhat context specific. If your manager has let you know about an important feature that needs to get out, and you insist on renaming some variables instead, that's probably not legitimate refactoring - you should get the feature out. If the only tasks in the feature queue are things like copy changes and style tweaks, you've probably got time to proactively go in and move some code around. Those are the edge cases though, ignoring the majority is how you get into trouble.
The majority? Yes. It takes day to day refactoring to stay on top of problems. If you ignore developing code smells, long methods, crazy hacks that were done to solve a problem quickly, they'll start to accumulate. Pretty soon you cross over the line from "I wish I had a few hours to clean up these small things" to "I really can't get any work done at all because this is such a huge mess". The key is to explain to your managers that there's nothing wrong about that. Even the best programmers will accumulate code cruft if things aren't revisited from time to time.
Requirements change over time. Best practices are moving targets. Web frameworks and languages add new features. New libraries are released that solve old problems in new elegant ways. And so on.
Get a boss that understands that. Work for clients that trust you on those facts. Give them my phone number if they don't believe you.
What to do
Here are some things we do to encourage code quality and commit to ongoing improvement:
Explain the agile process to clients. Specifically, explain how things are in an ordered queue, and how they have a priority. Once that's done, you can work as a peer to make sure that refactoring and maintenance tasks get into the queue and prioritized alongside new feature work.
Do periodic code reviews. If you can do this weekly, great. At worst, do it monthly. On some projects, we've made it a requirement to use a "feature branch" in the git repository, and required that another developer review literally every commit before it makes it's way into master. This can be overkill on a small project, but it's worked great once you get more than a few developers.
Have written code standards that evolve over time. It's important to write them down, because it creates an objective reference to point to when you see someone else do something weird, or aren't sure what to do yourself. It's important that they can evolve, because when there's legitimate disagreement about how to best do something on the team you need to take a moment to revise the standards to account for whatever differences you have. When that fails, call in a third party to help sort it out.
Explain the TDD process. Make sure clients and bosses understand why you write tests. When you explain Red Green Refactor …. don't skip the refactor step!
But how do I show the client what I did?
It's all about trust and expectations. If you haven't explained how this works to clients who don't know any better, they have every right to demand that every hour of time you put into their project result in something new they can see. If you do explain how maintenance and refactoring and constant quality improvement works, then you've set the expectation correctly and they shouldn't be surprised when tasks that "just change code you already wrote" are showing up in the queue, the changelog and the invoices.
Do any of you have stories of success (or failure) in explaining this issue to your clients, managers or coworkers?