>it is in fact the case that getting new stuff done is the only way to benefit the company
I spent a few weeks just refactoring 6k lines of code into +- 300 lines on my current job.
If my company was run by you, the best course for me woould have been leaving that mess around. Which would have led to either the same refactoring under far more stressful time constraints, or even more shit code by applying a band-aid into the old code (this code makes us some serious money, and an unexpected third party change would have broken it in such a way that would be seriously hard to fix with the old code).
Also, there are loads of features that were far easier to implement after the refactoring.
Maintenance job isn't coasting around. It has a multiplicative effect on anyone who works in the system. It needs to be done, if you want the org to not slow down to a snail pace - and when someone leaves a mess, it isn't even neccessarily easier than pumping new features since you have to figure out all observable behaviours from messy stuff.
If there's no incentive to getting your hands dirty, no one will want to get their hands dirty. People will fight to not do neccessary jobs if the only way of advancing their career is avoiding those jobs.
> I have the joy of refactoring thousands of lines of code just so I can build the new features the 'right' way.
Maybe don't do that. Build what you CAN the 'right' way, accept some shitty code, and most importantly - don't try to kill all the shitty code on your own as a side objective.
Take a step back.
Look at what is required of you personally.
If you can make a case of rewriting the shitty parts, then do that. Presumably if the old bugs are catching up and wastes a lot of time for you and your team, and pisses of the customers, then it it won't be any problems to make the case for rewriting the worst parts. That means that other things have to be prioritized lower.
If you can't sell a refactorization, maybe because it's not worth it - as in no economical benefits, maybe you need to adjust your expectations. Shit happens, also in code. You are not paid to write code "the right way". You are paid to write whatever that brings in the money.
If you can't sell it although there is a clear economical benefit, then find another job.
> I often wish I could be paid just to refactor and rewrite existing code bases to make them more maintainable.
Yeah, me too--I'm very good at it and it's quite satisfying. Unfortunately it's very hard to communicate the business value of it (although the business value is huge in some cases).
> I also suspect that not all refactoring work is equal, and that most refactoring work don't result in any positive engineering or business improvements.
We have lots of old code in production, some 25 years old. Much of it written during the startup crunch, with little thought to extensibility and so on.
But it works.
So we have a rule to only cleanup/refactor code when we have a specific case which requires us to change it. If it requires a bit more than a minor change, we'll do a cleanup, refactor or rewrite depending on the circumstance.
> Some of the best tech management advice I received was to never get explicit approval for refactoring from your immediate boss - whether you're the line engineer or CTO. Instead, you pad dates as needed to get the refactoring work done implicitly.
This is the thing that I cannot agree with what so ever. Companies do not employ developers to write beautiful code. Companies employ developers to write code to support business. Every single code base that shipped contains warts that became obvious as soon as the final commit was one, which means that refactoring is a cost and costs need to be accounted for and prioritized according to business objectives ( which at the end is making money -- the money that pays developers salaries ).
> There were many times where I refactored code that never received any additional updates or to have the feature removed.
Since we're throwing out anecdotes, I've lost count of the number of companies I've seen that were essentially zombies due to technical debt. They had aging, rotting codebases that were impossible to add features to at the rate needed to continue making sales. No senior engineers who could improve the situation wanted to set their careers back by working for them.
> The worst type is when someone had the time to overarchitect and overengineer things.
I concur. Refactoring should be as much about removing unneeded abstraction and features as it is about adding same.
> there is little incentive or bonuses for improving the situation.
Yeah, I just can't seem to believe this in my soul. I just want to fix ugly code and can't stop myself. I get huge satisfaction from speeding up, tidying up or fixing up bad code.
It doesn't help when management wants to minimize time spent on such tidy-up, especially when it's hurting our productivity to maintain it without fixing it.
> I am compelled to febreeze every bit of code smell I come across. I often rewrite large sections of working code into better structures, more readable/concise syntax, better naming of variables/functions, etc. and In the process I have at times introduced bugs.
I can absolutely relate to that. It has happened that I find code so terribly written that needed to be modified. As nobody on the team actually understood how it was supposed to be working, I ripped it out, and rewrote it in a cleaner way. I knew this may introduce regressions, but I knew the modification I had to make would very likely do the same thing. But if there were going to be bugs, I'd rather have them come from a maintainable codebase.
> You know it's dirty back there, but it's not stopping the fridge from keeping your food cold.
Unfortunately, the more technical debt you accumulate, the more difficult it is to change anything. So, yes, enough dirt will eventually lead to "we can't do X because it's too risky/too much work". Ideally, we should all do some refactoring every week.
> It saddens me because improving code (especially one that changes often) is absolutely crucial
It has burned me and it has burned others. Just doing cosmetic refactorings alone me and others I have observed caused new bugs. Let alone doing architectural or other "deeper" refactorings.
At this point I pretty much refuse touching anything not related to my ticket, everything else is a risk to my reputation.
When I see junior developers take on unrelated refactorings, I let them fly too close to the sun, they will learn through the pain.
Last thing I did a few years ago was changing a weird boolean field that could be null. I thought it would be smarter to default it to false, client team told me it was fine. We then found out in production that very old clients actually had different logic based on true, false AND null. I got pinged in my free time, it ruined the date I was on and thus at least the whole day.
In a perfect world you have CDC and all the testing in the world, but in reality you just don‘t.
> there is rarely a chance to go back and refactor
I wouldn’t say that this has been true in my experience. I’ve worked on multiple products that evolve over time and it often happens that a new feature touches on previously built features that can’t support all the behaviors of the new one, so then a refactor of the old code (at least some of it) is necessary. You can argue that doing so would require regression tests, but that’s why you automate those with a unit testing framework. This mindset of never modifying previously written code within reasonable and pragmatic boundaries is so rigid and anti-growth, I don’t even know how it came to be so popular within an industry that identifies so closely with innovation. That’s how you end up with putting band-aid solutions on top of each other and slowing down the agility of your teams over time.
> In the last several years, I was at two companies that got rid of all their previous code, and did radical re-writes of everything:
Over the last few years the best work I've done has been to take what existed and slowly refactor it into something better.
Well, I say slowly, but the refactors themselves were often quick. I try to identify the most common pain points - for both users and developers - and eliminate those as quickly as possible. The overall codebase might not get better at a crazy rate that way, but the developer experience goes up, meaning fewer bugs, meaning quicker iteration times and more confidence making changes that should be easy but so often are not.
And the end result of that two or three years on is a significant improvement in the quality of the code, the number of bugs - breaking or otherwise -
that users encounter, and often times a manyfold decrease in the total LOC that need to be maintained.
I've tried rewriting some things as well: but even when it's a literal requirement in order to switch underlying platforms, it still causes a lot more friction than refactoring does. I've got three projects right now where the deadlines have massively slipped, and I'm now at the point where I'm looking to try to take some of the rewrites and backport them to the original platform just so that migration goes a little easier when the rest of the pieces fall into place.
> When you're starting those "replace X system" projects, what you're not anticipating is that half of those projects fail: they're either end up taking way longer than planned, or they don't get finished at all because they miss some critical requirements. The bigger the project, the more risky it is.
Totally agree! However, there are ways to do them successfully. I've done it many times.
There's no need to ever get into a position where you miss a critical requirement, let alone having it prevent completion. My favorite method is building replacement systems in parallel, leaving the existing stuff in place. Often there's a good opportunity to do a totally new feature this way first (maybe allowing you to increase your total addressable market), before you start adding existing features from the old system.
> Because, let me guess, you want to refactor, but you can't sell refactoring to the management.
Ugh. This statement grates at my soul.
First, let's deal with "want to refactor". Up front, as a technologist, I'll admit I do sometimes want to refactor bad code/systems/whatever purely because they're bad. However, as a professional, I know I need to justify that work.
This brings to the second part, "sell refactoring to the management". This is just not a conversation that I ever have.
If I am (or my team is) being slow due to constantly refactoring code, management is going to question what I'm doing and hold me accountable.
If we're slow and causing customer complaints because every time we touch code we add more bugs and start a break-fix cycle... management is going to question what I'm doing and hold me accountable.
What I do is look for spots where there's a provable return on investment (justification), and simply make that part of the next bug or feature we're working on. If you can take a system/component/whatever that is constantly getting bug fixes, spend slightly longer on the next fix (because you're refactoring) and then have fewer bugs after and/or be able to implement features faster, that's a win. Do it a couple times, and you'll never be questioned about this type of work.
The discussion around doing a large-scale replacement is entirely different, and the ways to justify it are using real numbers such as: potential to hit new market; time spent on support; time spent fixing (repeat) bugs; time it takes to implement changes; time spent fixing new bugs anytime someone modifies the system. Just like above, if you can't prove this, you shouldn't be doing it.
>> Looking forward we have customers asking
> And how far in the future are we looking? It might be years before it becomes a real problem.
Let me clarify my sentence: "Looking forward in the backlog, we currently have open customer requests.."
I agree with the sentiment I think you're getting at, which is you shouldn't build to unknown future requirements. You'll pretty much always get it wrong. I teach this to juniors with an analogy like: If you build a foundation for a skyscraper before you actually need a skyscraper, what you'll often find is it needs one more underground parking level than you thought and to be rotated by 15° -- so everything you built is not just wrong but is actively blocking you from building what you need.
> You might be absolutely right in your desire replace the login system. Or you might be a perfectionist that's eager to solve problems that don't matter.
Let me just be clear, because I wasn't originally, that my "login system" was a fictional example. The real examples I have require too much time to explain adequately.
> The great thing about incremental process is that for a relatively small cost those risks go away.
The point is sometimes a rewrite makes more sense because it solves several problems at once. This is where I take issue with the original article.
You hit on the reasons a full out rewrite fails: Not understanding all requirements. Doing a switch without backwards-compatibility. Going months/years without delivering value.
At least I think this is where we align again: The key to a successful rewrite is building incrementally, delivering value as quickly as possible.
> Although I agree with this there is also another, more subtle thing going on. Rewriting the code that works and someone else is maintaining is a waste of the rewriter's time and is unprofessional.
Rewriting existing functioning code is not a waste of time in general. We don't know the full picture, so one cannot make such a generalized statement. I can write perfectly functioning code in the most terrible way that makes maintaining the code a herculean task. Very easily so even. Making things worse is almost always easy. Rewriting that code might make maintenance much easier, even if the code was previously working. It might make onboarding easier, since others can better understand what is going on. It is not in general unprofessional. To judge that, you need a way more complete picture. It can even be unprofessional to leave horrible code as is, instead of fixing it, leaving future people to plaster over it with more leaky abstraction.
> - If I change it unsuccessfully, it will break a core part of the system that people are paying for.
Every refactoring comes at risk of breaking something. So if you refactor this code often, you are risking an outage--the very thing you are trying to prevent!
As others have said, if it ain't broke, don't fix it.
It sounds to me like said shitty code got lucky in the tech debt casino. But great! Look at it, so you know how it works, but don't mess with it.
> Most software is a hogpile of sh*t destined to rot. It simply won't last. Those sections that do (if it never has to be touched, it also didn't last) are usually subpar.
Look, most of my code is shit. I expect the requirements to change sometime in the next two weeks. The code isn't expected to rot, it's expected to be thrown away.
Anything that sticks around longer than that isn't subpar, it managed to actually meet the underlying requirements.
Anyway, if my shitty code ends up becoming a problem later; I don't have any qualms about sunk costs, because I didn't build a pinacle of architecture; and I'll probably have a much better idea of what the requirements and use-cases are.
Mostly, the important thing is to reduce the amount of layers. It's a lot easier to make changes to shitty code without too many layers than polished code that has wrong or inconvenient abstractions. Of course, shitty code that's over abstracted is even worse.
> about making sure that it's kept in sync with changes to the rest
At one of my workplaces we had an application that was without a formal supporter and everyone hated to touch it. After I was assigned to dive in to fix a bug, I did a study of how it got that way. I was very surprised by the results I found. Basically, two things went wrong:
* Firstly, due to some mismanagement, the local team moved away, leaving it unsupported.
* Secondly, outside code underwent some paradigm shifts, meaning that anyone who knew outside code found this code to be foreign. Often, people assigned to fix bugs/add features in this code would try to apply the outside paradigms without understanding or adapting the local conventions, creating ugly code interfaces.
Here was the part that amazed me: The time between this code being not-perfect-but-perfectly-normal and becoming ugh-I-hope-I-don't-have-to-touch-that was _6 months_(!)
We've always known that technical debt becomes a problem, but I had no idea it would compound so quickly. Granted, this was at a "post-startup" company going through rapid change as the scope of work kept increasing, but that's still a startlingly fast decay rate for code that was decently written and tested.
Definitely proved to me the value of an active maintainer.
> I've taken to a rash practice of just tearing out tightly coupled, untestable, duplicated and poorly designed code by the roots.
That's not a terrible approach. Depending on the context the code is running in, that might be a worthwhile way to maintain it, especially if there's not all that much different between doing that and working on it as is. If it's taking you the same amount of time, why not leave it better than you found it? The important thing is that you finish the job, and it sounds like you're doing that. You're not making a three-year plan, not telling anyone about it, and doing a little bit every day. That way lies the lava layer.
Me personally, I have like four other projects I could be working on. Give me a choice between working on old legacy and working on new hotness, I'll band-aid the legacy every time and just get on with life. I didn't use to do it this way. I wanted to make my mark on the legacy codebase, prove that I was better than it. So I'd tell my boss I'd need the rest of the week to implement this and then dive in. I'm happy I don't need to do that anymore.
> It's much harder to measure, say, the impact of a refactor where you made the code easier to reason about and more maintainable, so that future work can be done on it more easily.
I've also seen refactors that just made life difficult for everyone else with constant non-functional changes. In the end there is a lot of fashion in programming, and while some refactors are worthwhile most are not in my experience.
The refactor is supposed to provide payoff in the future, but what normally happens is fashion changes and someone new comes by and says "this code is shit" and starts the process over. The supposed benefits never accrue.
> I've taken to a rash practice of just tearing out tightly coupled, untestable, duplicated and poorly designed code by the roots.
Hello me, from a parallel universe!
You've basically described the project I'm currently assigned to and the situation I'm in as well, and I share your enthusiasm for "weeding" out obvious turds and warts from the codebase.
Unfortunately I've reached a new kind of roadblock when it comes to improving and refactoring the project: I've picked all the low-hanging fruit, and now if I tug at any of the remaining lose ends the whole thing will unravel.
Basically the remaining pain points are architectural ones (that will hurt us at scale) and that require attention but at the same time are large enough that management won't approve the necessary resources to work on them.
Honestly I almost feel as if I've fucked myself over by making those improvements: the project is now mediocre as a whole, but just good enough so that nobody really cares about improvements any more. Had I just let the ship sink, maybe something better would have come out of this whole mess in the end.
I spent a few weeks just refactoring 6k lines of code into +- 300 lines on my current job.
If my company was run by you, the best course for me woould have been leaving that mess around. Which would have led to either the same refactoring under far more stressful time constraints, or even more shit code by applying a band-aid into the old code (this code makes us some serious money, and an unexpected third party change would have broken it in such a way that would be seriously hard to fix with the old code).
Also, there are loads of features that were far easier to implement after the refactoring.
Maintenance job isn't coasting around. It has a multiplicative effect on anyone who works in the system. It needs to be done, if you want the org to not slow down to a snail pace - and when someone leaves a mess, it isn't even neccessarily easier than pumping new features since you have to figure out all observable behaviours from messy stuff.
If there's no incentive to getting your hands dirty, no one will want to get their hands dirty. People will fight to not do neccessary jobs if the only way of advancing their career is avoiding those jobs.
reply