> That wasn't a conscious decision made in the past to do A instead of B knowing that B was the right choice, but not realistic at the moment.
I'm thinking of cases where it was a conscious choice. $deity knows that I've made plenty of such decisions in my career - chosing a hacky structure A to meet a next month's release, while articulating to management that after that release, I'll need to redo it to structure B. Sometimes a bunch of high-priority requests took precedence, which made the system stick to A for far longer than it should. I understand that to qualify as both "technical debt" and "architectural issue".
>they definitely need at least a year before they understand the problem space well enough for their offers of major architecture changes to be welcome.
IME, hubris usually correlates with architectural problems and higher than normal technical debt.
I can't see any good reason to analyze ideas about architectural overhauls on anything other than their technical merits. If there's a reason relating to the problem space for rejecting one particular idea that's fine, but unless your architecture is completely fucked that should be very rare.
It's perfectly normal to tear out the badly written guts of a software system and build up a better version, and a deep understanding of requirements should not really be required to do that in most cases.
> if slow compilation is such a pain point, I wonder why it takes seven years and counting to fix this bad decision
I'm not sure it was a "bad decision" if it was something that can be addressed after the fact and allowed them to ship working software more quickly.
The vast majority of the popular projects I am aware of have this kind of technical debt, likely due to survivorship bias. Project teams that refuse to take on technical debt are rarely successful enough to become popular.
> supposedly learned a lesson or two about managing large system and are more proactive about making easier to maintain system.
One thing you learn pretty quickly is that there is a massive difference between stated intentions and reality. It only takes one dissenting voice to get a big enterprise stuck in the past.
Usually far more are available.
(and I might even agree that from a business perspective it's usually the right call at some point. Software engineers will rewrite, rewrite, rewrite again and again and again)
> it’s simpler to rewrite part of the system than to even maintain it for a month.
This is often a career limiting decision. Rewrites are often much more difficult, particularly when you are dealing with a system that is not fully understood. This often leads to what should be a quick re-write taking > 3x more time and money than projected. In the end no one will remember why the technical reason for the rewrite, but they will remember who made the decision.
I've made this mistake a few times in my career, and I always thought, this project, it is the exception to the no-rewrites rule.
> we are creating more technical debt than we can possibly handle by doing X in Y way to meet Z deadline, this is going to bite you in the ass after we all switch teams/companies because nobody wants to deal with this
This one doesn't require you to read between the lines, people usually just tell me that directly :)
Your mileage may vary - in my experience this is another one of things where it's not actually true most of the time. Note how it's a really easy to claim to make, and basically impossible to disprove ("at some unknown time in the future something bad will happen unless you let me have my cake now"). To me that's a warning sign that it might be wrong.
As a leader (or CTO or whatever) your job is to make the call. And if it turns out you were wrong and there is literally nobody who wants to touch the mess, well the buck stops with you and you have to clean it up.
>somehow thinking before doing got a bad rap recently (for reasons beyond me).
I've seen plenty of software devs engage in detailed forward planning for a future that would be utterly different from what they expected.
Best case this rendered all that forward planning moot and a waste of time. Medium case is they did a lot of pointless work. Worst case they technologically straitjacketed themselves and dug multiple holes they couldnt extricate themselves from easily.
Then they'd pick themselves up and do it all over again thinking that if they (or more usually somebody else) just managed to predict the future better then this kind of shit wouldnt happen. E.g. those idiot PMs just need to provide better requirements.
It's a hard rut to get out of because in so many other spheres of life forward upfront planning is critical and the future IS predictable. Software intuitively feels like it should be too. But it's the exact fucking opposite of that.
This is, at least, why certain kinds of thinking before doing got a bad rap with me.
Those are a feature of Software Engineering, as opposed to Civil Engineering.
If there was a cheap way to turn Towers into Bridges (and the cost of failure was comparable to that of an app having an outage), our infrastructure would look a lot different.
I know quickly changing requirements are a pain for those who have to do the implementation, but, commercially, that flexibility has significant value.
> Most projects know pretty well where they will be in one or two years
I dispute that that is true. They may have a vague idea of a goal, but that's not applicable at the code level in general.
> Based on that knowledge you can make reasonable decisions and trade-offs now.
We aren't talking about making decisions, we're talking about stubbing out code for future needs, an entirely different thing. Nothing wrong with making reasonable decisions, but there is something wrong with stubbing out code you don't need because you think you'll need it later; if you need it later, add it later. It'll cost the same as adding it now. If you think it's going to cost more later, then revisit your architecture because it shouldn't. The implicit argument for adding it now is that's it's cheaper to add now than later, I'm saying that's nearly always a bunk argument.
> However, I admire your ability to write code without any forethought now that can be used perfectly in whatever form it will be needed later.
That's not what I said; the cost of change should be the same later as now. That doesn't mean you won't have to make a change, it just means it shouldn't be harder to add later than to add now.
If you are in charge of SW team, without too much SW knowledge, and you have to deliver something in the next 3 to 6 months to you customer, you would likely ask your architect/developers to do "same as previous product" but with a single modification for the new feature. This is the risk-free solution for the short term. Add to that that managers could change every 2 years, there are no incentive to take a big risk in rewriting or upgrading the technology. But this is also the boiling frog syndrome, in the long term you end up with a mess, and if you are in a competitive environment you are getting too slow and your company will be eaten alive. If you are relatively competitor free in your niche market, this could somewhat work...
If you hope to achieve something, you need to have a least one supporter in the upper management. It is still very possible that they are unaware how bad the situation is (or they are but they don't care for the long term). So you're first short term goal, should be to communicate clearly what is wrong, and above all what it costs the company every quarter in term of time lost, bugs, etc. And you should propose an alternate solution with some roadmap (even long term one). "In X years, we will have moved to Z and it will allow us to gain Y"
If you don't have at least one big ally, you'll never manage to have some time or resource to improve the situation. (It will probably be easier to convince some of your colleague).
So if you can't rally any ally in the next 3 months, or at least find potential allies that are aware of the situation, walk away.
PS : Even if this looks ugly, there might have been sound reasons that led to that mess. Every small decision in the past were maybe good for the time being. So don't judge too harshly your predecessors. Especially if you need to convince someone in upper management, that the decision he had taken 5 years ago, was absolutely horrible. (And since you are new, that could a very possible misstep)
Edit : if you have been hired to speed up the dev process, at least someone has started to think that maybe something was wrong, so there is some hope !
> The forces which have caused the previous project to accumulate technical debt are still in effect, so after the rewrite reaches feature parity you will probably have exactly the same amount of technical debt.
I would expect, based on both theoretical grounds and on past experience, that the rewrite will have more technical debt by the time it has reached feature parity.
Assuming your primary source of technical debt is all the little corners that get cut in order to meet a schedule, and assuming that the rewrite is being given far less time to get to a certain level of functionality than the original software took to get there, then one would expect the team to need to cut more corners to finish the rewrite. The code may be in a fancy new language, and it may be built on a more modern technology stack, but, in the mad rush to get a death march project done as quickly as possible, corners were cut.
If it feels otherwise, it may be because the project you're replacing was particularly messy. But probably it's because the design tradeoffs in the new system were made by yourself and your colleagues instead of that deranged pack of yahoos who were running things in the '00s.
That's the opposite of what I wrote. My scenario was you knew that it didn't make sense for that project but you still did it because of time/budget/resource reasons. You start paying off the day you can't scale it anymore.
> No, I meant the scenario where a company listens to the engineers and lets them put off business goals to spend weeks to months every so often refactoring, ending up with a modern, easily-maintained system instead of an arcane mess.
I've seen it happen, once. Almost two years of work, lots of architecture, design and all that. Built a beautiful codebase that implemented a product that noone actually wanted. It was a positive experience in some ways, but not something I want to see again.
> That should just be part of the deal; any exceptions should be temporary and carefully negotiated.
This feels like it's probably dependent on the culture in question. Writing tests, caring about readability and the long term sustainability of the codebase and even managing the technical debt all can be the exceptions themselves in certain teams.
Sure, you can turn around and run for the hills, but for many there, those are the circumstances that they put up with to get food on the table, or work with for different reasons, so those circumstances shouldn't be ignored.
Ergo:
> a) refusing to put debt in to begin with
Management: "Hey, what is this ticket taking so long to complete? Can't you just develop the solution more quickly, without spending so much time on refactoring, planning or whatever it is that you do? Developer X did something similar faster just last week!"
> b) for any new work, including necessary cleanup of what you'll be touching in the estimates
Management: "Why are you suggesting estimates that are 1.5x - 2x higher than the rest of the team? Refactoring? Integration tests? Test data generation? Logging? Do you really think that those are necessary for this many tickets? Don't you want to reconsider your estimate, not to be the outlier? No? Okay, we'll just take the averaged estimate of them all."
Personally, i believe that developers should be trusted more and just care about the long term sustainability of what they're developing beyond their probably employment duration at the company (given how popular job hopping is nowadays), yet sadly that's not been the reality in my experience, probably because of prioritizing functionality and business value now, rather than stability and reliability later.
> they definitely need at least a year before they understand the problem space well enough for their offers of major architecture changes to be welcome.
No offense intended, because I don't know your situation, but as someone who has worked in several industries, every time I've heard that kind of phrase it's boiled down to some combination of unnecessary complexity (of architecture and/or process) and undocumented tribal knowledge.
I can't imagine any problem space that would take a senior developer a full year to become competent.
> What project management system do you use? Part of the value of those is supposed to be locking in choices for some period, that allows for actual work to get done.
Jira, but it's easy to say priorities are getting changed, let's cancel this or give it to other guy and I need you here.
> Choosing to neglect (or underfund) the maintenance activities doesn't cost the business anything for some period of time (the delay). And performing them does not (within the period of that delay) bring in any revenue. It is only a cost to them, it has no visible upside within their time horizon. By the time the costs start catching up, it's death by a thousand cuts. Usually they'll go from 100/0 dev/maintenance to 99/1, then 98/2, and so on. Then one day it flips from 80/20 to 20/80 and they realized they fucked up. But by then they've been promoted, so who cares?
Pretty much what I noticed as well.
> You have to convince them
This is exactly the part that I hate. Good practices, code quality, design etc. are not a hobby of mine. The business is not doing me a personal favor by allowing me to practice these ideas.
> For one reason or another, sorting out architectural issues to scale more gracefully was something I could never convince Jeff to allocate resources to do. There were always too many customer-facing features that needed to be developed.
> The time tax is paid by not delivering a feature. Architecture is supposed to be there to help you, not fail deadlines.
Most deadlines are artificial. If it's not artificial, yeah, you're going to have to sacrifice something for the sake of the real deadline. But make no mistake - you are sacrificing something.
> Also, it seems you completely failed to read or get what I wrote. My point was that if a PR delivers a feature without defects then your team is far better off if you accept the PR and refactor later when you can spare the time
You don't ever have spare time. When you finish a feature there's another waiting for you. At least, that's how it works literally everywhere I've worked. Working this way is just another form of over-promising and under-delivering.
> then it is if you throw a hissy fit and refuse to get the feature done if you don't get your way
Rejecting a change doesn't mean you're throwing a hissy fit.
> You are paid to create value which is in delivering features and fixing bugs. Architecture is supposed to be there to help you.
It does help you. In the medium to longer term. In the short term, literally any standard out there will slow you down - unit tests, CI/CD, shit - testing at all. It all slows you down. It doesn't mean we should just code like its 90s era PHP - the fact that we don't should be a strong signal about the value of this whole hypothesis you have going here.
> You have to pass interfaces throughout a bunch of components, which require updating all sorts of tests, and result in a large code footprint.
Funny. Your tests are slowing you down, why don't you just comment them out to get your feature in?
> Look at what you're trying to claim. It's supposedly easy and trouble-free to put together a PR that respects your personal notion of what the software architecture should be.
It's not actually just personal. We have unit tests that enforce certain architectural rules.
> Yet, the work to refactor code that breaks it is so insurmountable that you're no longer able to refactor it back to shape?
If you don't follow standards, as a senior software engineer, why should anyone else? If no one is following standards, how large do you think that shitpile is going to get? And do you really think its just as easy to fix something after the fact?
Why are you even refactoring for reasons you view as subjective? I don't do subjective refactorings. Despite what you claim, at the point of a change, much of this stuff is very not subjective. If you show any developer a function call before a feature was added, and what it looks like after its been hacked it in some fucked way, no one is going to say the fucked version is better.
> It looks like the FANG engineer you tried to badmouth had a firmer grasp on things and on what it matters the most than you do.
Well maybe you're just so smart its trivial for you to navigate this stuff. If so, continue by all means. Until then, us mere mortals will have to adopt standards so our codebases don't outpace our ability to understand it.
I'm thinking of cases where it was a conscious choice. $deity knows that I've made plenty of such decisions in my career - chosing a hacky structure A to meet a next month's release, while articulating to management that after that release, I'll need to redo it to structure B. Sometimes a bunch of high-priority requests took precedence, which made the system stick to A for far longer than it should. I understand that to qualify as both "technical debt" and "architectural issue".
reply