The Perpetual "Software Crisis"
I’ve been listening to the recent webcast “Solving the Embedded Software Crisis” (see also Rich Nass’ column “The need for more programmers” in the May issue of the ESD magazine). Unsurprisingly, the main thrust of both the webcast and the column was that code-generating tools, such as LabVIEW from National Instruments, the webcast’s sponsor, can help alleviate the allegedly looming “software crisis,” as though our problem was not enough code.
The Software Glut Crisis
But tools or no tools, the real problem, in my view, isn’t creating new code so much as getting rid of old, obsolete code. At every company I’ve worked for, we maintained one broad code base for all the products in a given division. We only ever added to it, as new features, product variants, and entirely new products came along. We never removed anything.
Needless to say, the code base became a kitchen sink of everything the company had ever done, prototypes and dead ends included. Most of it was long obsolete, but it lived on forever.
Adding code is easy. Removing dead code is hard (without breaking the parts that are actually still in use).
Yet most managers don’t get it. I remember one day my boss came to my desk wanting to know how much code I’d just cranked out. I proudly showed him that I’d managed to remove an ugly function. He was clearly disappointed in my negative productivity.
The Real Software Crisis
In my whole career, I’ve never once experienced a software shortage. Quite the opposite: I’ve always seen too much of it.
Therefore, getting rid of code matters more than creating it. As I said, that’s hard; it requires careful planning and an actual design for obsolescence. But without mechanisms for dropping the old baggage, we face a real Software Crisis.
In the future installments of this blog, I plan to provide a few concrete design strategies to allow easy (or at least easier) removing of obsolete code. Stay tuned…
9 Responses
This is a great point, Miro. I’d love to see more talk in the engineering community about how to recognize no-longer-needed code. If dead code may have future value, it should be possible to store it in the version control vault for later use.
I think this is an excellent point. I’d also be interested in your thoughts on what it takes to bring old code “up to date”. In my experience one also gets more credit for writing new code than for making old code usable.
In Agile Development we call this refactoring. It is part of everyday work. Code is kept resume quality. Bad code is incrementally improved with the support of a suite of automated tests.Most developers are petrified of changing/deleting code for good cause. They can’t be certain of the code’s behavior. Test Driven Development results in a large suite of automated tests. So if you find some code that you think is no longer needed, delete it and run the test suite. You will know soon enough. (Yeah, I know I am simplifying and living in an idealist world) But often it just works the way I say.Nigel, I salute you for a job well done, wiping out some bad code!
I absolutely agree about the value of refactoring to keep the code lean and clean.However, the main point of my original post goes beyond refactoring and bringing the old code “up to date”. The central point is to get rid of old, no longer used code, such as obsolete features, no longer supported product versions, etc. I don’t want them refactored–I want them out of the active code base.The key question is: How do you design and implement systems so that you can relatively easily get rid of obsolete code? I’m not talking of deleting parts of the code here and there. I’m talking of very careful physical design, that is, the art of partitioning the code into directories and files. If you do this correctly, you can simply stop maintaining obsolete files.Physical design is one of the most unappreciated areas of our business. Countless books and article teach logical design, such as procedural design or object-oriented design, but hardly any book teaches good physical design. One notable exception is “Large-Scale C++ Software Design” by John Lakos. This book is very good, but is not specific to embedded systems. I don’t know of any other book tackling the issue of physical software design specifically for embedded software. Miro
The big problem with many systems I see is that the design has deteriorated and lost all its conceptual integrity. There is a big free for all going on in the code. Functions grow to thousands of lines. Functions with parameter list of double digit length. If statements with compound-compound conditionals.How did these problems happen? How does a 1000 line function get to be1000 lines long. One line at a time. Programmers make code incrementally worse. We need to reverse the trend. Developers need pride in workmanship. They need professional practices that make what you are suggesting possible, practical and safe.Well designed (a.k.a modular, loosely coupled) code can be pruned like you say, but there is not much of that kind of code out there. But even if there were, you would not be allowed to delete it because there is no way to be sure that the code is not needed unless there is a comprehensive set of automated tests and a team that knows the code.Developers are afraid to delete code because they don’t know for sure if it will break something. They won’t extract complex conditionals into understandable and testable helper functions because of some misguided notion of performance or lack of precedent.One way to achieve your goal of being able to delete unneeded code is to use Test Driven Development and Refactoring. Things change over time, design is never done. We need evolutionary design techniques. TDD is the best technique for achieving what you are looking for that I have seen.Code and tests are written concurrently, so the code has to be testable. As it turns out, testable designs are good designs; modular and loosely coupled. Discover some untestable code; it’s an early warning of a design problem. Applying TDD/Refactoring throughout the life if a product will result in a good design. Changes, including deleting oxbow code, can be made with confidence.As far as the John Lakos’ book not being targeted to embedded, that’s part of our industry’s problem as well. Too many embedded developers seem to think that good advice like he and others offer does not apply to embedded.The “we do embedded, we’re different” attitude needs to change. There are a lot of advances going on in software development. Sure there are different challenges. But many are common challenges that have decent solutions that the embedded community is not learning from.
Hi MiroMaybe you just have not gotten to reviewing my post, but…I meant no disrespect to you in my reply to your reply. If you send it back to me, I’ll make sure that is evident.I’d like to have a dialog about things we both care about.thanks, James
James,Thank you for your thoughtful comments. I think that we are in violent agreement. Actually, I have devoted a separate post to the test-driven approach (see Agile Embedded Development).However, the key point of this post (“Embedded Software Crisis or Embedded Software Glut?”) is that the main problem we face is too much code, not too little, as everybody asserts. Engineers and managers need to realize that adding new code is very easy compared to removing code. Yet, removing dead code is far more productive. Once this realization sinks in, we can start paying more attention to structuring the code (both logical and physical design) to allow us to perform cleanup more effectively.
*In the future installments of this blog, I plan to provide a few concrete design strategies to allow easy (or at least easier) removing of obsolete code. Stay tuned.*I am waiting Miro ….
Very good post, and very good follow-up comments.
I have been skimming the web for details of what Miro calls physical design but unfortnately to no avail.
Any insightfull contribution brewing on the subject, Miro?