How do you define scalability?
“Scalable” is an interesting word that gest thrown around all too much in the development\information technology world. I recently had an argument a discussion with a co-worker about the possibility of building a brand new set of services on top of our old, rotting code base. My opinion is that we should take a step back and take a serious look at the current code base. I don’t believe that it is a good idea to build an important new service (or set of services) on top of the current code because it was “architected” by several out-sourced junior developers 5 years ago.
The code is… garbage. Some of the recent stuff is good and usable because it was developed by seasoned engineers. The older stuff is so poorly written that every time we are required by our product team to add a new feature or enhance an existing one, the time it takes to get the product out the door is months when it should be weeks and weeks when it should be days.
The co-worker in question continually argued that the current application has scaled quite a bit over the past 5 or 6 years and is fully equipped to scale further. He argued that over the lifetime of the codebase, we have added so many new features. The thing that kept jumping out at me was his use of the word “scalable” to describe our applications ability to handle more traffic, and nothing more.
This made me think – is the application the only thing that needs to be scalable? I think the answer is unequivocally “No.”
In our case, we can scale the application to handle more requests but we cannot scale the development team. That is our problem – scaling the development of our software is impossible. Like hardware scaling, development scaling has two options and for the sake of simplicity we’ll call them by the same names.
Vertical scaling: Hire more developers. Its easy to find low quality people to work on low quality code, but they still cost a decent amount of money. Single point of failure because since the code is so complicated, its more difficult (if not impossible) to find people who can quickly learn the old code to a point that they can build something on top of it. In my experience, few developers were willing to work in such a disaster of code. We hired a number of developers who quit after just a couple of months. Its not fun or interesting to be in “bug fix” mode forever.
Horizontal scaling: Use the engineers\architects\developers you already employ and take the time to fix the code (or write new code). This is harder and cant always be done (in our case, it’s a viable option). Its harder also because you would most likely need to consider all the things your software does now and recreate that experience. You eliminate the points of failure noted above because, if done right (and hopefully it is, since that’s why you're doing it in the first place), you’ll be able to hire any eligible developer and they should be able to pick up the new, modern, well architected code much quicker than above.
The ability to re-build from scratch is rarely an option in the software development world but when it is, you should seriously consider it. When you hit that point where you can no longer scale your development to a level at which you can easily and quickly iterate on your application or build new features, you should think about your options and find out if re building makes sense. It might not and you might be stuck working with unscalable software, even if it does handle the load and generate cash flow.