Most likely most readers have heard of the Human Singularity as originally described in the novels from Vernor Vinge and popularized by Ray Kurzweil.
Are we approaching a software singularity? Everywhere I look certain trends jump out at me.
More and more software development is done by layering existing complex packages on top of each other. The first step in most new development is to scour the internet for open-source packages. Original development effort is then invested to glue these packages together, plaster over everything which isn’t needed and fill in the remaining gaps. If the building blocks being used are fully understood, this may well be an optimal solution. Unfortunately, understanding is often superficial and the solution is by no means robust.
Each new generation of each new product adds a few new functions. As a rule of thumb, there is generally an exponential relationship between the number of functions offered and the complexity of software. This explosive growth of complexity requires corresponding efforts to be manageable. These efforts might be invested in partitioning into simple subsystems or into the use of a platform. Is it possible that Agile management methods are so successful because they offer a structure with which projects can be terminated when they have become “good enough”?
Another result of increasing complexity is the growth in software development teams. Everywhere we look good software engineers are desperately sought after. At the same time, when one speaks with the engineers themselves, their loudest complaint is always about the inefficiencies with which they are confronted within the organization. Inefficient structures and inefficient processes result in wasted time and effort for the engineers. This is something that software development standards (e.g. CMMI and ASPICE) are supposed to counter. Unfortunately, most organizations are fully occupied with their current development efforts and have no free capacity for internal improvement.
Development cycles are lasting longer than in the past. Something which required two years in the past can now be expected to require three years. This seems to be coupled with another symptom, that the ability to produce a high-quality product is a fading memory of the past. This is unfortunately coupled with a meteoric rise in importance for the issues of cybersecurity and functional safety. Although much literature exists on both subjects, most industries are guilty of a profound neglect of these issues. The EU is now slowly waking up to these problems and is assuming some regulatory responsibility. It is my hope that this trend will continue and strengthen.
Yes, a software development singularity is coming. IMHO, the keys to survival of the singularity are:
– Mature development processes that are implemented to maximize efficiency and simplicity
– Mature re-use concepts for software that make evolutionary improvement possible
– A stronger focus on “what is needed” rather than “what is possible”
It will be interesting to revisit this in 10 years and ask how accurate I was.
