18. The Roles of Software in Society

IT systems have a life-cycle in which maintenance is the dominant activity. Maintaining (i.e. extending and debugging) an IT system that you did not develop exacerbates all of the IT-system problems, and is the major programming effort. At a certain point in IT-system complexity, further debugging is a self-defeating exercise. We all live within a mesh of IT systems: some offer conveniences that we can either accept or reject, others do not give us a choice. The outputs of an IT system may be viewed as a useful guide to be accepted cautiously, or as the ultimate truth to be accepted blindly; the reality of all IT systems will lie somewhere in between these two extremes. IT systems are not extended or amended lightly, because change is so dangerous; hence, we tend to ‘work around’ their failings. Many IT systems improve our lives. Some IT systems have negative impacts, all the way from minor aggravation to life threatening. There are no easy answers (as far as we know). So for IT systems in safety-critical roles where failures have the potential to cause deaths and/or disasters, the best advice is too lessen the criticality of their roles and to limit expectation of what they can deliver.