Bainbridge and the Ironies of Automation

Lisanne Bainbridge wrote a paper in 1983 that is more relevant today than ever. “Ironies of Automation” now has over 1800 citations¹ and describes a fundamental problem.

The more we automate, the harder the remaining human tasks become.

The central irony concerns the mindset of system designers. Bainbridge observed that they view humans as unreliable and inefficient and therefore want to remove them from the system.² But these same fallible humans design the system and must step in when automation fails.

By taking away the easy parts of the task, automation can make the difficult parts of the human operator’s task more difficult.³

There is also an attention problem. Those who only monitor screens instead of actively controlling lose their feel for the process. Automatic systems can mask failures by compensating for deviations until it is too late.⁴ The skills needed for emergencies atrophy through lack of practice.

The final irony is the most bitter:

It is the most successful automated systems, with rare need for manual intervention, which may need the greatest investment in human operator training.⁵

The better automation works, the more we need to invest in human skills that are almost never used.