Assessing the usefulness of a visual programming IDE for large-scale automation software

Industrial control applications are usually designed by domain experts instead of software engineers. These experts frequently use visual programming languages based on standards such as IEC 61131-3 and IEC 61499. The standards apply model-based engineering concepts to abstract from hardware and low-level communication. Developing industrial control software is challenging due to the fact that control systems are usually unique and need to be maintained for many years. The arising challenges, together with the growing complexity of control software, require very usable model-based development environments for visual programming languages. However, so far only little empirical research exists on the practical usefulness of such environments, i.e., their usability and utility. In this paper, we discuss common control software maintenance tasks and tool capabilities based on existing research and show the realization of these capabilities in the 4diac IDE. We performed a walkthrough of the demonstrated capabilities using the cognitive dimensions of notations framework from the field of human–computer interaction. We then improved the tool and conducted a user study involving ten industrial automation engineers, who used the 4diac IDE in a realistic control software maintenance scenario. Based on lessons learnt from this study, we adapted the 4diac IDE to better handle large graphical models. We evaluated these changes in a reassessment study with automation engineers from seven industrial enterprises. We derive general implications with respect to large-scale applications for developers of IDEs that we deem applicable in the context of (visual) model-based engineering tools.