Effectiveness and Cost-Benefit Ratio of Weekly User Group Sessions

Early user involvement is a central part of a user-centered design process. In a project to create a UX design for an engineering tool for railways, weekly one-hour user group sessions have been used to elicit domain insights and to gather user feedback for proposed design options. This paper explores the effectiveness of such weekly user group sessions, by answering three research questions: RQ1: How much did the participating users effectively influence the UX design? RQ2: What was the impact of the participating users on efficiency improvements? RQ3: What was the return-on-investment (ROI) for the weekly group sessions? During 18 weekly user group sessions, 64 design decisions were made. Out of the 64 decisions, in 24 cases (38%) the participating users have chosen a different design option than the design option, initially selected by the UX team. Out of the 24 decisions, 14 decisions led to low, 10 to medium and 0 to high efficiency improvements. Comparing the hourly effort of preparing and holding such weekly sessions (cost of investment) with the avoided rework (gain of investment), the ROI is 12%. The conclusion is that weekly user group sessions have been a worthwhile approach to make the user supported design choices early in the development process.