Beyond Problem Identification: Valuing methods in a ‘system of usability practice’
Historically, usability evaluation methods (UEMs) have been evaluated on their capability for problem identification. However, the relevance of this approach has been questioned for applied usability work. To investigate alternative explanations of what is important for method use a grounded theory of usability practitioners was developed (9 interviews from the website domain and 13 in the safety-critical domain). The analysis proceeded in bottom-up and top-down stages. The bottom-up stages produced insight from the data in an exploratory and inductive manner. This highlighted the importance of contextual factors and the need for system descriptions: UEM adoption and adaptation cannot be fully understood devoid of context. The top-down stages used Distributed Cognition and Resilience Engineering conceptual frameworks as leverage for exploring the data in a deductive manner. These were chosen for their functional descriptions of systems. To illustrate the importance of context we describe three models: 1) where previous research has highlighted the downstream utility of UEMs we expand the metaphor to consider the landscape through which the stream flows, where the landscape represents the project's context; 2) where information propagation and transformation in a project is influenced by social, information flow, artefact, physical and evolutionary factors; and 3) where the functional couplings between parts of the system of usability practice can be monitored and managed to positively resonate with each other, thereby improving the performance of the system overall. The concept of 'Positive Resonance' is introduced to describe how practitioners adapt to the context to maximise their impact under constrained resources. The functional couplings are described in a functional resonance model of HCI practice. This model is validated by interviewees and other practitioners outside of the study. This research shows that problem identification is limited for valuing UEMs. Instead, functional couplings of UEMs should be considered to improve system performance, which influence UEM adoption and adaptation in practice.