Leveraging Activity Recognition to Enable Protective Behavior Detection in Continuous Data

Chongyang Wang, Y Gao, Akhil Mathur, ACDC Williams, Nic Lane, N Bianchi-Berthouze
in Journal article


Protective behavior exhibited by people with chronic pain (CP) during
physical activities is the key to understanding their physical and emotional
states. Existing automatic protective behavior detection (PBD) methods rely on
pre-segmentation of activities predefined by users. However, in real life,
people perform activities casually. Therefore, where those activities present
difficulties for people with chronic pain, technology-enabled support should be
delivered continuously and automatically adapted to activity type and
occurrence of protective behavior. Hence, to facilitate ubiquitous CP
management, it becomes critical to enable accurate PBD over continuous data. In
this paper, we propose to integrate human activity recognition (HAR) with PBD
via a novel hierarchical HAR-PBD architecture comprising graph-convolution and
long short-term memory (GC-LSTM) networks, and alleviate class imbalances using
a class-balanced focal categorical-cross-entropy (CFCC) loss. Through in-depth
evaluation of the approach using a CP patients' dataset, we show that the
leveraging of HAR, GC-LSTM networks, and CFCC loss leads to clear increase in
PBD performance against the baseline (macro F1 score of 0.81 vs. 0.66 and
precision-recall area-under-the-curve (PR-AUC) of 0.60 vs. 0.44). We conclude
by discussing possible use cases of the hierarchical architecture in CP
management and beyond. We also discuss current limitations and ways forward.