Recognition of fine-grained walking patterns using a smartwatch with deep attentive neural networks

Generally, people do various things while walking. For example, people frequently walk while looking at their smartphones. Sometimes we walk differently than usual; for example, when walking on ice or snow, we tend to waddle. Understanding walking patterns could provide users with contextual information tailored to the current situation. To formulate this as a machine-learn-ing problem, we defined 18 different everyday walking styles. Noting that walking strategies sig-nificantly affect the spatiotemporal features of hand motions, e.g., the speed and intensity of the swinging arm, we propose a smartwatch-based wearable system that can recognize these prede-fined walking styles. We developed a wearable system, suitable for use with a commercial smart-watch, that can capture hand motions in the form of multivariate timeseries (MTS) signals. Then, we employed a set of machine learning algorithms, including feature-based and recent deep learning algorithms, to learn the MTS data in a supervised fashion. Experimental results demonstrated that, with recent deep learning algorithms, the proposed approach successfully recognized a variety of walking patterns, using the smartwatch measurements. We analyzed the results with recent at-tention-based recurrent neural networks to understand the relative contributions of the MTS signals in the classification process.