0
Research Papers

Human Arm Motion Tracking by Orientation-Based Fusion of Inertial Sensors and Kinect Using Unscented Kalman Filter

[+] Author and Article Information
Arash Atrsaei

Department of Mechanical Engineering,
Sharif University of Technology,
Tehran 1458889694, Iran
e-mail: atrsaei@mech.sharif.edu

Hassan Salarieh

Department of Mechanical Engineering,
Sharif University of Technology,
Tehran 1458889694, Iran
e-mail: salarieh@sharif.edu

Aria Alasty

Department of Mechanical Engineering,
Sharif University of Technology,
Tehran 1458889694, Iran
e-mail: aalasti@sharif.edu

1Corresponding author.

Manuscript received March 15, 2016; final manuscript received July 2, 2016; published online August 2, 2016. Assoc. Editor: Pasquale Vena.

J Biomech Eng 138(9), 091005 (Aug 02, 2016) (13 pages) Paper No: BIO-16-1097; doi: 10.1115/1.4034170 History: Received March 15, 2016; Revised July 02, 2016

Due to various applications of human motion capture techniques, developing low-cost methods that would be applicable in nonlaboratory environments is under consideration. MEMS inertial sensors and Kinect are two low-cost devices that can be utilized in home-based motion capture systems, e.g., home-based rehabilitation. In this work, an unscented Kalman filter approach was developed based on the complementary properties of Kinect and the inertial sensors to fuse the orientation data of these two devices for human arm motion tracking during both stationary shoulder joint position and human body movement. A new measurement model of the fusion algorithm was obtained that can compensate for the inertial sensors drift problem in high dynamic motions and also joints occlusion in Kinect. The efficiency of the proposed algorithm was evaluated by an optical motion tracker system. The errors were reduced by almost 50% compared to cases when either inertial sensor or Kinect measurements were utilized.

Copyright © 2016 by ASME
Your Session has timed out. Please sign back in to continue.

References

Zhou, H. , and Hu, H. , 2008, “ Human Motion Tracking for Rehabilitation—A Survey,” Biomed. Signal Process. Control, 3(1), pp. 1–18. [CrossRef]
Bachmann, E. R. , McGhee, R. B. , Yun, X. , and Zyda, M. J. , 2001, “ Inertial and Magnetic Posture Tracking for Inserting Humans Into Networked Virtual Environments,” ACM Symposium on Virtual Reality Software and Technology (VRST '01), Banff, AB, Canada, Nov. 15–17, pp. 9–16.
Zhang, Z. , 2012, “ Microsoft Kinect Sensor and Its Effect,” IEEE MultiMedia, 19(2), pp. 4–10. [CrossRef]
Ahmadi, A. , Destelle, F. , Monaghan, D. , Moran, K. , O'Connor, N. E. , Unzueta, L. , and Linaza, M. T. , 2015, “ Human Gait Monitoring Using Body-Worn Inertial Sensors and Kinematic Modeling,” IEEE SENSORS 2015, Busan, South Korea, Nov. 1–4, pp. 540–547.
Zhang, Z.-Q. , and Wu, J.-K. , 2011, “ A Novel Hierarchical Information Fusion Method for Three-Dimensional Upper Limb Motion Estimation,” IEEE Trans. Instrum. Meas., 60(11), pp. 3709–3719. [CrossRef]
Tao, G. , Huang, Z. , Sun, Y. , Yao, S. , and Wu, J. , 2013, “ Biomechanical Model-Based Multi-Sensor Motion Estimation,” IEEE Sensors Applications Symposium (SAS), Galveston, TX, Feb. 19–21, pp. 156–161.
El-Gohary, M. , Holmstrom, L. , Huisinga, J. , King, E. , McNames, J. , and Horak, F. , 2011, “ Upper Limb Joint Angle Tracking With Inertial Sensors,” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEMBS), Boston, MA, Aug. 30–Sept. 3, pp. 5629–5632.
Renaudin, V. , Afzal, M. H. , and Lachapelle, G. , 2010, “ Complete Triaxis Magnetometer Calibration in the Magnetic Domain,” J. Sens., 2010, p. 967245. [CrossRef]
Luinge, H. J. , and Veltink, P. H. , 2005, “ Measuring Orientation of Human Body Segments Using Miniature Gyroscopes and Accelerometers,” Med. Biol. Eng. Comput., 43(2), pp. 273–282. [CrossRef] [PubMed]
Le, T.-L. , Nguyen, M.-Q. , and Nguyen, T.-T.-M. , 2013, “ Human Posture Recognition Using Human Skeleton Provided by Kinect,” International Conference on Computing, Management, and Telecommunications (ComManTel), Ho Chi Minh City, Vietnam, Jan. 21–24, pp. 340–345.
El-laithy, R. A. , Huang, J. , and Yeh, M. , 2012, “ Study on the Use of Microsoft Kinect for Robotics Applications,” IEEE/ION Position Location and Navigation Symposium (PLANS), Myrtle Beach, SC, Apr. 23–26, pp. 1280–1288.
Gabel, M. , Gilad-Bachrach, R. , Renshaw, E. , and Schuster, A. , 2012, “ Full Body Gait Analysis With Kinect,” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), San Deigo, CA, Aug. 28–Sept. 1, pp. 1964–1967.
Pfister, A. , West, A. M. , Bronner, S. , and Noah, J. A. , 2014, “ Comparative Abilities of Microsoft Kinect and Vicon 3D Motion Capture for Gait Analysis,” J. Med. Eng. Technol., 38(5), pp. 274–280. [CrossRef] [PubMed]
Clark, R. A. , Pua, Y.-H. , Fortin, K. , Ritchie, C. , Webster, K. E. , Denehy, L. , and Bryant, A. L. , 2012, “ Validity of the Microsoft Kinect for Assessment of Postural Control,” Gait Posture, 36(3), pp. 372–377. [CrossRef] [PubMed]
Dutta, T. , 2012, “ Evaluation of the Kinect Sensor for 3-D Kinematic Measurement in the Workplace,” Appl. Ergon., 43(4), pp. 645–649. [CrossRef] [PubMed]
Webster, D. , and Celik, O. , 2014, “ Experimental Evaluation of Microsoft Kinect's Accuracy and Capture Rate for Stroke Rehabilitation Applications,” IEEE Haptics Symposium (HAPTICS), Houston, TX, Feb. 23–26, pp. 455–460.
Bo, A. , Hayashibe, M. , and Poignet, P. , 2011, “ Joint Angle Estimation in Rehabilitation With Inertial Sensors and Its Integration With Kinect,” 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC '11), Boston, MA, Aug. 30–Sept. 3, pp. 3479–3483.
Feng, S. , and Murray-Smith, R. , 2014, “ Fusing Kinect Sensor and Inertial Sensors With Multi-Rate Kalman Filter,” IET Conference on Data Fusion & Target Tracking: Algorithms and Applications (DF&TT 2014), Liverpool, UK, Apr. 30, pp. 1–8.
Tian, Y. , Meng, X. , Tao, D. , Liu, D. , and Feng, C. , 2015, “ Upper Limb Motion Tracking With the Integration of IMU and Kinect,” Neurocomputing, 159, pp. 207–218. [CrossRef]
Destelle, F. , Ahmadi, A. , O'Connor, N. E. , Moran, K. , Chatzitofis, A. , Zarpalas, D. , and Daras, P. , 2014, “ Low-Cost Accurate Skeleton Tracking Based on Fusion of Kinect and Wearable Inertial Sensors,” 22nd European Signal Processing Conference (EUSIPCO), Lisbon, Portugal, Sept. 1–5, pp. 371–375.
Kalkbrenner, C. , Hacker, S. , Algorri, M.-E. , and Blechschmidt-Trapp, R. , 2014, “ Motion Capturing With Inertial Measurement Units and Kinect-Tracking of Limb Movement Using Optical and Orientation Information,” International Conference on Biomedical Electronics and Devices (BIODEVICES 2014), Angers, France, Mar. 3–6, pp. 120–126.
Lizarralde, F. , and Wen, J. T. , 1996, “ Attitude Control Without Angular Velocity Measurement: A Passivity Approach,” IEEE Trans. Autom. Control, 41(3), pp. 468–472. [CrossRef]
Yun, X. , Lizarraga, M. , Bachmann, E. R. , and McGhee, R. B. , 2003, “ An Improved Quaternion-Based Kalman Filter for Real-Time Tracking of Rigid Body Orientation,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003), Las Vegas, NV, Oct. 27–31, Vol. 2, pp. 1074–1079.
Simon, D. , 2006, Optimal State Estimation: Kalman, H, and Nonlinear Approaches, Wiley, Hoboken, NJ.
Shuster, M. D. , and Oh, S. , 1981, “ Three-Axis Attitude Determination From Vector Observations,” J. Guid., Control, Dyn., 4(1), pp. 70–77. [CrossRef]
Yun, X. , Bachmann, E. R. , and McGhee, R. B. , 2008, “ A Simplified Quaternion-Based Algorithm for Orientation Estimation From Earth Gravity and Magnetic Field Measurements,” IEEE Trans. Instrum. Meas., 57(3), pp. 638–650. [CrossRef]
Sabatini, A. M. , 2006, “ Quaternion-Based Extended Kalman Filter for Determining Orientation by Inertial and Magnetic Sensing,” IEEE Trans. Biomed. Eng., 53(7), pp. 1346–1356. [CrossRef] [PubMed]
Julier, S. J. , and Uhlmann, J. K. , 1997, “ New Extension of the Kalman Filter to Nonlinear Systems,” AeroSense'97, Conference on Photonic Quantum Computing, Orlando FL, Apr. 20–25, Proc. SPIE, 3068, pp. 182–193.

Figures

Grahic Jump Location
Fig. 1

The proposed human model (a) human body model and (b) human arm model

Grahic Jump Location
Fig. 2

The angle between Kinect x axis and the line passing through the shoulder joints

Grahic Jump Location
Fig. 3

The flowchart diagram of the proposed algorithm

Grahic Jump Location
Fig. 5

The sampling frequency of the inertial sensor (long lines), which is constant and Kinect (short lines), which is varying

Grahic Jump Location
Fig. 6

The fusion results for the first stage of the test (a) elbow X, (b) wrist X, (c) elbow Y, (d) wrist Y, (e) elbow Z, and (f) wrist Z

Grahic Jump Location
Fig. 7

The inertial sensors results for the first stage of the test without data fusion (a) elbow X, (b) wrist X, (c) elbow Y, (d) wrist Y, (e) elbow Z, and (f) wrist Z

Grahic Jump Location
Fig. 8

Kinect results for the first stage of the test without data fusion, (a) elbow X, (b) wrist X, (c) elbow Y, (d) wrist Y, (e) elbow Z, and (f) wrist Z

Grahic Jump Location
Fig. 9

The quaternion rotation of the forearm by the fusion method

Grahic Jump Location
Fig. 10

The quaternion rotation of the upper arm by the fusion method

Grahic Jump Location
Fig. 11

The norm of the accelerometers

Grahic Jump Location
Fig. 15

The fusion results for the second stage of the test (a) elbow X, (b) wrist X, (c) elbow Y, (d) wrist Y, (e) elbow Z, and (f) wrist Z

Grahic Jump Location
Fig. 14

Kinect results for the second stage of the test without data fusion (a) shoulder X, (b) shoulder Y, (c) shoulder Z, (d) elbow X, (e) elbow Y, (f) elbow Z, (g) wrist X, (h) wrist Y, and (i) wrist Z

Grahic Jump Location
Fig. 13

The second stage of the test

Grahic Jump Location
Fig. 12

The effect of utilizing the second measurement equation (a) without constraint and (b) with constraint

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In