Hand Centered Studies of Human Movement Project

Technical Report 94-1

Human movement tracking technology

A discussion of general issues which must be considered in evaluating human movement tracking systems and a summary of currently available sensing technologies for tracking human movement

Prepared by: Axel Mulder, School of Kinesiology, Simon Fraser University, July 1994

Acknowledgement: This work was supported in part by a strategic grant from the Natural Sciences and Engineering Research Council of Canada.

© Copyright 1994 Simon Fraser University. All rights reserved.

You can also get this paper here.


Contents

Summary

General system description

Specifying human movement tracking system

Inside-in human movement tracking

Inside-out human movement tracking

Outside-in human movement tracking Tracking a part of the human body

References (you can also search for more references)

Search for tracking products

Search for tracking product vendors


Summary

Human movement tracking systems can be classified as inside-in, inside-out and outside-in systems.

Inside-in systems are defined as those which employ sensor(s) and source(s) that are both on the body (e.g. a glove with piezo-resistive flex sensors). The sensors generally have small form-factors and are therefore especially suitable for tracking small body parts. Whilst these systems allow for capture of any body movement and allow for an unlimited workspace, they are also considered obtrusive and generally do not provide 3D world-based information.

Inside-out systems employ sensor(s) on the body that sense artificial external source(s) (e.g. a coil moving in a externally generated electromagnetic field), or natural external source(s) (e.g. a mechanical head tracker using a wall or ceiling as a reference or an accelerometer moving in the earth's gravitational field). Although these systems provide 3D world-based information, their workspace and accuracy is generally limited due to use of the external source and their formfactor restricts use to medium and larger sized bodyparts.

Outside-in systems employ an external sensor that senses artificial source(s) or marker(s) on the body, e.g. an electro-optical system that tracks reflective markers, or natural source(s) on the body (e.g. a videocamera based system that tracks the pupil and cornea). These systems generally suffer from occlusion, and a limited workspace, but they are considered the least obtrusive. Due to the occlusion it is hard or impossible to track small bodyparts unless the workspace is severely restricted (e.g. eye movement tracking systems). The optical or image based systems require sophisticated hardware and software and are therefore expensive.

The strengths and weaknesses of the principles underlying the tracking systems are discussed. A list with descriptive, dynamic, static, precision, interfacing, computational, operational and economic measures is given. Some consideration is given to the application of tracking systems to track various bodyparts. A list of human movement tracking systems, commercial and R&D, and mainly classified by the system's use of medium between sensor and source (acoustical, electromagnetical, mechanical, optical etc.) is available upon request from the author.


General system description

Human movement tracking systems are systems that generate in real-time data that represent the measured human movement. In general such systems consist of the following items, some of which can be omitted, depending on the technology involved (see figure below):

Movement of an object is always relative to a point of reference. In order to measure such movement a sensor attached to the object (e.g. a bodypart) can sense (measure the distance to, or orientation or position of) a source attached to the reference or, the sensor, attached to the reference, senses a source that is attached to the object. The human body consists of many moving parts, so that, taking one bodypart as a reference for another, the source and sensor can both be attached to the body. Also, the source can be either natural (e.g. the earth's gravitational field) or artificial (light emitted by an LED). Thus, a taxonomy of human movement tracking systems is possible that extends an existing taxonomy (e.g. Meyer et al 1992):

In the case of outside-in systems, artificial sources are usually called markers or beacons, which can be either passive (e.g. light reflectors) or active (e.g. IR LED's).

Other taxonomies have been proposed. The medium of the sensing technology (i.e. acoustical, optical, electromagnetical, mechanical) is often used as a classifier (Ferrin 1991, Meyer et al 1992 and many others). However, inside-in tracking systems are generally not included in this taxonomy. The bodypart (i.e. head, hand, finger, leg, knee, ankle, foot, spine, face, eye, arm, elbow, chest, pelvis, etc.) is also frequently used as a classifier.


Specifying human movement tracking systems

The following measures are proposed for the specification of a human movement tracking system. See also Kalawsky (1993).

Descriptive measures

Static measures

Dynamic measures

Precision measures

Interfacing measures

Computational measures

Operational measures

Economic measures

Using the above specification measures that are most critical inside-in, inside-out and outside in systems can be compared. However, the comparison is rather crude. See the following pages for more information.

Inside-in systems Inside-out systems Outside-in systems
Spatial resolution ~ 0.5 - 1 deg ~ 0.005 - 8 mm;
~0.025 - 0.1 deg
~ 0.0015 - 0.2 %
of field of view
Spatial accuracy <= 5 deg ~ 0.8 - 5 mm;
~ 0.1 to 3 deg
~ 0.004 - 0.5 %
of field of view
Bandwith <= 80 Hz <= 150 Hz <= 125 Hz (normally)
Latency ~ 1 ms (?) ~ 1 - 40 ms ~ 1 ms (?)
Precision medium to high high ~ 0.0055 - 0.02 %
of field of view
Data frame
of reference
joint/muscle-based world-based world-based
Formfactor ca. 1 cc / sensor-
source pair
1-10 cc / sensor;
> 10 cc / source
< 1 cc / sensor;
> 10 cc / source
Workspace size unlimited natural src.: unlimited
art. src.: 1-2 m radius
1-4 m radius
Price (US$) 1-10 k (glove),
30 k (suit)
1-10 k 20-150 k


Inside-in human movement tracking

General remarks

Many of these technologies do not allow for registration of joint-axial rotation (e.g. pronation/supination of the wrist).

All the technologies use body-centered (joint-angle, eye-rotation, muscle-tension) coordinates.

There is no external source or reference necessary, i.e. the workspace is in principle unlimited.

Due to the fact that inside-in systems are worn on the body they are generally considered obtrusive.

Resolution, static/dynamic range, bandwith and latency are all limited by the interface circuitry, generally not by the sensors.

Most of the technologies have small form-factors and are therefore especially suitable for small body parts (finger, eye, toe). For larger bodyparts the accuracy of these technologies may be reduced due to bodyfat.

Bending or flexing sensors across joints involves a transfer of joint angle to the bend angle of the strip which may reduce the accuracy of the technology, although the sensor itself may have a high repeatability. Each individual sensor must be calibrated for each individual user.

See also Sturman (1994) and Kalawsky (1993).

Available technologies

  • Piezo-resistive flex (Virtex Cyberglove, TCAS Datawear, Mattel/Nintendo Powerglove, Penny & Giles goniometers, GLAD-IN-ART glove, various force-ball device)

  • Piezo-electric flex (Mulder 1988)

  • Fiber-optic flex (VPL Dataglove, W Industries Spaceglove)

  • Light-tube flex (Sayre glove, see Sturman (1994))

  • Cable extension (+ potentiometer / optical encoder / .. )

  • Potentiometers, optical encoders of rotation, goniometers (Virtual reality news WrighTrac)

  • Short range mutual inductance (Motion Orchestration Systems suit and glove)

  • Hall effect (Exos Dexterous Handmaster)

  • Electro-myographic (BioControl systems EMG interfaces, Bodysynth suit, NTT Cyberfinger glove)

  • Electro-oculographic (BioControl systems EOG interfaces)

  • Nerve potential via nerve cuff


    Inside-out human movement tracking

    General remarks

    The external source however does provide in most cases 3D, world-based information, i.e. joint-axial rotations can be measured.

    The form-factor is in most cases fairly large so that the technologies usually apply to larger bodyparts (i.e. not for eye, finger or toe), imply some obtrusiveness and may have limited accuracy due to inertia of the sensor/receptor (the receiver may shift due to skin/muscle movements). Additionally, there will be some offset introduced due to the receiver size.

    Most of the technologies involve some computing which may increase response latency.

    Resolution, static/dynamic range, bandwith are all limited by the interface circuitry, generally not by the sensors.

    The technologies that use an artificial external source have a limited workspace.

    See also Kalawsky (1993), Bhatnagar (1993), Meyer et al (1992) and Ferrin (1991).

    In UNC technical report 92-027 a specification for a position and orientation (inside-out) sensor has been proposed that should allow for "ideal" human movement tracking:

    Available technologies (natural source)

  • Piezo-electric accelerometer (Ladin, 1989 and Doyle, 1993)

  • Gyroscope (solid state, micromechanical or laser, see Doyle, 1993)

  • Inclinometer

  • Magnetic fluxgate compass (Doyle, 1993)

  • Magneto-resistive compass (Doyle, 1993)

  • Altitude sensor

    Available technologies (artificial source)

  • Potentiometer / optical encoder, externally attached via mechanical linkage (Shooting Star Technology head tracker, Fake Space BOOM, Immersion Probe, Sutherland headtracker, various mice)

  • DC EM pulse (Ascension Technology 6DOF tracker)

  • AC EM field strength (Polhemus 6DOF tracker)

  • AC EM field, phase coherent

  • Acoustic time of flight of a pulse (Logitech 2D/6D mouse, Mattel/Nintendo Powerglove, Lincoln Laboratory Wand, Science Accesories Corporation Space Pen)

  • Acoustic, phase coherent (Piltdown acoustic compass, Seitz-Pezaris head tracker)

  • On body videocamera capturing IR LED's (U. of North Carolina head tracker)

  • Infrared sensing of synchronized rotating beams (Honeywell laser beam helmet tracker)

  • Global positioning systems


    Outside-in human movement tracking

    General remarks

    These tracking technologies are generally the least obtrusive of movement tracking technologies.

    Videocamera-based technologies are limited by occlusion. For movements of larger bodyparts this may be solvable, but for e.g. fingers, two closely interacting hands, or two closely interacting persons it remains a major problem.

    Videocamera-based technologies are computing intensive due to difficulties with staying locked onto the bodypart or marker and/or the involved transformations of data, so that response latency may be high (especially relevant for eye-tracking).

    The performance of videocamera-based technologies is dependent on the type of lens or the field of view of the camera. Videocamera-based technologies are operational in a limited workspace only due to the field of view of the camera(s). If the field of view of one camera is increased, resolution is decreased.

    Illumination of the environment may be interfering with proper operation of the system.

    Conventional 30 or 60 frames per second technology provides insufficient bandwith, i.e. special highspeed cameras are required.

    The amount of instrumentation of image based technologies generally remains the same independent of the number of points tracked.

    On-body passive or active markers or beacons have to be attached to the bodypart which introduces an offset. Special care has to taken to select and position a marker.

    See also Kalawsky (1993), Bhatnagar (1993), Meyer et al (1992) and Ferrin (1991).

    Available technologies (natural source)

  • Body image analysis

  • Pupil tracking (ISCAN RK426, LC technologies Eyegaze, Micromeasurements system)

  • 1st-4th Purkinje trackers

  • Scleral and limbus reflection

  • Shadow image analysis (Artificial reality corp. Videoplace, Vivid Mandala)

  • Moire pattern

  • Electrostatic field measurement (as used by Lev Termen, currently researched by Tom Zimmerman and Neil Gershenfeld in the MIT Medialab)

    Available technologies (artificial source)

  • Reflective marker (BTS ELITE, Adaptive Optics Associates Multi Trax, Oxford Metrics VICON, Charnwood dynamics CODA, Optikon MacReflex, Peak Performance technologies PEAK5, HCS Vision Technology Primas, Motion Analysis ExpertVision 3D, Yaman Optfollow 7100, Hentschel HSG, Optron 5600, Origin Instruments head tracker)

  • Infrared LED (Northern Digital Optotrak, Log-In COSTEL, Selspot SELSPOT, Honeywell LED array helmet tracker, United Detector Technology Instrument Group Opeye)

  • B/W marker pattern (Honeywell videometric helmet tracker)

  • Color marker pattern (Columbus Instruments Biovision, Dorner 1994)


    Tracking a part of the human body

    The size of the bodypart can be used to distinguish three different classes of applications of human movement tracking systems: large to medium sized body parts (head, hand, arm, leg, foot, hip, trunk, shoulder), medium to small body parts (finger, toe, jaw), small to tiny bodyparts (eye, lips, nose, ears).

    Movements of medium to large bodyparts are usually registered with either inside-out or outside-in systems, although inside-in systems also have been built for this application(e.g. VPL Datasuit).

    Movements of medium to small bodyparts are usually registered with inside-in systems.

    Registration of eye movements requires specialized solutions. Hallet (1986) and O'Donnel (1986) give a useful, fairly complete, overview of available technologies and Hallet (1986) suggests the following requirements for an eye movement tracking system:

    Commonly used technologies for the tracking of eye movements include:

    Error sources of eye movement tracking systems include:


    References

    For more references try searching my bibliography (ca. 140K).

    Newsgroup sci.virtual-worlds

    (author unknown) Research directions in virtual environments, Report of an NSF invitational workshop, March 23-24, 1992, University of North Carolina at Chapel Hill UNC Technical Report No. TR92-027, [available through anonymous ftp in ftp.cs.unc.edu: /pub/publications/techreports and by e-mailing netlib@cs.unc.edu and sending the request "get 92-027.ps from techreports"]. Chapel Hill, NC: Dept. of Computer Science, University of North Carolina.

    Ahlers, R-J., Lu, J., (1989). Stereoscopic vision - An application oriented overview, Proceedings of the SPIE - The International Society for Optical Engineering Optics, Vol. 1194. Illumination and Image Sensing for Machine Vision IV. (pp. 298-308). Bellingham, WA: SPIE.

    Bhatnagar, D.K. (1993). Position Trackers for Head Mounted Display Systems: A Survey. UNC Technical Report No. TR93-010, [available through anonymous ftp in ftp.cs.unc.edu:/pub/publications/techreports and by e-mailing netlib@cs.unc.edu and sending the request "get 93-010.ps from techreports"]. Chapel Hill, NC: Dept. of Computer Science, University of North Carolina.

    Bryson, S., (1993). Virtual reality hardware, In: Implementing Virtual Reality, Coursenotes #43, ACM SIGGRAPH 93, p1.3.16-1.3.24. New York, NY: ACM.

    Burdea, G., Zhuang, J., (1991). Dextrous telerobotics with forcefeedback - an overview. Part 1: Human factors, Robotica v9 pp 171-178

    Burdea, G., Zhuang, J., (1991). Dextrous telerobotics with forcefeedback - an overview. Part 2: Control and implementation, Robotica v9 (1991) pp 291-298

    Calvert, T.W., Chapman, A.E., (1994). Analysis and synthesis of human movement, In: Handbook of pattern recognition and image processing: Computer vision, ? (Ed), p431-474. London, UK: Academic Press.

    Dorner, B., (1994). Chasing the colour glove, MSc. Thesis, School of Computer Science, Simon Fraser University, British Columbia, Canada

    Doyle, K., (1993). Comp.robotics Frequently Asked Questions. [available through anonymous ftp at rtfm.mit.edu in pub/usenet/news.answers/robotics-faq/part1, part2 and part3] 70+ pages.

    Eglowstein, H., (1990). Reach out and touch your data, Byte (July) pp 283-290

    Ferrin, Frank J. (1991). Survey of helmet tracking technologies. SPIE - The International Society for Optical Engineering - Large-Screen Projection, Avionic, and Helmet-Mounted Displays, Volume 1456 (pp. 86-94). Bellingham, WA, USA: SPIE.

    Gradecki, J.D. (1994). Survey of Available Head Trackers. PCVR Magazine, (13) (January), p. 24.

    Hallet, P.E., (1986). Eye movements. In: Boff, K.R., Kaufman, L., Thomas, J.P., (Editors). Handbook of perception and human performance, section 10 pp 25-28. New York: Wiley.

    Kalawsky, R.S., (1993). The science of virtual reality and virtual environments. pp 277-291, 135-164, 187-197. Wokingham, England: Addison-Wesley.

    Krueger, M.W., (1990). Artificial reality. 2nd ed. Reading, MA USA: Addison Wesley

    Ladin, Z., Flowers, C. and Messner, W., (1989). A quantitative comparison of a position measurement system and accelerometry. Journal of Biomechanics 22, p 295 - 308

    Macellari, V., Leo, T., Chistolini, P., Bugarini, M., (1985). Comparison among remote sensing systems for human movement measurement. MELECON '85 vol 1: Bioengineering, Luque, A., Figueiras Vidal, A.R., Delgado, J.M.R. (Editors). Amsterdam, The Netherlands: Elsevier Science Publishers B.V. (North-Holland).

    Meyer, K., Applewhite, H.L., and Biocca, F.A. (1992). A Survey of Position-Trackers. Presence: Teleoperators and Virtual Environments. 1 (2) (spring 1992), 173-200.

    Mulder, A.G.E., (1988). A piezo-electric flex transducer for a hand gesture interface. Unpublished technical report, Department of physics, State university of Groningen, The Netherlands.

    O'Donnel, R.D. (1986). Workload assessment methodology. In: Boff, K.R., Kaufman, L., Thomas, J.P., (Editors). Handbook of perception and human performance, section 42 pp 38-49. New York: Wiley.

    Samuelson, B., Wangenheim, M., Wos, H., (1987). A device for three-dimensional registration of human movement, Ergonomics v30 n12 pp 1655-1670

    Shimoga, K.B., (1993). A survey of perceptual feedback issues in dextrous telemanipulation: part I. Finger force feedback, proceedings of the IEEE Virtual realit y annual international symposium (Seattle Washington, september 18-22 1993) pp263-270

    Shimoga, K.B., (1993). A survey of perceptual feedback issues in dexterous telemanipulation: part II. Finger touch feedback, proceedings of the IEEE Virtual reality annual international symposium (Seattle Washington, september 18-22 1993) pp 271-279

    Sturman, D.J., and Zeltzer, D. (1994). A Survey of Glove-Based Input. IEEE Computer Graphics and Applications, 14 (1) (january), 30-39.

    Thompson, J., (1993). Virtual reality: an international directory of research projects. Westport: Meckler.

    Tyson, J.N., Das, B., (1990). A comparative analysis of anthropometric and kinematic measurement systems. In: Advances in industrial ergonomics and safety II, Das, B., (editor). pp 301-308. London, UK: Taylor & Francis.