Comparing between two skeleton tracking on processing 3

I am currently doing my dissertation which would involve in having 2 people a professional athlete and an amateur. First with the image processing skeletonization I would like to record the professional athlete while performing the squat exercise , then when the amateur performs the exercise I want to be able to compare the professional skeleton with that of the amateur to see if it is properly formed.

Please I m open for any suggestions and opinions , Would gladly appreciate some help

Here lies your question:

properly formed.

What does properly performed actually mean ? How can this be quantified ?

Bare in mind I'm not an athletic/experienced in this field. If I were given the task I would counter-intuitively go in the opposite direction: moving away form Processing 3/kinect/computer. I would instead:

1. find a professional athlete
2. find a skilled with trainer with functional mobility training.
3. find an amateur (probably easiest)

Item 2 will be trickier.For example FMS seems to put a lot of emphasis on correct exercising and mobility (to enhance performance and reduce risk of injuries). I'm not sure if that's the only approach or the best. You might want to check opinions on Physical Fitness, consult with people studying/teaching exercise science, etc. Do check credentials as it feels like a field where everyone has an opinion/preference.

The idea is to understand how a professional educated trainer asses correct movement. Take note of how that works in the real world and try to systemise it.

What are the cues for a correct execution ?

• is the key poses
• the motion in between
• how the skeletal and muscular system work together/ the weights/forces applied/etc.

Having a better understanding of how this works in the real world should lead you to things you can start quantifying/comparing numerically on a computer.

Try to make a checklist/score system manually using a pen and paper based on the information you gather. If this works you already have a system you can start programming.

The next step is acquiring the data. This is probably where the kinect comes, but bare in mind:

• the second version of the kinect is more precise than the first
• there is a Kinect2 SDK wrapper for Processing 3: use that if you can (windows only). There is a way you can get libfreenect2 working with OpenNI on osx/linux and therefore with SimpleOpenNI in Processing, but it's not straight forward and you won't have the same precision on the skeleton tracking algorithm
• use data that is as precise as possible:
  • you can get the accuracy of a tracked skeleton joint
  • use an environment that doesn't contain a complex background (makes it easy to segment users and detect/track skeletons with little change of mistaking it for something else). prefer artificial non-incandescent light (less of a problem with kinect v2, but still you want as little IR interference as possible).
  • comparing orientation matrices or joints on single poses might not be enough to get the full picture: how do you capture/quantify motion taking into account the things that the kinect can't easily see: muscles flexing/forces applied/moving centre of gravity/etc.
  • try to use a grid system that will make it simple to pair the digital values with real world measurements. Check out how people used to study motion in the past, for example Étienne-Jules Marey or Eadweard Muybridge

Motion capture by Étienne-Jules Marey

Motion study by Eadweard Muybridge (notice the grid)

It's a pretty full on project to get right involving bits of anatomy/physics/kinematics/etc.

Start with the research first:

• how did people study this in the past ?
• what are the current developments ?
• how does it work in the real world (without computers) ?

Take your constraints into account:

• what resources (people/gear/etc.) can you use ?
• how much time do you have available ? Given the above, what topic/section of the project can be realistically be tackled to get useful results.

Overall probably something along these lines:

• background research
• real world studies
• comparison system has feature which can be measured both with kinect and by a person
• record data (real world data + mobility comparison evalutation and kinect data + mobility comparison)
• compare data
• write evaluation of findings (how effective is the system? what are limitations ? what could be improved (future work) ? etc.)

In short be aware of the kinect limitations: skeleton tracking is probability based: it's not 100% accurate. use data that's as clean/correct as possible to begin with (make it easy to acquire good data if you can control the capture environment). From what a real trainer would track, what could you track with a kinect ? do a comparison of the intersecting measurements.