Week 06 | Tools & Technology #2 |

Week 06 | Tools & Technology #2 |

Monday, February 10, 2024

Room 3420 (Marsio), Computer classrooom

Inspiration #

This week will be our first introduction to working with tracking people and their bodily movements in a space. These are some examples I have been working on previously.

The projects below are from a time before the Kinect existed and these kinds of things were much harder to do. Couple of them use the basic blob tracking technique we will build today.

Animoitu Liike, 2009 #

Mortimer, 2009 #

Mortimer includes some scenes where we used real-time dynamic projection mapping. This was achieved using infrared camera and LEDs.

Mortimer Full Performance

Giants of the Hoods, 2010 #

Giants of the Hoods was a project where we went to different neighborhoods in Helsinki and Espoo and projected real-time animations on the facades of buildings. The characters were controlled by people dancing

People were also able to have themselves photographed and they could decide which part of their body they would lioke to donate to the “Giant” projected on the wall. The character basically ended up being a collage of the people who participated.

This was based on a similar setup used in the Reverse Shadow Theatre and used the Animata software.

Animoitu Liike, 2011 #

After the original Kinect was released, new experimental projects popped into existence everyday. This is my update on the Animoitu Liike project.

AHNE, 2011 #

Ahne was a research project created in the SOPI research group. It started out as an exploration to create spatial interface that would not have any visual feedback. The only feedback would be sound and haptics. The user would wear a glove that had a haptic feedback motor and wireless sensors.

Body Tracking (Pose Detection) #

Kinect (Azure) #

Pros:

  • Works in complete darkness since it uses infrared
  • Works directly in TouchDesigner
  • You also get depth image and point cloud

Cons:

  • Expensive sensor
  • Windows only
  • Has been discontinued

Although the Azure Kinect is discontinued by Microsoft, there are a couple of options that are almost direct replacements. Orbbec makes two cameras that use the samer Body Tracking models as the Kinect.

In TouchDesigner, you can find the Kinect Azure TOP and CHOP that give you the image and tracking data from the Kincet Azure sensor.

ZED #

Pros:

  • Actively maintained and supported
  • Works directly in TouchDesigner
  • You also get depth image and point cloud

Cons:

  • Expensive sensor
  • Windows only

MediaPipe #

MediaPipe is the simplest option in the sense that it doesn’t require any special hardware and works with different operating systems.

Pros:

  • No special hardware needed (any webcam that is able to output a 720p image will work)
  • Does also much more beyond the pose tracking (face tracking, hand tracking, object tracking, image classification, image segmentation)
  • Multiplatform

Cons:

  • No true 3D information. The Z-coordinate is a relative coordinate based on the center-of-mass
  • The TouchDesigner plugin is sometimes buggy and doesn’t load the camera. Especially with the Aalto computers.

Homework #

Final Project Ideation #

If you haven’t already, now is the time to start planning your final project for the course. See the guidelines here. We will go over the details in class today and get this process started.

Watch #