Movement Study
My central gesture is a pick–inspect–place loop. I repeat the same task so that changes in my pace, posture, attention, and fatigue become visible.
The Unreal robot will mirror my motion. If I show a reinforcement-learning dashboard, I will label it as a parallel training process, unless I build a direct connection between my captured motion and that system.
| ID | Movement | What I want to show | Variations | Digital house |
|---|---|---|---|---|
| M01 | Clock in: enter, take my position, lower my gaze. | My body becoming available for work. | Eager, hesitant, exhausted. | The robot assumes a service pose. |
| M02 | Pick: reach, grasp, and lift an object. | Anticipation of weight and adjustment after contact. | Near or far; either hand. | The robot lifts a luxury object. |
| M03 | Inspect: rotate the object and decide where it belongs. | Judgment as labor, even when motion data barely records it. | Quick glance or prolonged uncertainty. | Inspection looks like admiration. |
| M04 | Sort: place the object in one of two bins and return. | Repetition, asymmetry, and practiced efficiency. | Left or right; correct or mistaken. | The robot arranges a display. |
| M05 | Assemble: align and press two pieces together. | Coordination and material resistance. | Smooth fit or repeated attempts. | The robot appears to work effortlessly. |
| M06 | Reset: move the trays and clear my station. | The hidden work that enables the next cycle. | Routine, rushed, or paused. | The room returns to a pristine state. |
| M07 | Correct: retrieve an item and redo an action. | Error and repair as part of production. | Self-correction or external cue. | I show the motion with and without the correction. |
| M08 | Pause: freeze mid-reach, look toward a cue, resume. | Who controls my pace, and the effort of holding still. | Short or long hold; resume or restart. | The robot freezes in an elegant pose. |
| M09 | Accumulate: repeat M02–M06 for several minutes. | How a real body changes through repetition. | Comfortable, imposed fast, then slower. | Early and late takes appear side by side. |
| M10 | Refuse / exit: stop mid-cycle and leave. | What happens when I stop supplying motion. | Set the object down or walk away holding it. | The robot stops, loops, or remains posed. |
Capture units: I will record M01–M08 and M10 as short takes, starting and ending in a neutral stance. M09 will be one continuous take. I will log the props, bin positions, pace, repetitions, and prompts, and keep imperfect takes as part of the database.
Rehearsal and shot sequence
| Sequence | Physical view | Digital view | Purpose |
|---|---|---|---|
| 1. Establish | Wide shot of my body and station. | Wide shot of the house. | Establish the mismatch between spaces. |
| 2. Learn the loop | Side view of the full task. | Robot performs the corresponding motion. | Make the live connection clear. |
| 3. Show what capture misses | Close video of my fingers, grip, and corrections. | Skeleton or robot beside it. | Compare lived action with captured motion. |
| 4. Repeat | Fixed, uncut wide shot. | Early and current takes, or an archive of loops. | Let small bodily changes become visible. |
| 5. Stop | Empty station after I leave. | Robot’s behavior after I stop. | End on the question of whether the output continues without me. |
I will shoot the close views on a regular video camera, separate from OptiTrack. They can preserve touch, facial expression, and hesitation that may never reach the avatar.
Capture and preproduction
I will draw a floor plan, mark my standing point and reach limits, and rehearse with large, lightweight, matte props. I need an open station that keeps my markers visible, especially when I cross my arms or lean over the bins. I will test my worker uniform over the marker suit and avoid reflective fabric or shoes. OptiTrack camera placement · Skeleton tracking
I will prioritize full-body actions: reaching, lifting, turning, transferring, resetting, holding, and walking away. If precise finger movement matters, I will need separate hand tracking; otherwise, I will show it in video or animate the robot’s grip deliberately. Glove setup
My test day will include calibration, a slow loop, a normal loop, a long take, and an occlusion test. I will ask someone to watch me and the avatar together and note where they diverge. I may also seek feedback from a movement practitioner and someone familiar with repetitive work.
Preservation and erasure
I can preserve timing, posture, repetition, and correction. The avatar may erase touch, resistance, uncertainty, and the conditions of work. I want that gap to remain visible: a difficult assembly on video might sit beside a robot that completes it smoothly.
I will keep the raw and failed takes alongside the cleaned animation, and label motion as live, recorded, edited, or looped. If I use Xiaomi’s public RL display, I will identify it as a separate process. The question I want to leave open is: what survives when my labor becomes reusable motion, and what disappears?