The work,
in motion.
Two minutes inside the work: simulations, signals, particles, and language in motion.
English audio. Captions are available in all ten site languages; translations are machine-assisted.
Read the transcript
00:01 Hello everyone, I'm Michael, a senior at Kehillah High School.
00:05 This is my personal portfolio and I'm going to walk through some projects together with you.
00:10 It starts on a lake.
00:11 This is the school's Sailbot simulator running on a PC with the autopilot steering.
00:16 Underneath, the code works out the forces on the boat, moves it a tiny step and repeats.
00:21 Motion is just a state updating on a clock.
00:24 Next came a class investigation, shown here in its raw log.
00:27 If you write a graph as a grid and multiply it by itself, each number counts the routes between two points.
00:33 Sign-Translator's skeleton is that kind of graph.
00:37 The plain Fourier transform takes almost 9 seconds on 8,000 samples.
00:42 When you split the signal in half and recurse, the FFT takes just 26 milliseconds.
00:48 Later on, a spectrum like this revealed that Sign-Translator's generated head was shaking twice as hard as a real signer.
00:55 The next project was just for art, 75,000 particles pulled around by hand landmarks.
01:01 Those same landmarks later became one of the data paths in Sign-Translator.
01:06 All that curiosity led me to Sign-Translator.
01:09 Lines from two cameras meet at each joint, which gives its position in 3D.
01:14 Because bones don't change length, they took the focal length from over 40% wrong to within 1%.
01:20 The motion is then compressed into codes, like a vocabulary, and a transformer writes them from English.
01:27 Then nearly everything failed.
01:28 The whole sentence generators read no better than standing still.
01:32 One of them passed 95 out of 95 motion checks and still couldn't carry a sentence.
01:38 So now the system generates each word separately and stitches the clips together.
01:43 Words it doesn't know simply vanish.
01:45 To score it, a model reads the motion back into English.
01:49 Blank motion scores 2.05, the generated motion 2.4, and real signing about 4.
01:55 It's a narrow gain, but it's a start.
01:57 Thanks for watching.
Playback could not start. Try the video controls or download the video.
Behind the demonstrations.
The film shows experiments and work in progress. Explore the project pages for methods, context, and limitations.
More from my video channel.
YouTube