This Robot Camera Can Film ‘How-To’ Tutorial Videos

Researchers from the University of Toronto have created an intuitive robot camera designed to film “How-To” tutorial videos for YouTubers and video creators.

Dubbed “Stargazer,” the interactive camera follows implicit and explicit cues from the person in front of the camera allowing creators who don’t have access to a film crew to create more dynamic videos than if they were just filming on a static camera.

“The robot is there to help humans, but not to replace humans,” explains lead researcher Jiannan Li, a Ph.D. candidate in the University of Toronto’s Department of Computer Science. “The instructors are here to teach. The robot’s role is to help with filming — the heavy-lifting work.”

Stargazer robot
Computer science Ph.D. candidate Jiannan Li with the robot camera Stargazer | Photo by Matt Hintsa

In a published paper, researchers explain that Stargazer uses a single camera on a robot arm armed with seven independent motors that allows it to track regions of interest.

The system’s camera behaviors can be adjusted based on subtle cues from instructors, such as body movements, gestures, and speech that are detected by the prototype’s sensors.

The instructor’s voice is recorded with a wireless microphone and sent to Microsoft Azure Speech-to-Text, a speech-recognition software. The transcribed text, along with a custom prompt, is then sent to the GPT-3 program, a large language model which labels the instructor’s intention for the camera.

Robot camera filming a tutorial video
The robot camera films a tutorial video on skateboard maintenance.

The researchers say that the video-makers need not worry about guiding the robot as the clever machine can interpret natural speech.

“For example, the instructor can have Stargazer adjust its view to look at each of the tools they will be using during a tutorial by pointing to each one, prompting the camera to pan around,” Krystle Hewitt explains in a press release.

“The instructor can also say to viewers, ‘If you look at how I put A into B from the top,’ Stargazer will respond by framing the action with a high angle to give the audience a better view.”

“The goal is to have the robot understand in real time what kind of shot the instructor wants,” adds Li.

“The important part of this goal is that we want these vocabularies to be non-disruptive. It should feel like they fit into the tutorial.”

Stargazer robot filming a tutorial
An instructor using Stargazer to film a video on how to build a 3D-lamp.

The researchers invited six instructors to test the robot camera, including a skateboard maintenance tutorial and interactive sculpture-making.

According to the researchers, they all completed their tutorials within two takes without needing any additional controls, and each was satisfied with the quality of the video.

The team is hoping to explore putting the technology into other forms of cameras, such as drones and robots with wheels to help film in different environments other than a tabletop.

Future research could also investigate methods to detect more diverse and subtle triggers from the tutors by combining simultaneous signals from an instructor’s gaze, posture, and speech.

Image credits:Photos courtesy of the University of Toronto.