Real Steel: Real teleoperation (plus video review)

What it gets right about robots: teleoperation

Recommendation: Watch it! It’s a family friendly mash up of Rocky and absentee father bonding with son tropes with lots of great actors and good effects.

Did you miss the movie Real Steel when it came out in 2011? If you did, you’re not to blame. It certainly looked a bit lame… sort of rock ‘em, sock ‘em robots with a mash-up of the estranged father-son and Rocky underdog tropes. Sure it had Hugh Jackman and Evangeline Lily but it seemed so… formulaic.

Except it was actually pretty good and it gets extra points as a good illustration of teleoperation, an important concept in robotics.

The movie was based on the 1956 short story Steel by Richard Matheson (yep, the I Am Legend link to movie Richard Matheson). It was directed by Shawn Levy (the director for Night at the Museum series and some of the great episodes of Stranger Things) and while the screenplay was a bit muddled (where was Carrie Fisher when you needed her to be a script doctor?), the acting and the special effects carry it past the goal line.

The basic plot in terms of robotics is that there is a huge World Wide Wrestling Federation type of fighting event, only with tricked out robots, not people. The robots are teleoperated, which gives the plot its main robot theme: the robot reflects or amplifies the personality and cognitive style of the operator. You’re a feisty boy? Your robot embodies that feistiness in the fighting ring. And, as we all know from Rocky, all you need is heart and a lot of hard work to get some respect. Watching it may make you wish for Scalzi’s imagined teleoperated team sport of Hilketa in his book Head On, but the Real Steel robots are a good stopgap.

The plot of Real Steel isn’t particularly original and teleoperating robots isn’t either, but that’s a good thing. Teleoperation has been around since the dawn of robotics during WWII for nuclear handling. Teleoperation means that someone is operating (operation) the robot remotely (teleo).

Historically, roboticists have viewed teleoperation as a degenerate case of autonomy- you couldn’t make the robot autonomous so you had to settle for it being operated by a human. That certainly can be true for some applications, but other applications actually don’t want autonomy. Let’s think about telehealth for COVID. You aren’t trying to automate all the things a doctor or nurse does, you’re enabling the doctor or nurse to work with infected patients from that nice clean room next door. Or telepresence- you’re using the robot to talk to granny in the nursing home. For work, you can use a robot to roll through the factory floor or if you’re a real estate agent, let clients see a house. For my field of search and rescue, responders want to see and act in distal environments in real-time. Not press a button, hope the robot comes back in four hours and says it found a person bleeding out that had 2 hours to live at the time the robot found them.

Roboticists are now grudgingly acknowledging that teleoperation might be OK. What many roboticists miss is just how hard teleoperation is. As you watch the movie, notice that each robot team uses a different user interface. Clearly a good user interface makes it easier for the operator to control the robot. It also helps, or thwarts, the operator in seeing the “bigger picture”- what is the condition of the robot, what are the blind spots, and so on. That is called situation awareness in the cognitive literature. It’s not trivial to get the interface right.

And it’s not just an interface, it’s interaction. In the movie, you see the teams try to delegate more of the routine moves to the robot as a macro or automated script to speed up execution. No matter how good the interface, there is a delay caused by the operator comprehending the situation, reacting and sending control signals via the user interface, and the robot actually executing those commands. It’d be great to eliminate that delay. If you are trying to teleoperate a robot in space, there’s a communication lag on top of that- Tom Sheridan was a pioneer in that. Plus teleoperation is tiring; the mental effort of working through a computer is like working with a low-grade fever: You might not notice that you’re not 100% but you find that even regular activities exhaust you. You want the interface to be more than just a dashboard, so we call that different, better way of getting things done “interaction.”

You’ve probably read about “Zoom fatigue”, that teleconferencing is more tiring because of the extra cognitive work in working through a computer interface. Well, it is way worse controlling robots. My colleagues and I have looked at this for nearly two decades since robots for disasters are teleoperated and our rubric is: 20 minutes of teleoperating a robot at a disaster is the fatigue equivalent of 2 hours of mental work. My colleagues Drs. Ranjana Mehta and Camille Peres found that pilots flying small unmanned aerial systems at disasters after just 1 day show the cognitive impairment of 0.05% blood alcohol concentration - and these are expert pilots who fly disasters!

The point is: you want to see and work through the robot but you want the robot to be smart enough that you can delegate as much as you can to the robot in order to speed up execution and reduce your fatigue.

Ah- but at what point does the robot move from being teleoperated, which seems unintelligent, to being intelligent? Plus being intelligent in an assistant way is very different in terms of programming from being intelligent in a Jarvis Ironman artificial general intelligence way.

We roboticists don’t have a precise line that once the robot design crosses over it, it no longer just teleoperated. But we do have a name for it: we call this “shared autonomy.” Shared autonomy may not be as hard as creating an artificial general intelligence, like Commander Data in Picard or Ava in Ex Machina, but it’s pretty hard. You have to enable the person and robot to work seamlessly together. Be honest, most of us haven’t figured out how to have people work with other people seamlessly.

You can read more about teleoperation in Chapter 5 of my textbook Introduction to AI Robotics or Chapter 1 in Robotics Through Science Fiction, and our RTSF Topics page on Teleoperation has links to other movies, books, and stories— Sanjeev and Robotwallah is one of my favorites.

In the meantime, watch Real Steel. It’d be great to watch on Father’s Day- if you’re a dad, you might not be as good looking as Hugh Jackman’s character but you’re definitely a better father. If you’re a boy, you’ll love the fight scenes. And if you are the rest of the family, you will be charmed, perhaps reluctantly but charmed nonetheless. And you can count it as an educational experience on teleoperation.

Watch the video review here:

Or buy Real Steel from Amazon here.