Interview with Dr. Stan Schneider, CEO of Real-Time Innovations, April 2023
Stan, could you introduce yourself and roles at the Teleoperation Consortium as well as RTI?
I’m CEO of Real-Time Innovations (RTI). RTI is the world’s largest architectural software provider for smart distributed systems. RTI software runs nearly 2000 projects, all of them making real-world “things” run smarter. It does that by connecting devices like sensors and actuators (motors) with smart algorithms and remote operation control centers.
We run some large systems, like the largest power plant in North America (Grand Coulee dam), Siemens wind turbine farms, the Canadian Air Traffic Control system, NASA's launch control system, nearly all US Navy ships, and GE Healthcare's hospital device networks. Most of our business is smaller systems, but all are mission critical. They include many surgical robots, trains and metro control systems, and over 250 autonomous vehicle designs.
Most of these systems have some remote operational control or oversight, so in 2021 I joined the board of the Teloperation Consortium (TC). The Teleoperations Consortium is studying the problem of how to control or direct “real world” systems from afar. I’m also on the board of the Autonomous Vehicle Computing Consortium (AVCC) and the advisory board for the IoT Solutions World Congress. I was on the Industrial IoT Consortium (IIC) Steering Committee for 6 years, including as Vice Chair.
Before founding RTI, I managed a large Stanford University robotics laboratory, led a systems software team for personal computers, and developed software for telecommunications equipment. I started out in the automotive industry, working on safety systems, impact testing, occupant protection, and data acquisition.
I did a PhD in EE/CS at Stanford in autonomous systems at the Stanford Aerospace Robotics Laboratory.
Autonomous vehicles have yet to really take off with 5G. Do you agree with this and, if so, why do you think that is? What should/could have been done differently to improve adoption of 5G for autonomous vehicles and larger scale roll-outs?
Fundamentally, there are challenges before 5G can become the backbone of human-transport autonomy. A few months ago, I wrote a blog called “Autonomy is like Happiness”. It’s a good analogy here: there is always room for more happiness. And, there will always be opportunities for more autonomy. You can’t be “fully autonomous” any more than you can be “fully happy”. Also, the environment really matters. You can be happy at home but not at work. Your car can work well in autonomous mode on a controlled freeway but is incapable of navigating a chaotic parking lot.
So for an autonomous vehicle to benefit from 5G capabilities, it will need dedicated infrastructure. There are several reasons to enable 5G-based advanced driver assistance systems (ADAS), the most relevant of which is cost efficiency. The real challenge of road autonomy is increasing levels of autonomous operations in complex environments. You can’t delegate driving via remote technology in most situations, because safety, among other things, should be handled locally. Most challenges in autonomy don’t require 5ms latency. So, 5G doesn’t directly help autonomous vehicles, except perhaps within a private network.
Vehicle communications are still very important, though. As recently demonstrated by Cruise vehicles stranded by felled trees in San Francisco, autonomous vehicles can get stuck. For the foreseeable future, these systems need external support now and then (just like happiness!). The role of telecommunications in these systems is to provide some level of sensor “telepresence” as well as some ability to control the system. That doesn’t mean remote driving, but it does require remote instructions to the vehicle. Better connections for this helps, but it’s not usually a compelling use case for 5G over 4G technologies.
There are benefits for remote driving in applications like Agriculture, Construction, and Mining (ACM), where the environment is both controlled (no pedestrians) and hostile (remote location, dirty, etc.). There are also applications with less dangerous settings, like smaller, slower delivery robots.
Beyond vehicles, you are also involved in automation in multiple other sectors, from medical robotics to defense and beyond. How much impact has 5G made in these sectors, regarding automation?
RTI customers are running distributed applications in all of these sectors. Today’s medical robotic systems incorporate live operation or monitoring by a local surgeon. However, that is changing quickly due to the promise of teleoperated systems, where remote operations can virtually bring trained surgeons into areas where local expertise is not available. These applications will need 5G and 6G high-bandwidth, low-latency control loops.
RTI is also working on many other challenges of anytime, anywhere communications. One of the most interesting is how to handle dynamic local interactions. The discovery capabilities of potential conflicting vehicles, for instance, is really hard at the metro-area level. In a city, there are thousands of cars to check for an impending collision at any given time. However, by putting some of the discovery logic into the communication tower, the region of interest is much smaller, and so the set of possible interacting vehicles is much smaller. These applications still face standardization and adoption hurdles. Ad-hoc smaller networks also have many uses that don’t require such wide spread, including emergency response to a disaster like a tornado, oil-spill cleanup, forward-deployed battlefields, and more.
So why does this need 5G? The 5G spec is a lot more than a bigger, better wireless pipe. 5G SA (Stand Alone, as compared to NSA: non-stand alone) allows for virtualized everything in the 5G Core. It allows for the deployment of services that exist at the edge. One of those services could be Ad-Hoc Mobile Discovery.
5G also provides network guarantees in the form of 5G QoS (Quality of Service) that guarantees performance on several dimensions. These pair well with the QoS parameters found in the Data Distribution Service (DDS) middleware standard. Putting these together could enable run time requests, negotiation and adaptability as the wireless environment changes. In the longer term, multiple layers of software and communications can make 5G far more usable.
One thing holding 5G back is that most people think that the bigger, better pipe already exists. It runs in NSA mode and is publicly deployed. But most of these features are far from deployed, nor easily deployable, on public infrastructure.
What role will 6G play in autonomous vehicles? Would you expect it to be the gamechanger technology that will live up to what 5G didn’t, and why?
6G may be a game-changer, but that depends on the game. Autonomous public-road vehicles need to handle the real-world unpredictability. Safety is the biggest challenge, but adding faster communications, whether 5G, 6G, or eventually 10G won’t provide that; safety should be assured under local control. Most of the intelligence and data processing has to be local to the vehicle. Fortunately, despite the ups and downs of the press, the technology to at least exceed the distressing record of human drivers is not that far away. But 6G isn’t the game changer in safety.
What off-board telecommunications can do is provide ancillary benefits, such as offering help to disabled or stuck vehicles, connecting fleets that are in constant motion, and recording data for later analysis. So, if the safety game is not changeable through wireless connectivity, helping autonomous vehicles navigate all the unforeseeable conditions can greatly accelerate adoption and use cases. Teleoperation is a real game changer for practical adoption in the real world.
Other domains also have use cases for 6G. Healthcare, mining, agriculture, defense and many more industries face challenges that are more easily handled remotely.
You will be delivering a presentation on The software defined future at Big 5G’s 6G Summit. Could you give us a sneak peak of what your speech will address?
AI can and will run cars, traffic control, urban air mobility, trains, renewable energy, hospital devices, surgical robots, naval systems, air defense, avionics, simulation and training. It will make the entire planet run better. But it’s not that simple. Today’s AI systems (like ChatGPT) run in the cloud. And cloud AI by itself can’t change the way the world actually runs.
To work in the real world, AI needs a connection to the sensors and actuators (motors) that let it sense and act on things. And the connection must be reliable enough to trust, fast enough to act, scalable enough to handle huge distributed infrastructure, and conceptually clean enough to organize thousands or millions of components. Intelligent real-world systems need to interface a lot of software in ways that are fundamentally different from all the other software systems on the planet.
The era of evolving artificial intelligence in the real world is more aptly called the “software defined” world, because although the explosion of hardware capability enabled it, AI is software that increasingly defines where it can be used. And the most important characteristic of that software is how it controls and processes dataflow. Getting the right data to the right place at the right time enables edge intelligence.
My talk will go over these challenges, look at a few current real-world use cases, and explore ways the evolving wireless technology can play in the software-defined future.