“I think that I probably as guilty as everyone else,” admits Max Bajracharya, senior vice president of robotics at the Toyota Research Institute (TRI). “It’s like, now our GPUs are better. Oh we have machine learning and now you know we can do this. Oh okay, maybe that was harder than we thought.”
Ambition is, of course, an important aspect of this job. But there is also a great and inevitable tradition of relearning mistakes. The smartest people in the room can tell you a million times why a specific problem hasn’t been solved, but it’s still easy to convince yourself that this time, with the right people and the right tools, things will be different.
For TRI’s in-house robotics team, the impossible task is home. The lack of success in the category has not been for lack of trying. Generations of robotics specialists have agreed that there are many problems waiting to be automated, but so far, successes have been limited. Beyond the robotic vacuum, there has been little progress.
For a long time, the TRI robotics team has focused on the home. That’s in large part because it chose elderly care as a “north star” for the same reason that Japanese companies are so far ahead of the rest of the world in the category. Japan has the world’s highest proportion of citizens aged 65 and over, behind only Monaco, a microstate in Western Europe with a population of less than 40,000.
In a world where our health and well-being are so closely tied to our ability to work, it is a problem bordering on crisis. It’s the kind of thing that makes Yale assistant professors New York Times headlines for suggesting a mass suicide. That’s obviously the most sensational of the “fixes”, but it’s still a problem in search of a meaningful solution. As such, many Japanese robotics specialists have turned to robotics and automation to address issues like home healthcare, food preparation, and even loneliness.
Image Credits: brian heater
The first professionally produced videos featured robotics in the home, performing complex tasks such as cooking and cleaning a wide range of surfaces. When TRI opened the doors of its South Bay labs to selected press this week to showcase a range of its different projects, the homey element was conspicuously absent. Bajracharya showed a couple of robots. The first was a modified off-the-shelf arm that moved boxes from a stack to nearby conveyor belts, in a demo designed to unload trucks, one of the most difficult tasks to automate in an industrial warehouse environment.
The second was a robot on wheels that goes shopping. Unlike the warehouse example, which had stock parts with a modified gripper, this system was designed in-house largely out of necessity. The robot is dispatched to retrieve different products on the shelf based on barcodes and general location. The system can be extended to the top shelf to find items, before determining the best method to grab the wide range of different objects and place them in your basket. The system is a consequence of the team’s pivot away from home-specific robots.
Image Credits: brian heater
Next to both robots is a simulated kitchen, with a gantry system set up on top of its walls. An almost humanoid robot hangs, motionless and lifeless. It’s not recognizable for the duration of the demos, but the system will look familiar to anyone who’s seen the team’s early concept videos.
“Home is very hard,” says Bajracharya. “We choose challenging tasks because they are difficult. The problem with the house is not that it was too hard. It was that it was too difficult to measure the progress we were making. We tried many things. We tried to make a mess procedurally. We put flour and rice on the tables and tried to clean them. We put things all over the house to keep the robot tidy. We were deploying to Airbnbs to see how well we were doing, but the problem is that we couldn’t get the same home every time. But if we did, we would fit too much into that house.”
Moving to the supermarket was an effort to address a more structured environment while also addressing a pressing issue for the senior community. Testing the product, the team moved from Airbnbs to a local family-owned grocery store.
Image Credits: brian heater
“To be totally honest, the challenge problem doesn’t matter,” explains Bajracharya. “The DARPA robotics challenges were made-up tasks that were difficult. That is also true for our challenge tasks. We like the home because it is representative of where we eventually want to be helping people at home. But it doesn’t have to be home. The grocery market is a very good representation because it has great diversity.”
In this case, some of the learnings presented in this scenario translate to the broader needs of Toyota.
What, precisely, constitutes progress for a team of this nature is a difficult question to answer. However, it is one of the most important, as large corporations have begun to cut roles in long-running research projects that have yet to deliver tangible, monetizable results. When I asked Gill Pratt the question yesterday, the head of TRI told me:
Toyota is a company that has worked hard so that employment does not follow the economic cycle. The auto business is one that booms and busts all the time. You may know the history of Toyota is to try not to fire people when times are tough, but to go through a thing or two. One is shared sacrifice, where people join the cause. The second is to take advantage of those times to invest in maintenance, plans and education to help people train.
Image Credits: brian heater
Toyota is known in the industry for its “no layoffs” policy. It’s an admirable goal, to be sure, especially as companies like Google and Amazon are in the midst of layoffs numbering in the tens of thousands. But when the goals are more abstract, as is the case with TRI and other research wings, how does a company measure the relevant milestones?
“We were making progress at home, but not as quickly or as clearly as when we moved to the grocery store,” the executive explains. “When we move into the grocery store, it really becomes very apparent how well you’re doing and what the real issues are in your system. And then you can really focus on solving those problems. When we toured Toyota’s manufacturing and logistics facilities, we saw all these opportunities where they’re basically the challenge of grocery shopping, except they’re a little different. Now, instead of the parts being grocery items, the parts are all the parts in a distribution center.”
As is the nature of research projects, Bajracharya adds, sometimes the beneficial results are unexpected: “The projects are still looking at how we ultimately amplify people in their homes. But over time, as we choose these challenging tasks, if things come up that are applicable to these other areas, that’s where we’re using these short-term milestones to show the progress in the research that we’re making.”
The path to producing such breakthroughs can also be confusing at times.
“I think we now understand the landscape”, Bajracharya. “Maybe I was naive at first to think that, okay, we just need to find this person that we’re going to pass the technology on to a third party or someone within Toyota. But I think what we’ve learned is that whatever it is, whether it’s a business unit, a company, a startup or a unit within Toyota, they don’t seem to exist.”
Startups, similar to what Alphabet has done with its X Labs, is certainly on the table, though it’s not likely to be the main path to production. However, it is not yet clear what form that path will ultimately take. Although robotics as a category is currently much more viable than when TRI was founded in 2017.
“Over the last five years, I feel like we’ve made enough progress on this very challenging problem that we’re now starting to see it develop into these real-world applications,” says Bajracharya. “We have consciously changed. We’re still pushing 80% of the state of the art with research, but now we’ve allocated maybe 20% of our resources to find out if that research is as good as we think it is and if it can be applied to reality. -global applications. We could fail. We might find that we think we’ve made some cool strides, but it’s just not reliable or fast enough. But we are putting 20% of our effort into trying.”
