Can Vision and Artificial Intelligence Make Every Robot Collaborative?

June 7, 2019 6:31 am

That is the aim of this Boston-area startup. This year, it expects to come to market with technology to make even fast and powerful industrial robots safe to approach. The technology promises to eliminate the need for guarding around them — safety measures that might not be as safe as you think.

Originally titled ‘Unguarded Automation’

Article From: 6/2/2019 Modern Machine Shop, Peter Zelinski, Editor-in-Chief

Automated manufacturing facilities are full of physical barriers — guarding and fences — because of our fears.

That is, because of our entirely legitimate fears. Automated systems, industrial robots in particular, are capable of moving fast and unexpectedly, imperiling a human who comes too close. A fence is a reasonable precaution. However, Patrick Sobalvarro points out that fences are far from fully effective as safety measures. People are quick and clever, and often too effective at finding workarounds for physical barriers. He looks forward to the day when people can work safely alongside even powerful robots without fences because vision technology and artificial intelligence (AI) provide for greater safety than a physical barrier can deliver.

And the day when this is possible might arrive yet this year.

No employees were harmed in the making of this photo. New technology using vision and AI can make any robot collaborative, its developers say, eliminating the need for guarding around automation. Relevant robot safety standards already address it. Veo Robotics’ Gwenn Ellerby stood close to the robot for this photo.

Dr. Sobalvarro is one of the cofounders of Cambridge, Massachusetts-based Veo Robotics, a company that has been working to develop a collaborative robot or “cobot” system based not on force and speed limitation—typical of cobots in use today—but instead on speed and separation monitoring achieved reliably using AI and advanced vision technology. Veo’s “Freemove” system can make any robot collaborative, he says, providing the robot with the ability to operate close to a person without threat to that person’s safety. In fact, during my visit to Veo’s headquarters, I saw demonstrations involving standard industrial robots from brands I recognize. Existing robot safety standards, including ISO 10218, ISO/TS R1506 and IEC 61508, all recognize this system’s approach to realizing operator safety, and Veo expects to have a commercial version of Freemove for production facilities on the market by the end of the year.

The system answers what will soon be recognized as a growing need for collaborative automation, he says, a need the most commonly used cobots available today cannot address alone.

Collaborative automation “gives you the option to have a human in the cycle,” he says, and this option—as opposed to full automation—is becoming more valuable. Various trends in manufacturing make full automation increasingly problematic, including mass customization, shorter product cycles and tightening quality demands. “For a growing number of manufactured products, you will never amortize the cost of full, dedicated automation before you have to change the process,” he says. Rather, the option that makes economic sense in these cases is partial, redeployable automation safely operating alongside a human being. Collaborative automation is the way to achieve this.

Yet cobots typically realize this safety in part by operating at speed and force levels below what might cause harm, thereby limiting the maximum payload to perhaps 10 kilograms including the cobot’s end effector. Dr. Sobalvarro points out that only a sliver of industrial robot applications involve loads this light. Veo’s system aims to allow robots that are fast and powerful enough to seriously injure a human to be controlled so as to become collaborative as well.

The vision technology enabling this possibility arrived only within the last five years, he says. One of its earliest uses was video games. Safety is no game, of course, but the motion-tracking vision capability used by Veo is an industrial, longer-range version of the kind of motion sensing used by the Xbox Kinect game accessory.

The Freemove system uses machine vision to locate objects it recognizes, such as the robot, as well as occlusions that might represent the presence of a human. The system responds to the latter by slowing or stopping the robot to avoid potential harm.

Observing Occlusions

Dr. Sobalvarro explains how the system relying on this sensing technology works. Veo’s collaborative automation employs four or more vision cameras mounted high above the robot that have overlapping view fields. Each camera repeatedly sends infrared flashes to map every object in its field of view by means of the IR reflection. The resolution of the cameras developed by Veo is fine enough to distinguish a human finger at 10 meters. Thirty times per second, the collaborative system maps the vicinity of the robot by combining all the cameras’ data. Within this combined field, he says, the system is looking for occlusions.

That is, the system is looking for every volume of space that it cannot see because something is blocking its view. Any occlusion a little larger than a loaf of bread, or bigger, is assumed to be able to contain a human (meaningpotentially a very small human), and analysis of these occlusions is used to detect safety concerns. Phase transitions logged through the 30 samples per second track the progress of any occlusion. If the course of movement of an occlusion can provide clearance for a human to have entered the work zone and come within range of the robot, then the space is considered “infected” by the occlusion and the system responds accordingly. From this occlusion’s current position, the Freemove software then compares future states based on possible moves of the robot and possible moves of the suspected human at a presumed (fast) speed of 2 meters per second. If the event horizons of these two actors lead to any possible state in which the human comes in contact with the robot, then the robot is slowed down or brought to a stop.

Patrick Sobalvarro (background) discussed the vision-based collaborative technology with me, as did Veo co-founder Clara Vu, speaking with me here.

Crucially, the system employs AI, but not machine learning, Dr. Sobalvarro says. Machine learning is probabilistic, where the relevant safety standards make clear no statistical approaches for triggering safety measures are allowed. “Even very good machine learning could have one fail in 10,000, which is too large where human safety in a manufacturing environment is concerned,” he says. The AI instead entails rapid computation for occlusion analysis combined with classification according to model recognition.

The latter component, classification, makes the occlusion analysis more efficient. From the vision data, the system is equipped to identify the robot, fixturing and workpiece, ruling them out as suspect occlusions. This classification also helps make the system quick to install—a valuable side benefit. No precise locating of cameras is required, because once the cameras are mounted, the system can “zero” itself by locating the base of the robot.

Human Motives

Why is this artificial intelligence system safer than a fence? Because fences are vulnerable to human intelligence, Dr. Sobalvarro says. Humans follow conflicting motives that put them in danger. “The career of a person working in manufacturing involves a lot of training in safety, but it involves even more training in how important it is to keep production moving,” he says. In an instant, these two priorities can come in conflict. The worker who dropped an item that fell behind the safety fence might resist interrupting production just to retrieve that item. That worker might see how to compromise or move over or under the fence—a move made in an instant that might prove injurious or deadly. By contrast, there is no similar way to defeat the vision and AI system with just a single moment’s poor impulse. If the worker tried to do so by covering the camera, then this would produce an occlusion that would lead to a response from the system. With the vision and AI system, there just isn’t a fence that could be jumped.

I broke away and tried to catch the robot by surprise myself. It was moving at normal speed before this picture was taken, but quickly slowed to a stop as I drew this close.

There have been non-contact systems before for speed and separation monitoring of robots. Light curtains, for example. However, devices such as these can only be conservative, keeping the barrier of triggering far away from the robot’s reach. They don’t allow for collaboration, for the human and robot easily working together in close proximity so that the automation does not have to accomplish every detail.

Indeed, Dr. Sobalvarro notes there is nothing about the Veo system that suits it specifically to a robot. The ultimate hope of the company is that any automated industrial machinery might be governed by a system such as this, producing unguarded factory floors through which people can move freely, in safety and without fear.

Article From: 6/2/2019 Modern Machine Shop, Peter Zelinski, Editor-in-Chief

MMSOnline Blog

Click Here for a free subscription to Modern Machine Shop magazine.
Click Here to subscribe to the MMS Extra monthly e-newsletter.

This article is reprinted with permission from Modern Machine Shop Magazine, and is protected under U.S. and international copyright laws. Modern Machine Shop Magazine is protected under U.S. and international copyright laws. Before reproducing anything from this Web site, call Gardner Business Media, Inc. (513) 527-8800.

Leave a Reply