Key trends in computer vision and robotics for the agri-food sector

"By bringing together various disciplines such as computer vision, robotics, spectral imaging and Artificial Intelligence (AI) such as machine learning, we can develop smart integrated computer vision and robotics systems for the agri-food sector that benefit both people and planet," says Paul Goethals. He is business development manager at Vision+Robotics, a research program within Wageningen University & Research where these disciplines come together and work with industry on concrete applications. In this article, he explains the latest trends and developments in this field.

"The use of all kinds of sensors in agri and food has taken off and will continue to increase in the coming years," Goethals states. "In this regard, it is important that the large amounts of data generated by these sensors can be analyzed quickly and accurately. Computer vision is an example of this, using increasingly sophisticated cameras to determine the internal and external quality of agri and food products. Thanks to fast and accurate data models, supported by AI, you can arrive at objective management information to support operational decisions faster and better, among other things."

Hyperspectral imaging

One of the focus areas of the Vision+Robotics program is hyperspectral imaging. "Using special cameras, products are scanned, revealing changes in the internal molecular structure. This can add tremendous value to the food sector. For example, you can measure the sweetness of an apple without having to cut into it. And you can also closely monitor the moisture, protein or starch content of agri and food products. In the food industry, this allows you to adjust processes in production lines in real time as soon as certain deviations occur. Equipment is getting better and faster and models are becoming more powerful. AI and more specifically machine learning are helping to interpret and process images."

Flexible workstations

Another development is creating more flexibility in production lines. "Traditional production lines handle large quantities of the same product, but due to more diverse consumer demand, there is a trend toward smaller production batches with greater variation. This results in considerable changeover times in production lines. Flexible workstations can reduce this problem. Currently, flexible workstations are still in their infancy, but they will gain ground in the coming years. With flexible workstations, food processing companies can have robots automatically perform different operations, driven directly by the specifications of a given production batch."

Training

Being able to easily program robots is therefore also a research area within the Vision+Robotics program. "We expect that robots and cobots will be trained based on human behavior. Workers perform actions, a camera records everything and then the images are automatically analyzed and transferred to the robot that can work with them. The goal is that manual programming will no longer be needed. We are currently running a project investigating this. This has enormous potential. And if you can transfer operations to 1 robot, you can then easily roll it out further to other robots in your production line or even at other production sites."

Synthetic data

Data is becoming increasingly important. "The food sector is increasingly using digital twins, where production lines are simulated in a virtual environment. But you can take this a step further. We are currently doing research on how to digitize the movements of variable products such as vegetables, fruit or fish on a production line. How do the products fall out of a crate on a production line, how do they lay on top of each other and how do they make their way to the workstation? If you can digitize this too, you can simulate the entire process in the computer in advance and train actions of robots accordingly. A lot is already possible in this field with regard to logistical processing, such as packing and moving boxes with a fixed shape. We are now in the process of moving to a variable product so that robots can also grip this type of product more easily with grippers."

Grippers

Grippers are and will continue to be important. "There are many developments going on here as well. Recently, it was unthinkable to grab delicate products such as berries or strawberries with a robot since they are easily damaged. More and more soft-grippers are coming on the market, which are pneumatically driven, made of soft material and equipped with the necessary sensors, among other things. This makes it possible to set the right sensitivity and ensure that the products are not damaged. These types of grippers are currently taking the market by storm and we are also using them in our robot development."

Combinations

Eventually, several technologies will converge. "For example, think of flexible robot stations with computer vision that can quickly change advanced grippers and be programmed based on imitation of human behavior. This is only possible with a lot of data about the robot's environment. Future robotics is therefore highly data-driven. A fast powerful data infrastructure is therefore also necessary to realize this. Fortunately, the necessary steps forward are also being made in this area."