Calling all web developers, AI and machine vision specialists, coders and hardware and electronic fanatics we need your help to develop a robot picker to fill the labour gap in the agriculture sector.

The problem

The shortage of labour available to the agricultural sector has led to crop losses and increased costs in crop production requiring selective harvesting.

The proposed solution

A robot picker to act as a one-to-one replacement for human pickers in the field. The key features of the robot will be its ability to drive itself along a row of crops, identify target crops which are ready to be harvested, position itself over the target crops, and successfully pick and store the crop ready for transport to a sorting centre.

Our progress so far

We have developed the mechanical structure of a machine which can be controlled programmatically to drive (steering angle and speed) which has an onboard gantry system for moving a picking head. Two cameras, one for navigation and one for crop identification, are fitted to the robot which are to be used as the main inputs to the system.

Key areas for development during this hackathon

• Web dev: the robot should operate free from cables and in relatively rough environments. When in operation, the robot will have access to the mobile data network. Therefore, a web-based interface is needed to interact with the robot. An operator at a separate location will need to be able to know the status of the robot, which could be through images from the camera, status information showing the state of the robot, and more. The operator will need to be able to send commands, such as driving instructions, back to the robot.
• Artificial intelligence/ machine learning: identifying which crops are ready to be harvested can be a difficult problem as lighting and weather conditions often change in the field. Machine learning may provide the answer to a reliable detection system which is tolerant of these changing conditions. Cameras and other sensors may be used to build a dataset which can be trained for this task.
• Machine vision: a camera is fitted exclusively to help the robot navigate along a row of crops and avoid obstacles. Extracting the optimal driving instructions from this camera feed will lead to a reliable self-driving system which will increase the safety and efficiency of the robot while positioning itself over the crop.
• General python coding skills/ system integration: all this needs to be linked together to form a comprehensive control system for the robot, which is a difficult task in itself. Anyone with skill in python programming language or a willingness to learn is invited to support this effort.
• Hardware and electronics: efforts have been made to ensure the hardware is fully capable of solving the problem at hand, but participants will have access to electronics equipment and components along with 3D printers and are encourages to make improvements to the hardware over the course of the event if desired.