DeepFruits: A Fruit Detection System Using Deep Neural Networks

Introduction

This paper presents a novel approach to fruit detection using deep convolutional neural networks. The aim is to build an accurate, fast and reliable fruit detection system, which is a vital element of an autonomous agricultural robotic platform; it is a key element for fruit yield estimation and automated harvesting. Recent work in deep neural networks has led to the development of a state-of-the-art object detector termed Faster Region-based CNN (Faster R-CNN). We adapt this model, through transfer learning, for the task of fruit detection using imagery obtained from two modalities: colour (RGB) and Near-Infrared (NIR). Early and late fusion methods are explored for combining the multi-modal (RGB and NIR) information. This leads to a novel multi-modal Faster R-CNN model, which achieves state-of-the-art results compared to prior work with the F1 score, which takes into account both precision and recall performances improving from 0.807 to 0.838 for the detection of sweet pepper. In addition to improved accuracy, this approach is also much quicker to deploy for new fruits, as it requires bounding box annotation rather than pixel-level annotation (annotating bounding boxes is approximately an order of magnitude quicker to perform). The model is retrained to perform the detection of seven fruits, with the entire process taking four hours to annotate and train the new model per fruit.

Prior Works

• Automated Bell Pepper Harvesting using Robotic Vision System
  Silpa Ajith Kumar, J. S. Kumar, 2019
• Automatic Fruits Classification System Based on Deep Neural Network
  Khadija Munir, A. I. Umar, Waqas Yousaf, 2020
• Convolutional Neural Networks (CNN) for Detecting Fruit Information Using Machine Learning Techniques
  Fouzia Risdin, P. Mondal, Kazi Mahmudul Hassan, 2020
• Deep fruit detection in orchards
  Suchet Bargoti, J. Underwood, 2016
• DetSSeg: A Selective On-Field Pomegranate Segmentation Approach
  Shubham S. Mane, Prashant Bartakke, Tulshidas Bastewad, 2023
• Visual detection of occluded crop: For automated harvesting
  C. McCool, Inkyu Sa, Feras Dayoub, Christopher F. Lehnert, Tristan Perez, B. Upcroft, 2016
• Fruit Quantity and Ripeness Estimation Using a Robotic Vision System
  Michael Halstead, C. McCool, Simon Denman, Tristan Perez, C. Fookes, 2018
• Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards
  Suchet Bargoti, J. Underwood, 2016
• Deep Count: Fruit Counting Based on Deep Simulated Learning
  Maryam Rahnemoonfar, Clay Sheppard, 2017
• Building Efficient Fruit Detection Model
  Pavan N. Kunchur, V. Pandurangi, Madhu Hollikeri, 2019

How to read the graph

Each node is an academic paper related to the origin paper.

• Papers are arranged according to their similarity (this is not a citation tree)
• Node size is the number of citations
• Node color is the publishing year
• Similar papers have strong connecting lines and cluster together.