Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Intelligent Service Robotics
Published in: Intelligent Service Robotics 2/2022

01-12-2021 | Original Research Paper

Pepper to fall: a perception method for sweet pepper robotic harvesting

Authors: Marsela Polic, Jelena Tabak, Matko Orsag

Published in: Intelligent Service Robotics | Issue 2/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In this paper we propose a robotic system for picking peppers in a structured robotic greenhouse environment. A commercially available robotic manipulator is equipped with an RGB-D camera used to detect a correct pose to grasp peppers. The detection algorithm uses the state-of-the-art pretrained CNN architecture. The system was trained using transfer learning on a synthetic dataset made with a 3D modeling software, Blender. Point cloud data are used to detect the pepper’s 6DOF pose through geometric model fitting, which is used to plan the manipulator motion. On top of that, a state machine is derived to control the system workflow. We report the results of a series of experiments conducted to test the precision and the robustness of detection, as well as the success rate of the harvesting procedure.
previous article Feedforward operational stiffness modulation and external force estimation of planar robots equipped with variable stiffness actuators
next article Deep latent-space sequential skill chaining from incomplete demonstrations

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
2.
Bac CW, van Henten EJ, Hemming J, Edan Y (2014) Harvesting robots for high-value crops: state-of-the-art review and challenges ahead. J Field Robot 31(6):888–911CrossRef
3.
Barth R, Hemming J, van Henten EJ (2016) Design of an eye-in-hand sensing and servo control framework for harvesting robotics in dense vegetation. Biosys Eng 146:71–84CrossRef
4.
Fu L, Gao F, Wu J, Li R, Karkee M, Zhang Q (2020) Application of consumer rgb-d cameras for fruit detection and localization in field: a critical review. Comput Electron Agric 177:105687CrossRef
5.
Lin G, Tang Y, Zou X, Xiong J, Fang Y (2020) Color-, depth-, and shape-based 3d fruit detection. Precision Agric 21(1):1–17CrossRef
6.
Tu S, Pang J, Liu H, Zhuang N, Chen Y, Zheng C, Wan H, Xue Y (2020) Passion fruit detection and counting based on multiple scale faster r-cnn using rgb-d images. Precision Agric 21(5):1072–1091CrossRef
7.
Wang Z, Walsh KB, Verma B (2017) On-tree mango fruit size estimation using rgb-d images. Sensors 17(12):2738CrossRef
8.
Zhang J, Karkee M, Zhang Q, Zhang X, Yaqoob M, Fu L, Wang S (2020) Multi-class object detection using faster r-cnn and estimation of shaking locations for automated shake-and-catch apple harvesting. Comput Electron Agric 173:105384CrossRef
9.
Nguyen TT, Vandevoorde K, Wouters N, Kayacan E, De Baerdemaeker JG, Saeys W (2016) Detection of red and bicoloured apples on tree with an rgb-d camera. Biosys Eng 146:33–44CrossRef
10.
Perez RM, Cheein FA, Rosell-Polo JR (2017) Flexible system of multiple rgb-d sensors for measuring and classifying fruits in agri-food industry. Comput Electron Agric 139:231–242CrossRef
11.
Andujar D, Ribeiro A, Fernández-Quintanilla C, Dorado J (2016) Using depth cameras to extract structural parameters to assess the growth state and yield of cauliflower crops. Comput Electron Agric 122:67–73CrossRef
12.
Milella A, Marani R, Petitti A, Reina G (2019) In-field high throughput grapevine phenotyping with a consumer-grade depth camera. Comput Electron Agric 156:293–306CrossRef
13.
Lehnert C, English A, McCool C, Tow AW, Perez T (2017) Autonomous sweet pepper harvesting for protected cropping systems. IEEE Robot Autom Lett 2(2):872–879CrossRef
14.
Kang H, Chen C (2020) Fruit detection, segmentation and 3d visualisation of environments in apple orchards. Comput Electron Agric 171:105302CrossRef
15.
Arad B, Kurtser P, Barnea E, Harel B, Edan Y, Ben-Shahar O (2019) Controlled lighting and illumination-independent target detection for real-time cost-efficient applications. the case study of sweet pepper robotic harvesting. Sensors,19(6): 1390
16.
Lehnert C, Sa I, McCool C, Upcroft B, Perez T (2016) Sweet pepper pose detection and grasping for automated crop harvesting. In: 2016 IEEE international conference on robotics and automation (ICRA), pp 2428–2434, IEEE
17.
Sa I, Lehnert C, English A, McCool C, Dayoub F, Upcroft B, Perez T (2017) Peduncle detection of sweet pepper for autonomous crop harvesting-combined color and 3-d information. IEEE Robot Autom Lett 2(2):765–772CrossRef
18.
Hinterstoisser S, Pauly O, Heibel H, Martina M, Bokeloh M (2019) An annotation saved is an annotation earned: using fully synthetic training for object detection. In: Proceedings of the IEEE/CVF international conference on computer vision workshops
19.
Khan S, Phan B, Salay R, Czarnecki K (2019) Procsy: Procedural synthetic dataset generation towards influence factor studies of semantic segmentation networks. In: CVPR workshops, pp 88–96
20.
Di Cicco M, Potena C, Grisetti G, Pretto A (2017) Automatic model based dataset generation for fast and accurate crop and weeds detection. In: 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 5188–5195, IEEE
21.
Olatunji J, Redding G, Rowe C, East A (2020) Reconstruction of kiwifruit fruit geometry using a cgan trained on a synthetic dataset. Comput Electron Agric 177:105699CrossRef
22.
Zhang K, Wu Q, Chen Y (2021) Detecting soybean leaf disease from synthetic image using multi-feature fusion faster r-cnn. Comput Electron Agric 183:106064CrossRef
23.
Barth R, Isselmuiden J, Hemming J, Van Henten EJ (2018) Data synthesis methods for semantic segmentation in agriculture: a capsicum annuum dataset. Comput Electron Agric 144:284–296CrossRef
24.
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: Single shot multibox detector. In: European conference on computer vision, pp 21–37, Springer
25.
Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510–4520
26.
Maric B, Polic M, Tabak T, Orsag M (2020) Unsupervised optimization approach to in situ calibration of collaborative human-robot interaction tools. In: 2020 IEEE international conference on multisensor fusion and integration for intelligent systems (MFI), pp 255–262, IEEE
27.
Hess R (2010) Blender foundations: the essential guide to learning blender 2.6. Focal Press
28.
AliceVision, Meshroom: A 3D reconstruction software., 2018
Metadata
Title
Pepper to fall: a perception method for sweet pepper robotic harvesting
Authors
Marsela Polic
Jelena Tabak
Matko Orsag
Publication date
01-12-2021
Publisher
Springer Berlin Heidelberg
Published in
Intelligent Service Robotics / Issue 2/2022
Print ISSN: 1861-2776
Electronic ISSN: 1861-2784
DOI
https://doi.org/10.1007/s11370-021-00401-7

Other articles of this Issue 2/2022

Intelligent Service Robotics 2/2022 Go to the issue

Original Research Paper

Feedforward operational stiffness modulation and external force estimation of planar robots equipped with variable stiffness actuators

Original Research Paper

What if there was no revisit? Large-scale graph-based SLAM with traffic sign detection in an HD map using LiDAR inertial odometry

Original Research Paper

Object manipulation system based on image-based reinforcement learning

Original Research Paper

Enabling landings on irregular surfaces for unmanned aerial vehicles via a novel robotic landing gear

Editorial

Special Issue on the 2021 Ubiquitous Robots Conference

Original Research Paper

Deep latent-space sequential skill chaining from incomplete demonstrations