Smart Robot Vision for a Pick and Place Robotic System

Mohammed, Momena M.; AL-Khafaji, Mohanned M. H.; Abbas, Tahseen F.

doi:10.30684/etj.2023.135966.1292

	Smart Robot Vision for a Pick and Place Robotic System
Engineering and Technology Journal
Article 2, Volume 41, Issue 6, 2023, Pages 756-770 PDF (1.22 M)
Document Type: Research Paper
DOI: 10.30684/etj.2023.135966.1292
Authors
Momena M. Mohammed^* ; Mohanned M. H. AL-Khafaji; Tahseen F. Abbas
Production Engineering and Metallurgy Dept., University of Technology-Iraq, Alsina’a street, 10066 Baghdad, Iraq
Abstract
The main contribution of this paper is to develop an innovative algorithm to accurately detect and identify the shape and color of objects under various light intensities and find their location to be manipulated by a pick-and-place robotic arm. Workpieces of various shapes and colors are dispersed on the robot's work plane and manipulated according to its specifications. The proposed algorithm utilizes the HSV color model to distinguish between different object colors and shapes. The S channel is used to detect the shapes of objects. After that, a series of filters (Median, Bilateral, and Gaussian) are applied to reduce the noise of the segmented image to make the process of discovering the shape and coordinates of the objects successful. The draw-contour method is used to discover the object’s shapes. After the shape of the object is discovered, the centroid coordinates are calculated. After extensive testing on 354 images that are captured in various lighting conditions in the range of (5-7000 lux), the overall system performance of 93.83% is achieved, and the average execution time is 2.21s. Finally, we had a dependable flexible automatic pick and place system that could correctly detect and identify the objects based on their features.
Highlights
The movement of the 5 DOF robotic arm was controlled through the geometric approach in inverse kinematics analysis HSV color space, a series of filters (Median, Bilateral, and Gaussian), and the draw-contour method to discover objects’ colors, shapes, and centroid were implemented. The shapes and colors of the objects in different lighting conditions ranging from 5 to 7000 lux with an accuracy of 93.83% were discovered.
Keywords
Robot vision; Raspberry pi; Arduino; different illumination conditions; HSV

References
[1] V. Batra and V. Kumar, Real-Time Object Detection and Localization for Vision-Based Robot Manipulator, SN Computer Science. 2 (2021) 1-10. https://doi.org/10.1007/s42979-021-00561-4 [2] P. Tsarouchi, S. A. Matthaiakis, G. Michalos, S. Makris and G. Chryssolouris, A method for detection of randomly placed objects for robotic handling, CIRP J Manuf Sci Technol. 4 (2016) 20-27. https://doi.org/10.1016/j.cirpj.2016.04.005 [3] T. F. Abaas, A. A. Khleif and M. Q. Abbood, Computer Vision-Based System for Classification and Sorting Color Objects, IOP Conf. Series: Materials Science and Engineering. 745, 2020, 012030. https://doi.10.1088/1757-899X/745/1/012030 [4] G. D. Leo, C. Liguori, A. Pietrosanto and P. Sommella, A vision system for the online quality monitoring of industrial manufacturing, Opt Lasers Eng.89 (2016) 162-168. https://doi.org/10.1016/j.optlaseng.2016.05.007 [5] R. V. Sharan and G. C. Onwubolu, Automating the Process of Work-Piece Recognition and Location for a Pick-and-Place Robot in a SFMS, in: International Journal of Image, Graphics and Signal Processing (IJIGSP). 6 (2014) 9-17. https://doi.org/10.5815/IJIGSP.2014.04.02 [6] A. A. Ata, S. F. Rezeka, A. El-Shenawy and M. Diab, Design and Development of 5-DOF Color Sorting Manipulator for Industrial Applications, in: World Academy Sci. Eng. Technol. Int. J. Mech. Aerosp. Ind. Mechatron. Manuf. Eng. 7 (2013) 2457-2464. [7] Y. P. Loh, X. Liang and C. S. Chan, Low-light image enhancement using Gaussian Process for features retrieval, Signal Process Image Commun. 74 (2019) 175-190. https://doi.org/10.1016/j.image.2019.02.001 [8] C. Li, J. Guo, F. Porikli and Y. Pang, LightenNet: A Convolutional Neural Network for weakly illuminated image enhancement, Pattern Recognit Lett. 104 (2018) 15-22. https://doi.org/10.1016/j.patrec.2018.01.010 [9] R. Kumar, S. Kumar, S. Lal and P. Chand, Object detection and recognition system for pick and place robot, in: Asia-Pacific World Congress on Computer Science and Engineering, IEEE. 2014,1-7. https://doi.org/ 10.1109/APWCCSE.2014.7053853 [10] H. M. Qul’am, T. Dewi, P. Risma, Y. Oktarina and D. Permatasari, Edge detection for online image processing of a vision guide pick and place robot, in: 2019 International Conference on Electrical Engineering and Computer Science (ICECOS) IEEE.2019, 102-106. https://doi.org/10.1109/ICECOS47637.2019.8984522 [11] S. Chakole and N. Ukani, Low-Cost Vision System for Pick and Place application using camera and ABB Industrial Robot, in: 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT) IEEE. 2020, 1-6. https://doi.10.1109/ICCCNT49239.2020.9225522 [12] P. Shankar, V. Babureddy, S. C. S. Vigneshwaran and C. A. Prakash, Design and analysis of pick and place robot for colour based sorting applications, in: AIP Conference Proceedings.2247, 2020, 020025. https://doi.org/10.1063/5.0004429 [13] M. H. Ali, A. K., Y. K. and Z. T. a. A. O, Vision-based Robot Manipulator for Industrial Applications, Procedia Comput. Sci.133 (2018) 205–212. https://doi.org/10.1016/j.procs.2018.07.025 [14] F. S. Hameed, H. M. Alwan and Q. A. Ateia, Pose Estimation of Objects Using Digital Image Processing for Pick-and-Place Applications of Robotic Arms, ETJ. 38 (2020) 707-718. [15] T. F. Abaas, A. A. Khleif and M. Q. Abbood, Inverse Kinematics Analysis and Simulation of a 5 DOF Robotic Arm using MATLAB, KECBUJ.16 (2020) 1- 10. [16] P. V. Garad, Object Sorting Robot Based On the Shape, IJARIIT. 3 (2017) 129-134. [17] A. Wang, W. Zhang and X. Wei, A review on weed detection using ground-based machine vision and image processing techniques, Comput Electron Agric.158 (2019) 226-240. https://doi.org/10.1016/j.compag.2019.02.005 [18] H. O. Velesaca, P. L. Suarez, R. Mira and A. D. Sappa, Computer vision based food grain classification: A comprehensive survey, Comput Electron Agric. 187 (2021) 106287. https://doi.org/10.1016/j.compag.2021.106287 [19] K. Muhammad, M. Sajjad, S. Rho and S. W. Baik, Image steganography using uncorrelated color space and its application for security of visual contents in online social networks, Future Gener Comput Syst. 86 (2018) 951-960. https://doi.org/10.1016/j.future.2016.11.029 [20] G. Bargshady, X. Zhou, R. C. D. J. Soar, F. Whittaker and H. Wang, The modeling of human facial pain intensity based on Temporal Convolutional Networks trained with video frames in HSV color space, Appl. Soft Comput. 97 (2020) 106805. https://doi.org/10.1016/j.asoc.2020.106805 [21] K. B. Shaik, G. P, V.Kalist, B.S.Sathish and J. M. Jenitha, Comparative Study of Skin Color Detection and Segmentation in HSV and YCbCr Color Space, Procedia Comput. Sci. 57 (2015) 41-48. https://doi.org/10.1016/j.procs.2015.07.362 [22] F. G. Lamont, J. Cervantes, A. López and L. Rodriguez, Segmentation of images by color features: A survey, Neurocomputing.292 (2018) 1-27. https://doi.org/10.1016/j.neucom.2018.01.091 [23] D. Wu and D.-W. Sun, Colour measurements by computer vision for food quality control - A review, Trends Food Sci Technol.29 (2013) 5-20. https://doi.org/10.1016/j.tifs.2012.08.004 [24] S. Kotte, P. R. Kumar and S. K. Injeti, An efficient approach for optimal multilevel thresholding selection for gray scale images based on improved differential search algorithm, Ain Shams Eng. J. 9 (2018) 1043-1067. https://doi.org/10.1016/j.asej.2016.06.007 [25] D. Bulanon, T. Kataoka, Y. Ota and T. Hiroma, A segmentation algorithm for the automatic recognition of Fuji apples at harvest, Biosyst Eng. 83 (2002) 405-412. https://doi.org/10.1006/bioe.2002.0132 [26] T. Y. Goh, S. N. Basah, H. Yazid, M. J. A. Safar and F. S. A. Saad, Performance analysis of image thresholding: Otsu technique, Measurement. 114 (2018) 298-307. https://doi.org/10.1016/j.measurement.2017.09.052 [27] U. Erkan, L. Gökrem and S. Enginoglu, Different applied median filter in salt and pepper noise, Comput. Electr. Eng. 70 (2018) 789-798. https://doi.org/10.1016/j.compeleceng.2018.01.019 [28] B. Goyal, A. Dogra, S. Agrawal, B. Sohi and A. Sharma, Image denoising review: From classical to state-of-the-art approaches, Information Fusion. 55 (2020): 220-244. https://doi.org/10.1016/j.inffus.2019.09.003 [29] J. Joseph and R. Periyasamy, An image driven bilateral filter with adaptive range and spatial parameters for denoising Magnetic Resonance Images, Comput. Electr. Eng. 69 (2018) 782-795. https://doi.org/10.1016/j.compeleceng.2018.02.033 [30] A. K. Sat and T. Tint, Object detection and recognition system for pick and place robot, in: International Conference on Big Data Analysis and Deep Learning Applications, Springer.2018, 315-323. https://doi.org/10.1007/978-981-13-0869-7_35
Statistics Article View: 834 PDF Download: 608