Abstract

Ball catching by a robot is one of the challenging and complex control tasks that is extensively studied to achieve human-like skills in robots. Over the last decade, several ball-catching robot designs have attained benchmarks in visual tracking and control algorithms. However, the coordination between the ball’s path tracking and the robot’s motion planning remains highly sensitive to environmental parameter changes. In general, ball-catching robots require a noise-free background with good lighting and multiple off-board tracking cameras. Also, a common failing point of these systems is the short flight time (or high-speed) of the ball and the uncertainties of throwing direction. To address these issues, in this study, we propose a ball-catching platform system that can rapidly orient the platform towards the throwing direction by utilizing two onboard cameras with multi-threading. A graphical user interface platform has been developed to implement the orientation algorithm and mask the ball with high accuracy. Our experimental results show that the proposed orientation platform system can be used in a low-light noisy background, and the overall ball-catching rate increases from 50% to 90% compared to the baseline design. The new system can also avoid erratic platform movements when masking is done in a noisy environment.

References

1.
Peper
,
L.
,
Bootsma
,
R. J.
,
Mestre
,
D. R.
, and
Bakker
,
F. C.
,
1994
, “
Catching Balls: How to Get the Hand to the Right Place at the Right Time
,”
J. Exp. Psychol. Hum. Percept. Perform.
,
20
(
3
), pp.
591
612
.
2.
Kim
,
S.
,
Shukla
,
A.
, and
Billard
,
A.
,
2014
, “
Catching Objects in Flight
,”
IEEE Trans. Rob.
,
30
(
5
), pp.
1049
1065
.
3.
Bäuml
,
B.
,
Birbach
,
O.
,
Wimböck
,
T.
,
Frese
,
U.
,
Dietrich
,
A.
, and
Hirzinger
,
G.
,
2011
, “
Catching Flying Balls With a Mobile Humanoid: System Overview and Design Considerations
,”
Proceedings of the 2011 11th IEEE-RAS International Conference on Humanoid Robots
,
Bled, Slovenia
,
Oct. 26–28
, pp.
513
520
.
4.
Bäuml
,
B.
,
Wimböck
,
T.
, and
Hirzinger
,
G.
,
2010
, “
Kinematically Optimal Catching a Flying Ball With a Hand-Arm-System
,”
Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems
,
Taipei, Taiwan
,
Oct. 18–22
, pp.
2592
2599
.
5.
Bujarbaruah
,
M.
,
Zheng
,
T.
,
Shetty
,
A.
,
Sehr
,
M.
, and
Borrelli
,
F.
,
2020
, “
Learning to Play Cup-and-Ball With Noisy Camera Observations
,”
Proceedings of the 2020 IEEE 16th International Conference on Automation Science and Engineering (CASE)
,
Hong Kong (Virtual)
,
Aug. 20–21
, pp.
372
377
.
6.
Nemec
,
B.
,
Zorko
,
M.
, and
Žlajpah
,
L.
,
2010
, “
Learning of a Ball-in-a-Cup Playing Robot
,”
Proceedings of the 19th International Workshop on Robotics in Alpe-Adria-Danube Region (RAAD 2010)
,
Budapest, Hungary
,
June 23–25
, pp.
297
301
.
7.
Li
,
S.
,
2020
, “
Robot Playing Kendama With Model-Based and Model-Free Reinforcement Learning
,” arXiv: abs/2003.06751.
8.
Lee
,
K.
,
Batz
,
G.
, and
Wollherr
,
D.
,
2008
, “
Basketball Robot: Ball-On-Plate With Pure Haptic Information
,”
Proceedings of the 2008 IEEE International Conference on Robotics and Automation
,
Pasadena, CA
,
May 19–23
, pp.
2410
2415
.
9.
JohanLink
,
2021
, “
Ball-Balancing-PID-System
,” https://github.com/JohanLink/Ball-Balancing-PID-System
10.
Oka
,
T.
,
Komura
,
N.
, and
Namiki
,
A.
,
2017
, “
Ball Juggling Robot System Controlled by High-Speed Vision
,”
Proceedings of the 2017 IEEE International Conference on Cyborg and Bionic Systems (CBS)
,
Beijing, China
,
Oct. 17–19
, pp.
91
96
.
11.
Carter
,
E. J.
,
Mistry
,
M. N.
,
Carr
,
G. P. K.
,
Kelly
,
B. A.
, and
Hodgins
,
J. K.
, “
Playing Catch With Robots: Incorporating Social Gestures Into Physical Interactions
,”
Proceedings of the 23rd IEEE International Symposium on Robot and Human Interactive Communication
, pp.
231
236
.
12.
Namiki
,
A.
, and
Itoi
,
N.
, “
Ball Catching in Kendama Game by Estimating Grasp Conditions Based on a High-Speed Vision System and Tactile Sensors
,”
Proceedings of the 2014 IEEE-RAS International Conference on Humanoid Robots
, pp.
634
639
.
13.
Mülling
,
K.
,
Kober
,
J.
, and
Peters
,
J.
, “
A Biomimetic Approach to Robot Table Tennis
,”
Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems
, pp.
1921
1926
.
14.
Li
,
H.
,
Wu
,
H.
,
Lou
,
L.
,
Kühnlenz
,
K.
, and
Ravn
,
O.
,
2012
, “
Ping-Pong Robotics With High-Speed Vision System
,”
Proceedings of the 2012 12th International Conference on Control Automation Robotics & Vision (ICARCV)
,
Guangzhou, China
,
Dec. 5–7
, pp.
106
111
.
15.
Wang
,
Z.
,
Boularias
,
A.
,
Mülling
,
K.
,
Schölkopf
,
B.
, and
Peters
,
J.
,
2017
, “
Anticipatory Action Selection for Human–Robot Table Tennis
,”
Artif. Intell.
,
247
(Special Issue on AI and Robotics), pp.
399
414
.
16.
Arif
,
T. M.
, and
Ito
,
T.
,
2011
, “
S121013 Master-Slave Robotic Arm Manipulation for Communication Robot
,”
Proceedings of the Mechanical Engineering Congress
,
Tokyo, Japan
,
Sept. 11–14
, pp. S121013-1–S121013-5.
17.
Innovate-Educate, Q.
,
2022
, “
2 DOF Ball Balancer
,” https://www.quanser.com/products/2-dof-ball-balancer/
18.
ACROME
,
2022
, “
Ball Balancing Table, 2-DoF Control Platform for Teaching and Research
,” https://acrome.net/product/ball-balancing-table
19.
Link
,
J.
,
2018
, “
Ball Balancing PID System
,” https://www.thingiverse.com/thing:3310602
21.
Virtanen
,
P.
,
Gommers
,
R.
,
Oliphant
,
T. E.
,
Haberland
,
M.
,
Reddy
,
T.
,
Cournapeau
,
D.
,
Burovski
,
E.
, et al
,
2020
, “
SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python
,”
Nat. Methods
,
17
(
3
), pp.
261
272
.
22.
Dorf
,
R. C.
, and
Bishop
,
R. H.
,
2021
,
Modern Control Systems
,
Pearson Education
,
New York
, pp.
450
457
.
23.
Ellis
,
G.
,
2012
, “Chapter 6—Four Types of Controllers,”
Control System Design Guide
, 4th ed.,
G.
Ellis
, ed.,
Butterworth-Heinemann
,
Boston
, MA, pp.
97
119
.
24.
Summerfield
,
M.
,
2015
,
Rapid GUI Programming With Python and Qt: The Definitive Guide to PyQt Programming
,
Prentice Hall
,
Upper Saddle River, NJ
,
Chap 19 Multithreading
.
25.
Tazehkandi
,
A. A.
,
2018
,
Computer Vision With OpenCV 3 and Qt5: Build Visually Appealing, Multithreaded, Cross-Platform Computer Vision Applications
,
Packt Publishing
,
Birmingham, UK
.
26.
Ramos
,
L. P.
,
2022
, “
Use PyQt's QThread to Prevent Freezing GUIs
,” https://realpython.com/python-pyqt-qthread/
27.
Fitzpatrick
,
M.
,
2021
, “
Multithreading PyQt5 Applications With QThreadPool
,” https://www.pythonguis.com/tutorials/multithreading-pyqt-applications-qthreadpool/
28.
Bradski
,
G.
,
2000
, “
The OpenCV Library
,”
Dr. Dobb's J. Softw. Tools Prof. Program.
,
25
(
11
), pp.
120
123
.
29.
Van Der Walt
,
S.
,
Schönberger
,
J. L.
,
Nunez-Iglesias
,
J.
,
Boulogne
,
F.
,
Warner
,
J. D.
,
Yager
,
N.
,
Gouillart
,
E.
, and
Yu
,
T.
,
2014
, “
Scikit-Image: Image Processing in Python
,”
PeerJ
,
2
, pp.
e453
e453
.
30.
Rosebrock
,
A.
,
2016
, “
OpenCV Center of Contour
,” https://pyimagesearch.com/2016/02/01/opencv-center-of-contour/
31.
Adams
,
C.
,
2019
, “
Ergonomic Lighting Levels by Room for Residential Spaces
,” https://www.thoughtco.com/lighting-levels-by-room-1206643
32.
Arif
,
T. M.
,
2020
,
Introduction to Deep Learning for Engineers: Using Python and Google Cloud Platform
,
Morgan & Claypool
,
California, USA
, pp.
17
18
.
You do not currently have access to this content.