Vision Based Hand Gesture Control of Dobot Magician
DOI:
https://doi.org/10.15662/IJEETR.2026.0802199Keywords:
Hand Gesture Recognition, Computer Vision, Human–Computer Interaction, Gesture-Based Control, Vision-Based Robotics, Image Processing, Real-Time Object Detection, Feature Extraction, Machine Learning, Deep LearningAbstract
Human–robot interaction is an important area in modern robotics, where intuitive and efficient control methods are required for operating robotic systems. Traditional robot control approaches usually depend on keyboards, controllers, or manual programming, which can make interaction less flexible and more complex for users. To overcome these limitations, this project presents a vision-based hand gesture control system for the Dobot Magician robotic arm. The system allows users to control the robot using natural hand gestures captured through a camera, enabling a more interactive and contactless control mechanism. By using computer vision techniques, the system detects and interprets hand gestures in real time, allowing users to perform robot movements without the need for physical control devices
The proposed system integrates computer vision and robotic control to enable real-time gesture-based interaction with the Dobot Magician robotic arm. Hand gestures are detected using OpenCV and MediaPipe, which track hand landmarks from the live video stream. The recognized gestures are converted into robot commands through a command mapping module for controlling arm movements. The system uses the ROS 2 communication framework to transmit commands between modules efficiently
This approach enhances the intuitiveness and accessibility of robotic systems and can be applied in areas such as robotics education, industrial automation, human–robot interaction, and assistive technologies. The developed system demonstrates the potential of vision-based gesture interfaces as an efficient and user-friendly method for robotic control
References
1. V. Pavlovic, R. Sharma, and T. S. Huang, “Visual Interpretation of Hand Gestures for Human–Computer Interaction: A Review,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 677–695, 1997.
2. J. Wachs, M. Kölsch, H. Stern, and Y. Edan, “Vision-Based Hand Gesture Applications,” Communications of the ACM, vol. 54, no. 2, pp. 60–71, 2011.
3. G. Bradski and A. Kaehler, Learning OpenCV: Computer Vision with the OpenCV Library, O’Reilly Media, 2008.
4. R. Szeliski, Computer Vision: Algorithms and Applications, Springer, 2010.
5. F. Chollet, Deep Learning with Python, Manning Publications, 2017.
6. M. Quigley et al., “ROS: An Open-Source Robot Operating System,” IEEE International Conference on Robotics and Automation Workshop, 2009.
7. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
8. Google Research, “MediaPipe: A Framework for Building Perception Pipelines,” 2019. [Online]. Available: https://mediapipe.dev
9. B. Siciliano and O. Khatib, Springer Handbook of Robotics, Springer, 2016.
10. S. Mitra and T. Acharya, “Gesture Recognition: A Survey,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 37, no. 3, pp. 311–324, 2007.
11. C.Nagarajan and M.Madheswaran - ‘Stability Analysis of Series Parallel Resonant Converter with Fuzzy Logic Controller Using State Space Techniques’- Taylor &Francis, Electric Power Components and Systems, Vol.39 (8), pp.780-793, May 2011. DOI: 10.1080/15325008.2010.541746
12. C.Nagarajan and M.Madheswaran - ‘Experimental verification and stability state space analysis of CLL-T Series Parallel Resonant Converter’ - Journal of Electrical Engineering, Vol.63 (6), pp.365-372, Dec.2012. DOI: 10.2478/v10187-012-0054-2
13. C.Nagarajan and M.Madheswaran - ‘Performance Analysis of LCL-T Resonant Converter with Fuzzy/PID Using State Space Analysis’- Springer, Electrical Engineering, Vol.93 (3), pp.167-178, September 2011. DOI 10.1007/s00202-011-0203-9
14. S.Tamilselvi, R.Prakash, C.Nagarajan,“Solar System Integrated Smart Grid Utilizing Hybrid Coot-Genetic Algorithm Optimized ANN Controller” Iranian Journal Of Science And Technology-Transactions Of Electrical Engineering, DOI10.1007/s40998-025-00917-z,2025
15. S.Tamilselvi, R.Prakash, C.Nagarajan,“ Adaptive sliding mode control of multilevel grid-connected inverters using reinforcement learning for enhanced LVRT performance” Electric Power Systems Research 253 (2026) 112428, doi.org/10.1016/j.epsr.2025.112428
16. S.Thirunavukkarasu, C. Nagarajan, 2024, “Performance Investigation on OCF and SCF study in BLDC machine using FTANN Controller," Journal of Electrical Engineering And Technology, Volume 20, pages 2675–2688, (2025), doi.org/10.1007/s42835-024-02126-w
17. C. Nagarajan, M.Madheswaran and D.Ramasubramanian- ‘Development of DSP based Robust Control Method for General Resonant Converter Topologies using Transfer Function Model’- Acta Electrotechnica et Informatica Journal , Vol.13 (2), pp.18-31,April-June.2013, DOI: 10.2478/aeei-2013-0025.
18. C.Nagarajan and M.Madheswaran - ‘DSP Based Fuzzy Controller for Series Parallel Resonant converter’- Springer, Frontiers of Electrical and Electronic Engineering, Vol. 7(4), pp. 438-446, Dec.12. DOI 10.1007/s11460-012-0212-0.
19. C.Nagarajan and M.Madheswaran - ‘Experimental Study and steady state stability analysis of CLL-T Series Parallel Resonant Converter with Fuzzy controller using State Space Analysis’- Iranian Journal of Electrical & Electronic Engineering, Vol.8 (3), pp.259-267, September 2012.
20. C.Nagarajan and M.Madheswaran, “Analysis and Simulation of LCL Series Resonant Full Bridge Converter Using PWM Technique with Load Independent Operation” has been presented in ICTES’08, a IEEE / IET International Conference organized by M.G.R.University, Chennai.Vol.no.1, pp.190-195, Dec.2007
21. Suganthi Mullainathan, Ramesh Natarajan, “An SPSS and CNN modelling based quality assessment using ceramic materials and membrane filtration techniques”, Revista Materia (Rio J.) Vol. 30, 2025, DOI : https://doi.org/10.1590/1517-7076-RMAT-2024-0721
22. M Suganthi, N Ramesh, “Treatment of water using natural zeolite as membrane filter”, Journal of Environmental Protection and Ecology, Volume 23, Issue 2, pp: 520-530,2022
23. Rajasekar, M., Nahar, G., Jagatheeswaran, S., Chinthamani, S. A. M., Mohammed, S. H., & Al-Hilali, A. (2024, May). Retraction Notice: The Roadmap to Classify Malware Using ML Algo Through IOT Based SN. In 2024 4th International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE) (pp. 1-1). IEEE.
24. Kiran, A., Rubini, P., & Kumar, S. S. (2025). Comprehensive review of privacy, utility and fairness offered by synthetic data. IEEE Access.
25. Anbazhagan, K. (2025). AI Driven Zero Trust Security Model for Enterprise Data Protection and Intelligent Infrastructure Management. International Journal of Technology, Management and Humanities, 11(03), 101-107
26. Chiranjeevi, Y., Sugumar, R., & Tahir, S. (2024, November). Effective Classification of Ocular Disease Using Resnet-50 in Comparison with Squeezenet. In 2024 IEEE 9th International Conference on Engineering Technologies and Applied Sciences (ICETAS) (pp. 1-6). IEEE.
27. Mathew, A. (2023). Cybercrime-as-a-service & AI-enabled threats. International Journal of Computer Science and Mobile Computing, 12(1), 28-31.
28. Kumar, S. A., & Anand, L. (2025). A Novel EEG-Based Deep Learning Framework for Enhancing Communication in Locked-In Syndrome Using P300 Speller and Attention Mechanisms. KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS, 19(11), 3841-3855.
29. Loganayagi, S., Balakrishnan, T. S., Vimal, V. R., & Thangam, S. A. (2024, November). Assessing the Efficacy of ML Techniques for Forecasting Healthcare Consumer Readmission: A Comparative Analysis of Risk Factors and Healthcare Interventions. In 2024 International Conference on Smart Technologies for Sustainable Development Goals (ICSTSDG) (pp. 1-7). IEEE.
30. Anbazhagan, K. (2024). Trustworthy and Adaptive AI Systems for Enterprise Analytics Cybersecurity and Decision Optimization Using API-First and Cloud-Native Architectures. International Journal of Technology, Management and Humanities, 10(03), 65-74.
31. Padmapriya, V. M., Thenmozhi, K., Hemalatha, M., Thanikaiselvan, V., Lakshmi, C., Chidambaram, N., & Rengarajan, A. (2025). Secured IIoT against trust deficit-A flexi cryptic approach. Multimedia Tools and Applications, 84(9), 5625-5652.
32. Soundappan, S. J. (2020). Big Data Analytics in Healthcare: Applications for Pandemic Forecastin. International Journal of Advanced Research in Computer Science & Technology (IJARCST), 3(1), 2248-2253.
33. Mathew, A., & Alex, H. (2023, January). Hyper automation and augmented intelligence. In 2023 5th International Conference on Smart Systems and Inventive Technology (ICSSIT) (pp. 1230-1234). IEEE.
34. Murugeshwari, B., Sudharson, K., Panimalar, S. P., Shanmugapriya, M., & Abinaya, M. (2020). SAFE–Secure Authentication in Federated Environment using CEG Key code.
35. Anand, L., Tyagi, R., & Mehta, V. (2024, January). Food recognition using deep learning for recipe and restaurant recommendation. In Proceedings of Eighth International Conference on Information System Design and Intelligent Applications (pp. 269-279). Singapore: Springer Nature Singapore.
36. Kumar, K. A., Vimal, V. R., & Karthick, V. (2024, June). Error Rate Increment of Quantum Computer using Quantity Computational Algorithm Comparing with Deutsch Jozsa Algorithm. In 2024 OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 4.0 (pp. 1-5). IEEE.
