Li, Z.Huang, J.Zhang, P.Shi, P.2024-10-292024-10-292025IEEE Transactions on Automation Science and Engineering, 2025; 22:534-5451545-59551558-3783https://hdl.handle.net/2440/143032Date of Publication: 04 July 2024In this paper, an upper limb prosthesis has been furnished with a novel vision-based manipulation and grasping strategy. The proposed whole-body safety-critical control design includes vision servoing, multiple tasks planning with strict priorities, which can be formulated as an hierarchical multi-task optimization (HMO) problem with safety conditions—expressed as control barrier functions (CBF). Firstly, a modified YOLOv7 algorithm with key points detection is developed to determine the grasping pattern of the object and extract its edge contour information using a depth camera. An HMO-based strategy with a notion of CBF, providing inequality constraints in the control input, is proposed to handle multiple prioritized tasks with various constraints to offer guarantees of safety with the whole-body motion in consideration. Then the HMO problem is solved by a neuro-dynamics optimization solution online. Finally, experiments are implemented by using a self-developed upper limb prosthesis. Experimental results validate the performance of the proposed whole-body control strategy.en© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.Whole-body control; upper limb prosthesis; task priority; neuro-dynamics optimization; vision-based coordinationJournal article10.1109/TASE.2024.34128232024-10-29701527Shi, P. [0000-0001-6295-0405] [0000-0001-8218-586X] [0000-0002-0864-552X] [0000-0002-1358-2367] [0000-0002-5312-5435]