Graphical Abstract Figure
Graphical Abstract Figure
Close modal

Abstract

In order to deal with the complex dynamics and control problems involved in space debris removal, a trajectory planning technique for a spatial robotic arm based on twin delayed DDPG (TD3) in deep reinforcement learning is proposed, and it can accomplish an end-to-end control effect comparable to that of human hand gripping objects. The trajectory planning method for capturing space debris by a floating-base space robotic arm is realized using a space robotic arm task simulation platform built on MuJoCo and using trajectory planners, trajectory trackers, and joint and end-effector control strategies formulated with seven different weighted reward functions. This makes it easier to complete spacecraft in-orbit servicing and maintenance missions. The experiment results demonstrate that the capture strategy can maintain a capture success rate of more than 99%, and debris capture can be mostly finished in three stages when taking the stability of the floating base into consideration by continuously modifying the trajectory.

References

1.
Xu
,
K.
,
Dong
,
G. Y.
,
Qiao
,
A. W.
,
Han
,
L. L.
, and
Ding
,
X. L.
,
2022
, “
Review and Prospect of On-Orbit Serving Spacecraft Capture Device Technology
,”
Aerosp. Shanghai (Chinese & English)
,
39
(
6
), pp.
12
28
.
2.
Mohanty
,
R. R.
,
Raina
,
A. S.
,
Chaudhuri
,
S.
,
Quek
,
F.
,
Sueda
,
S.
, and
Krishnamurthy
,
V. R.
,
2023
, “
Spatial Manipulation in Virtual Peripersonal Space: A Study of Motor Strategies
,”
ASME J. Comput. Inf. Sci. Eng.
,
23
(
2
), p.
021004
.
3.
Spensieri
,
D.
,
Åblad
,
E.
,
Kressin
,
J.
,
Carlson
,
J. S.
, and
Andersson
,
A.
,
2021
, “
Collision-Free Robot Coordination and Visualization Tools for Robust Cycle Time Optimization
,”
ASME J. Comput. Inf. Sci. Eng.
,
21
(
4
), p.
041011
.
4.
Yoshida
,
K.
,
Hashizume
,
K.
, and
Abiko
,
S.
,
2001
, “
Zero Reaction Maneuver: Flight Validation With ETS-VII Space Robot and Extension to Kinematically Redundant Arm
,”
IEEE International Conference on Robotics and Automation
,
Seoul, South Korea
,
May 21–26
, Vol. 1, pp.
441
446
.
5.
Matsueda
,
T.
,
Kuraoka
,
K.
, and
Goma
,
K.
,
1991
, “
JEMRMS System Design and Development Status
,”
NTC’91—National Telesystems Conference Proceedings
,
Atlanta, GA
,
Mar. 26–27
, pp.
391
395
.
6.
Xu
,
L.
,
Hu
,
Q.
, and
Zhang
,
Y.
,
2017
, “
Performance Control of Robot Manipulators With Kinematics, Dynamics and Actuator Uncertainties
,”
J. Robust Nonlinear Control
,
27
(
6
), pp.
875
893
.
7.
Huang
,
X. H.
,
Jia
,
Y. H.
,
Xu
,
S. J.
, and
Lu
,
S.
,
2017
, “
Trajectory Planning of a Space Manipulator With Constant Zero-Disturbance to Base Attitude
,”
J. Beijing Univ. Aeronaut. Astronaut.
,
43
(
3
), pp.
488
496
.
8.
Meng
,
G.
,
Han
,
L.
, and
Zhang
,
C.
,
2021
, “
Research Progress and Technical Changes of Space Robot
,”
Acta Aeronaut. Astronaut. Sin.
,
42
(
1
), pp.
8
32
.
9.
Haug
,
E. J.
,
2017
, “
An Ordinary Differential Equation Formulation for Multibody Dynamics: Nonholonomic Constraints
,”
ASME J. Comput. Inf. Sci. Eng.
,
17
(
1
), p.
011009
.
10.
Hu
,
J. C.
, and
Wang
,
T. S.
,
2018
, “
Minimum Base Attitude Disturbance Planning for a Space Robot During Target Capture
,”
ASME J. Mech. Rob.
,
10
(
5
), p.
051002
.
11.
Zhao
,
Y.
,
Guan
,
G. S.
,
Guo
,
J. F.
,
Yu
,
X. Q.
, and
Yan
,
P.
,
2021
, “
Trajectory Planning of Space Manipulator Based on Multi-agent Reinforcement Learning
,”
Acta Aeronaut. Astronaut. Sin.
,
42
(
1
), pp.
266
276
.
12.
Zhang
,
T. T.
,
2019
, “
Research on Trajectory Planning and Intelligent Control of Space Manipulator Capture for Non-cooperative Targets
,”
M.S. thesis
,
Harbin Institute of Technology
,
Harbin
.
13.
Wang
,
Q. C.
,
Quan
,
Q. Q.
,
Deng
,
Z. Q.
, and
Hou
,
X. Y.
,
2016
, “
An Underactuated Robotic Arm Based on Differential Gears for Capturing Moving Targets: Analysis and Design
,”
ASME J. Mech. Rob.
,
8
(
4
), p.
041012
.
14.
Rajendran
,
P.
,
Thakar
,
S.
,
Bhatt
,
P. M.
,
Kabir
,
A. M.
, and
Gupta
,
S. K.
,
2020
, “
Strategies for Speeding Up Manipulator Path Planning to Find High Quality Paths in Cluttered Environments
,”
ASME J. Comput. Inf. Sci. Eng.
,
21
(
1
), p.
011009
.
15.
Yang
,
J. Z.
, and
Howard
,
B.
,
2020
, “
Prediction of Initial and Final Postures for Motion Planning in Human Manual Manipulation Tasks Based on Cognitive Decision Making
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
1
), p.
011007
.
16.
Liu
,
S.
,
Wu
,
S. N.
,
Liu
,
Y. F.
,
Wu
,
Z. G.
, and
Mao
,
Z. M.
,
2019
, “
Autonomous Reinforcement Learning Control for Space Robot to Capture Non-cooperative Targets (in Chinese)
,”
Sci. Sin.: Phys. Mech. Astron.
,
49
(
2
), pp.
113
122
.
17.
Liu
,
S.
,
2019
, “
Intelligent Control and Recognition of Space Robot Capturing Non-cooperative Targets
,”
M.S. thesis
,
Dalian University of Technology
,
Dalian
.
18.
Xu
,
W.
, and
Lu
,
S.
,
2019
, “
A Study of Spatial Robotic Arm Path Planning Based on Sarsa(λ) Reinforcement Learning
,”
J. Astronaut.
,
40
(
4
), pp.
435
443
.
19.
Amarjyoti
,
S.
,
2017
, “
Deep Reinforcement Learning for Robotic Manipulation—The State of the Art
,”
CsRo
.
20.
Liang
,
B. Y.
,
Chen
,
Z. H.
,
Guo
,
M. S.
, and
Wang
,
Y.
,
2021
, “
Space Robot Target Intelligent Capture System Based on Deep Reinforcement Learning Model
,”
J. Phys. Conf. Ser.
,
1848
(
1
), pp.
1
7
.
21.
Cao
,
Y. X.
,
Wang
,
S. J.
,
Zheng
,
X.
, and
Liu
,
L.
,
2021
, “
Planning and Control of Space Robot in Capture Operation Based on Reinforcement Learning
,”
Proceedings of the 40th Chinese Control Conference
,
Shanghai
,
July 26–28
.
22.
Sun
,
K.
,
Wang
,
Y. B.
,
Du
,
D. S.
, and
Qi
,
N. M.
,
2020
, “
Capture Control Strategy of Free-Floating Space Manipulator Based on Deep Reinforcement Learning Algorithm
,”
Manned Spaceflight
,
26
(
6
), pp.
751
757
.
23.
Yan
,
X. L.
,
Shan
,
M. H.
, and
Shi
,
L. L.
,
2023
, “
Adaptive and Intelligent Control of a Dual-Arm Space Robot for Target Manipulation During the Post-Capture Phase
,”
Aerosp. Sci. Technol.
,
142
(
B
), p.
108688
.
24.
Caballero
,
A.
,
Sanchez-Cuevas
,
P. J.
,
Bejar
,
M.
,
Heredia
,
G.
,
Trujillo
,
M. A.
, and
Ollero
,
A.
,
2020
, “
An Aerodynamic Extension for Motion Planning With Dynamics Awareness in Aerial Long-Reach Manipulators
,”
Int. J. Aerosp. Eng.
,
2020
(
1
), p.
6348035
.
25.
Kurrek
,
P.
,
Zoghlami
,
F.
,
Jocas
,
M.
,
Stoelen
,
M.
, and
Salehi
,
V.
,
2020
, “
Q-Model: An Artificial Intelligence Based Methodology for the Development of Autonomous Robots
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
6
), p.
061006
.
26.
Du
,
D. S.
,
2019
, “
Research on Capture Control Strategy of Space Manipulator Based on Reinforcement Learning
,”
M.S. thesis
,
Harbin Institute of Technology
,
Harbin
.
27.
Liu
,
Q. Y.
,
2019
, “
Deep Reinforcement Learning Based Object Grasping of Dual-Arm Robot
,”
M.S. thesis
,
Shandong University
,
Jinan
.
28.
Dong
,
Y.
,
2021
, “
Research on Autonomous Exoskeleton Control Based on Reinforcement Learning
,”
M.S. thesis
,
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
,
Shenzhen
.
29.
Li
,
W.
,
Deng
,
C. H.
,
Ge
,
J. M.
,
Liu
,
W.
, and
Li
,
C. Y.
,
2022
, “
Research Progress of Joint Space Trajectory Planning for Robots
,”
Mach. Des. Manuf. Eng.
,
51
(
10
), pp.
15
23
.
30.
Dai
,
Z. D.
,
2016
, “
Progress and Key Technologies in Several Frontiers of Space Robots
,”
Manned Spaceflight
,
22
(
1
), pp.
9
15
.
31.
Ren
,
B.
,
Wang
,
Y.
, and
Chen
,
J.
,
2020
, “
Trajectory-Tracking-Based Adaptive Neural Network Sliding Mode Controller for Robot Manipulators
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
3
), p.
031009
.
32.
Kober
,
J.
, and
Peters
,
J.
,
2011
, “
Policy Search for Motor Primitives in Robotics
,”
Mach. Learn.
,
84
(
1–2
), pp.
171
203
.
You do not currently have access to this content.