iros2019國際學(xué)術(shù)會(huì)議論文集visual servoing of miniature magnetic film swimming robots for 3darbitrary path following_W

1、IEEE Robotics and Automation Letters (RAL) paper presented at the2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Macau, China, November 4-8, 2019Visual servoing of miniature magnetic film swimming robots for 3D arbitrary path followingChenyang Huang1,2,3, Tiantian Xu1

2、,2,4,5, Jia Liu1,2,3, Laliphat Manamanchaiya1,2,3 and Xinyu Wu1,2,6,Abstract Soft swimming microrobots that can be powered and guided remotely by magnetic field show greater potential for numerous applications than traditional rigid counterparts, due to their soft and flexible architectures. However

3、, main challenges in closed-loop control remain to be overcome for the soft robots to reach the accuracy and repeatability in applications. This paper proposes a closed-loop control method of soft swimming microrobots for 3D arbitrary path following at low Reynolds numbers by visual servoing, where

4、the path curve is divided into a series of line segments. Different complicated paths drawn by users through a 3D mouse without the input of parametric equations are followed by swimming robots during experiments. The control method with friendly user interaction and good performance is able to be i

5、ntegrated easily into any generic purposely non-holonomic robots.Index Terms soft robotics, motion control, 3D path follow- ing, magnetic actuation, visual servoing.I. INTRODUCTIONUntethered microrobots can perform numerous tasks at small scales, from the biomedical area 14, such as targeted deliver

6、y of drugs and minimally invasive surgery, to the industrial area 5 6, such as the construction of heterogeneous microparts and cargo transport, due to their microsize which gives them the possiblity to enter the narrow environment. The locomotion in liquids at the microscale is fundamentally differ

7、ent from that at the macroscale because of the negligible inertial forces compared with drag forces 7. As a result, propulsive motions of microorganisms using cilia or flagella to execute corkscrew-like movements or beating and waving have been mimicked to enable micro- robots to swim in liquids, fo

8、r example, through the use of externally applied magnetic fields 8 9. In the field of microrobots, there are main challenges of developing effective microrobots, such as materials design and remotely control, thus limiting robotic approaches 10.Many studies of microrobots show that a significant tra

9、nsition from rigid architectures to soft bodies has been1Guangdong Provincial Key Laboratory of Robotics and Intelligent System, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China; , , , laliphatsiat.ac.c

10、n, 2Chinese Academy of Sciences (CAS) Key Laboratory of Human- Machine Intelligence-Synergy Systems, Shenzhen 518055, China3University of Chinese Academy of Sciences, Beijing 100049, China4Shenzhen Key Laboratory of Minimally Invasive Surgical Robotics and System, Shenzhen Institutes

11、of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China5SIAT Branch, Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen 518055, China6Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong, ChinaCorrespond

12、ence: ; ; Tel.: +86-755-8639-2164growing because of great advantages of the latter including the high deformability and multiple degrees of freedom 11 14. Hu et al. 15 have proposed a shape-programmable elastic soft-bodied millirobot, whose magnetization profile can be

13、programmed to deliver time-varying shapes and achieves seven different types of locomotions in the external controllable magnetic field. Huang et al. 16 have combined thermally actuated hygromorphic materials and magnetic particles to enhance mobility of the soft microrobot. These works have well de

14、monstrated the soft microrobot with reconfigurable shape and controllable motility, but precise locomotion control is still under investigation.Recently, researchers have shown an increased interest in locomotion control at the microscale 1719. Xu et al.20 proposed a velocity-independent planar path

15、 following algorithm which enables a helical microswimmer to follow a straight line path with a stereo vision system. Guan et al. 21 demonstrated arbitrary planar path following of a helical mi- croswimmer using features directly presented in image space as feedbacks. Compared to the above 2D path f

16、ollowing, Oulmas et al. 22 designed a 3D path following controller for guiding a magnetically actuated helical swimmer with the kinematics expressed in the Serret-Frenet frame rather than global frame. However, the complicated parametric equations of the reference path are not repeatable and conveni

17、ent for non-smooth path following tasks in application scenarios and human-machine interaction manipulation.In this paper, we develop a 3D closed-loop motion control algorithm of a millimeter-scale soft film robot for 3D arbitrary path following based on the real-time visual tracking, where the 3D p

18、ath curve is divided into a series of line segments. Because of the decomposition of the 3D path following, the reference path can be easily drawn by users through a 3D mouse without the input of parametric equations. Such a soft film robot can be deformed as a chiral shape and produces a thrust for

19、ce from the surrounding viscous fluid with a uniform rotating magnetic field applied on in order to be wirelessly actuated in 3D space. A stereo vision system is used to measure the real-time position and advance direction of the robot. The proposed control algorithm was tested experimentally in dif

20、ferent complex path following. Friendly interactivity and robust performance proved in the experiments is especially significant in obstacle avoidance scenarios like targeted delivery of drugs.In the following parts of this paper, section II presents the modeling of the film microrobot, including th

21、e swimming model, the dynamics model and the 3D path following framework. Section III introduces the magnetic actuationCopyright 2019 IEEEng(a)8niformxHeadmagnetizi goyfieldzBmx zoyHeadInclination angle of the field(b)x yHeadxzMtzyHeadMsxyzMt _ netMs _ net(c)yBv pxzMtRotatingPlane of BMt(d) BMsv pMs

22、yxzRotating Plane of B(e)v pDecomposition ofMt _ netthemagnetizationMprofilez Bs _ nety ox Helical Propulsion Model system and the 3D stereo visual measurement. Section IV presents the results and analyses, including the overall path following error, the influences from model mismatch, as well as th

23、e stability of the control algorithm. Finally, section V concludes the paper, and presents the perspectives of future works.II. MODELINGA. Swimming modelThe schematic representation of a triangular film swim- ming robot is depicted in Fig.1. A specific magnetization profile of the robot can be creat

24、ed by bending the film into a quarter arc and placing it in a uniform magnetizing field Bm , as illustrated in Fig.1(a). The configuration will be further discussed in Section III. As shown in Fig.1(b), this magnetization profile can be represented as the sum ofcomponents magnetization profile Mt in

25、 x y plane andmagnetization profile Ms in y z plane.M (s) = m sin (w s) m cos (w s) 0 Tttsts(1)Ms (s) = 0 0 ms T2Lwhere mt, ms and ws represent the magnetization magnitude of the robot in xy plane, in yz plane and the spatial angular frequency, respectively, with ws = . s is the robots length with a

26、 range of 0-L.1) Magnetic actuation: The magnetization profiles of all magnetized objects will lead to magnetic torques with corre- sponding magnitudes by the field, which can be determined by:dTm = M BdVm(2)where dTm is the distributed magnetic torque, M is the magnetization related to the position

27、 along the films long axis, B is the externally imposed magnetic field, Vm is the volume related to the cross-sectional area of film.2) Dynamic morphology: A quasi-static analysis is used to describe how the film robot deforms in an external mag-Fig. 1: (a)Schematic of the miniature film swimming ro

28、bot and the magnetization process. (b) Concept of the robot with a specific magnetization profile which can be divided into magnetization profiles in xy plane and yz plane. Deformations of the robot within a uniform rotating magnetic field: (c) in xy plane, and (d) in xz plane. (e) A helical shape f

29、ormed in a uniform rotating magnetic field and the propulsion model of the robot at low Reynolds numbers.moments of the film robot. In this case of the radius of rotation of the head and the length of the tail are considered to be the same order of magnitude. Thus, the deformation of the film robot

30、can be described by twist angle (l) which can be represented by 23:dlnetic field B. For the deformation of the robot in x y plane( Fig.1(c), static deflections on an element can be described bythe Euler-Bernoulli equation where the rotational deflection=dlGJlwhere G is the shear modulus of the mater

31、ial and Jl is the(4)along the robot (s) is expressed as 15:2torsional rigidity. l is the rotation axis of the film robot. Thenet magnetic moment l of the head can be solved from Eq.2. Therefore, a helical shape of the robot is formed in 3D space 0 0 1 Mt(s) B A(s) = EI s2(3)where A(s), E and I are t

32、he cross-sectional area, Youngs modulus of the robot and second moment of area, respec- tively. Similarly, the deformation of the robot in xz plane can also be described( Fig.1(d).and the propulsion model is depicted in Fig.1(e).3) Helical propulsion model: Following 22, the relation- ship between t

33、he non-fluidic applied force F and torque T and the robot propulsion velocity and angular velocity in a helical propulsion model can be represented by:When placed in a uniform rotating magnetic field, the deflection of the robot will change continuously until the flexible film bents into a steady-st

34、ate shape. ConsideringFABvpT=BTC(5)the difference in cross-sectional area, the segment close to the head of the film robot will be more active to the static actuation. Therefore, the net magnetic moment Mt net and Ms net of the head can be represented by the total magneticwhere A, B and C are 3x3 su

35、bmatrices forming the mobility matrix which encapsulate the structural and environmental characteristics of the helical swimmer. Because of the low scale and uniform magnetic field, the non-fluidic appliedforce F given by the sum of gravity and magnetic forces is considered as a disturbance that wil

36、l be corrected by the kinematic controller developed in the next section.B. Kinematics equationsIn order to guide the robot to move along the desired direction in 3D space, a kinematic model is developed using inertial frame Uwith origin O, as illustrated in Fig.2. The position P of the robot is cha

37、racterized by the projection of the geometric center onto its rotation axis. The movement of the robot is decided by the steering angle vector which consist of the yaw angle and the pitch angle . The yaw angle is defined as an angle between the advance velocity and the z-axis, while the pitch angle

38、is defined as an angle between the axis of the advance velocity projected onto the horizontal plane xOy and the x-axis. Thus, the kinematic equation can be expressed as:x= v cos cos y= v cos sin xOnGPitch angle Yaw angle TUyzC21PCvBpvdD Ck 1CkReferencepath(n)C0kCnFig. 2: Modeling for arbitrary 3D pa

39、th following.be explained later in the paper. The notations are depicted as follows, as shown in Fig.2.z = v sin = z = y(6) (n) is a sequence of key points used to describe the reference 3D arbitrary path. Ck and Ck+1 are the starting and the ending key points of the current active segment on the re

40、ference path.is the speed ofwdehsecrreibethde inpoisniteirotnialvefrcatmore of Uthe, raonbdotvP = x, y, zT isthe robot, while and are the yaw and pitch angular velocity, respectively.One such non-ideality is the drift of the robot due to its own weight. Considering that the film robot is typically h

41、eavier than its fluid medium, the real advance velocity is not aligned with the propulsion direction of the robot which follows synchronously the axis of the rotating magnetic field generally (Fig.2). To simplify the velocity control problem without loss of generality, a gravity compensation is pro-

42、 posed, which can be expressed as:v = vp + nG(7)where v represents the real advance velocity vector, v P represents the position of the robot which character- ized by the projection of the geometric center onto its rotation axis. D is the projection of P onto the segment between Ck and Ck+1. The uni

43、t vector of CkCk+1 can be written as k. d is defined as the distance error which is the distance between point P and point D. n is the desired guiding direction vector of the robot for the next moment. vp is the vector of the propulsion velocity of the robot. v is the vector of the real advance velo

44、city of the robot. e and e are yaw angle error and pitch angle errorbetween the vector v and the vector n, which can be defined as the steering errors replaced by a 21 matrix:prepresents the propulsion velocity vector of the robot which is generally parallel to the rotation axis of the rotating magn

45、etic field, and nG represents a certain vector parallel to the gravity of the robot. The gravity compensation will be estimated in the experiments below.C. Arbitrary 3D path followingBased on the model of the robot above, a 3D arbitrary path following algorithm with friendly user interaction and goo

46、d performance is designed, which is also suitable for path following of most mobile robots with nonholonomic constraint in 2D or 3D space. A sequence of key points is able to be used to describe 3D arbitrary path drawn by users with a 3D mouse instead of complex descriptions like parametric equation

47、s, of which each two adjacent points are connected into a straight line to approximate a segment on the path. A closed-loop control method is designed to induce the robot to align with the desired path, which wille =e e.T out and B are the desired steering angle vector of the robot for the next mome

48、nt and the rotation axis of the external magnetic field, respectively.The 3D arbitrary path following can be considered as a combination of many straight segments path following, which is designed as an iterative process. When the distance between the position P of the robot and the ending point Ck+

49、1 of the current active path segment is less than a certain threshold, the next segment path following will be activated. The goal of closed-loop control for 3D path following is to drive the advance velocity v to align with the tangent vector of the path and track a reference profile at the same ti

50、me. The two-layered control framework proposed for the goal is depicted in Fig.3. The path following controller is to obtain the desired guiding direction of the robot, while the steering controller is to steer the robot in this direction. Thestate vector of the closed-loop control is then written a

51、s:Reference 3D path(n)Disturbancen*out3D Helmholtz B systemP , vv3D visual measurementLPFPath following controllerSteering controllerSwimmerP , vCalibrationFig. 3: Block diagram for the 3D following of the soft microswimmer.=qdeLet the advance velocity of the robot v be characterized by the yaw angl

52、e and pitch angle . There is:(8)#atan2(v , v )F(v) =yx(9)v22+varctan( vz) xywhere vx, vy, vzT is the coordinate of the vector v in the inertial frameU , and F( ) is a mapping relationship from the three-dimensional vector to the Euler angles.The real-time position of the soft swimming robot is estim

53、ated by a 3D stereo vision system. The real advance velocity of the robot can be represented by a vector given by the subtraction of the current position P and previous position Ppre of the robot, such that:Fig. 4: A vector field of the desired guiding direction (n) at different position in a straig

54、ht path segments vide a priority choice between the two velocity, which determines the speed of alignment to the reference path.cscsP=w R csP+w T v = P Ppre(10)A simple Lyapunov candidate and its derivative which are used to find stable conditions of this control solution can be express

55、ed as:wherew Rcs and w Tcs are the rotation and translation ma- trix from the side camera to the world coordinate system, respectively. csP is the coordinate of point P in the side1VT= d d(13)2camera system which can be calculated in the 3D stereo vision system 20. Thus, d and e can be expressed by:

56、V = 1 (dT d + d T d) = dT d2(14)d = D P = F(n) F(v)(11)eThe core of closed-loop control is minimizing the state vector (q) to zero. To find the desired guiding direction of the robot at any position in every straight path segments following, a possible control solution which consists of the desired guiding direction vector (n) as the resultant velocity of two components is shown in Fig.4.V is a positive scalar when d is not zero or null otherwise, and then V should be a negati