Graphical Abstract Figure
Graphical Abstract Figure
Close modal

Abstract

Weld seams of in-service pressure storage equipment, such as spherical tanks, require regular inspection to ensure safe operation. Wall-climbing robots can replace manual operations, increasing inspection efficiency and reducing maintenance costs. High precision and fast weld seam identification and tracking are beneficial for improving the automated navigation and spatial positioning of wall-climbing robots. This study proposes a weld seam recognition and tracking method with the omnidirectional wall-climbing robot for spherical tank inspection. Based on deep learning networks, the robot has a front-mounted camera to recognize weld seams and extract weld paths. Weld seam deviation data (drift angle and offset distance) were used in real time to provide feedback on the robot's relative position. For the robot to quickly correct deviations and track weld seams, a seam path-tracking controller based on sliding mode control was designed and simulated. Weld recognition experiments revealed that the robot can accurately recognize and extract weld paths, and the recognition time for each image was approximately 0.25 s. In the weld seam tracking experiments, the robot could successfully track longitudinal and transverse weld seams at different speeds (from 0.05 to 0.2 m/s). During the process of weld seam tracking, the robot angle error was kept within ±3 deg, and the maximum offset distance was less than ±35 mm. Field tests on a 3000-m3 spherical tank were conducted to verify the practicability and effectiveness of the weld seam tracking system. This robotic system can autonomously complete weld seam identification and tracking, which promotes the automation of spherical tank inspection and maintenance.

1 Introduction

Spherical tanks are commonly used to store chemical liquids and liquefied petroleum natural gas in the petroleum, chemical, and power industries. Multiple metal arc plates are used for welding spherical tanks in large-scale pressure vessel equipment. The weld seams between these metal plates may develop some internal defects, such as cracks, pores, and slag inclusions after a long period of service. Regular inspection and maintenance of weld seams on spherical tank surfaces can ensure safe production and operation. Compared with manual work, wall-climbing robots can adsorb and climb on wall surfaces, improving inspection efficiency and reducing maintenance costs. However, it is difficult for inspection robots to automatically identify and track weld seams because of the large space size of spherical tanks and the harsh operating environment.

Wall-climbing robots using various adsorption methods, such as negative pressure adsorption [14], magnetic adsorption [59], thrust adsorption [1012], and bionic adsorption [1315], have been developed and deployed. Metal walls are typically used for wall-climbing robots with permanent magnet adsorption. Permanent magnet adsorption can provide a greater adsorption force and no energy consumption than other methods. Magnetic adsorption wall-climbing robots have significant advantages in the application of spherical tank inspection. However, unlike climbing on planar walls, spherical tanks of different diameters have different surface curvatures. Wall-climbing robots for spherical tank inspection must adapt to curved working surfaces to stably adsorb and climb, which requires more reliable and flexible adaptation mechanisms.

Weld seams are the detection target, and the accurate weld seam recognition by the wall-climbing robot is the premise of automatic spherical tank inspection. Several weld identification methods have been tested and applied, such as weld line localization method using a cross-structured light device [16,17], an improved Canny algorithm to detect the edges of seam and extract the characteristic parameters of welding images [18], a seam tracking system based on dual cameras with auxiliary LED lights to present real-time image processing algorithms [19], and a novel intelligent system for the automatic visual inspection of vessels based on an ensemble of classifiers [20]. Because of the painted surface of spherical tanks and various disturbances, such as rust, corrosion, smudges, and scratches, it is difficult to accurately identify weld seams only by image processing algorithms. Deep learning methods as emerging identification technologies based on convolutional neural networks (CNNs) can achieve faster and more accurate target detection and classification. Some welding identification methods based on deep learning have also been developed and studied [21,22]. During the wall-climbing robot inspection on the curved surface of the spherical tanks, weld seam recognition and weld path tracking are indispensable capabilities of the robot, which contribute to more automated navigation operations and improve the efficiency of inspection and maintenance.

In our earlier studies, a series of wall-climbing robots have been developed and applied [23,24], which can effectively climb on tank surfaces. Because the surface of the spherical tanks is spray-painted and may be covered with corrosion and stains, it is difficult to accurately judge the weld path using conventional image processing methods. This study presents a weld seam recognition and tracking method for the designed omnidirectional wall-climbing robot. The wall-climbing robot with an adjustable frame and suspension mechanisms can adequately absorb curved surfaces with different diameters. Mecanum wheels enable the robot to climb omnidirectionally on spherical surfaces. Based on deep learning networks, the robot can quickly identify weld seams and fit weld paths for tracking using a front-mounted camera. A complete weld seam tracking control model of the robot was proposed and simulated based on the robot’s kinematic model and sliding mode control. Multiple experiments on a cylindrical tank were performed to test the efficiency and accuracy of weld seam recognition and tracking by the robot. Field tests on a 3000 m3 spherical tank were conducted to verify the feasibility and practicality of the seam recognition and tracking robotic system.

2 Omnidirectional Wall-Climbing Robot

Wall-climbing robots can be used to carry inspection equipment or maintenance tools to replace manual work on spherical tanks. These spherical tanks are welded by multiple arc-shaped plates, so the contact surfaces of wall-climbing robots are curved or spherical, and the weld seams are also distributed in the form of spatial curves, which puts higher requirements on climbing adaptability and flexibility of the robot. In this study, to achieve stable adsorption and flexible movement on the arc-shaped wall of the spherical tanks, an omnidirectional wall-climbing robot was designed.

2.1 Robot Mechanism Design.

The schematic of the developed wall-climbing robot for the weld seam inspection on spherical tanks is shown in Fig. 1(a). The omnidirectional wall-climbing robot includes the following components (Fig. 1(b)): four Mecanum wheels, four drive motors, an adjustable robot frame, four permanent magnets, four shock-absorbing suspension mechanisms, as well as robot control computer.

Fig. 1
Omnidirectional wall-climbing robot for spherical tank inspection: (a) spherical tank inspection and (b) developed omnidirectional wall-climbing robot
Fig. 1
Omnidirectional wall-climbing robot for spherical tank inspection: (a) spherical tank inspection and (b) developed omnidirectional wall-climbing robot
Close modal

The driving form of four Mecanum wheels enables the wall-climbing robot to move in all directions on tank surfaces, and the robot can adjust its position and speed more flexibly. When climbing on spherical tanks, which is more difficult than flat metal walls, the four wheels of the robot must adjust their inclination angle to adapt to spherical surfaces; otherwise, stable adsorption cannot be guaranteed. The adjustable robot frame can change the inclination angle of four Mecanum wheels to ensure full contact with spherical tank surfaces of different diameters. Four permanent magnets are located near the Mecanum wheels to provide adsorption force, and their positions change with the wheel's inclination angles. When surmounting obstacles or weld seams, four independent shock-absorbing suspension mechanisms keep the robot stable. The performance parameters of the omnidirectional wall-climbing robot are as follows: the outer dimension is 466 × 520 × 498 mm, the mass is 13.75 kg, the maximum payload is 10 kg, and the maximum climbing velocity is 0.23 m/s. Robot force analysis and climbing tests indicate that the robot can stably adsorb and climb on spherical surfaces with different diameters [25].

A camera is installed at the front of the robot to capture weld seam images on tank surfaces. The position parameters of the Mecanum wheels and camera are shown in Fig. 2. The Mecanum wheel radius r is 63.5 mm, the distance between the front and rear wheels Lw is 400 mm, and the distance between the left and right wheels Ww is 300 mm. hc and lc represent the installation height and distance of the camera, and their values are 200 and 100 mm, respectively. These robot design parameters determine the relative geometric relationship between the captured weld seam image and the center of the robot and are used to calculate the deviations during robot tracking weld paths.

Fig. 2
Robot dimensions and position parameters: (a) side view and (b) top view
Fig. 2
Robot dimensions and position parameters: (a) side view and (b) top view
Close modal

2.2 Control System Composition.

Figure 3 shows the weld recognition and robot control system. The entire robotic system includes an omnidirectional wall-climbing robot, a wireless router, and a remote computer. The electronic hardware of the wall-climbing robot consists of a robot control computer, a camera, a central controller, and encoders. Real-time images of weld seams are captured using the camera. The robot control computer is used to process these images and identify weld seams. A NVIDIA RTX2060 graphics processing unit (GPU) is equipped to accelerate deep learning network learning and weld seam recognition. The main functions of the robot control computer include image processing, weld seam recognition, robot position estimation, robot motion control, and data communication. The central controller is used to control the rotation speed of four motors and calculate the robot’s running speed through four encoders.

Fig. 3
Weld recognition and robot control system
Fig. 3
Weld recognition and robot control system
Close modal

Workers can use the remote control unit and remote computer to send commands and control the motion of the omnidirectional wall-climbing robot. The remote computer communicates with the robot through the wireless router. The remote computer can display real-time weld seam paths, motion states, and the spatial position of the robot.

3 Weld Path Identification

To realize automated weld seam inspection by the wall-climbing robot, weld seam recognition and path extraction are one of the most important tasks. After capturing the weld seam images, the robot needs to complete weld seam recognition, weld path fitting, and path deviation calculation before it performs autonomous weld tracking and inspection operations.

3.1 Weld Seam Recognition.

After a long period of outdoor work, the surface of spherical tanks is covered in dirt and rust, affecting the recognition accuracy of weld seams. Weld seam images on the tank surface captured by the wall-climbing robot are shown in Fig. 4. Weld seams on tank surfaces are classified as single transverse, single longitudinal, crossed, and T-shaped weld seams based on the characteristics of distribution locations. Different categories make fitting weld paths for tracking more difficult.

Fig. 4
Weld seam images captured by the omnidirectional wall-climbing robot
Fig. 4
Weld seam images captured by the omnidirectional wall-climbing robot
Close modal

Accurate identification of weld paths is beneficial to the automatic inspection and maintenance of wall-climbing robots. However, weld seam identification and path line extraction with high precision have always been arduous tasks. It is difficult to obtain intact seam path trajectories only by image processing because of the small distinction between weld seams and surroundings. Furthermore, the recognition method of image processing is very sensitive to illumination changes and image disturbances. Even if some features are obtained, the path information is mostly discontinuous, unclear, and distorted.

With the development of artificial intelligence in recent years, deep learning methods have gradually become a feasible solution to achieve more accurate distinction of weld objects through training and learning. Deep learning is generally composed of multilayer CNNs, which can classify objects more accurately after learning and training. The instance segmentation algorithms can achieve pixel-level classification and segmentation, which provides new means for weld seam recognition and extraction.

Figure 5 shows the basic process of weld seam recognizing and weld path fitting. First, weld seam images were captured by the camera mounted on the wall-climbing robot. Mask R-CNN [26] was chosen to identify weld seams from captured images and perform instance segmentation. Compared with PA-Net [27], RetinaMask [28], FCIS [29], and Mask R-CNN have excellent processing speed (116.3 ms) and mask mean average precision (35.7%). Some deep learning networks (such as YOLACT [30,31]) have faster recognition speed, but the mask mean average precision still does not exceed Mask R-CNN. Subsequently, weld paths were extracted and fitted using image processing and the least squares method.

Fig. 5
Work procedure of weld seam recognition and path fitting
Fig. 5
Work procedure of weld seam recognition and path fitting
Close modal

Mask R-CNN is a deep learning network based on CNNs. It can accomplish the instance segmentation of targets in images. Mask R-CNN's weld seam recognition process consists of feature extraction, candidate region processing, RoIAlign layer, classification, regression, and mask generation. First, a series of CNN layers (such as ResNet50 and ResNet101) combined with feature pyramid networks [32], as backbone networks, was used to extract different depth feature maps from weld seam images. The region proposal network can generate regions of interest (RoI). The corresponding features of each RoI on the feature maps were extracted by RoIAlign with bilinear interpolation to avoid deviations.

However, after the RoIAlign layer, RoIs are fed into two fully connected layers for image classification and bounding box regression to determine categories and boxes of the objects. Meanwhile, a mask prediction branch uses fully convolutional networks [33] to segment objects in the image in pixels, and predicts m × m masks from each RoI.

The wall-climbing robot has collected 1967 images of weld seams on the surface of a tank, with a resolution of 320 × 240 pixels. Smaller sizes can reduce training and recognition time during the training and testing process, but it has little impact on weld path fitting. The contours of the weld seams in these acquired images are labeled and masked in advance. Data augmentation methods, such as Gaussian blur, geometric transformation, brightness adjustment, and noise addition, are used to expand the training dataset. After 20 times of data augmentation, the training dataset includes 39,340 weld seam images. The number of categories in the dataset was 2, representing background and weld seam.

The network model was trained on the robot control computer with NVIDIA GeForce RTX2060 for GPU acceleration. Its parameters were set as follows: The initial learning rate of the network was 0.001; the weight attenuation coefficient was 0.0005; the momentum coefficient was 0.8. After 30,000 iterations of training, the total loss of the network model is less than 0.06, and the average accuracy is greater than 0.98. As shown in Fig. 6, the recognition results of weld seams include classification probabilities, regression boxes, and pixel-level masks. Based on the deep learning network model, the weld seams can be classified and recognized for further weld path extraction.

Fig. 6
Testing results of weld seam recognition
Fig. 6
Testing results of weld seam recognition
Close modal

3.2 Weld Path Fitting.

After weld seams were recognized, the binarized weld seam images were obtained through image processing methods such as color distinction, binarization processing, and Gaussian filter, and then, the least squares method was used to fit the centerlines of weld paths. Two-position parameters of weld paths were estimated, such as αp and dp. Figure 7 shows the fitted weld path and path parameters. αp is the drift angle of the identified weld path in the image and dp is the transverse distance (pixel) from the weld seam to the center of the image. When the robot reaches an intersected weld point, two weld seams will be recognized. The robot tracks one weld path according to manual selection or path planning.

Fig. 7
Position parameters of weld paths: (a) original weld image, (b) recognized weld image, and (c) weld path image and position parameters
Fig. 7
Position parameters of weld paths: (a) original weld image, (b) recognized weld image, and (c) weld path image and position parameters
Close modal

The line equation of a weld path is given as h(x)=kx+b. When the quadratic sum G(x) of distances from all weld points to a line is the smallest, the line is considered the centerline of the weld path.

In the images of single weld seams, if the coordinates of weld points are (xi,yi) and their total number is m, G(x) is expressed as follows:
(1)
when G(x) takes its minimum value, the corresponding k and b can be calculated. The solution of the minimum value of G(x) can be transformed into finding the extremum of G(x). Because it is a linear equation, the partial derivatives of G(x) to k and b are expressed as follows:
(2)
when G/k=0 and G/b=0, the calculation equation can be obtained as follows:
(3)
where p=i=1mxi2,q=i=1mxi,r=i=1mxiyi,u=i=1mxi,v=m,andw=i=1myi.
The values of k and b are solved as follows:
(4)
Then, the drift angle αp and offset distance dp are calculated as follows:
(5)
where (xp,yp) is the center coordinate of weld seam images.

3.3 Robot Tracking Error.

After the weld seam path was fitted, the positional deviation of the robot relative to the weld seam can be calculated. Because the camera was installed on the front of the robot, the drift angle αp and offset distance dp should be converted into the deviation relative to the robot center. The camera can capture 120-deg wide-angle images without distortion, with a focal length of 2.8 mm. When the installation height of the camera hc is 200 mm, the actual width that the captured image can cover is 205 mm. When the image resolution is set to 320 × 240 pixels, we can obtain the following conversion formula:
(6)
where αm is the actual drift angle of the fitted weld path relative to the camera and dm is the actual offset distance of the weld path relative to the center of the camera.

Figure 8 shows the positional relationship between the tracked weld path and the wall-climbing robot. Point Oc is the center of the robot, point Om is the center of the camera, and the robot coordinate system xOcy is established. According to the robot design parameters, the distance between point Oc and point Om is lc+Lw/2. αc and dc are the drift angle and offset distance of the robot center relative to the tracked weld path.

Fig. 8
Positional deviation of the wall-climbing robot relative to weld paths
Fig. 8
Positional deviation of the wall-climbing robot relative to weld paths
Close modal
According to the geometric relationship, αc and dc can be calculated as follows:
(7)
According to Eqs. (6) and (7), we can obtain
(8)

The deviation conversion in Eq. (8) indicates that αc and αp are linearly dependent, and changes in αc or αp will cause changes in dc. The image range captured by the camera is limited, which places higher requirements on the robot weld seam tracking algorithm. If the robot cannot quickly correct its position and angle, the camera may lose weld seam images, and the robot will fail to track the weld path. At the beginning of weld seam tracking, the tracked weld seam should be within the maximum camera range.

4 Weld Path-Tracking Control

The weld recognition method based on deep learning can identify and extract the weld paths, and the arc-shaped welds on the surface of the spherical tank bring more difficulty to the robot's autonomous operation. Fast and stable path-tracking control is conducive to improving the robot's inspection efficiency and reducing energy consumption. Based on the omnidirectional motion form of the robot and the calculation of the weld path deviations, the control method of tracking weld paths directly affects the accuracy of the robot's automatic inspection operation.

4.1 Kinematic Model.

Based on four Mecanum wheels, the robot can move and rotate in any direction. Assembled encoders are used to measure the rotation speed of each wheel. Four independent proportion integral differential (PID) controllers were developed and used to accurately control wheel speeds. During the robot motion process, the inverse kinematics equation is expressed as follows:
(9)
(10)
where θ˙i are the angular velocities of the driving wheels and vi are the corresponding linear velocities. vx, vy, and ωc are the forward velocity, traverse velocity, and angular velocity of the robot, respectively. βw is the angle between the roller and the axis of the Mecanum wheel.

4.2 Robot Postural Adjustment.

Based on the adjustable robot frame and shock-absorbing elastic suspension mechanisms, the robot can climb flexibly on the surface of spherical tanks. Because the robot is much smaller than spherical tanks, the weld seam tracking motion on tank surfaces can be simplified as a plane motion. During the weld seam tracking process, the wall-climbing robot continuously corrects the drift angle and offset distance to ensure that the robot’s climbing direction coincides with the weld path direction.

Assuming that weld paths are distributed in a global coordinate system XOY, the initial state vector of the robot is qc=[xcycac]T, where xc and yc represent the current coordinate values of the robot, and ac represents the heading angle of the robot. The desired state vector is qd=[xdydad]T, which is the ideal state for weld path tracking by the robot. During the weld seam tracking process, the robot state error vector is qe=[xeyeae]T, where xe, ye, and ae represent the coordinate and angle errors, respectively. According to the coordinate transformation, we can obtain:
(11)
xe, ye, and ae are as follows:
(12)
Differentiate Eq. (12):
(13)
where vx, vy, and ωc are the current velocities and angular velocity of the robot, respectively. vxd, vyd, and ωd are the desired velocities and angular velocity, respectively.

Figure 9 shows the position correction analysis during the robot weld seams tracking process. The weld seam coordinate system XOwY is established, and the tracked weld path is distributed along the Y coordinate axis. The starting point of the robot is Oc, and the desired point is Od. The distance between point Oc and point Ow is dc, the angle between the robot's forward direction and the Y coordinate axis is ac. In the coordinate system XOwY, the current state vector of the robot is qc=[dcsinacdccosacac]T. When tracking the weld seam, the robot should reach point Od on the weld path (Y coordinate axis), and the next desired state vector is qd=[xd00]T.

Fig. 9
Weld seam tracking analysis of omnidirectional wall-climbing robot
Fig. 9
Weld seam tracking analysis of omnidirectional wall-climbing robot
Close modal
When the robot tracks the weld path, combined with Eq. (12), xe, ye, and ae can be expressed as follows:
(14)

4.3 Sliding Mode Controller.

Figure 10 shows the control diagram of the weld seam tracking by the omnidirectional wall-climbing robot using sliding mode control. In the initial stage, after the weld path was identified, the robot calculates and converts the deviations of the weld path. The sliding mode controller is used to adjust the robot's posture and position, and the three velocities (vx,vy,andωc) of the robot will be corrected simultaneously. The velocities of robot wheels are modified based on the robot kinematic model. During the wheel speed regulation process, four independent PID controllers are used to complete the closed-loop control of wheel speeds using the data recorded by four encoders.

Fig. 10
Weld seam tracking of the omnidirectional wall-climbing robot based on the sliding mode control
Fig. 10
Weld seam tracking of the omnidirectional wall-climbing robot based on the sliding mode control
Close modal
Because the omnidirectional wall-climbing robot has three degrees of freedom, the sliding mode controller must control three velocity variables. ev=[xeyeae]TR3×1 is defined as the difference between the desired state vector and the real state vector of the robot. The sliding mode surface vector. S=[s1s2s3]TR3×1 is defined as
(15)
where K=[k1k2k3]T is a positive integral gain matrix.
The derivative of S is given as the following:
(16)
After choosing the reaching law, the following can be obtained:
(17)
where Q=diag(Q1,Q2,Q3) and P=diag(P1,P2,P3) are diagonal positive definite matrices.
Combined with Eqs. (13), (16), and (17), we obtain the following:
(18)
Simplified to
(19)
Finally, we obtain the following:
(20)
From Eq. (20), the control velocities of the robot can be solved when tracking weld paths. The desired forward velocity vxd of the robot tracking weld path can be given manually, desired traverse velocity vyd and rotational angular velocity ωd are set as follows:
(21)

vyd and ωd are linearly related to dc and ac, respectively. When there is no offset distance and drift angle between the robot and the tracked weld path, vyd and ωd are 0.

The stability analysis is as follows:
(22)
The derivative of V is given below:
(22)

If Qi0 and Pi0, V is negative semidefinite. According to Barbalat's lemma, S0 as t0, when S0, ev=Kevdt, and ev0 as t0.

4.4 Simulation.

Simulation experiments were performed to verify the correctness and feasibility of the weld path-tracking controller. During the weld path-tracking process, the maximum forward velocity and traverse velocity of the robot is 0.23 m/s, and the maximum rotational angular velocity is 0.94 rad/s. The simulation parameters are set as follows: the initial drift angle αp in the weld image is π/6 rad, the initial offset distance dp is 80 pixels, and the given tracking velocity vxd is 0.1 m/s. The parameters of the sliding mode controller are as follows: kω=0.1, ky=0.8, Q1=Q3=0.5, Q2=0.75, P1=P2=P3=0.01, and k1=k2=k3=1.

The simulation results of the robot tracking weld path are shown in Fig. 11. The drift angle and offset distance of the robot can be quickly corrected, the robot state errors ye and ae are also corrected accordingly, and the robot tracking error xe is kept at 0.1 m. The simulation results indicate that the designed controller can accurately adjust robot velocities based on the weld path deviations and the robot can successfully track the weld path. In subsequent experiments and field tests, the relevant parameters of the seam path-tracking controller will be optimized and adjusted to ensure that the actual wall-climbing robot stably tracks the weld seams on tank surfaces.

Fig. 11
Simulation results of welding seam tracking based on sliding mode control
Fig. 11
Simulation results of welding seam tracking based on sliding mode control
Close modal

5 Experiments and Field Tests

To verify the accuracy of weld seam recognition and tracking, we conducted experiments with the climbing robot on an experimental platform. The experiments mainly include weld recognition experiments and weld seam tracking experiments. In addition, field tests on a real spherical tank were also carried out to verify the feasibility and stability of the weld seam recognition and tracking control method as well as the developed robotic system in application.

5.1 Weld Recognition Experiments.

The experimental platform, which is a cylindrical tank, was welded from multiple arc-shaped plates (Fig. 12). The cylindrical tank has a diameter of 4000 mm, a height of 2.8 m, and a wall thickness of 10 mm. Multiple longitudinal and transverse weld seams are distributed on the tank surface for robot recognition and tracking. The omnidirectional wall-climbing robot can stably adsorb and climb on the vertical curved surface of the cylindrical tank.

Fig. 12
Experimental platform and omnidirectional wall-climbing robot
Fig. 12
Experimental platform and omnidirectional wall-climbing robot
Close modal

In the weld seam recognition experiments, the robot used the camera to capture weld seam images of the tank surface, recognized weld seams, and extracted weld paths based on deep learning networks. The robot's real-time recognition results of weld seams are shown in Fig. 13. The results indicate that the weld seam recognition method based on deep learning can accurately identify weld seams and is not easily disturbed by the surrounding environment, such as rust, stains, and scratches.

Fig. 13
Real-time recognition results of weld seam by omnidirectional wall-climbing robot
Fig. 13
Real-time recognition results of weld seam by omnidirectional wall-climbing robot
Close modal

Figure 14 shows the real-time weld path extraction results, including original weld images and binary path images. The robot could extract the angle and offset distance of weld paths in real time, which were then used to feedback on the position of the robot relative to the weld seam. The total processing time of each image was kept between 0.22 and 0.25 s, including image loading loss time (0.006 s), deep learning identification time (0.21–0.23 s), and image processing time (0.018 s). The experimental results indicate that the robot can successfully identify and extract weld paths. The robotic system can obtain at least 4 sets of weld path data per second, which can meet the inspection requirements of the wall-climbing robot.

Fig. 14
Real-time weld path extraction results by omnidirectional wall-climbing robot: (a) angle: 7.6 deg, distance: 104.88 pixels, (b) angle: −10.54 deg, distance: −66.36 pixels, (c) angle: 1.33 deg, distance: 47.72 pixels, and (d) angle: −2.31 deg, distance: −50.89 pixels
Fig. 14
Real-time weld path extraction results by omnidirectional wall-climbing robot: (a) angle: 7.6 deg, distance: 104.88 pixels, (b) angle: −10.54 deg, distance: −66.36 pixels, (c) angle: 1.33 deg, distance: 47.72 pixels, and (d) angle: −2.31 deg, distance: −50.89 pixels
Close modal

5.2 Weld Seam Tracking Experiments.

Weld seam tracking experiments were performed to verify the tracking accuracy of the wall-climbing robot. In the experiments, the robot adsorbed on the vertical arc-shaped wall of the experimental platform and automatically recognized and tracked the weld seams. The experiments included longitudinal weld seam tracking experiments and transverse weld seam tracking experiments. The given desired tracking velocity of the robot included ±0.05 m/s, ±0.1 m/s, ±0.15 m/s, and ±0.2 m/s. The real-time drift angle and offset distance of the robot relative to the tracked weld seam were recorded during weld seam tracking.

Figure 15 shows the process of longitudinal weld seam tracking experiments. In the beginning, it is necessary to ensure that the robot camera can capture images of weld seams. The initial drift angle and offset distance were random. The robot tracked a longitudinal weld seam upward at different velocities and then tracked downwards (without turning around) at the same velocity.

Fig. 15
Longitudinal weld seam tracking experiments: (a–e) tracking upward and (f–j) tracking downward
Fig. 15
Longitudinal weld seam tracking experiments: (a–e) tracking upward and (f–j) tracking downward
Close modal

Figure 16 shows the experimental results of the robot longitudinal seam tracking at different velocities. The robot angle (drift angle) and offset distance fluctuate significantly during the initial stage of tracking weld seams, but they are quickly adjusted back. In subsequent stages, the robot can stably track weld seams, and the robot angle and offset distance are continuously corrected. During the upward tracking weld seams, the robot angle deviations at different velocities are kept within ±3 deg (Fig. 16(a)). As shown in Fig. 16(b), at low tracking velocities (0.05 and 0.1 m/s), the maximum offset distances of the robot are less than ±20 mm. The offset distances slightly increase as the tracking velocity increases (0.15 and 0.2 m/s), but it can still be maintained within ±30 mm. Just like upward tracking seams, the downward tracking seam performance of the robot is also excellent (Figs. 16(c) and 16(d)). In longitudinal seam tracking experiments, the robot could flexibly adjust its velocities and position to smoothly track the weld seams with high precision. The experimental results indicate that the wall-climbing robot with a designed weld seam tracking controller can accurately track longitudinal seams on vertical tank surfaces.

Fig. 16
Experimental results of longitudinal weld seam tracking by the wall-climbing robot: (a) drift angle when upward tracking; (b) offset distance when upward tracking; (c) drift angle when downward tracking; and (b) offset distance when downward tracking
Fig. 16
Experimental results of longitudinal weld seam tracking by the wall-climbing robot: (a) drift angle when upward tracking; (b) offset distance when upward tracking; (c) drift angle when downward tracking; and (b) offset distance when downward tracking
Close modal

Figure 17 shows the experimental process of transverse weld seam tracking by the robot. The transverse weld seams are distributed in the middle of the experimental tank. Experimental results of robot transverse weld seam tracking at different velocities are shown in Fig. 18. The robot can correct deviations and track the weld seam with high precision for diverse initial angles and distances. Whether tracking the weld seam forward or backward, the drift angle of the robot can still be maintained within ±3 deg. When the robot tracking weld seam at 0.05 and 0.1 m/s, the offset distances are maintained within ±20 mm. The offset distance increases with the tracking velocity. For example, when the robot tracking velocity is 0.2 m/s, the maximum offset distance of the robot reaches −35 mm. The reason for this is that affected by gravity, the robot may slip slightly during transverse weld seams tracking, increasing the offset distance. The experimental results indicate that the maximum offset distance of transverse weld seams tracking at low velocity is ±20 mm, and the maximum offset distance at high velocity is ±35 mm. The weld seam tracking controller can enable the robot to track weld seams with high precision and correct the deviation caused by the sliding.

Fig. 17
Transverse weld seam tracking experiments: (a–e) tracking forward and (f–j) tracking backward
Fig. 17
Transverse weld seam tracking experiments: (a–e) tracking forward and (f–j) tracking backward
Close modal
Fig. 18
Experimental results of transverse weld seam tracking by the wall-climbing robot: (a) drift angle when forward tracking, (b) offset distance when forward tracking, (c) drift angle when backward tracking, and (b) offset distance when backward tracking
Fig. 18
Experimental results of transverse weld seam tracking by the wall-climbing robot: (a) drift angle when forward tracking, (b) offset distance when forward tracking, (c) drift angle when backward tracking, and (b) offset distance when backward tracking
Close modal

In the weld seam tracking experiments, based on the designed weld seam tracking controller, the wall-climbing robot could consistently track the longitudinal and transverse weld seams with high precision. After receiving the tracking command, the robot could quickly correct the random initial drift angle and offset distance. During the stable stage of weld seam tracking, the robot continuously adjusted its posture and corrected the deviations. The robot could still maintain stable tracking at a higher velocity, and the maximum drift angle and offset distance were maintained within ±3 deg and ±35 mm. The omnidirectional wall-climbing robot with high-precision weld seam tracking capability can improve the automatic inspection of spherical tanks.

5.3 Field Tests.

Field tests were conducted to verify the adaptability and stability of automatic weld seam tracking for the omnidirectional wall-climbing robot in engineering applications. As shown in Fig. 19, the wall-climbing robot was tested on a real liquefied petroleum gas (LPG) spherical tank. It has a volume of 3000 m3, a diameter of 18 m, a maximum height of 20 m, and a thickness of 49 mm. Multiple weld seams are distributed on the surface of the spherical tank, the total length of weld seams is 455.46 m. Because of the large volume of industrial spherical tanks, the traditional manual inspection method will be expensive and time-consuming. Manual inspections often require a large amount of scaffolding to detect all weld seams, whereas the wall-climbing robot can flexibly reach any weld seam position for inspection.

Fig. 19
Field tests on LPG spherical tank
Fig. 19
Field tests on LPG spherical tank
Close modal

The robot captured weld images of the spherical tank to expand the training dataset. Before the field tests, new 2172 weld images on the surface of the spherical tank were collected. After data augmentation, the training dataset was added with 43,440 weld images. On other different tanks, the training dataset of Mask R-CNN will continue to increase. Deep learning is a method of continuous learning. With more training and learning, the networks can recognize more types of weld seams. Presently, the robot has trained numerous weld images of different spherical tanks.

Figure 20 shows the real-time image results of the robot weld seam tracking on the spherical tank surface. Based on deep learning, the robot can accurately extract the weld paths on the spherical tank, the average recognition time of each weld image was approximately 0.25 s. Figure 21 shows the test process of robot tracking weld seams on the spherical tank. In the test, the robot tracked multiple longitudinal and transverse weld seams. The results indicate that the robot can quickly identify and extract weld paths with high precision, and automatically run along the weld seams. When the robot reaches one intersected weld point, the robot will automatically stop waiting for the control command, and workers can select the next tracked weld seam through the remote computer. If the robot fails to recognize or misidentifies weld seams, the robot will be switched to the remote control mode to resolve the issue.

Fig. 20
Real-time weld seam tracking by the robot on the LPG spherical tank: (a) angle: 1.88 deg, distance: 7.93 pixels, (b) angle: 0.31 deg, distance: −31.67 pixels, (c) angle: 2.3 deg, distance: 3.91 pixels, (d) angle: −3.5 deg, distance: 26.56 pixels, (e) angle: 4.78 deg, distance: −35.07 pixels, and (f) angle: −1.44 deg, distance: 54.8 pixels
Fig. 20
Real-time weld seam tracking by the robot on the LPG spherical tank: (a) angle: 1.88 deg, distance: 7.93 pixels, (b) angle: 0.31 deg, distance: −31.67 pixels, (c) angle: 2.3 deg, distance: 3.91 pixels, (d) angle: −3.5 deg, distance: 26.56 pixels, (e) angle: 4.78 deg, distance: −35.07 pixels, and (f) angle: −1.44 deg, distance: 54.8 pixels
Close modal
Fig. 21
Weld seam tracking tests on LPG spherical tank
Fig. 21
Weld seam tracking tests on LPG spherical tank
Close modal

The field tests verified the efficiency of the weld seam recognition and control method in engineering applications. The developed robot control system enhances the practicability of the wall-climbing robot and improves the automatic inspection of spherical tanks. Equipped with inspection and maintenance mechanisms, the wall-climbing robot can complete more inspection and maintenance tasks.

5.4 Discussion.

There remains a lack of effective implementation methods for tracking weld seams and automatic inspection operation by the robots in real spherical tanks. In this study, we proposed a weld seam recognition and weld path-tracking control method suitable for the developed omnidirectional wall-climbing robot to implement weld seam inspection on arc-shaped working surfaces of cylindrical tanks and spherical tanks. The designed omnidirectional wall-climbing robot could provide flexible movement forms for automated weld seam tracking, and the recognition method-based deep learning networks enabled more accurate extraction of welding seam paths. By analyzing the weld path deviations, the control method based on sliding mode control was proposed and applied to adjust the movement velocities of the robot during the weld seam tracking process. In the cylindrical tank experiments and field tests of the real spherical tank, the robot could complete the recognition and tracking of the weld seams, with a maximum drift angle of ±3 deg and a maximum offset distance of ±35 mm. During the process of the robot tracking the weld seams, due to the continuous adjustment of the running velocities and angular velocity, the tracking route fluctuated slightly, but it could be corrected in time. This weld seam recognition and weld path-tracking control method is conducive to improving the autonomous operation capabilities of the spherical tank inspection robots.

There may be some possible limitations in this study, the first limitation is that the weld seam training dataset may be insufficient, and the deep learning networks can be continuously optimized to improve the accuracy and speed of weld seam recognition and extraction. The second limitation concerns that more spherical tanks should be tested and implemented to improve the recognition accuracy of weld seams. More in-depth follow-up research and applications will be conducted to optimize the performance of robot tracking weld seams, including establishing more comprehensive weld seam datasets, optimizing deep learning models, and conducting more field tests.

6 Conclusion

In this study, a weld seam recognition and tracking control method with the developed omnidirectional wall-climbing robot for spherical tank inspection is proposed. The designed wall-climbing robot adopts Mecanum wheels to achieve omnidirectional movement. With the adjustable robot frame and shock-absorbing suspension mechanisms, the robot can be applied to spherical tanks of different diameters. The robot captures real-time images of weld seams through the front-mounted camera. It can be controlled and monitored by a remote computer. The robot can quickly recognize weld seams and accurately extract weld paths based on deep learning networks. Real-time drift angle and offset distance are used to feedback on the position of the robot relative to the tracked weld seam. Based on the kinematic model and sliding mode control, the robot can quickly correct deviations and adjust its position to track weld seams. In the weld seam recognition experiments, the wall-climbing robot could identify weld seams and extract weld paths in real time, and the processing time of each weld image was approximately 0.25 s. In the weld seam tracking experiments, at different velocities, the robot successfully and continuously tracked longitudinal and transverse weld seams in two directions. At low velocity, the robot could achieve more accurate weld seam tracking, and the maximum drift angle and offset distance were ±3 deg and ±20 mm. The experimental results indicate that the robot can recognize and track weld seams with high accuracy.

Furthermore, field tests on a 3000-m3 spherical tank were conducted to verify the practicability and reliability of the developed robotic system. High precision and fast weld seam recognition and tracking facilitate automated and intelligent inspection of the wall-climbing robot. More tests on different spherical tanks will be conducted in the future to optimize the performance of the robotic system. More weld seam images will be collected to extend the training dataset, which will benefit the recognition accuracy and application range of the inspection robot.

Acknowledgment

This project is supported by the Natural Science Foundation of Jiangsu Province, China (BK202300367), the Science and Technology Project of the State Administration for Market Regulation, China (2023MK040), the Open Project of Anhui Province Key Laboratory of Special and Heavy Load Robot (TZJQR002-2024), and the Open Project of China International Science and Technology Cooperation Base on Intelligent Equipment Manufacturing in Special Service Environment (ISTC2023KF06).

Conflict of Interest

There are no conflicts of interest.

Data Availability Statement

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.

References

1.
Ge
,
D.
,
Tang
,
Y.
,
Ma
,
S.
,
Matsuno
,
T.
, and
Ren
,
C.
,
2020
, “
A Pressing Attachment Approach for a Wall-Climbing Robot Utilizing Passive Suction Cups
,”
Robotics
,
9
(
2
), p.
26
.
2.
Chen
,
N.
,
Shi
,
K.
, and
Li
,
X.
,
2020
, “
Theoretical and Experimental Study and Design Method of Blade Height of a Rotational-Flow Suction Unit in a Wall-Climbing Robot
,”
ASME J. Mech. Rob.
,
12
(
4
), p.
045002
.
3.
Sakagami
,
N.
,
Yumoto
,
Y.
,
Takebayashi
,
T.
, and
Kawamura
,
S.
,
2019
, “
Development of Dam Inspection Robot With Negative Pressure Effect Plate
,”
J. Field Rob.
,
36
(
8
), pp.
1422
1435
.
4.
Fujita
,
M.
,
Ikeda
,
S.
,
Fujimoto
,
T.
,
Shimizu
,
T.
,
Ikemoto
,
S.
, and
Miyamoto
,
T.
,
2018
, “
Development of Universal Vacuum Gripper for Wall-Climbing Robot
,”
Adv. Rob.
,
32
(
6
), pp.
283
296
.
5.
Zhang
,
W.
,
Zhang
,
W.
, and
Sun
,
Z.
,
2021
, “
A Reconfigurable Soft Wall-Climbing Robot Actuated by Electromagnet
,”
Int. J. Adv. Rob. Syst.
,
18
(
2
), p.
1729881421992285
.
6.
Zhang
,
X.
,
Zhang
,
M.
,
Jiao
,
S.
,
Sun
,
L.
, and
Li
,
M.
,
2024
, “
Design and Optimization of the Wall Climbing Robot for Magnetic Particle Detection of Ship Welds
,”
J. Mar. Sci. Eng.
,
12
(
4
), p.
610
.
7.
Fan
,
J.
,
Xu
,
T.
,
Fang
,
Q.
,
Zhao
,
J.
, and
Zhu
,
Y.
,
2020
, “
A Novel Style Design of a Permanent-Magnetic Adsorption Mechanism for a Wall-Climbing Robot
,”
ASME J. Mech. Rob.
,
12
(
3
), p.
035001
.
8.
Li
,
B.
,
Ushiroda
,
K.
,
Yang
,
L.
,
Song
,
Q.
, and
Xiao
,
J.
,
2017
, “
Wall-Climbing Robot for Non-Destructive Evaluation Using Impact-Echo and Metric Learning SVM
,”
Int. J. Intell. Rob. Appl.
,
1
, pp.
255
270
.
9.
Dissanayake
,
M.
,
Sattar
,
T. P.
,
Lowe
,
S.
,
Pinson
,
I.
, and
Gan
,
T. H.
,
2018
, “
Adaptable Legged-Magnetic Adhesion Tracked Wheel Robotic Platform for Misaligned Mooring Chain Climbing and Inspection
,”
Ind. Robot Int. J.
,
45
(
5
), pp.
634
646
.
10.
Zhou
,
Q.
, and
Li
,
X.
,
2018
, “
Experimental Investigation on Climbing Robot Using Rotation-Flow Adsorption Unit
,”
Rob. Auton. Syst.
,
105
, pp.
112
120
.
11.
Chang
,
S.
,
Yang
,
F.
,
Qian
,
X.
,
Cui
,
S.
,
Wang
,
J.
,
Zhao
,
L.
, and
Ren
,
R.
,
2024
, “
Motion Characteristics of a Reverse Thrust Adsorption Wall-Climbing Robot With Multi-Degree-of-Freedom Propeller
,”
Int. J. Rob. Autom.
,
39
(
2
), pp.
137
149
.
12.
Vega-Heredia
,
M.
,
Mohan
,
R. E.
,
Wen
,
T. Y.
,
Siti'Aisyah
,
J.
,
Vengadesh
,
A.
,
Ghanta
,
S.
, and
Vinu
,
S.
,
2019
, “
Design and Modelling of a Modular Window Cleaning Robot
,”
Autom. Constr.
,
103
, pp.
268
278
.
13.
Xu
,
F.
,
Meng
,
F.
,
Jiang
,
Q.
, and
Peng
,
G.
,
2020
, “
Grappling Claws for a Robot to Climb Rough Wall Surfaces: Mechanical Design, Grasping Algorithm, and Experiments
,”
Rob. Auton. Syst.
,
128
, p.
103501
.
14.
Bian
,
S.
,
Wei
,
Y.
,
Xu
,
F.
, and
Kong
,
D.
,
2021
, “
A Four-Legged Wall-Climbing Robot With Spines and Miniature Setae Array Inspired by Longicorn and Gecko
,”
J. Bionic Eng.
,
18
(
2
), pp.
292
305
.
15.
Huang
,
X.
,
Zhang
,
C.
,
Feng
,
W.
,
Zhang
,
X.
,
Zhang
,
D.
, and
Liu
,
Y.
,
2024
, “
A Bionic Starfish Adsorption Crawling Soft Robot
,”
J. Bionic Eng.
,
21
(
1
), pp.
149
165
.
16.
Yang
,
L.
,
Liu
,
Y.
, and
Peng
,
J.
,
2020
, “
Advances Techniques of the Structured Light Sensing in Intelligent Welding Robots: A Review
,”
Int. J. Adv. Manuf. Technol.
,
110
(
3
), pp.
1027
1046
.
17.
Dong
,
Z.
,
Mai
,
Z.
,
Yin
,
S.
,
Wang
,
J.
,
Yuan
,
J.
, and
Fei
,
Y.
,
2020
, “
A Weld Line Detection Robot Based on Structure Light for Automatic NDT
,”
Int. J. Adv. Manuf. Technol.
,
111
(
7–8
), pp.
1831
1845
.
18.
Ye
,
H.
,
Liu
,
Y.
, and
Su
,
K.
,
2021
, “
A Modified Method for Welding Seam Location of Tube-Sheet Welding Based on Image Edge Segmentation
,”
2021 IEEE Fourth Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC)
,
Chongqing, China
,
June 18–20
, Vol. 4, IEEE, pp.
1913
1917
.
19.
Wang
,
Z.
,
Zhang
,
K.
,
Chen
,
Y.
,
Luo
,
Z.
, and
Zheng
,
J.
,
2017
, “
A Real-Time Weld Line Detection for Derusting Wall-Climbing Robot Using Dual Cameras
,”
J. Manuf. Processes
,
27
, pp.
76
86
.
20.
Maglietta
,
R.
,
Milella
,
A.
,
Caccia
,
M.
, and
Bruzzone
,
G.
,
2018
, “
A Vision-Based System for Robotic Inspection of Marine Vessels
,”
Signal, Image Video Process.
,
12
(
3
), pp.
471
478
.
21.
Ma
,
Y.
,
Fan
,
J.
,
Zhou
,
Z.
,
Zhao
,
S.
,
Jing
,
F.
, and
Tan
,
M.
,
2024
, “
WeldNet: A Deep Learning Based Method for Weld Seam Type Identification and Initial Point Guidance
,”
Expert Syst. Appl.
,
238
, p.
121700
.
22.
Wang
,
J.
,
Mu
,
C.
,
Mu
,
S.
,
Zhu
,
R.
, and
Yu
,
H.
,
2023
, “
Welding Seam Detection and Location: Deep Learning Network-Based Approach
,”
Int. J. Press. Vessels Pip.
,
202
, p.
104893
.
23.
Li
,
J.
,
Dong
,
L.
,
Tian
,
M.
,
Tu
,
C.
, and
Wang
,
X.
,
2023
, “
Adjustable Magnetic Adsorption Omnidirectional Wall-Climbing Robot for Tank Inspection
,”
Proceedings of 2022 International Conference on Intelligent Systems Design and Engineering Applications
,
Tokyo, Japan
,
May 13–15
, pp.
67
80
.
24.
Gui
,
C.
,
Li
,
J.
,
Tu
,
C. L.
,
Wang
,
X. S.
, and
Zheng
,
K.
,
2021
, “
Design and Analysis of Curvature Adaptive Wall Climbing Robot
,”
2021 IEEE Far East NDT New Technology & Application Forum (FENDT)
,
Kunming, China
,
Dec. 14–17
, IEEE, pp.
177
184
.
25.
Li
,
J.
,
Feng
,
H.
,
Tu
,
C.
,
Jin
,
S.
, and
Wang
,
X.
,
2019
, “
Design of Inspection Robot for Spherical Tank Based on Mecanum Wheel
,”
2019 Far East NDT New Technology & Application Forum (FENDT)
,
Qingdao, China
,
June 24–27
, pp.
218
224
..
26.
He
,
K.
,
Gkioxari
,
G.
,
Dollár
,
P.
, and
Girshick
,
R.
,
2017
, “
Mask r-cnn
,”
Proceedings of the IEEE International Conference on Computer Vision
,
Venice, Italy
,
Oct. 22–29
, pp.
2961
2969
.
27.
Liu
,
S.
,
Qi
,
L.
,
Qin
,
H.
,
Shi
,
J.
, and
Jia
,
J.
,
2018
, “
Path Aggregation Network for Instance Segmentation
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT
,
June 18–23
, pp.
8759
8768
.
28.
Fu
,
C. Y.
,
Shvets
,
M.
, and
Berg
,
A. C.
,
2019
, “
RetinaMask: Learning to Predict Masks Improves State-of-the-Art Single-Shot Detection for Free
,” arXiv:1901.03353.
29.
Li
,
Y.
,
Qi
,
H.
,
Dai
,
J.
,
Ji
,
X.
, and
Wei
,
Y.
,
2017
, “
Fully Convolutional Instance-Aware Semantic Segmentation
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Honolulu, HI
,
July 21–26
, pp.
2359
2367
.
30.
Bolya
,
D.
,
Zhou
,
C.
,
Xiao
,
F.
, and
Lee
,
Y. J.
,
2019
, “
Yolact: Real-Time Instance Segmentation
,”
Proceedings of the IEEE/CVF International Conference on Computer Vision
,
Seoul, South Korea
,
Oct. 27–Nov.2
.
31.
Bolya
,
D.
,
Zhou
,
C.
,
Xiao
,
F.
, and
Lee
,
Y. J.
,
2022
, “
YOLACT++ Better Real-Time Instance Segmentation
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
44
(
2
), pp.
1108
1121
.
32.
Lin
,
T. Y.
,
Dollár
,
P.
,
Girshick
,
R.
,
He
,
K.
,
Hariharan
,
B.
, and
Belongie
,
S.
,
2017
, “
Feature Pyramid Networks for Object Detection
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Honolulu, HI
,
July 21–26
, pp.
2117
2125
.
33.
Shelhamer
,
E.
,
Long
,
J.
, and
Darrell
,
T.
,
2017
, “
Fully Convolutional Networks for Semantic Segmentation
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
39
(
4
), pp.
640
651
.