Abstract

An intelligent system for spatial visual feedback is presented, which enables the robot's autonomy for a range of robotic assembly tasks, in particular for arc welding, in an unstructured and “fixtureless” environment. The robot's autonomy is empowered by an embedded inductive inference-based machine learning module which learns a welded object's structural properties in the form of geometrical properties. In particular, the system tries to recognize line segments, using a spatial (three-dimensional) visual sensor in order to autonomously execute the objective task. The innovative result is that the recognition of the geometric primitives is done without a predefined Computer-Aided Design (CAD) model, significantly improving the system's autonomy and robustness. The system is validated on real-world welding tasks.

Introduction

Welding is a fundamental process in industrial assembly technology [1]. Although the category of welding processes is very broad (DIN 1910-2008: Welding and allied processes), arc welding is the most frequently used in metalworking industry. The most common types are the gas shielded tungsten arc welding (GTAW) and the gas shielded metal arc welding (GMAW) [1].

Present-day technology of robotic arc welding is generally developed to automatically execute only nominal tasks, with very limited capabilities of adaptation, autonomy, and collaborative interaction with human operators. If used, adaptive behavior is, as a rule, almost exclusively related to various techniques of compensation of small offsetting errors of the weldment and small variations of seam geometry—seam finding and seam tracking, mainly enabled by feed-backing visual information acquired using laser triangulation sensors [24], or by using machine vision systems, designed exclusively for robotic arc welding applications [5], or even multiple vision systems with sensory data fusion [6], including well-known techniques based on tactile sensors or welding arc sensors [7]. An important drawback is that the actual robotic arc welding technology is entirely designed for capital-intensive, large-scale or mass-production industries, resulting in complex systems, which are costly and incapable of properly responding to mass customization [8,9], and for SMEs which are forced to choose between inappropriate automation and hard-to-provide skilled welders, thus bringing them into “automation trap” [10].

A new paradigm of affordable and highly flexible robotic arc welding technology, which is consistent with the manufacturing paradigm shift from mass production to mass customization, and which meets the specific requirements of SMEs, is urgently needed. The solution could be based on the concept of semi-autonomous hybrid systems [11] and Collaborative Robotics (COBOTs) which allows multimodal physical and cognitive human–robot interaction and teamwork without fences.

Semi-autonomous behavior of the robotic system can be achieved through the concept of Programming by Demonstration (PbD), which is essentially based on the natural mechanism of learning by imitation. This concept has the potential to remarkably simplify human–robot interaction, and thus establish functional and productive hybrid (human and robot) welding system with a significant reduction of required robot programming skills and efforts. A typical example of the PbD concept achieved through a direct physical human–robot interaction (pHRI), is shown in Fig. 1 [10,12]. Manual guidance (dragging) which is required for seam tracking by the attached welding torch and its accurate positioning in task-relevant points is enabled through an appropriate force feedback interface device which is coupled with a back drivable robot arm controlled by the unified passivity-based joint position and impedance control framework [14,15], and real-time acquisition of robot joint or Tool Center Point coordinates (TCP), (the trajectory recording).

Fig. 1
The arc welding scenario for practical validation of the Programming by Demonstration concept achieved through direct human–robot physical interaction and passivity-based control of the robot arm motion as a key enabling technology [13]
Fig. 1
The arc welding scenario for practical validation of the Programming by Demonstration concept achieved through direct human–robot physical interaction and passivity-based control of the robot arm motion as a key enabling technology [13]
Close modal

Although it looks very attractive because it allows moderately trained human workers with no programming knowledge and skills to program complex robot motions, thus automate quasi-repetitive arc welding tasks without writing and debugging complex job codes, in reality, this approach is not so productive because PbD based on direct physical human–robot interaction suffers from three major drawbacks:

  1. Recorded trajectory of the welding torch generated by the human worker guidance hardly can satisfy geometrical allowances imposed by the arc welding process (typically in the range of less than 1 mm).

  2. The demonstration process is far from the human natural way of communication and knowledge/skill transfer, especially for complex process requirements for achieving and keeping accurate angles of the torch orientation and its stand-off distance (not so human friendly);

  3. The demonstration process takes considerable time and should be repeated for each subsequent assembly to be welded, even if it is seemingly identical to its precedent, in order to adapt previously recorded trajectory to the small geometrical variations of assembly positioning and seam geometry.

Programming by Demonstration based on direct physical human–robot interaction requires a CAD model of the assembly to be welded and a complex graphical user interface for editing recorded motions, definition of process-specific data, and verification of the generated task by graphical simulation of the robot TCP motion. In general, Programming by Demonstration based on direct physical human–robot interaction only, has very limited capabilities and does not allow any kind of robot autonomy to be developed, and gradual improvement of performances to be achieved by knowledge acquisition and learning through generalization instead of imitation. Therefore, the interaction space has to be extended towards the higher stratum, i.e., the complementary cognitive layer (cognitive human–robot interaction—cHRI), which includes the perception of the working environment, extensive processing of acquired sensory data, decision-making and even thinking, which all together enable more human-like interaction to be established—shifting toward the extended notion of Collaborative Robotics. Contrary to human welders, with their flexibility gained from environment perception based on visual and acoustic sensory feedback, and intelligence for complex, real-time processing of acquired sensory data, contemporary arc welding robots in most cases perform their tasks literary “blindly,” or at least “semi-blindly,” without, or with very rudimentary processing of acquired data. Low cognitive performances and, consequently, negligible capabilities for autonomous decision-making in uncertain and non-well-structured environment are the fundamental drawbacks, even for the state-of-the-art robotic arc welding technology. This heavily hinders the potentials for its ubiquitous use in small- and medium-sized enterprise (SME) environment and, more generally, creates a technological gap, i.e., considerable inconsistency with mass customization and mass personalization production paradigm. Consequently, PbD based on pHRI should be extended from rudimentary learning by imitation, which in fact means replying to previously recorded and by human operator slightly adapted motions, to PbD based on the intertwining of pHRI and cHRI, which enables the acquisition of a human welder’s behavior through a combination of two processes: learning by imitation and learning by generalization. To achieve this, manipulating robots must be transformed into cognitive machines, which possess human-like cognitive abilities—Cognitive Collaborative Robotics (or CoCOBOTs).

This paper presents results that contribute to overcoming the identified acute shortage of welding professionals and facilitate adoption and implementation of arc welding automation based on the use of industrial robots, particularly in SMEs, through the development of a new paradigm of affordable and highly flexible robotic arc welding technology, which is consistent with mass customization as well as specific demands imposed by SMEs.

In this paper, an intelligent system for spatial visual feedback is presented. The system enables a robot’s autonomy for a range of robotic assembly tasks, in particular for arc welding, in an unstructured and “fixtureless” environment (meaning elimination of the need for precise positioning of elements required in fixed automation, normally not used in manual arc welding). The robot’s autonomy is based on an embedded Inductive Inference-based Learning Machine (IILM) which learns a product's geometrical primitives’ structural properties, in particular the line segments, to be recognized by the spatial (three-dimensional or 3D) visual sensory system. Another innovative result is that the recognition of the geometric primitives is a Computer-Aided Design (CAD) model-free (meaning elimination of needs for a product's CAD model as a reference for the recognition process by the vision-based sensory system). This significantly improves the system's autonomy and robustness. The system is validated on real-world welding tasks.

For the recognition process, an inductive inference-based machine learning algorithm is applied. More specifically, an algorithm to inductively learn a formal grammar, in other words, the grammatical inference-based approach [16], was applied, wherein the learned model is represented by a regular grammar which is equivalent to a Finite State Automata (FSA) [17]. Although the automata theory was at the heart of computer science and grammatical inference was one of the major research areas that contributed to the development of the field of artificial intelligence, the fields got lesser attention in the years that saw huge developments in deep learning and other statistical algorithms of machine learning. However, recently, grammatical inference and automaton learning have received renewed attention for the applications of artificial intelligence [18,19].

Following are two scientific hypotheses of the proposed approach: (1) the spatial visual feedback enforced by IILM enables “fixtureless” environment and CAD model-free capabilities for robotic arc welding, and (2) the automata theory and formal grammars (besides the “traditional” analytical and statistical pattern recognition methods) provide sufficient analytical framework to effectively learn, generalize, and recognize specific topological relations, the structure within the feed-backed spatial point cloud of seemingly unorganized data set, presented to the IILM.

The Framework Generic Architecture

The framework of the proposed solution is based on the concept of semi-autonomous hybrid systems [11] and collaborative cognitive robots [20], which allow multimodal physical and cognitive interaction of spatially co-located human workers and welding robots. Transfer of knowledge and skills within such framework is to be provided through Programming by Demonstration (PbD), based on learning by generalization mechanism (instead of learning by imitation [10]).

Acquired raw sensory data, elementarily processed, XP*2 (including the high-level functions like 3D point cloud registration and similar), are fed to the IILM through the spatial visual sensory feedback channel and thus provide the intelligent task/action planner with the required real-world data of the associate Robotic Welding System (RWS). In order to make the overall process sufficiently robust and reliable, the IILM is coupled with the human supervisor through the PbD Generalized Gateway (see Fig. 2). Abbreviations used in Fig. 2 are described in the Nomenclature section.

Fig. 2
Proposed generic architecture of the semi-autonomous hybrid system for enabling effective human–robot collaboration
Fig. 2
Proposed generic architecture of the semi-autonomous hybrid system for enabling effective human–robot collaboration
Close modal

The IILM learning module does the following two jobs: (1) synthesizing grammar to represent a class of physical object geometrical primitives, during the learning phase, and (2) recognizing whether a given object geometry represented by a spatial point cloud belongs to the synthesized class of specific geometrical type by iteratively parsing geometrical data, acquired from the real physical object and presented in the form of a string of codified geometrical primitives, through the synthesized grammar. In the context of the presented work, this implies verifying if the presented preprocessed point cloud string represents a geometrical primitive of the type “straight line” (of any inclination) or not (see Fig. 3).

Fig. 3
Inductive Inference Learning Machine (IILM) for topological model/structure learning (sub-module A), and recognition in real-world spatial point clouds (sub-module B)
Fig. 3
Inductive Inference Learning Machine (IILM) for topological model/structure learning (sub-module A), and recognition in real-world spatial point clouds (sub-module B)
Close modal

The IILM module has two modes of operations, having accordingly two sub-modules (sub-modules A and B), as shown in Fig. 3. The first mode of operation, executed by the sub-module/block A in Fig. 3, denominated “Model/Structure Learning,” corresponds to the so-called, “training/learning” the model phase. The inputs in this sub-module are the acquired raw sensory data, elementarily processed, XP*2. These data are converted, or transduced, into the ordered dataset, X, through a geometry encoding system by the function “Strings Transducer.” Following this, the ordered dataset X is compressed into a string, before it is passed to grammar synthesis by inductive inference. The model for the data conversion and compression, as well as the learning algorithm, is detailed in the section “Primitives Representation and Learning Algorithm”. The sub-module/block A “Model/Structure Learning” outputs learned grammar, which will serve for the process of recognition if the raw sensory data XP*2 represents a geometric primitive of the robot's expected trajectory, i.e., of the welding seam. It is important to notice that each type of the geometric primitives, such as “line,” “arc,” etc., corresponds to a particular grammar.

The recognition of the structural properties of the raw sensory data XP*2, i.e., the recognition of the raw sensory data, XP*2 represents one of the geometric primitives, is realized in the sub-module/block B in Fig. 3, denominated “Task/Action Replanner.” This phase of IILM operation corresponds to the so-called “model deployment” phase of the general machine learning “process flow.” The recognition is realized by the “classical” string parsing, with the result “yes/no,” i.e., confirming or refuting the structural property of the raw sensory data set XP*2, i.e., that of the geometric features’ raw sensory data set XP*2, is the geometric primitive corresponding the grammar used. The output of this sub-module/block B “Task/Action Replanner” is labeling of data that will be used for the automatic programming of the robot. Examples of the string parsing are given in the subsequent section, presenting the experimental validation of the model.

The automata theory and formal grammars are adopted as a representation system for the IILM, in order to prove the feasibility of this approach, theoretically and practically, i.e., to test the formulated hypotheses. The problem of recognizing linear topological relations between the finite set of coplanar data points is studied. Although the chosen line primitive is elementary, it is sufficiently representative to prove the posed hypotheses, and thus open an entirely new methodological framework for designing many of the building blocks of intelligent task/action planning modules, like for instance, semantic segmentation [21].

Primitives Representation and Learning Algorithm

As shown in Fig. 3, the elementarily processed unordered raw dataset XP*2 is converted to the ordered dataset, X, through the geometry encoding system in the Strings Transducer. Representation of the line primitive segments and subsequent recognition of the new segments within the ordered set X, consisting of n coplanar data points x(i) ∈ X, require definition of appropriate set F of k topological features f(i) ∈ F. Based on that, construction of representation scheme R enables transduction of specific topological relations, which exist between the data points x(i) ∈ X, into a string of symbols from an alphabet V, which is a finite set of m symbols v(j) ∈ V. In case of recognition of line segments s(i), a predictor function p = p(X) has to be introduced in the form of a linear polynomial estimate of the point set X. Chosen predictor p represents the global context of the entire point set X and is expressed by a single parameter ap, i.e., the line slope parameter.

Local structural relations among the data points can also be expressed by a set of single parameters ai, defined by the polynomial form of the line equations associated with the line segments (Eq. (1)) that physically connect neighboring points of X.

Necessary and sufficient feature set F for topology transduction into a string is defined by using intimate relation between the ap and the ai parameters, which is in fact a quantitative measure related to the angular deviation of the segment si from the predictor p. If the features space is defined as Eq. (2), and the domain of f1 is Df1 as in Eq. (3), then required representation scheme R can be derived through granulation of the domain Df1 into a finite set of m subdomains Df1 = {d1, d2, …, dm} and by applying bijective relation R as defined in Eq. (4)
si=si(x(i),x(i+1)),1in1
(1)
F={f1|f1=apai}
(2)
Df1=(f1+)
(3)
R:d(i)Df1v(i)V,i=(1,m)
(4)

The above-described representation scheme R is graphically depicted in Fig. 4(a). Figure 4 also includes two examples for m = 6, one positive, Fig. 4(b), and one negative, Fig. 4(c).

Fig. 4
(a) Abstract Representation model, (b) codified Geometrical Primitives of positive example “dddccecdccddcbe,” and (c) codified Geometrical Primitives of negative example “dcbcccbdddddecd”
Fig. 4
(a) Abstract Representation model, (b) codified Geometrical Primitives of positive example “dddccecdccddcbe,” and (c) codified Geometrical Primitives of negative example “dcbcccbdddddecd”
Close modal

As explained earlier, the Strings Transducer transforms a sequence of coplanar datapoints x(i) into a sequence of symbols (string) ω, considering the angle that a segment makes with the line that connects the first and the last points. For the presented case of arc welding, the angles between subsequent datapoints are within the range of −90 deg to +90 deg. These angles are divided into intervals of 30 deg, and each interval is assigned a symbol/letter, i.e., the codification process.

The resulting string ω is then passed to the Structure Learning phase. The learning process itself involves generating a formal grammar, describing a model to represent line segments where the straightness of a segment is of an “accepted” level as indicated by a “supervisor” through positive examples. Hence, the task is essentially a grammatical inference task, for which an inductive inference-based learning algorithm is applied. Regular grammar is synthesized through an automated synthesis process.

Another additional step is embedded in IILM which compresses the string ω, before it is passed to grammar synthesis or parsing stages (Fig. 3). The string compression process is parameterized by a compression factor k, where k = {1, |ω|}. The compression factor k implies considering symbol sequences of up to k adjoining occurrences of the same symbol as a single symbol in the string considered. The consecutive occurrences of a symbol are split into blocks of k length (or lesser length if the sequence ends before k occurrences). For example, any of the symbol sequences “ccc,” “cc,” and “c” (in that order of priority consideration) are rewritten as “c” only. Hence, “ccccddc” will be compressed to “ccdc” (more examples in Tables 1 and 2).

Table 1

Input examples for grammar inference

NoisePositive examples for learningExamples after compression
±20%cccccccccccccccccccc
dcccddcccccccdcdcdcccdc
dccdcccdccddccddcdcdcdcd
±40%cddbdccccddcdcdcdbdccdcdcd
dcdccdcdcddcdccdcdcdcdcdcdc
dccdbddcebddcdcdcdbdcebdcdc
dccececcdccdddcdcececdcdc
dddccecdccddcbedcecdcdcbe
NoisePositive examples for learningExamples after compression
±20%cccccccccccccccccccc
dcccddcccccccdcdcdcccdc
dccdcccdccddccddcdcdcdcd
±40%cddbdccccddcdcdcdbdccdcdcd
dcdccdcdcddcdccdcdcdcdcdcdc
dccdbddcebddcdcdcdbdcebdcdc
dccececcdccdddcdcececdcdc
dddccecdccddcbedcecdcdcbe
Table 2

Examples of string parsing by grammar G1

Input stringsCompressed input stringsPassing result
Positive example strings
cddcccdcdccccdccdcdcdccdcAccepted
ddcccdcdcdccccddcdcdcdccdAccepted
cccdccdccdddcdccdcdcdcdcAccepted
dcdcddccddcdcdcdcdcdcdcdcdcAccepted
dddccccdcdcdcdcdccdcdcdcdcAccepted
Negative example strings
cccccccddddddddcccdddRejected
ccdddddddddddddcdddddRejected
ccccccccccdddddccccddRejected
ccccccccccdddddccccddRejected
bbbbbbdddddddddbbdddRejected
Input stringsCompressed input stringsPassing result
Positive example strings
cddcccdcdccccdccdcdcdccdcAccepted
ddcccdcdcdccccddcdcdcdccdAccepted
cccdccdccdddcdccdcdcdcdcAccepted
dcdcddccddcdcdcdcdcdcdcdcdcAccepted
dddccccdcdcdcdcdccdcdcdcdcAccepted
Negative example strings
cccccccddddddddcccdddRejected
ccdddddddddddddcdddddRejected
ccccccccccdddddccccddRejected
ccccccccccdddddccccddRejected
bbbbbbdddddddddbbdddRejected

The learning algorithm used for grammar synthesis for the given task was Successor Method-based algorithm, originally developed by two different teams [22,23] and then by [24], probably independently of each other. During the learning mode, this algorithm should be fed only positive examples to inductively infer regular grammar. A string is denoted as a “positive example” if it describes a line segment. The algorithm reads the strings by one symbol at a time. As the strings are generated from left to right in the presented case, the grammar will be right-linear. A regular grammar is a finite state automaton (FSA). In this case, each symbol is considered as an input for the next state of FSA. Each new symbol generates a new grammar rule. If the symbol is followed by another symbol, that is, an intermediate state of the system, the resulting rule will have a terminal (small letters by convention) as well as a non-terminal symbol (capital letters by convention) on the right side (e.g., CcC), else the resulting rule will have only a terminal symbol (e.g., Cc). In the end, only unique rules are saved. A detailed explanation of the algorithm can be found in the above-cited literatures. Once the grammar is generated, during the “online” or “inference/recognition” mode, it is used to parse strings. The strings, which are not parsed successfully till the end, are declared as “negative examples” by the system, i.e., not representing a straight line.

The learning algorithm’s outline is following:

Input: strings set I; compression factor k;

   Rewrite strings applying compression by factor k;

   Analyse I to extract alphabet V;

   Successor Method to infer grammar G:

     For each v(i) ∈ V

     Assign a state for each successor of v(i);

     Merge states with common successor states;

     Generate automata A;

     Generate productions set P;

Output: grammar G.

The application of Successor Method to infer grammar was done through an automated process, using a grammar inference tool (see Fig. 5) for the tool's Graphical User Interface (GUI) [25].

Fig. 5
Grammar synthesis tool graphical user interface [25]
Fig. 5
Grammar synthesis tool graphical user interface [25]
Close modal

Experimental Validation

The experimental validation is performed on both, the computer simulation space, where grammar synthesis and its effectiveness are tested, and physical space, where the effectiveness of IILM is tested in real-world physical space.

Validation by Simulation.

For the purpose of grammar synthesis, two representational samples, for two levels of data noises, were selected from the set of positive examples. This is in order to identify the accepted noise level for the grammar synthesis/learning phase. Examples provided for this task are given in Table 1. Two grammars, G1 and G2, were synthesized, one for each noise level, see Eqs. (5) and (6), respectively. In Eqs. (5) and (6), V is the Terminal Alphabet, N is the set of Non-terminal Symbols, S is the Start Symbol, and P is the set of Productions/Rewriting Rules
G1={V,N,S,P},whereV={c,d},N={C,D},S={S},P={SC,SD,CcC,CcD,DdC,Cc,Dd}
(5)
G2={V,N,S,P},whereV={b,c,d,e},N={B,C,D,E},S={S},P={SC,SD,BbD,BbE,CcB,CcC,CcD,CcE,DdB,DdC,EeB,EeC,Cc,Dd,Ee}
(6)

The recognition process was tested on both, positive and negative, types of examples, which were not used for the learning/grammar synthesis process. Table 2 presents some results of the recognition process by the grammar G1 which was created using a positive sample of less noise. This also implies that for the grammar synthesis purpose, it is recommended to use more representative examples in order to create a more robust grammar or a representation class.

Table 3 presents the summary of the experimental results and evaluation of the robustness of grammars G1 and G2 for various noise levels.

Table 3

Robustness evaluation of G1 and G2 for various noise levels

Noise levelSample sizeRecognition by G1Recognition by G2
Correctly recognized%Correctly recognized%
±10%20019798.519798.5
±20%20019597.519597.5
±30%2001889418894
±40%2009346.516683
Noise levelSample sizeRecognition by G1Recognition by G2
Correctly recognized%Correctly recognized%
±10%20019798.519798.5
±20%20019597.519597.5
±30%2001889418894
±40%2009346.516683

Validation by Simulation.

Real-world space experimental validation is conducted on an industrial robotic arc welding system which is installed in a laboratory environment for research purposes. The experimental setup is shown in Fig. 6 and exactly corresponds to the physical layer of the proposed hybrid system shown in Fig. 2. The system consists of six-degrees-of-freedom articulated robot arm controlled by an open architecture control system and of Stratified Spatial Machine Vision system (SSMV), whose function and streamed sensory data are handled by the dedicated controller and preprocessor. The SSMV system consists of two functional strata. The first stratum provides a visual perception of the robot's wider workspace, with the basic task of quickly recognizing the position and orientation of the assembly to be welded. Its output is a massive 3D point cloud (approximately half a million points or more) whose spatial resolution is on the order of 10 mm, which is enough for a satisfactorily accurate location of the weldment. The secondary stratum provides 3D visual perception with high spatial resolution, 0.1 mm or rather, focused on a narrow zone where the welding process takes place. Visual perception on the secondary stratum is multimodal, providing in parallel: (a) a complete color image of the welding zone, (b) a seam section profile, and (c) point triangulation in some part of the field of view that can be freely selected as needed. The SSMV system is the result of its own development at the University of Belgrade and is not commercially available. Sensor controller and preprocessor modules are synchronized with the robot controller, and readings of the robot joint coordinates are delivered in real-time to the image preprocessor. Joint coordinates, coupled with the generated point clouds, are streamed to the IILM module.

Fig. 6
Laboratory setup for experimental validation: (a) structure of the physical system, and (b) corresponding photo [26]
Fig. 6
Laboratory setup for experimental validation: (a) structure of the physical system, and (b) corresponding photo [26]
Close modal

In the first stage, plane sections of the weldment in the form of coplanar point clouds are extracted as subsets from acquired spatial point clouds. Points that are topologically organized in the form of the line segments are recognized in each plane section using IILM. Then, in the second stage of sensory data processing, corresponding virtual seam points are generated by intersecting two neighboring coplanar line segments in each plane section. Thus, the entire welding seam which refers to the real seam geometry is recognized as a set of generated seam points. The result of the validation process in real-world space from both stages is shown in Fig. 7. Conformity test of the recognized welding seam geometry with the geometry measured by coordinate measuring machine (CMM) is high, steadily within the ±1 mm tolerance field. The used test piece, i.e., the welding sample, was taken from an industrial company.

Fig. 7
Results of the validation process: (a) assembly to be welded used as a test object, size of 810 × 370 × 60 mm, and (b) recognized line segments, non-line segments and welding seam points on used test weldment (every third plane sections shown)
Fig. 7
Results of the validation process: (a) assembly to be welded used as a test object, size of 810 × 370 × 60 mm, and (b) recognized line segments, non-line segments and welding seam points on used test weldment (every third plane sections shown)
Close modal

Discussion

In the proposed adoption of the successor method, one critical step is the transformation of raw sensory data into codes or symbols which can be effectively fed to the system to synthesize simple grammar rules. These rules are “transparent,” as opposed to “black-box” type models, and provide robustness in terms of predictability. This quality is important in industrial robotics where safety and quality in production are of utmost importance. The adopted codification process gave an alphabet of six codes {a, b, c, d, e, f}. However, the number of symbols can be changed depending on the “resolution” or size of intervals of angles. If the intervals are reduced from 30 deg to 20 deg, the alphabet will be of size 9. However, in the experiments, this extra resolution did not yield any benefits but, on the contrary, created more complexity and poor results. Hence, the interval of 30 deg was finalized.

Another innovation was in string compression. This simplified the strings, while increasing the efficiency and accuracy of the pattern recognition task. The synthesized grammars contain recursive rules. Meaning, that the strings can be large, with practically no control over their length. The challenge is in the parsing of these strings. The parser needs to distinguish between a positive and a negative example. The string compression mechanism facilitated this process as it highlighted the symbols representing “extreme” angles. Longer sequences of subsequent symbols representing extreme angles, in compressed strings, are clear candidates for a rejection by the parser.

Furthermore, the proposed method also demonstrated robustness against high levels of noise. For the sake of testing the system's performance, artificial noise was added to the geometrical primitives, to create worse scenarios than found in reality in the welded sheets. However, as shown in the results, even with a noise level of 40%, the system performed fairly.

Above all, the biggest advantage of the proposed approach is in its simplicity both in terms of explainability and implementation and execution. The solution works with very little computing resources and gives very high-quality results.

Conclusions and Future Work

The two scientific hypotheses formulated previously are experimentally proved using sensory data, acquired from the real-world industrial setups.

The conducted experiments showed that the learning results are influenced by the context, e.g., by the data noise levels. This was experimentally demonstrated through the synthesis of two grammars G1 and G2. This is in accordance with the well-known “language identification in the limit” paradigm [27], which represents at the same time the criteria for stopping the learning as an asymptotic process.

The proposed concept has the potential to remarkably simplify human–robot interaction, and thus to establish a functional and productive hybrid welding system with a significant reduction of required robot programming skills and efforts, making it more affordable for SMEs, which was one of the utilitarian objectives of the system proposed.

Resuming, the objective of this paper was to demonstrate the applicability of Inductive Machine Learning (by extension: Inductive Inference-based Machine Learning) in further enchantment of robot autonomy. This objective was completely fulfilled. However, the work presented was developed for the geometric primitive “line” and not for other primitives. Learning other primitives’ grammars, i.e., structural descriptions, should just follow the methodology, including the validation methodology, which could not be considered of particular scientific contribution. Another task, which is to be considered a scientific contribution, should be learning grammars for so-called “complex features,” which are composed of two or more primitive features and compose the welding trajectory.

Consequently, in this paper, the objective was not to develop the model up to the level of physical welding, which could have different approaches, and that is the subject of future work as referred to below.

The effectiveness of the approach presented is demonstrated in the video which could be accessed through the link.2

The video does not show the physical welding process, but instead of this, the effectiveness of the approach is demonstrated by the laser pointer which is embedded in the welding torch nozzle, thus replacing the welding wire. It is important to note that weldment is not precisely positioned in the robot’s workspace using a welding fixture as it is a typical setting in robotic welding. The weldment, which is preassembled with the tack welding, is freely located. In this way, the objective of the approach presented to provide the capability of the “fixtureless welding” (FLW), for the investment benefit of SME, as well as cost-effectiveness for the lot-size one production, is demonstrated.

Future work should address several problems such as learning other geometrical primitives and complex geometrical and technological features in the direction to achieve higher levels of robot cognitive capacities, integration into cyber-physical systems, ubiquity, collaborativeness, and lowering acquisition and operational costs.

Footnote

Funding Data

  • Fundação para a Ciência e Tecnologia (FCT) within the Research & Development (R&D) Units Project Scope (Grant No. UIDB/00319/2020; Funder ID: 10.13039/501100001871).

  • Serbian Ministry for Science and Technology Development (Grant No. TR35007; Funder ID: 10.13039/501100004564)—Smart Robotic Systems for Customized Manufacturing.

Conflict of Interest

There are no conflicts of interest.

Data Availability Statement

The authors attest that all data for this study are included in the paper.

Nomenclature

XP*1 =

demand inputs to the Intelligent Task and Actions Planner

XP*2 =

raw sensory data inputs, elementarily processed, acquired from the Robotic Welding System

X_RWS* =

inputs to the physical Robotic Welding System (preassembled weldments and other systemic inputs which can be classified as material, energy, and information)

Y_ RWS* =

outputs of the Robotic Welding System (finished weldments and other systemic outputs which can be classified as material, energy, and information).

Z_P1 =

product and process data bidirectional exchange relevant for nominal/model-based planning of the welding process, supporting a human supervisor interaction with the Intelligent Task and Actions Planner through the PbD Generalized Gateway interface

Z_P2 =

product and process data bidirectional exchange, supporting a human supervisor interaction with the adaptive and learning functions of the Intelligent Task and Actions Planner through the PbD Generalized Gateway interface

Z_RWS =

product and process data bidirectional exchange with the Robotic Welding System, supporting SCADA functions which are embedded in the PbD Generalized Gateway interface

YP =

outputs of the Intelligent Task and Action Planner

References

1.
Pires
,
J. N.
,
Loureiro
,
A.
, and
Bölmsjo
,
G.
,
2006
,
Welding Robots; Technology, System Issues and Applications
,
Springer-Verlag
,
London
.
2.
Lanzetta
,
M.
,
Santochi
,
M.
, and
Tantussi
,
G.
,
2001
, “
On-Line Control of Robotized Gas Metal Arc Welding
,”
CIRP Ann. Manuf. Technol.
,
50
(
1
), pp.
13
16
.
3.
Zou
,
Y.
,
Zhu
,
M.
, and
Chen
,
X.
,
2021
, “
A Robust Detector for Automated Welding Seam Tracking System
,”
ASME J. Dyn. Syst. Meas. Control
,
143
(
7
), p.
071001
.
4.
Wang
,
Z.
, and
Xu
,
Y.
,
2020
, “Vision-Based Seam Tracking in Robotic Welding: A Review of Recent Research,”
Transactions on Intelligent Welding Manufacturing. Transactions on Intelligent Welding Manufacturing
,
S.
Chen
,
Y.
Zhang
, and
Z.
Feng
, eds.,
Springer
,
Singapore
.
5.
Naso
,
D.
,
Turchiano
,
B.
, and
Pantaleo
,
P.
,
2005
, “
A Fuzzy-Logic Based Optical Sensor for Online Weld Defect-Detection
,”
IEEE Trans. Ind. Inform.
,
1
(
4
), pp.
259
273
.
6.
You
,
D.
,
Gao
,
X.
, and
Katayama
,
K.
,
2014
, “
Multisensor Fusion System for Monitoring High-Power Disk Laser Welding Using Support Vector Machine
,”
IEEE Trans. Ind. Inform.
,
10
(
2
), pp.
1285
2955
.
7.
Jaeschke
,
B.
,
2015
, “
Seam Tracking With Arc Sensors Using Welding Power Sources for Gas Metal Arc Welding (GMAW) With Consumable Electrodes
,” Technical Documentation No.: Y00.0110.1-00, Lorch Schweißtechnik GmbH, Auenwald, Germany.
8.
Jovane
,
F.
,
Koren
,
Y.
, and
Boër
,
C. R.
,
2003
, “
Present and Future of Flexible Automation: Towards New Paradigms
,”
CIRP Ann. Manuf. Technol.
,
52
(
2
), pp.
543
560
.
9.
Westkämper
,
E.
,
2014
,
Towards the Re-Industrialization of Europe—A Concept for Manufacturing for 2030
,
Springer Verlag
,
Berlin/Heidelberg
.
10.
Schraft
,
R. D.
, and
Meyer
,
C.
,
2006
, “
The Need for an Intuitive Teaching Method for Small and Medium Enterprises
,”
Proceedings of the Joint Conference on Robotics ISR 2006—ROBOTIK
,
Munich, Germany
,
May 15–17
, p.
10
, (CD-ROM), Abstract p. 95, (VDI-Berichte 1956).
11.
Takata
,
S.
, and
Hirano
,
T.
,
2011
, “
Human and Robot Allocation Method for Hybrid Assembly Systems
,”
CIRP Ann. Manuf. Technol.
,
60
(
1
), pp.
9
12
.
12.
Perzylo
,
A.
, et al
,
2019
, “
SMErobotics: Smart Robots for Flexible Manufacturing
,”
IEEE Robot. Autom. Mag.
,
26
(
1
), pp.
78
90
.
13.
European Commission
,
n.d.
, The European Robotics Initiative for Strengthening the Competitiveness of SMEs in Manufacturing by Integrating Aspects of Cognitive Systems. https://cordis.europa.eu/project/id/287787
14.
Ott
,
C.
,
2008
,
Cartesian Impedance Control of Redundant and Flexible-Joint Robots
,
Springer Verlag
,
Berling/Heidelberg
.
15.
Albu-Schäffer
,
A.
,
Ott
,
C.
, and
Hirzinger
,
G.
,
2007
, “
A Unified Passivity-Based Control Framework for Position, Torque and Impedance Control of Flexible Joint Robots
,”
Int. J. Robot. Res.
,
26
(
1
), pp.
23
39
.
16.
Angluin
,
D.
,
1982
, “
Inference of Reversible Languages
,”
J. ACM
,
29
(
3
), pp.
741
765
.
17.
Hopcroft
,
J. E.
, and
Ullman
,
J. D.
,
1979
,
Introduction to Automata Theory, Languages and Computation
,
Addison-Wesley
,
Reading, MA
.
18.
Aichernig
,
B. K.
, and
Tappler
,
M.
,
2018
, “
Efficient Active Automata Learning via Mutation Testing
,”
J. Autom. Reason.
,
63
(
4
), pp.
1103
1134
.
19.
Krichen
,
M.
,
2019
, “
Improving Formal Verification and Testing Techniques for Internet of Things and Smart Cities
,”
Mobile Netw. Appl.
,
28
(
2
), pp.
732
743
.
20.
Vongbunyong
,
S.
,
Kara
,
S.
, and
Pagnucco
,
M.
,
2013
, “
Application of Cognitive Robotics in Disassembly of Products
,”
CIRP Ann. Manuf. Technol.
,
62
(
1
), pp.
31
34
.
21.
Shapovalov
,
R.
,
Vetrov
,
D.
, and
Kohli
,
P.
,
2013
, “
Spatial Inference Machines
,”
Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Portland, OR
,
June 23–28
, IEEE Computer Society 2013, pp.
2985
2992
.
22.
Rodger
,
R. S.
, and
Rosebrugh
,
R. D.
,
1979
, “
Computing a Grammar for Sequences of Behavioural Acts
,”
Anim. Behav.
,
27
(
3
), pp.
737
749
.
23.
Richetin
,
M.
, and
Vernadat
,
F.
,
1984
, “
Efficient Regular Grammatical Inference for Pattern Recognition
,”
Pattern Recognit.
,
17
(
2
), pp.
245
250
.
24.
Miclet
,
L.
,
1987
, “Grammatical Inference,”
Syntactic and Structural Pattern Recognition, Theory and Applications
,
H.
Bunke
, and
A.
Sanfeliu
, eds.,
World Scientific
,
Singapore
. Series in Computer Science—Vol. 7.
25.
Shah
,
V.
, and
Putnik
,
G. D.
,
2016
, “
Software Tools for Understanding Grammatical Inference Algorithms: Part I—Tools for Regular Grammars and Finite-State Automata
,”
FME Trans. Fac. Mech. Eng. Belgr.
,
44
(
1
), pp.
83
91
.
26.
Danilov
,
I.
,
Petrovic
,
P. B.
,
Korac
,
F.
, and
Lukic
,
N.
,
2015
, “
Stratified Visual 3D Feedback for Adaptive Robotic Arc-Welding
,”
Proceedings of 2nd International Conference on Electrical, Electronic and Computing Engineering, IcETRAN
,
Silver Lake, Serbia
,
June 8–11
, p.
ROI4.3 1–6
.
27.
Gold
,
E. M.
,
1967
, “
Language Identification in the Limit
,”
Inf. Control
,
10
(
5
), pp.
447
474
.