## Abstract

This paper presents a long short-term memory (LSTM)-based ensemble learning approach for time-dependent reliability analysis. An LSTM network is first adopted to learn system dynamics for a specific setting with a fixed realization of time-independent random variables and stochastic processes. By randomly sampling the time-independent random variables, multiple LSTM networks can be trained and leveraged with the Gaussian process (GP) regression to construct a global surrogate model for the time-dependent limit state function. In detail, a set of augmented data is first generated by the LSTM networks and then utilized for GP modeling to estimate system responses under time-dependent uncertainties. With the GP models, the time-dependent system reliability can be approximated directly by sampling-based methods such as the Monte Carlo simulation (MCS). Three case studies are introduced to demonstrate the efficiency and accuracy of the proposed approach.

## 1 Introduction

Reliability is the probability that an engineered system successfully performs its intended functionality with the consideration of various uncertainties. Reliability analysis is of critical importance for engineering product development, and research efforts have been devoted to accurately estimating system reliability. Most probable point (MPP)-based methods such as the first-order reliability method (FORM) and second-order reliability method (SORM) [1–5] have been utilized for analytically estimating the reliability, where the limit state function is transformed into the standard normal space and the MPP is defined as the point on the failure surface that is closest to the origin. With the iterative MPP searching process, reliability can be approximated based on the first- or second-order Tayler expansion. To reduce the computational cost for high-dimensional problems, dimension reduction methods decompose the high-dimensional limit state function into multiple low-dimensional component functions to form a global surrogate model. Sampling methods such as Monte Carlo simulation (MCS) [6–8] and importance sampling [9–11] have been developed to enhance the accuracy of reliability estimations, which are commonly integrated with surrogate modeling techniques [12–15] for alleviating the computational burden.

Conventional reliability analysis methods are not applicable for time-dependent problems, in which the limit state function involves time-dependent uncertainties such as system performance degradation, stochastic operation conditions, and system aging. With the time parameter and stochastic processes, performing time-dependent reliability analysis is much more complex and difficult compared with estimating reliability for static problems. The most challenging part is to efficiently yet accurately account for the time-dependent correlation of the system responses. In the literature, different approaches have been investigated [16–19], including extreme value and outcrossing rate based approaches, for handling time-dependent reliability analysis problems. Extreme value-based approaches focus on extracting the worst performance of an engineered system over a certain period. Failure occurs if the extreme value is greater than or less than the threshold within a time interval. By identifying the extreme system performances, a time-dependent reliability problem can be transformed into a time-independent case, and then conventional reliability analysis methods can be applied for assessing the time-dependent reliability. However, it is intractable to obtain the probabilistic characterization of the extreme value analytically for practical problems. Though simulation-based methods can be used to estimate the extreme value distribution, the required computational resources are still significant due to the inevitable large number of function evaluations. In the outcrossing rate-based approaches, the outcrossing event occurs when the response reaches its limit state and the outcrossing rate is then defined as the rate of change in the probability with respect to time. The time-dependent reliability can be approximated by the integration of the outcrossing rate. However, the existing approaches such as Rice’s formula [20] can only applied to obtain the outcrossing rate for Gaussian random processes. In addition, Rice’s formula assumes that all the outcrossing events are independent. Based on Rice’s formula, different methods have been developed for time-dependent reliability analysis. Hu and Du [21] extended the joint outcrossing rate method to limit state functions with both random variables and stochastic processes, then used both the single and joint outcrossing rates to approximate the time-dependent probability of failure. In the PHI2 method, Andrieu-Renaud et al. [22] estimated the outcrossing rate based on the correlation coefficient function of the limit state at different time instants and point-in-time reliability indices.

Most of the aforementioned methods rely on the first- or second-order approximation of the limit state, which may introduce significant errors for time-dependent problems with highly nonlinearity. Therefore, surrogate model methods have been investigated to improve the accuracy of the time-dependent reliability analysis. Wang and Wang [23] proposed a nested extreme value response method for efficient time-dependent reliability analysis. The Gaussian process (GP) modeling technique was utilized to predict extreme system responses, and a double-loop sampling scheme was developed to iteratively update the GP model. Instead of using an optimization loop for identifying the global extreme responses, Hu and Mahadevan [24] proposed a single-loop Kriging modeling approach to sequentially refine surrogate models by identifying sample points for both random variables and time at the same level. Shi et al. [25] proposed a multiple-Kriging-surrogate method to determine the best regression trend. The adaptive sampling strategy was established to identify new training samples and determine the ensemble of surrogate models.

Inspired by the biological neural network of the human brain, deep learning utilizes a layered structure of algorithms to make informed decisions based on the knowledge learned from data. In recent years, deep learning techniques have been successfully applied in a variety of applications such as image processing [26,27], natural language processing [28], medical information processing [29], and robotics and control [30,31]. Recurrent neural networks (RNNs) [32,33] are a class of neural networks that allow previous outputs to be used as inputs while having hidden states. To deal with issues associated with gradient exploding and vanishing for the training of RNN, long short-term memory (LSTM) [34,35] has gained tremendous successes to make predictions based on time-series data. In this paper, an LSTM-based ensemble learning approach is developed for time-dependent reliability analysis involving stochastic processes. With the capability of modeling the time-dependent relationship, LSTM networks are employed to learn the time-dependent behavior of the system response with respect to the stochastic processes while fixing the random variables. The benefit of constructing the LSTM models lies in that they can accurately predict system responses given any new random realizations of the stochastic processes. As a result, a set of augmented data can be collected based on multiple LSTM models. The Gaussian process regression technique is then adopted for modeling the time-dependent system response. With specified stochastic processes and time instant, GP models can be constructed to predict the system response. By employing the MCS, the proposed approach can be utilized to estimate the time-dependent reliability. The remainder of this paper is organized as follows. Section 2 introduces the background of time-dependent reliability analysis and realization of stochastic processes. Section 3 introduces the details of the LSTM-based ensemble learning approach. The effectiveness of the proposed approach is demonstrated using three examples in Sec. 4, while the conclusions are presented in Sec. 5.

## 2 Background

*G*(

**x**,

**z**(

*t*),

*t*), where

**x**∈ R

^{nr}denotes the time-independent random variables,

**z**(

*t*) represents the stochastic processes, and

*t*is the time parameter. For time-dependent reliability analysis, failure occurs if system performance at any time instant falls below a threshold, written as

*T*] represents the system life cycle. Therefore, the probability of failure evaluated based on the time-dependent system response can be defined as

*z*(

_{G}*t*), it can be fully characterized by three time-dependent functions, including the mean function

*μ*(

_{Y}*t*), standard deviation function

*σ*(

_{Y}*t*), and autocorrelation function

*ρ*(

_{Y}*t*). By discretizing the overall time interval into

*s*time nodes, the covariance between two time nodes is defined as

**Σ**=

**QDQ**, where

^{T}**Q**= [

**Q**

_{1},

**Q**

_{2}, …,

**Q**

*] represents the matrix of eigenvectors and*

_{s}**D**is a diagonal matrix with eigenvalues. Given a specified criterion, the number of dominated eigenfunctions can be determined, then the original Gaussian process z

*(*

_{G}*t*) can be simulated as

*m*is the number of dominated eigenfunctions and

**p**= [

*p*

_{1},

*p*

_{2}, …,

*p*] are a set of uncorrelated standard normal random variables.

_{m}## 3 LSTM-Based Ensemble Learning

As shown in Fig. 1, multiple LSTM networks are first trained to learn conditional limit state functions based on a set of random realizations of the input variables and the stochastic processes. To handle the time-independent random variables, a set of Gaussian process models are leveraged with the LSTM models to predict the global time-dependent system response under uncertainty. With the GP models, time-dependent reliability can be approximated using the MCS method. The LSTM-based conditional limit state modeling is introduced in Sec. 3.1. Section 3.2 presents the global surrogate modeling using GP regression, and the numerical procedure of employing the proposed approach for time-dependent reliability analysis is explained in Sec. 3.3.

### 3.1 LSTM-Based Conditional Limit State Modeling.

The RNN has been developed to learn insights from time-series data, which has a feed-back loop to store prior information. A major limitation remains that it cannot provide accurate predictions if the time-series data have long-term dependencies. All RNNs have the form of a chain of repeating modules. To remember information for long periods of time, three gated units in LSTM are used in the repeating modules, including an input gate, output gate, and forget gate. The core idea of LSTM lies in that the information flow is represented by the cell state, which is shown as the top horizontal line in Fig. 2.

*G*(

**x**,

**z**(

*t*),

*t*), the concept of conditional limit state functions is introduced for LSTM modeling. In the proposed approach,

*n*random realizations of the input variables and the stochastic processes are generated, denoted as

**X**= [

**x**

_{1},

**x**

_{2}, …,

**x**

*] and*

_{n}**Z**= [

**z**

_{1}(

*t*),

**z**

_{2}(

*t*), …,

**z**

*(*

_{n}*t*)], respectively. For each sample in

**X**, one conditional limit state function

*g*(

_{i}**z**(

*t*),

*t*) can be expressed as

**x**

*represents the*

_{i}*i*th realizations of the random variables

**x**. As shown in Eq. (6), each conditional limit state function is a simplified version of the original limit state function with fixed random variables. For the

*i*th conditional limit state function, the time-dependent system responses

**y**

*are evaluated for the inputs [*

_{i}**x**

*,*

_{i}**z**

*(*

_{i}*t*),

*t*]. To train an LSTM model for this conditional limit state function, the input of training data is expressed as a matrix

**z**

*(*

_{i}*t*) represents the stochastic processes value at the

_{j}*j*th time instant. Accordingly, the training label for the LSTM is a vector with

*s*elements, expressed as

**y**

*= [*

_{i}*y*(

_{i}*t*

_{1}),

*y*(

_{i}*t*

_{2}), …,

*y*(

_{i}*t*)]. In the repeating module, the inputs at the particular time instant

_{s}**η**

*and the previous cell output*

_{j}**h**

_{j−}_{1}are provided to the forget gate, which is multiplied with weight matrices and followed by bias, expressed as

**f**

*represents the forget gate output,*

_{j}**W**

*,*

_{f}**R**

*, and*

_{f}**b**

*represent the input weights, recurrent weights, and bias, respectively, and*

_{f}*σ*(.) stands for the activation function. Usually, the sigmoid function is adopted as the activation function of the forget gate. Next, we need to decide what new information should be stored in the cell state based on two parts. First, the input gate determines which state will be updated while a hyperbolic tangent layer creates a vector candidate values that should be added to the state. The computation process can be summarized as

**W**

*,*

_{i}**R**

*, and*

_{i}**b**

*) and (*

_{i}**W**

*,*

_{C}**R**

*, and*

_{C}**b**

*) are the input weights, recurrent weights, and bias of the input gate and the cell state, respectively. Then, all the outputs from Eqs. (8) and (9) are utilized to update the old cell state, written as*

_{C}**h**

*. By applying the hyperbolic tangent function to the cell state, the cell output*

_{j}**h**

*of LSTM at time instant*

_{j}*t*can be achieved as

_{j}**o**

*represents the results of the output gate, and*

_{j}**W**

*,*

_{o}**R**

*, and*

_{o}**b**

*represent the input weights, recurrent weights, and bias, respectively. For the regression task, the output*

_{o}**h**

*is directly linked to the training labels with a linear activation function. The identical module is repeated at each time instant. Based on the LSTM structure introduced above, the gradients of weights and biases term can be computed accordingly. Given the training data, optimization algorithms such as stochastic gradient descent, root mean square propagation, and Adam can be utilized for determining the weight matrices and biases of the LSTM. In this work, the mean square error (MSE) is adopted as the loss function, and*

_{j}*n*LSTM models are constructed to, respectively, capture the relationship between the stochastic processes and time-dependent responses of the conditional limit state functions.

### 3.2 Gaussian Process-Based Global Surrogate Modeling.

The LSTMs constructed in Sec. 3.1 can provide accurate predictions for the conditional limit state functions at certain local locations. In other words, these LSTMs lack the capability of modeling the global limit state function with the time-independent random variables. To address this difficulty, Gaussian process regression is adopted for global surrogate modeling, which aims at predicting the time-dependent system responses given any random realizations of time-independent variables **x**′ and stochastic processes **z**′(*t*).

**z**′(

*t*) is provided to the constructed

*n*LSTM models for estimating the time-dependent responses of the corresponding conditional limit state functions. As a result, the response predictions from all the LSTM models can be collected as

**z**′(

*t*) to the

*i*th conditional limit state function

*g*, expressed as

_{i}**x**′, GP models are constructed at each time instant based on the augmented data. Given a specified time instant

*t*a GP model is built based on the training labels $y^\u2032(tk)$ extracted from the augmented data, which is expressed as

_{k},**X**= [

**x**

_{1},

**x**

_{2}, …,

**x**

*]. The GP model $Mk\u2032$ is formulated as*

_{n}**h**(

**x**)

**β**and covariance function

*V*(

**x**,

**x**′) =

*σ*

^{2}

*R*(

**x**,

**x**′). The term

**h**(

**x**) is the vector of predefined polynomial functions and

**β**is the vector of corresponding coefficients. In the GP model, the covariance function

*V*(

**x**,

**x**′) is expressed as

**ω**= [

*ω*

_{1},

*ω*

_{2}, …,

*ω*] is the vector of roughness parameters that capture the nonlinearity of the process,

_{k}*d*is the dimension of the input

**x**, and

*σ*

^{2}is an unknown variance. Therefore, the unknown hyperparameters

**β**,

*σ*

^{2}, and

**ω**fully characterize the GP model, which can be approximated by the maximum likelihood estimation (MLE) method based on the training data. Once the hyperparameters are achieved, the GP model is capable of predicting the response at point

**x**′ as a normal distribution with mean

**r**is the correlation vector between

**x**′ and the training points

**X**, and

**H**is a unit vector if the prior mean function is a constant. The resultant mean prediction $\mu k\u2032$ is actually the response estimation at

*G*(

**x**′,

**z**′(

*t*),

_{k}*t*). With the identical training inputs

_{k}**X**= [

**x**

_{1},

**x**

_{2}, …,

**x**

*], a set of GP models can be constructed based on the data set*

_{n}**ŷ**′(

*t*) with

_{k}*k =*1, …,

*s.*As a result, the time-dependent responses for function

*G*(

**x**′,

**z**′(

*t*),

*t*) can be estimated by collecting the results from all

*s*GP models. By combining the multiple LSTMs and the set of GP models, time-dependent system response predictions can be achieved with the consideration of uncertainties due to input variations and stochastic processes. Given any random realization of the time-independent variables and stochastic processes, the proposed ensemble learning framework can provide response predictions in a global sense.

### 3.3 Time-Dependent Reliability Analysis.

The procedure of employing the proposed ensemble learning framework is summarized in a flowchart as shown in Fig. 3. According to the statistical information of the input variables and the stochastic processes, *N* random realizations are generated as the MCS samples, denoted as **U**^{mcs} = [**X**^{mcs}, **Z**^{mcs}(*t*)], where **X**_{mcs} = [**x**_{1}* ^{m}*,

**x**

_{2}

*, …,*

^{m}**x**

*] and*

_{N}^{m}**Z**

^{mcs}(

*t*) = [

**z**

_{1}

*(*

^{m}*t*),

**z**

_{2}

*(*

^{m}*t*), …,

**z**

*(*

_{N}^{m}*t*)]. To prepare the data for training multiple LSTM models, Latin hypercube sampling (LHS) is employed to generate

*n*random samples of the time-independent variables as

**X**= [

**x**

_{1},

**x**

_{2}, …,

**x**

*] and thus lead to*

_{n}*n*conditional limit state functions. With

*n*random realizations of the stochastic processes

**Z**= [

**z**

_{1}(

*t*),

**z**

_{2}(

*t*), …,

**z**

*(*

_{n}*t*)], the time-dependent responses are directly evaluated based on the conditional limit state functions. Following the procedure introduced in Sec. 3.1,

*n*LSTM models can be constructed while the

*i*th LSTM model is built based on the input [

**z**

*(*

_{i}*t*),

*t*] and output

**y**

*. In the proposed approach, the LSTM models are constructed based on the “Keras” library in python 3.6. The MSE is used as the loss function, where the “Adam” optimizer with default learning rate 0.001 is adopted for the training process of all LSTM models.*

_{i}**Z**

^{mcs}are provided to all the LSTM models for estimating the time-dependent responses for the local region. As shown in Fig. 1, the response predictions for the

*j*th random realization

**z**

*(*

_{j}^{m}*t*) using

*n*LSTM models are collected as

**ŷ**

*denotes the time-dependent response predictions from the*

_{i}^{j}*i*th LSTM model. As introduced in Sec. 3.2,

*s*GP models

**M**

*= [*

^{j}*M*

_{1}

*,*

^{j}*M*

_{2}

*, …,*

^{j}*M*] are, respectively, constructed based on the augmented data

_{s}^{j}**ŷ**

*(*

^{j}*t*) = [

_{k}**ŷ**

_{1}

*(*

^{j}*t*),

_{k}**ŷ**

_{2}

*(*

^{j}*t*), …,

_{k}**ŷ**

*(*

_{n}^{j}*t*)],

_{k}*k*= 1,2, …,

*s*. As a result, time-dependent response predictions for

*G*(

**x**

*,*

_{j}**z**

*(*

_{j}^{m}*t*),

*t*) are achieved by providing

**x**

*to the GP models*

_{j}**M**

*. The same procedure will be repeated for all the MCS samples, therefore, a total number of*

^{j}*N*

*×*

*s*GP models will be constructed for global time-dependent reliability analysis. Once all the time-dependent response predictions have been obtained, the minimum response corresponding to each MCS sample is extracted for reliability analysis. The MCS sample will be classified as failure or safe by an indicator function, given as

*µ*represents the GP model response prediction of the

_{k}^{j}*j*th MCS sample at the

*k*th time instant. This equation shows that a failure event occurs when the worst performance over the given time period is less than zero. After evaluating all the MCS samples, the time-dependent reliability can be approximated by

*N*represents the number of failure samples classified by the indicator function.

_{f}## 4 Case Studies

In this section, three examples are used to demonstrate the effectiveness of the proposed approach for solving time-dependent reliability analysis problems.

### 4.1 Case Study I: A Mathematical Problem.

**x**= [

*x*

_{1},

*x*

_{2}] following a normal distribution, and each stochastic process in

**z**(

*t*) is assumed to follow a stationary Gaussian process. The autocorrelation for the

*i*th stochastic process can be expressed as

*λ*are assigned to be 0.01, 0.005, 0.005, respectively. The statistical information of the random variables and the stochastic processes are summarized in Table 1. The time interval [0, 1] for this example is discretized into 60 nodes evenly, and 10

^{5}random realizations of the stochastic processes are generated for time-dependent reliability analysis.

The first step of employing the proposed approach is to prepare the training data for the LSTM models. As introduced in Sec. 3.3, the LHS is employed to generate ten samples of random variables **x**. Each LHS sample is combined with a random realization of the stochastic processes **z**(*t*), then the corresponding time-dependent responses are evaluated based on Eq. (22). Given the training data sets, ten LSTM models can be constructed, respectively. For each LSTM model, 10^{5} MCS samples of stochastic processes are provided for predicting the time-dependent system responses of the conditional limit state functions. Based on the achieved augmented data, GP models can be constructed by specifying a time node with the realization of the stochastic processes. Eventually, the time-dependent responses corresponding to the 10^{5} MCS samples can be obtained based on the GP models, which are further utilized for estimating the time-dependent reliability.

Ten random realizations of the stochastic processes that are used to train multiple LSTM models are shown in Figs. 4(a)–4(c), respectively, which plot the random realizations for each stochastic process, and the resultant time-dependent responses are depicted in Fig. 4(d). As a result, ten LSTM models are trained based on these training data, where the number of neurons for each LSTM model is set to 40. All the LSTM models are well trained by using Adam optimizer to minimize the MSE with 3000 epochs. To demonstrate that the LSTM is capable of accurately capturing the relationship between the stochastic processes and the time-dependent responses. The comparison of the actual and predicted time-dependent responses is depicted in Fig. 5, where Fig. 5(a) shows the response comparison of the first random realization **z**_{1}* ^{m}*(

*t*) using the first LSTM, and Fig. 5(b) shows the response comparison of the tenth random realization

**z**

_{10}

*(*

^{m}*t*) using the tenth LSTM. The results demonstrate that the LSTM models have been well trained, and thus are capable of accurately predicting the time-dependent responses of the conditional limit state functions for any realizations of the stochastic processes.

To demonstrate the accuracy of time-dependent response predictions using GP models, Fig. 6 shows the comparison of the accurate and predicted time-dependent responses for the 25th and 75th MCS samples, respectively. The results demonstrate that the constructed GP models can effectively handle the randomness of the time-independent variables, and they can provide accurate response predictions for the whole time-series data. For comparison purposes, the “equivalent stochastic process transformation (eSPT)” method [36] is adopted for approximating the time-dependent reliability of the mathematical example. To validate the accuracy of reliability estimation, the actual time-dependent system responses for the 10^{5} MCS samples are calculated, and the resultant accurate reliability 0.9754 is treated as a reference. The reliability approximations achieved by the proposed approach and eSPT are given as 0.9732 and 0.9726, respectively. By converting the stochastic processes into time-independent variables, the eSPT method requires 88 time-series data to obtain the reliability estimation. Note that the time interval consists of 60 time nodes. In the proposed approach, we constructed multiple LSTM models by using only ten time-series data. Therefore, the computational cost required by the proposed approach is much less than the eSPT method. By specifying different time intervals within [0, 1], time-dependent reliability given different time periods can be approximated based on the predicted responses. The comparison between the reliability approximations using the proposed approach and accurate results computed by direct MCS is summarized in Table 2, which indicates that the proposed approach is capable of accurately capturing the variation of time-dependent reliability with respect to time.

### 4.2 Case Study II: A Corroded Beam Problem.

*b*

_{0}and height

*h*

_{0}. Due to the corrosion, the size of cross section decreases with time, where the time-dependent behavior can be modeled as

*b*(

*t*) and

*h*(

*t*) are modeled as two time-dependent random variables. A stochastic load

*F*(

*t*) is applied at the middle span, which follows a stationary Gaussian process. The yield strength of the material is denoted by

*σ*, and the failure event occurs when the maximum stress exceeds the yielding limit of the beam. Therefore, the limit state function of the corroded beam problem is expressed as

_{y}The time interval is given as [1, 30] month, which is evenly divided into 59 time nodes. Following the numerical procedure, the proposed approach is employed to solve the corroded beam problem. For reliability analysis, 10^{5} MCS samples are generated according to the statistical properties of the random variables and the stochastic process. Fifteen training data sets are utilized for training the LSTM models, and the augmented data are obtained by using the LSTM models to predict the time-dependent responses given the 10^{5} random realizations of the stochastic process. Accordingly, GP models can be constructed to model the time-dependent responses corresponding to each random realization of *F*(*t*). As a result, time-dependent response predictions for the MCS samples are obtained based on the GP models.

For comparison purposes, the eSPT method is employed for time-dependent reliability analysis with a predefined cumulative confidence level 0.999. The reliabilities calculated by direct MCS, proposed approach, and the eSPT are given as 0.9756, 0.9748, and 0.9778, respectively. The results show that both the eSPT and the proposed approach can achieve an accurate time-dependent reliability estimation. In the eSPT method, 84 time-series data are utilized for updating process while the number of time-series data required by the proposed approach is 15. The results indicate that the proposed approach can achieve accurate time-dependent reliability estimations in an efficient manner.

In this study, the LSTM models are trained with a maximum epochs 3000, and the convergence of the MSE loss function for the first LSTM is shown in Fig. 8. It has been observed that all the LSTM models have similar convergence curves. To validate the effectiveness of the LSTM, the comparison of the actual and predicted responses using LSTM models is shown in Fig. 9, where Fig. 9(a) shows the response comparison of the first random realization *F*_{1}* ^{m}*(

*t*) using the first LSTM, and Fig. 9(b) shows the response comparison of the sixth random realization

*F*

_{6}

*(*

^{m}*t*) using the sixth LSTM. The results demonstrate that the LSTM models can make accurate response predictions for the conditional limit state functions with any random realization of the stochastic process. Thought the augmented data are not collected by directly evaluating the conditional limit state functions, the accuracy is guaranteed due to the benefits of using the LSTM.

Once all the augmented data have been collected from the multiple LSTM models, GP models for modeling the time-dependent response can be constructed as introduced in Sec. 3.3. As a result, the time-dependent responses corresponding to each MCS sample can be predicted based on the GP models. The actual minimum responses for the first 50 MCS samples are compared with the minimum responses extracted from the GP predictions. As shown in Fig. 10, the estimated minimum responses are ensured to be accurate, which almost overlap the actual ones. Considering all the 10^{5} MCS samples, Fig. 11 shows the comparison between the PDFs of the actual and estimated minimum responses. The results reveal that the proposed ensemble learning framework can provide accurate predictions of the minimum value of the time-dependent responses within the time interval, thus lead to accurate time-dependent reliability approximation. By specifying different time intervals, time-dependent reliability can be approximated based on the overall time-dependent response predictions. The comparison between the reliability approximations and accurate results computed by direct MCS is summarized in Table 4, which indicates that the proposed approach is capable of accurately predicting the time-dependent reliability within any time intervals [0, *T*_{I}], *T*_{I} ≤ T.

To test the robustness of the proposed approach, time-dependent reliability estimations are performed for the corroded beam problem with different modifications. Four different scenarios are introduced according to the detailed configurations presented in Table 5. With the identical settings and procedure as previous, the proposed LSTM-based ensemble learning framework is employed for estimating the time-dependent reliability for each scenario. To demonstrate the accuracy of the proposed approach, the direct MCS method is also adopted for accurate reliability analysis, and all the results are presented in Table 5. For different scenarios, the performances of the proposed approach are quite stable as accurate reliability approximations can always be achieved with small relative errors. The results demonstrate that the proposed approach is capable of solving time-dependent reliability analysis problems involving stochastic processes.

### 4.3 Case Study III: A Truss Example.

*F*(

*t*) in

*kN*has been applied on the tenth node along the horizontal direction. The area of the cross section

*A*(10

^{−4}m

^{2}) and Young’s modulus

*E*(GPa) for the bars are treated as two independent random variables that follow normal distributions, where

*A ∼ N*(4, 0.5) and

*E*∼

*N*(70, 2), respectively. The stochastic load

*F*(

*t*) is modeled as a non-stationary Gaussian process, which is characterized by its mean function

*μ*(

_{F}*t*), standard deviation function

*σ*(

_{F}*t*), and autocorrelation function

*ρ*(

_{F}*t*), expressed as

*δ*

_{FEA_}_{max}(.) represents the maximum displacement evaluated by the finite element analysis. In this study, the time interval [0, 12] is evenly discretized into 50 time nodes, and 10

^{5}MCS samples are generated for time-dependent reliability analysis.

The proposed approach is employed for time-dependent reliability analysis of the truss example, where eight training data sets are utilized for training the LSTM models. The maximum epochs are set to 3000 during the LSTM training processes. To validate the accuracy of the LSTM models, the predicted responses for the first random realization *F*_{1}* ^{m}*(

*t*) using the first LSTM model are compared with the actual responses as shown in Fig. 13. After collecting the augmented data from all LSTM models, GP models are constructed to model the global time-dependent responses. As shown in Fig. 14, accurate time-dependent response predictions can be achieved for the MCS samples. In Fig. 15(a), the actual minimum responses for the first 50 MCS samples are compared with the predicted minimum responses that are extracted from the GP estimations, while Fig. 15(b) depicts the PDF comparison between the actual and estimated minimum responses of all MCS samples. With eight time-series data, the proposed approach provides a reliability estimation of 0.9467, while the actual reliability evaluated by direct MCS is given as 0.9408. The results demonstrate that the proposed approach is applicable to time-dependent problems involving non-stationary processes.

To investigate the performance when dealing with higher reliability levels, the mean value of the random variable *A* is modified to 4.7. The accurate reliability evaluated by 10^{5} MCS samples is given as 0.99118. With eight time-series data, the reliability approximation using the proposed approach is given as 0.99486. The estimated minimum responses for the first 50 MCS samples are compared with the actual values as shown in Fig. 16(a), and the overall PDF comparison is depicted in Fig. 16(b). The accuracy of the proposed approach is demonstrated through the comparison study, and the results show that the proposed approach can provide a reasonable reliability approximation when handling higher reliability levels.

## 5 Conclusion

In this paper, an LSTM-based ensemble learning framework has been established, where MCS is adopted for estimating the time-dependent reliability based on the combination of LSTM networks and the GP modeling technique. To employ the LSTM network for learning the relationship between stochastic processes and time-dependent system responses, conditional limit state functions are introduced by fixing the time-independent random variables. Based on one time-series data, an LSTM model can be constructed for modeling a specific conditional limit state function, which can provide accurate response predictions given any random realizations of the stochastic processes. The time-dependent response predictions collected from multiple LSTM models are reorganized according to the realizations of stochastic processes and time instant. Gaussian process models are constructed to specifically model the time-dependent response with respect to random variables. As a result, time-dependent response predictions for the MCS samples can be achieved based on the GP models. The results from three case studies demonstrate that the proposed approach can handle complex problems involving multiple stochastic processes. Moreover, the proposed approach is capable of accurately predicting the overall time-dependent responses, which ensures the accuracy of reliability estimation and enables the capability of depicting the change of reliability within different time intervals.

## Conflict of Interest

There are no conflicts of interest.