Tree-Based Ensemble Learning Models for Wall Temperature Predictions in Post-Critical Heat Flux Flow Regimes at Subcooled and Low-Quality Conditions

Post-critical heat flux (CHF) heat transfer is a critical phenomenon in the analysis of various industrial applications, such as cryogenic applications [1], metallurgical processing [2], conventional boilers [3], and water-cooled nuclear reactors [4]. The post-CHF regimes involve complicated flow and heat transfer processes. As an example, during the reflooding phase of a hypothetical loss of coolant accident in a light water-cooled nuclear reactor (LWR), post-CHF heat transfer may be encountered due to high wall temperatures of nuclear fuel rods. To prevent fuel rods from overheating, subcooled water from the reactor emergency core cooling system is injected into the reactor core. Inverted annular film boiling (IAFB) will usually occur just downstream of the quench front, where a thin vapor film develops on the fuel rods and a liquid core exists in the central region of the flow channel. As the void fraction increases and the liquid core breaks up into liquid slugs, the flow regime transitions to the inverted slug film boiling (ISFB) regime (for low mass flux conditions) or agitated inverted annular film boiling (AIAFB) regime (for high mass flux conditions) [5]. The IAFB and ISFB/AIAFB regimes are regarded as low-quality film boiling regimes. Further downstream, the liquid slugs eventually break up into small droplets and the flow enters a high-quality regime, i.e., dispersed flow film boiling (DFFB). These above-mentioned post-CHF processes are caused by the departure from nucleate boiling (DNB) mechanism, which usually occurs at subcooled and low-quality inlet conditions. Another major mechanism that leads to post-CHF is the dryout of a liquid film that contacts the heated wall at high-quality conditions, which directly leads to the DFFB regime without going through the IAFB or ISFB/AIAFB regimes. A common feature of these post-CHF regimes is that the heat transfer is significantly deteriorated, causing high wall temperatures that can potentially lead to damage or failure of the heated surface. Therefore, accurately predicting wall temperatures in the post-CHF regimes is of high importance in the design and safety analysis of light water-cooled nuclear reactors and a variety of other industrial applications.

In recent years, many experimental and theoretical studies have been performed on the heat transfer characteristics in the post-CHF flow regimes [6–11]. A commonly used method to calculate the heated wall temperatures is to use empirical heat transfer correlations, which are typically high-order nonlinear functions of multiple empirical parameters, including the flow channel geometry, fluid properties, and flow conditions. These correlations can be characterized into three groups: (1) modified Bromley's model [12], (2) modified Dittus-Boelter model [13], and (3) laminar flow model [10,14]. These heat transfer correlations are useful and convenient for engineering applications and, to some extent, reveal physical mechanisms of the film boiling phenomena. However, due to the complex physical nature and flow regime transitions in the post-CHF flow regimes, these correlations are usually complicated functions with multiple empirical parameters that are calibrated based on experimental measurements. As a result, most of these correlations are only applicable to relatively narrow ranges of flow conditions and cannot be readily extended to beyond their range of validity, limiting their applications in practice.

Considering the thermal nonequilibrium characteristics in film boiling regimes, one-dimensional (1-D) two-fluid model is widely applied to predict wall temperature and void fraction in post-CHF flow regimes by solving the conservation equations for the liquid and vapor phases [15–18]. To solve the 1-D two-fluid model, closure models, including those for the wall heat transfer and friction, interfacial heat transfer and friction/shear/drag, etc., are needed as source terms to make the conservation equations solvable. However, due to the lack of local flow information in the post-CHF flow regimes, the empirical closure models currently used in the relevant computer codes, such as system-level and subchannel analysis codes, are usually extended from generally encountered two-phase flows. Such a mismatch in the closure models and flow regimes could cause large uncertainties in the simulations that involve post-CHF flow regimes. A robust and generalized model to predict the heat transfer and wall temperature in post-CHF regimes with good accuracy is, therefore, highly desired.

With the advances in algorithm and computing capabilities, data-driven modeling enabled by machine learning (ML) has received increasing attention and has been widely applied to classification and regression problems in many industrial applications in recent years [19–22]. A variety of ML methods have been used for data-driven modeling in engineering applications, including support vector machines (SVMs), Gaussian processes (GPs), decision trees (DTs), and artificial neural networks (ANNs). These ML methods have unique advantages in complex and nonlinear engineering problems over the conventional empirical correlation method. Instead of first identifying a fixed function form and then calibrating a few empirical parameters/coefficients, ML methods can directly learn from large datasets and make predictions with good accuracy and generalizability. In the field of nuclear thermal hydraulics, ML methods have been widely used to analyze a large amount of data obtained from experiments and/or numerical simulations for heat transfer problems [23–25]. ANNs and SVMs are two common ML methods used in this field [26,27]. However, both methods have some shortcomings. For example, it is difficult to determine the architecture and hyperparameters of ANNs, such as the number of hidden layers and the number of neurons in each hidden layer. In SVMs, the kernel function needs to be designed correctly to achieve good prediction results, however, selecting the right kernel could be tricky and largely depends on user's experience.

Compared to training a ML model with sophisticated architecture and many learnable parameters, such as deep neural networks, an alternative approach is to use the ensemble methods. The ensemble methods combine the predictions of multiple base estimators with relatively simple structure to improve generalizability and robustness over a single estimator. There are two groups of ensemble methods, i.e., averaging method (or bagging method) and boosting method. In the averaging method, the principle is to build many base estimators independently and form the final prediction by averaging their predictions with reduced variance, which is usually better than a single base estimator. In the boosting method, base estimators are built sequentially and fit on the residual of the former estimators to reduce the bias of the combined estimator. Random forest (RF) and gradient boosted decision trees (GBDT) are, respectively, representatives of the averaging method and boosting method, which combine a set of base estimators, i.e., DTs, to make more accurate predictions over a single DT. The model trained by the ensemble methods is robust and can give smoother predictions compared to a single complicated ML model, making it ideal for many engineering applications.

To the authors' best knowledge, although ML methods have been widely used for dryout point and CHF predictions [24–26], limited ML-related research work has been performed on wall heat transfer problems in the post-CHF flow regimes, especially at subcooled and low-quality conditions. In this paper, RF and GBDT models are developed, as a first attempt, to predict wall temperatures in the post-CHF flow regimes. We collected and summarized 22,813 experimental data points obtained from steady-state subcooled and low-quality film boiling experiments with water as the working fluid in vertical tubular test sections in the literature and consolidated them into a Low-quality Water Film Boiling (LWFB) database, which is then compared with three conventional heat transfer correlations for the film boiling regimes and is also used to train and validate the ML models. In addition, a statistical analysis is performed to evaluate the performance of the ML models by analyzing the relative errors between the measured and predicted wall temperatures for the GBDT and RF models.

To perform steady-state post-CHF experiments with water as the working fluid, especially at subcooled and low-quality conditions, Groeneveld [28] first applied an indirectly heated hot patch technique to achieve steady-state film boiling phenomena in a tubular test section in 1974. Then in 1984, Chen and Li [29] proposed a directly heated hot patch technique for their steady-state film boiling experimental investigation. Schematics of these two hot patch techniques are shown in Fig. 1. The test section is a Joule heated round tube. The indirectly heated hot patch is a copper block with high thermal inertia and a set of electrical cartridge heaters are included for the hot patch. In the directly heated hot patch technique, the wall thickness of the tubular test section is reduced over a short length near the tube ends to increase the local electrical resistance of the tube and therefore, increase the local heat flux at the reduced thickness section, which is called notch. In both techniques, high heat fluxes are realized for the hot patch, just upstream of the film boiling section, to form a dry patch in the hot patch and stabilize the quench front. The dry patch in the hot patch propagates to the downstream of the hot patch so that steady-state film boiling can be achieved in the test section. The function of the hot patch is to prevent the quench front from propagating along the test section and rewetting of the heated tube wall downstream of the quench front. With this technique, the average heat flux values required in the test section are relatively low and steady-state film boiling can be formed with the average wall heat flux considerably below the CHF values, which can avoid overheating the test section.

Both the indirectly heated and directly heated methods have been widely used in the literature for steady-state film boiling experiments with water as the working fluid in vertical tubular test sections. Relevant experimental data have been collected from steady-state post-CHF experiments that were performed in the 1980s and 1990s, available in the literature. All the experimental data include the wall temperature information but only part of the data have the void fraction information available. Table 1 summarizes some of these steady-state post-CHF experiments with water as the working fluid at subcooled and low-quality conditions [30–40], including the geometrical parameters (inner diameter, ID; outer diameter, OD; and test section length), flow conditions (system pressure, p; mass flux G; inlet subcooling ΔT_sub,in; wall temperature T_w; and wall heat flux $q″in$⁠), and the type of hot patch techniques. Tables 2 and 3 summarize the test matrices of these experiments. We collected a total of 22,813 experimental data points from the nine data sources in the accessible literature and consolidate them into our LWFB database, which covers a wide flow range of the system pressure from 0.1 to 9.0 MPa, mass flux from 25 to 2750 kg/m² s, and inlet subcooling from 1 to 70 °C.

where $σSB$ is Stefan-Boltzmann constant, 5.670367 × 10⁻⁸ (kg/s³-K⁴); and $εl$ and $εw$ are respectively the emissivity of the liquid water and heated wall inner surface.

The prediction of the wall temperature in the post-CHF regimes can be characterized as a regression problem in the supervised learning. In other words, we aim to train a regression ML model with a set of input–output pairs. In this work, the input features to the ML model are the flow conditions of a certain post-CHF experiments, while the output is the predicted wall temperature associated with the conditions. To evaluate the overall generalization capability of the ML model, we divide the full LWFB database into two parts: a training dataset used to train the ML model and a testing dataset to evaluate the ML's performance on unseen observations. While a wide range of supervised learning algorithms can be used in regression problems, the ensemble methods have been proven to be simple but powerful approaches that combine the predictions from multiple base estimators to improve the predictive performance. RF and GBDT are two robust tree-based ensemble learning methods with DTs as the base estimators. In this work, these two methods are used to predict wall temperatures in the post-CHF flow regimes. Both methods are computationally efficient with good generalization capabilities and do not require fine tuning of many hyperparameters.

The RF method was proposed by Breiman [41] and represents a prominent ensemble method that builds many individual trees in parallel. To grow multiple trees, RF uses the bagging or bootstrapping approach to generate samples with replacement from the training data. The sampled data are to grow DTs in parallel and all the DTs are independent and randomized. Since the data in each DT contain different portions of the original data, the corresponding trees differ across samples and therefore, form an ensemble of distinct DTs. After growing a RF, the final prediction is obtained by averaging the results over all the individual DTs for regression problems. Averaging usually achieves a better performance in prediction than a single DT because it decreases the variance and improves the accuracy of the model. Figure 2 shows a simplified RF structure for regression problems. The green nodes represent the optimal path. In this work, we train a RF model with 190 trees and the maximum depth of each tree is 16. The total number of the edges from the root node to the leaf node in the longest path is known as the depth of the tree. The tree depth controls the complexity of each tree and the computational cost.

RF is an averaging method, in which each DT is grown separately and independent of each other. GBDT represents another family of ensemble methods based on sequential learning proposed by Friedman [42]. Figure 3 shows a simplified GBDT structure for regression problems. In the GBDT method, each new tree is grown in sequence based on the pseudo-residuals of the previous tree and thus each tree depends on the results of its predecessor and provides a more accurate prediction. By adding sequential trees to the ensemble model (combination of all the previous trees), the loss function is minimized, and the overall error of prediction reaches a global minimum. In this work, our GBDT model has 400 trees, and the maximum depth of each tree is 6.

where N is the number of data points, y_m the observed value, and y_p the predicted value. The objective of the training process is to minimize the loss function by gradually updating the trainable parameters of the ML model. The trained ML model is then validated by the testing dataset. If the trained ML model does not have good performance on either the training or testing dataset, the hyperparameters are tuned to repeat the training process. Through multiple iterations, the ML model with optimal performance can be obtained. The experimental data we collected and the two ML models we developed can be accessed by readers through the link provided in the Appendix.

Based on the measured heat flux in the experiments, the wall temperature can be calculated from Eqs. (2) and (3) using these three empirical correlations.

Figure 5 shows comparisons of the wall temperatures calculated by these three correlations against those in the LWFB database, as well as the relative errors. The predictions of these correlations are similar and large discrepancies are observed between the experimental results and the three empirical correlations. The percentage of the data points that are within the ±50% error band is 68.9%, 69.8%, and 71.3% for the Bromley correlation, Berenson correlation, and CTF correlation, respectively. The relatively poor performance of these three correlations suggests that the conventional empirical correlations do not have good generalization capability to cover the wide range of flow conditions in the LWFB database. Moreover, these conventional heat transfer coefficient correlations were developed based on the fluid saturation temperature instead of the vapor bulk temperature, which may be considerably higher than the saturation temperature in the post-CHF flow regimes. However, the vapor temperature was not measured in the experiments included in Table 1, primarily due to the difficulties in accurately measuring the (high) vapor temperature. In summary, it is difficult for the conventional empirical correlations to accurately predict the wall temperatures in the post-CHF flow regimes over a wide range of flow conditions and therefore, a more robust model with improved generalization capabilities is needed.

In the present work, two data-driven tree-based ML models, based on GBDT and RF, are developed for wall temperature predictions in the post-CHF flow regimes using the LWFB database. The full LWFB database is split 80% and 20%, respectively, into a training dataset and a test dataset for model training and validation, as illustrated in Fig. 4. Figure 6 shows the predictions of the GBDT and RF models against the training data and test data, respectively. In general, the wall temperatures calculated by the RF and GBDT models match well with the experimental data. 98.7% of the training data points and 96.8% of the test data points are within the ±20% error band for the GBDT model and 98.4% and 96.1%, respectively, for the RF model. We observe that the ML models perform slightly better for the training data than the test data. This could be caused by the overfitting of the models, but such an overfitting is controlled well through the training process and can be deem as acceptable. Figures 6(b) and 6(d) show that the GBDT model performs slightly better than the RF model. The relative error slightly decreases as the wall temperature increases. Comparing Fig. 5 with Fig. 6, the predictive capabilities of these two ML models demonstrate significant improvement over the three conventional empirical correlations.

The ML models can be implemented into computational thermal-hydraulic codes of interest to replace the empirical model(s) for improved modeling capability. As a result, the wall temperature, vapor temperature, and heat transfer coefficient can be evaluated through an iterative process. A detailed procedure for such an implementation will not be discussed in this work, but the implementation can be similar to the subcooled flow boiling heat flux partitioning algorithm [46]. It should be noted that although the current ML models are trained based on a Python script and the scikit-learn library, no issue is expected to convert the models to another computer language, such as Fortran or C++, so the ML models can be compiled with the thermal-hydraulic codes as a data-driven closure relation. In this way, the ML models will be naturally fit into the physics-based model and its solution process. One such an example is the SAM-ML work [47], in which a Python-based neural network model is converted into C++ and is then integrated into the System Analysis Module (SAM) code [48] for sodium-cooled fast reactor safety analysis.

where T_sat is the saturation temperature (°C). The relative error of a good regression model should meet three criteria as follows:

1. The relative errors follow a normal distribution;

2. The relative errors are independent of the input features X (p, G, $ΔTsub,in$⁠, x_e, and D_h in this study). In other words, the relative errors should have no pattern relative to the input features X;

3. Most of the relative errors are close to zero.

To study the characteristics of the relative error of the ML models, the above three criteria are investigated by plotting the relative error in various ways. To study the first criterion, the histograms of the relative errors of the wall temperatures from the training and test datasets for the GBDT and RF models are shown in Fig. 7. The probability density function in the histogram is calculated by each bin's raw count divided by the total number of counts and the bin width. The normal distributions in Fig. 7 are obtained based on the mean and variance of the relative errors in the training and test datasets, respectively. The relative errors from these two ML models, in general, follow a normal distribution for both the training and test data, but the relative errors for the GBDT model are closer to the normal distribution than those for the RF model, indicating the GBDT model performs marginally better than the RF model.

We further use the quantile–quantile (Q–Q) plot, a commonly used visual method, to evaluate the relative error distribution of the predictions by the two ML models by plotting their quantiles against the ideal normal distributions, as depicted in Fig. 8. The values on the x axis are based on the normal distribution and those on the y axis are relative errors of the wall superheats. If the two distributions are similar, the points in the Q–Q plot will approximately lie on the line y = x. It can be seen from Fig. 8 that each of these two plots for the GBDT and RF models have an “S” shape. In the relative error range of [−0.1 0.2], the plots approximately lie on the line y = x for both the GBDT and RF models, which indicates that the distribution of the relative error of the predicted wall superheats follows a normal distribution in this range. Near the two ends of the plots, where the relative error is larger than 0.2 or smaller than −0.1, the trend of the Q–Q plot is steeper than the line y = x, indicating that distributions of the relative errors of the wall superheats have heavier tails than a normal distribution, which can also be seen in Fig. 7. Over 91% of the relative error data points are within [−0.1, 0.2] for the training data and over 86% for the test data for both the GBDT and RF models. Therefore, the relative errors can be regarded as, in general, following normal distributions based on the Q–Q plots for both the GBDT and RF models.

To investigate the second criterion, Fig. 9 shows the relative error of the wall superheats relative to five variables, i.e., p, G, $ΔTsub,in$⁠, x_e, and D_h, for the GBDT and RF models. The relative errors are randomly distributed along the x axis and do not follow any patterns relative to the x axis, which indicates the relative errors meet the second criterion. For the thermodynamic equilibrium quality in Fig. 9(d), the relative errors have slightly higher peaks near thermodynamic equilibrium quality value of zero. From Figs. 9(a) and 9(b), the absolute relative errors have large values, slightly over 60%, for low-pressure (<0.7 MPa) and low-mass flux (<550 kg/m² s) conditions while for high pressures and high mass fluxes, most of the absolute relative errors are within 20%. This is because most of the data in the LWFB database were obtained from low-pressure and low-mass flux conditions, i.e., 57.6% of the data points fall in the pressure range of [0.1, 0.7] MPa and 86.3% of the data in the mass flux range of [25, 550] kg/m² s. As described in Sec. 2, the LWFB database is consolidated based on nine different data sources, and this introduce uncertainties resulting from the use of different test facilities and the data collection by different researchers. Therefore, it appears reasonable that the more data we have, the more scattering the data will be, which will result in relatively larger prediction errors. From Fig. 9(d), the absolute relative errors have large values in the thermodynamic equilibrium quality range of [0, 0.1]. One of the reasons for this large relative errors is that similar to the above hypothesis, 54.6% of the data points fall in this range. In addition, the flow regime transition from the IAFB to the ISFB regimes may occur in this thermodynamic equilibrium quality range [49], which may also contribute to the observed large prediction uncertainties.

To assess the two models with respect to the third criterion, the cumulative probability distribution function (CPDF) of the absolute relative error for the two ML models are calculated and shown in Fig. 10. The CPDF curves for the training data rise slightly steeper than those for the test data. The curves rise rapidly when the absolute relative error is smaller than 10% and then tend to become flat. For both the GBDT and RF models, over 89% of the relative errors are within [−0.1, 0.1] for the training data and over 80% for the test data. Therefore, it is believed that the relative errors of the two models meet the third criterion.

The range of R is [−1:1].

The results for these three metrics are summarized in Table 4. Minor differences of these three metrics are observed for the three different DSRs. The two trained ML models showed good predictive capabilities for all different DSRs. Taking the (80%, 20%) split as an example, the RMSEs of both models are very small, i.e., 5.7% for the GBDT model and 6.2% for the RF model. The correlation coefficient is 97.1% for the GBDT model and 96.6% for the RF model. The 95% CI of the relative error is [−17.5%, 15.5%] for the GBDT model and [−18.8%, 16.7%] for the RF model. All these three metrics for the test data indicate that the GBDT and RF models have good predictive capabilities for the whole LWFB dataset while the GBDT model performs slightly better than the RF model, which is consistent with the discussions in Sec. 4.1. The applicable range of the developed GBDT and RD models is recommended to ensure the validity of the two ML models as: 0.1 $≤$p$≤$ 9.0 MPa, 25 $≤$G$≤$ 2750 kg/m² s, and 1 $≤$ ΔT_{sub, in}$≤$ 70 °C.

In this study, we propose a data-driven approach to develop machine learning models for wall temperature prediction in the post-CHF flow regimes. A general low-quality water film boiling database with 22,813 data points is consolidated from nine different data sources in the literature. Two tree-based ensemble models, i.e., GBDT and RF, are developed for the wall temperature prediction in the post-CHF flow regimes based on the LWFB database. The ML models take five flow features as the inputs, namely, the system pressure, mass flux, inlet subcooling, hydraulic diameter of the flow channel, and thermodynamic equilibrium quality, and predict the wall temperature as the output. The LWFB database is split into training data and test data to train and validate the GBDT and RF models, respectively. The performance of the two trained ML models is thoroughly evaluated and compared with three widely used empirical correlations with the full LWFB dataset. The conventional correlations show relatively poor performance in predicting the LWFB database with large errors, while the ML models show good accuracies for the entire database. A statistical analysis is further performed to evaluate the prediction uncertainty of the ML models. We found out that the relative errors between the measured and predicted wall superheats by the ML models approximately follow normal distributions. The relative errors are randomly distributed and have no clear patterns relative to the input variables. The cumulative probability distribution function within 10% error band covers over 80% of the test data points for both the GBDT and RF models. The RMSEs are 5.7% and 6.2% for the GBDT and RF models for the test data, respectively. The correlation coefficient is 97.1% for the GBDT model and 96.6% for the RF model. The 95% CI of the relative error is [-17.5%, 15.5%] for the GBDT model and [−18.8%, 16.7%] for the RF model. All these three metrics for the test data indicate that the GBDT and RF models have good predictive capabilities over a broad range of flow conditions for wall temperature in the post-CHF flow regimes, while the GBDT model has slightly better performance than the RF model. These two trained ML models have good predictive capabilities covering a broad range of conditions for vertical upward flow, i.e., the system pressure from 0.1 to 9.0 MPa, mass flux from 25 to 2750 kg/m² s, and inlet subcooling from 1 to 70 °C, and can obtain significantly more accurate wall temperature predictions than the currently used empirical correlations in the post-CHF flow regimes at subcooled and low-quality conditions.

The authors at the University of Michigan appreciate the financial support as well as the technical and administrative support provided by Mr. Andrew Ireland, Dr. Chris Hoxie, and Jennifer Dudek of the U.S. NRC.

• U.S. Nuclear Regulatory Commission (Award ID: 31310020C0009; Funder ID: 10.13039/100005187).

The data and information that support the findings of this article are freely available online.²

Part of the LWFB database used for training and the two tree-based ensemble models can be accessed through the following link given in footnote².