Thermal-Hydraulic Characterization of a Thermosyphon Cooling System for Highly Compact Edge MicroDataCenter¹

The delivery of computing services—including storage, databases, networking, software, analytics, artificial intelligence, and more—has started with cloud computing over the Internet, delivered from off-premise datacenters, running infrastructure at scale with a pay as you go service to their end customers. As the Internet of everything (IoE) continues to evolve, parts of these services are migrating from the centralized locations (cloud) to more decentralized locations closer to the end-user, e.g., fog computing, edge computing (EC), and only recently on to the end-user device itself, the mist computing. By 2025, Gartner predicts 75% of the data will be processed at the edge [1]. Applications are characterized by big data and near real-time processes in various areas, such as: energy management, highways, buildings, plant automation, supply chain, visual security, signage, and mobile-EC. Unlike for cloud systems, reference frameworks and standards for fog computing and edge computing are still in an early stage of development or nonexistent for mist computing.

As part of the European project BRAINE (a 28-partner consortium), a novel type of Edge MicroDataCenter (EMDC) providing cost-effective, powerful computing resources close to end-users and specifically designed for artificial intelligence elaborations was developed. The new very compact computer architecture had high heat fluxes to dissipate on the individual computer cards and a very high heat dissipation rate per unit volume in the dense array of these cards. This was a perfect match for two-phase microchannel cooling plates integrated into a passive thermosyphon (gravity-driven) cooling system, moving all the heat outside the server to be dissipated to indoor or outdoor air by a compact finned air-coil with a very low fan energy consumption. A patent for the new thermosyphon cooling system is pending.

The filling ratio of a thermosyphon (defined as the volume of liquid in the system over the total internal volume) is a critical parameter, and many experimental works have studied its impact on the thermal performance [2–4]. If the filling ratio is too low, this can induce an early dry-out in the evaporator, while if it is too high, the condenser can be flooded.

Two operational thermosyphon regimes have been identified in the literature [5]. The first regime is called the gravity dominant regime (GDR). In the GDR, buoyancy overbalances frictional forces when the heat load increases, and thus the working fluid mass flow rate increases as well. The second regime is called the friction dominant regime (FDR). In the FDR, frictional forces overbalance the increase in buoyancy caused by the high values of void fraction when the heat load increases and consequently, the mass flow rate decreases.

JJ Cooling's in-house solver validation has been presented in previous publications. In particular, a study presented in Ref. [6] compared the simulator against five independent databases, which included a wide range of operating conditions. Results from this study showed excellent predictive capabilities and robustness of the simulator. In Ref. [7], a comparison was made between predicted and experimental case temperatures for a 66 mm high air-cooled thermosyphon. The study considered the working fluids R1233zd(E) and R1234ze(E), heat loads ranging from 50 W to 700 W, and three different levels of air flow rate. The results showed that all 93 simulated datapoints were within an error band of ±10% when compared to the experimental measurements. In Ref. [8], the solver validation was performed using experimental data obtained with a 70 mm high air-cooled thermosyphon operating with R1233zd(E). The study considered heat loads ranging from 50 W to 500 W and three different air flow rates at 22 °C and 40 °C ambient temperatures. Out of the 72 simulated datapoints, 72% of the predictions for the case temperature were within ±2 °C of the experimental measurements.

The primary objective was the cooling of 11 nodes (which include CPUs, field programmable gate arrays, graphics processing units, and storage) considering one heat sink evaporator per node, but here the experimental results and solver validation results of a first demo EMDC which comprises four slots (i.e., four nodes) with several ceramic heaters mimicking the electronics components and its loop thermosyphon (LTS) cooling system are presented. The working fluids evaluated were the new environmentally friendly refrigerants R1233zd(E) and R1234ze(E).

The two-phase closed-loop thermosyphon cooling system and its instrumentation are presented in Fig. 1. It should be noted that although the liquid accumulator (LA) was used in some tests, it is not shown in the figure. In addition to the temperature measurements at the inlet and outlet of evaporator and condenser, 25 thermocouples were installed in the four nodes to monitor the temperature of the heaters mimicking the different components. The fan used at the condenser is model 9HV1224P1A001 from San Ace. Its maximum and minimum rated power consumption are 57.6 W and 8.88 W, respectively.

The internal structure of the solver involves modeling all the components of the thermosyphon, including the evaporator, riser, condenser, downcomer, and various fittings such as bends, contractions, and expansions. The solver takes into account the geometric characteristics of each component as well as the working fluid used, the total power dissipated, the evaporator saturation pressure, the subcooling at the inlet of the evaporator, and the ambient temperature. The solver combines models, correlations, and expertise that have been developed at JJ Cooling Innovation. Specifically, it focuses on two-phase flow pressure drop, as well as boiling and condensation heat transfer coefficients. Notably, the solver does not employ any scaling or empirical factors, it is for general use in our design of loop thermosyphon cooling systems. For each component, governing equations are solved, and the mass flow rate is then iteratively adjusted until Eq. (2) is satisfied.

In order to validate the solver, a comparison was presently made between the predicted pressure drop values of the evaporator, riser, condenser, and the maximum temperature of the heaters and the corresponding experimental measurements. Compared to the results presented in Ref. [9], simulations were re-run without the assumption that the downcomer is not full of liquid. In other words, operating points for which the downcomer is not full of liquid are now captured adequately.

where $VL$ is the volume of refrigerant in liquid phase, $VLTS$ is the loop thermosyphon internal volume, $m$ is the total mass of refrigerant in the loop thermosyphon, and $ρL$ is the saturated liquid density computed at 25 °C with refprop [10].

where $T¯CPU$ is the average temperature measured on all the heaters mimicking the CPUs, $Tair,in$ is the condenser air inlet temperature, and $Q$ is the total dissipated power at the evaporator.

The uncertainty related to temperature measurements was 0.150 °C, and the maximum uncertainty for the thermal resistance computed using the error propagation method [11] was 0.001 °C/W.

The filling ratio analysis in this work was performed at the system's nominal heat load, which is $Q$ = 600 W. The filling ratio effect on the thermal resistance for six different values of air volumetric flow rate (expressed as fan pulse-width modulation (PWM) percentage) is depicted in Fig. 2.

The lowest thermal resistance is observed for $FR$ = 60%. The optimum $FR$ is often associated with the minimum subcooling at the evaporator inlet. If the $FR$ is too high, it can overflood the condenser outlet, which in turn increases the subcooling and the length of the evaporator in single-phase flow. On the other hand, if the $FR$ is too low, it can lead to inadequate subcooling and result in poor flow distribution in the evaporator's channels, increasing the likelihood of dry-out. Therefore, it is essential to find the optimal $FR$ to ensure optimal performance and prevent issues such as dry-out or overflooding.

A significant impact of the air volumetric flow rate up to a PWM of 63% is observed as well with a maximum improvement in heat transfer. That is, beyond a certain value, the heat transfer improvement may saturate, and further increases in the PWM may not result in significant heat transfer enhancement. Thus, the fan should be operated appropriately to achieve the desired cooling with lower fan energy consumption.

Section 3.1, the best filling ratio was found to be 60%. Then, tests at different heat loads were performed at the fixed determined filling ratio to assess the impact of different operating conditions (e.g., idle state or overclocking) on the thermal performance. The results are shown in Fig. 3. Dividing by four, the heat load per node is obtained. The maximum uncertainty on the thermal resistance was 0.002 °C/W.

It is worth noting that the decrease in thermal resistance with increasing heat load suggests that the LTS is operating in the GDR [5], which is a desirable outcome as opposed to the FDR [5]. The steep decrease in thermal resistance from 100 W to 200 W is certainly due to a transition from pool boiling mode (no flow circulation in the loop) to thermosyphon mode (flow circulation in the entire loop). The flattening of the thermal resistance when higher values of heat load are applied means the LTS is still in GDR but close to transition to FDR. Finally, as it was observed during the filling ratio analysis, the impact of air volumetric flow rate on $R$ reached a saturation point for PWM values above 60%, which highlights the importance of air volumetric flow rate and its optimal range of operation.

A set of 99 experimental results for the loop thermosyphon charged with R1233zd(E) was considered. The experiments covered six different levels of air volumetric flow rate at the condenser and a range of heat loads from 300 W to 910 W. The $FR$ used for validation was 60%.

The percentage of datapoints within ±30% of the experimental pressure drop measurements for the different components was as follows: 100% for the evaporator, 76% for the riser, 45% for the condenser, and 91% for the total pressure drop.

Figure 4 presents the comparison between predicted and measured maximum temperature of the heaters mimicking CPUs. It can be observed that 94% of the predicted datapoints are within ±5 °C of the experimental measurements. The average absolute error is 1.7 °C.

To evaluate the impact of a LA on the thermal performance of the LTS, a specifically designed LA was installed at the outlet of the condenser, and the same experimental campaign as previously presented was run.

Figure 5 shows the results of the filling ratio analysis with R1233zd(E) at 600 W with the liquid accumulator installed. The maximum uncertainty for the thermal resistance was equal to 0.001 °C/W.

In Sec. 3, it was noted that there was an optimum $FR$ corresponding to a minimal thermal resistance and a maximum threshold for the PWM after which there was no significant effect on thermal performance. Here, for the LTS with LA, the same effects were observed, but with a higher optimal $FR$ of 70% instead of 60%, while the optimal PWM value remains around 60%.

The experimental results when the optimum $FR$ is considered are depicted in Fig. 6 for heat loads varying from 50 W to 950 W. The maximum thermal resistance uncertainty was 0.004 °C/W. At low values of air flow rate, the thermal resistance decreases with the heat load and seems to reach a limit value. However, for high values of air flow rate (fan PWM higher than 60%), the thermal resistance first decreases with the heat load and then it starts to increase slightly. The minimum value is reached at 600 W.

Figure 7 shows the experimental results of evaporator inlet subcooling for the LTS with and without the liquid accumulator at 60% filling ratio. Clearly, lower levels of subcooling are achieved when the LA is installed. This is highly desirable as it ensures better thermal performance of the evaporator and consequently of the entire LTS loop. The lower subcooling will guarantee that the entire or almost entire length of the evaporator will be in the two-phase flow region. It means higher heat transfer performance and more uniform temperature, since the working fluid remains at a constant temperature in the latent heat region.

It is also observed that when the liquid accumulator is not installed, the subcooling is highly dependent on the air flow rate (it is higher for the lower values), while when it is installed, the air flow rate has a much lower impact on the subcooling. In other words, the liquid accumulator reduces the subcooling variations with the fan operating conditions.

The thermal resistance comparison at optimal filling ratio with and without the liquid accumulator is presented in Fig. 8. It can be observed that the thermal resistance with the liquid accumulator installed is lower until 400 W. This suggests that the LTS with liquid accumulator is working throughout the entire range of heat load in thermosyphon mode: the liquid accumulator modulates the required charge of working fluid in the loop and increases therefore the LTS thermal performance.

The solver validation was performed considering 46 experimental results. The percentage of datapoints within ±30% of the experimental pressure drop measurements for the different components was as follows: 100% for the evaporator, 60% for the riser, 34% for the condenser, and 97% for the total pressure drop.

The comparison between predicted and measured maximum temperature of the heaters mimicking CPUs is plotted in Fig. 9. It can be observed that 100% of the predicted datapoints are within ±5 °C of the experimental measurements. The average absolute error is 1.1 °C.

The filling ratio analysis with working fluid R1234ze(E) and the liquid accumulator installed is presented in Fig. 10. The range of $FR$ evaluated was from 40% to 75% and PWM from 0% to 100% with steps of 20%. The maximal uncertainty observed for the thermal resistance was 0.001 °C/W. The thermal resistance evolution with the filling ratio is different with R1234ze(E) compared to R1233zd(E). Indeed, while R1233zd(E) exhibited a single minimum thermal resistance for a particular $FR$⁠, R1234ze(E) showed a plateau of minimum thermal resistance for $FR$s ranging from 50% to 75%. This suggests that R1234ze(E) may be more thermally effective than R1233zd(E) since it has a plateau of optimum filling ratio.

An optimal filling ratio of 75% was considered, and tests proceeded for heat loads from 100 W to 900 W. The thermal resistance evolution with the heat load for the different PWM levels is plotted in Fig. 11. The maximum uncertainty on the thermal resistance was 0.002 °C/W. The thermal resistance first decreases with the heat load and then stabilizes when high values of heat load are reached (Q ≥ 700 W). At low air flow rate values (0% and 20% PWM), the decrease in thermal resistance from 100 W to 200 W is steeper than for higher air flow rates.

Figure 12 shows the thermal resistance comparison at optimal filling ratio with working fluids R1234ze(E) and R1233zd(E) with the liquid accumulator. For lower values of air flow rate (0% and 20% PWM), both fluids yield very similar thermal resistance. However, for higher air flow rate, R1234ze(E) yields lower thermal resistances values than R1233zd(E) up to 200 W. Moreover, for fan PWM equal to 60%, 80%, and 100%, the thermal resistance increases only with R1233zd(E) for heat loads higher than 600 W. These observations suggests that R1234ze(E) is more efficient than R1233zd(E).

In total, 87 experimental results were considered for the validation. The percentage of datapoints within ±30% of the experimental pressure drop measurements for the different components was as follows: 98% for the evaporator, 91% for the riser, 72% for the condenser, and 98% for the total pressure drop.

Finally, Fig. 13 depicts the comparison between predicted and measured maximum temperature of the heaters mimicking CPUs. It can be observed that 98% of the predicted datapoints are within ±5 °C of the experimental measurements. The average absolute error is 1.2 °C.

Performance ratios up to 98 were observed. Additionally, a considerable impact on $η$ is observed when increasing the air volumetric flow rate. For example, a reduction of 3.7 times is noted at 800 W when the fan PWM is increased from 0 PWM to 60 PWM. Defining the power usage effectiveness to be equal to the ratio of total power consumption (power dissipated in the EMDC plus fan power consumption) to the power dissipated in the EMDC, the lowest value reached was 1.01 at 0% PWM at 800 W at 25 °C.

Table 1 provides a summary of all the pressure drop results presented previously, while Table 2 provides a summary of all the maximum temperature of the heaters mimicking CPUs results. It can be clearly seen that a significant percentage of the datapoints obtained for the evaporator, riser, condenser, and total pressure drop are within ±30% of the experimental measurements. Specifically, out of the 232 simulated points, 99%, 78%, 53%, and 95% of the datapoints for the evaporator, riser, condenser, and total pressure drop, respectively, are within this range which demonstrates the excellent predictive capabilities of the solver. It is noteworthy that no scaling or empirical factors were considered in the solver. Accurately predicting two-phase pressure drops is the key to predicting the flow rate within a loop thermosyphon, and thus achieving a robust design tool. The robustness of the design tool is obviously highlighted by the temperature results of Table 2.

The goal of this study was to present the experimental and solver validation results of a test campaign performed on a first demo four-slot Edge MicroDataCenter and its two-phase closed-loop thermosyphon cooling system.

The experimental campaign considered the two environmentally friendly working fluid R1233zd(E) and R1234ze(E) and evaluated the following aspects: optimal filling ratio determination, heat load effect, and air flow rate effect. Additionally, tests were conducted to compare the loop thermosyphon cooling system's performance with and without a liquid accumulator. The results showed that the loop thermosyphon with liquid accumulator worked throughout the entire heat load range as the liquid accumulator compensates for the lack or excess of working fluid charge in the loop. Finally, the tests revealed that working fluid R1234ze(E) had better thermal performances.

Concerning the solver validation, a total of 232 datapoints were simulated corresponding to different operating conditions (working fluid, filling ratio, heat load, and air flow rate) and two different configurations (with and without a liquid accumulator). The solver demonstrated excellent predictive capabilities with 97% of the datapoints within $±$5 °C of maximum measured CPUs temperature. It is worth noting that these predictions were achieved without any scaling or empirical factors.

This work was carried out as part of BRAINE Project funded under the INDUSTRIAL LEADERSHIP—Leadership in enabling and industrial technologies—Information and Communication Technologies (ICT) European Union Horizon 2020 Framework Programme which is gratefully acknowledged. The authors would like to thank also their colleagues at JJ Cooling Innovation for providing support in the review of this work.

• BRAINE Project funded under the INDUSTRIAL LEADERSHIP—Leadership in enabling and industrial technologies—Information and Communication Technologies (ICT) European Union Horizon 2020 Framework (No. 876967; Funder ID: 10.13039/100010661).

The authors attest that all data for this study are included in the paper.

FR =

filling ratio, %

g =

gravitational acceleration, m s⁻²

H =

height difference between the evaporator and the condenser, m

m =

mass, kg

P =

electrical power consumption, W

Q =

total dissipated power, W

R =

thermal resistance, ° C/W

$T¯$ =

average temperature, ° C

V =

volume, m³

Greek Symbols

$ΔP$ =

pressure drop, Pa

$ρ$ =

density, kg m⁻³

Superscripts and Subscripts

air =

air

CPU =

central processing unit

downcomer =

downcomer

driving =

driving

fan =

fan

friction =

friction

in =

inlet

L =

saturated liquid

LTS =

loop thermosyphon

momentum =

momentum

riser =

riser