Abstract

The data center's server power density and heat generation have increased exponentially because of the recent, unparalleled rise in the processing and storing of massive amounts of data on a regular basis. One-third of the overall energy used in conventional air-cooled data centers is directed toward cooling information technology equipment (ITE). The traditional air-cooled data centers must have low air supply temperatures and high air flow rates to support high-performance servers, rendering air cooling inefficient and compelling data center operators to use alternative cooling technology. Due to the direct interaction of dielectric fluids with all the components in the server, single-phase liquid immersion cooling addresses mentioned problems by offering a significantly greater thermal mass and a high percentage of heat dissipation. Single-phase liquid immersion cooling is a viable option for hyperscale, edge, and modular data center applications because, unlike direct-to-chip liquid cooling, it does not call for a complex liquid distribution system configuration and the dielectric liquid can make direct contact with all server components. Immersion cooling is superior to conventional air-cooling technology in terms of thermal energy management; however, there have been very few studies on the reliability of such cooling technology. A detailed assessment of the material compatibility of different electronic packaging materials for immersion cooling was required to comprehend their failure modes and reliability. For the mechanical design of electronics, the modulus and glass transition temperature (Tg) are essential material characteristics. The substrate is a crucial element of an electronic package that has a significant impact on the reliability and failure mechanisms of electronics at both the package and the board level. As per Open Compute Project (OCP) design guidelines for immersion-cooled IT equipment, the traditional material compatibility tests from standards like ASTM 3455 can be used with certain appropriate adjustments. The primary focus of this research is to address two challenges: The first part is to understand the impact of thermal aging on the thermomechanical properties of the halogen-free substrate core in the single-phase immersion cooling. Another goal of the study is to comprehend how thermal aging affects the thermomechanical characteristics of the substrate core in the air. In this research, the substrate core is aged in synthetic hydrocarbon fluid (EC110), polyalphaolefin 6 (PAO 6), and ambient air for 720 h each at two different temperatures: 85 °C and 125 °C, and the complex modulus and the glass transition temperature before and after aging are calculated and compared.

Introduction

Massive amounts of data are gathered, stored, processed, and distributed using data centers, which may be used for a variety of purposes, including business, amusement, and other requirements [1,2]. A single CPU's server power will normally be 140–190 W by 2020, whereas a single processor for high-performance computing will exceed 300 W. The heat dissipation of information technology equipment (ITE) accounts for 50–60% of the data center's heat source [3]. In 2025, data centers will consume 4.5% of the world's electricity, of which 40% will be used for cooling systems [4]. Globally, it is predicted that data centers used 91 billion kWh of energy annually in 2013, 140 billion kWh in 2020, and more in 2025. This would result in annual carbon emissions and operating electricity costing more than 150 million metric tons and $13 billion, respectively [5]. Because data centers consume a lot of electricity, particularly for cooling, decreasing energy use is a primary concern for IT companies and policy officials [6].

Despite significant technological advancement over the past few decades, managing the temperature of electronics or microprocessors still presents significant technical difficulties. The effective removal of increasing heat flux and extremely nonuniform power dissipation are the two key cooling problems [6]. Due to inadequate cooling capacity, high energy consumption, and high operational expenses, traditional air cooling has reached its limits. Immersion cooling offers several benefits such as higher thermal mass and high heat dissipation due to the dielectric fluids being in direct contact with all the components [7]. Single-phase immersion has several advantages such as it shields the ITE from the impact of contaminants, reduces failures due to vibration, and lowers the CapEx as it does not require sophisticated liquid distribution manifold construction, thus making data center architecture easier [7].

Reliability issues may arise when the heat-generating components are immersed in the dielectric fluids. Deformation, warpage, or delamination modes of failure that result in the failure of the complete package may be brought on by a slight variation in the material properties of the components [8]. While providing the material properties, most of the fluid suppliers and vendors send the soak test results. However, they do not consider operational reliability like the mechanical and electrical characteristics of the components unique to the application work environment [8]. The material compatibility may be divided into direct and indirect interactions. Direct interactions are processes that are generated by the fluid itself. They can occur in the fluid or when the fluid interacts with a component material. Dissolution, absorption/swelling, chemical contact of material, environmental stress cracking, fluid aging, and oxidation are the six basic ways for direct interactions to occur in single-phase fluids. Chemicals released or generated by direct contact with IT components induce indirect reactions rather than the fluid itself [9].

The thermomechanical characteristics of low loss printed circuit boards immersed in mineral oil for 720 h at 25, 50, 75, and 105 °C were studied by Chauhan et al. [10]. They found that at that the in-plane coefficient of thermal expansion values of the post-aged samples did not change much at these temperatures. Ramdas et al. [11] performed a similar study with FR-4 printed circuit board immersed in EC100 dielectric fluid. The results show that the modulus has decreased for post-aged samples which means that PCBs are less stiff and in turn decrease the warpage. Bhandari et al. [12] investigated the effect of single-phase immersion fluid EC100 on the mechanical properties of nonhalogenated substrates after immersion for 720 h at ambient temperature, 50 °C, and 75 °C. In this study, modulus of the immersed substrate is compared with the modulus of substrates exposed to air at the respective temperatures. It was observed that rising temperatures caused the modulus to decrease but at 75 °C in both air and EC100 fluid, a varying trend was observed.

The fundamental goal of this research is to quantify and characterize the material compatibility of immersed components. As per Open Compute Project (OCP) design guidelines for immersion-cooled IT equipment, the traditional material compatibility tests from standards like ASTM D471 and D2240 can be used with certain appropriate adjustments [9]. The present investigation is bifurcated into two distinct segments: The first stage involves comprehending the effects of thermal aging on the halogen-free substrate core within the dielectric fluid utilized for single-phase immersion cooling, with regards to its thermomechanical characteristics. Another goal of the research is to gain an understanding of the impact of thermal aging on the thermomechanical properties of the substrate core exposed in air at high temperature. The halogen-free laminate (IS550H) is utilized by individuals who necessitate exceptional thermal stability, as well as elevated power and voltage. The present study involves subjecting the substrate core to synthetic hydrocarbon fluid (EC110), polyalphaolefin 6 (PAO 6), and ambient air for a duration of 720 h at two distinct temperatures, namely, 85 °C and 125 °C. The complex modulus and the glass transition temperature (Tg) of the substrate core are subsequently measured using Dynamic Mechanical Analyzer (DMA) before and after the aging process, and a comparative analysis is performed.

Materials and Methods

The halogen-free substrate samples had a thickness of 0.25 mm and were trimmed to dimensions of approximately 50 mm × 5 mm. A set of 24 specimens were fabricated in preparation for DMA measurements. The selection of experimental parameters, namely, temperature (85 °C and 125 °C) and duration (720 h or one month), was made in accordance with the OCP guidelines and ASTM 3455. Figure 1 depicts the primary constituents of DMA, while Fig. 2 illustrates the sample mounted to the tensile probe. Figure 3 shows the thermal chamber which houses substrate samples that are submerged in EC100, PAO 6, and exposed to air. Table 1 provides information regarding the quantity of samples that are situated in each fluid.

Fig. 1
Schematic of DMA
Fig. 2
Sample mounted in the tensile attachment on DMA
Fig. 2
Sample mounted in the tensile attachment on DMA
Close modal
Fig. 3
Elastic modulus of sample exposed in air at 85 °C and 125 °C
Fig. 3
Elastic modulus of sample exposed in air at 85 °C and 125 °C
Close modal
Table 1

Aging of substrate samples in EC100, PAO 6, and air

Aging temperatureAging timeNo. of samples immersed in EC100No. of samples immersed in PAO 6No. of samples in the air
85 °C (complex modulus)∼720 h444
85 °C (glass transition temperature)444
125 °C (complex modulus)444
Aging temperatureAging timeNo. of samples immersed in EC100No. of samples immersed in PAO 6No. of samples in the air
85 °C (complex modulus)∼720 h444
85 °C (glass transition temperature)444
125 °C (complex modulus)444
The method of DMA is employed to quantify the kinetic properties of a given specimen, including its elasticity and viscosity [10]. Various DMA modules, including tension, bend, shear, and compression deformation attachments, can be employed to examine diverse material characteristics, depending on factors such as sample shape, modulus, and measurement objective. The DMA methodology is utilized to compute both the storage modulus and the loss modulus. Equations (1) and (2) provide the mathematical expression for the correlation among complex modulus, storage, and loss modulus [12]. The complex modulus that has been computed can be subjected to a comparison with Young's modulus. Following is the formula to calculate complex modulus:
(1)
(2)
(3)

where E* is the elastic modulus, E′ is the storage modulus, E″ is the loss modulus, and tan δ is the damping ratio.

The sample dimensions were measured utilizing a digital caliper that possessed an accuracy of 0.02 mm. The selection of the tensile attachment for the present study was based on the projected modulus derived from the sample dimensions and the material properties. Prior to being mounted onto the DMA for testing, the samples immersed in dielectric were appropriately cleaned using a paper towel. The DMA measurements were conducted with a fixed L amplitude of 10 μm and a force amplitude of 2000 mN for all samples. The experimental data were collected by exposing the specimens to a temperature range spanning from −40 °C to 280 °C, with a consistent ramp rate of 4 °C/min. Additionally, a range of frequencies, including 0.5, 1, 2, 5, and 10 Hz, were applied during the testing process. The auto-LN2 gas cooling unit facilitates the dispensation of liquid nitrogen to lower the temperature of the furnace below ambient temperature [13]. A thermal equilibrium was achieved by subjecting the system to an isothermal hold at the initial temperature, i.e., −35 °C. This resulted in an extension of the measurement duration of the DMA to approximately 2.5 h. Table 2 presents the parameters utilized for the purpose of testing.

Table 2

Testing parameters used in DMA

ParametersValues
Maximum force2000 mN
Temperature ramp3 °C
Sample thickness0.25–0.3 mm
Sample length20 mm
Sample width5 mm
ParametersValues
Maximum force2000 mN
Temperature ramp3 °C
Sample thickness0.25–0.3 mm
Sample length20 mm
Sample width5 mm

Results and Discussion

The specimens underwent analysis using DMA, and the complex modulus was determined through the examination of the storage modulus and loss modulus data gathered during the testing procedure. Four samples were used for each test case, and the average value was documented. The figures illustrate the values of elastic modulus, with the error bars indicating the standard deviation.

The elastic modulus of the specimens exposed air at 85 °C and 125 °C is illustrated in Fig. 3. The error bar shows the standard deviation with a value of ±1 GPa. The modulus of elasticity of the sample subjected to air at 125 °C is greater than that of the sample exposed to 85 °C. Furthermore, the modulus values of both sample variants exhibit a decreasing trend as the temperature is raised from −35 °C to 180 °C in the DMA. The consistency of the modulus variation between the two samples remains constant within the temperature range of −35 °C to 180 °C. The data depicted in Fig. 4 illustrate an identical trend in samples that were submerged in PAO 6 at both 85 °C and 125 °C. However, the samples that were immersed in EC100 at 85 °C and 125 °C chamber temperature as shown in Fig. 5 exhibit the same modulus at all temperature ranges from −35 °C to 180 °C.

Fig. 4
Elastic modulus of sample exposed in PAO 6 at 85 °C and 125 °C
Fig. 4
Elastic modulus of sample exposed in PAO 6 at 85 °C and 125 °C
Close modal
Fig. 5
Elastic modulus of sample exposed in EC100 at 85 °C and 125 °C
Fig. 5
Elastic modulus of sample exposed in EC100 at 85 °C and 125 °C
Close modal

The modulus trends of samples subjected to immersion in EC100, PAO 6, and air at 85 °C are presented in Fig. 6. Analysis reveals that the elastic modulus of substrates immersed in PAO 6 is higher compared to those immersed in EC100. A minor deviation in modulus values is detected between air-exposed samples and those immersed in EC100. As illustrated in Fig. 7, when the substrates were subjected to 125 °C, the modulus of samples immersed in PAO 6 was found to align closely with those exposed to air. Conversely, samples immersed in EC100 exhibited a lower modulus relative to both air-exposed and PAO 6-immersed samples. The observed variations in modulus values across samples subjected to thermal aging and air exposure at 85 °C raise concerns about material performance under varying environmental conditions. These fluctuations echo findings from Bhandari et al. [12], suggesting a consistent pattern across different studies.

Fig. 6
Elastic modulus of sample immersed in EC100, PAO 6, and air at 85 °C
Fig. 6
Elastic modulus of sample immersed in EC100, PAO 6, and air at 85 °C
Close modal
Fig. 7
Elastic modulus of sample immersed in EC100, PAO 6, and air at 125 °C
Fig. 7
Elastic modulus of sample immersed in EC100, PAO 6, and air at 125 °C
Close modal

Figure 8 displays the results window following the completion of DMA tests conducted at various frequencies. As depicted in the figure, DMA yields curves for damping ratio, loss modulus, and storage modulus. The Tg is determined by identifying the peak temperature of the loss modulus or the onset of storage modulus analysis. These features are often recognized for their sharp and smooth characteristics, which simplify the process of determining Tg [13]. However, the presence of various additives and processing techniques can broaden these peaks, complicating Tg determination. The physical significance of the loss modulus peak lies in molecular motion dynamics. As the material approaches this peak, energy dissipation intensifies due to the cooperative movement of large polymer segments. Paradoxically, this cooperative movement also renders the material more deformable. Thus, the observed peak reflects the intricate interplay between increased viscous behavior and enhanced deformability.

Fig. 8
Result created by DMA software
Fig. 8
Result created by DMA software
Close modal

As indicated in Table 1, the substrates underwent an extensive 720-h exposure to an elevated temperature of 85 °C, aimed at scrutinizing the influence of immersion fluid on the Tg. Figure 9 presents the average Tg outcomes for samples immersed in air, EC100, and PAO 6, determined by identifying the peak temperature of the loss modulus. Notably, the average Tg values for air, EC100, and PAO 6 are 203.5 °C, 200.3 °C, and 203.3 °C, respectively. These data show a remarkable consistency in Tg across diverse immersion fluid conditions, suggesting an insubstantial impact on the substrate's Tg. The comprehensive examination underscores the substrate's resilience in maintaining its Tg across diverse environmental conditions, promising versatility for various industrial and technological applications.

Fig. 9
Average Tg of the samples exposed to air, EC100, and PAO6
Fig. 9
Average Tg of the samples exposed to air, EC100, and PAO6
Close modal

Conclusion

The increasing adoption of immersion cooling can be linked to the growing demand for high-performance computing and the rise in power density. Reliability remains a top concern in the electronic packaging industry. This study focused on the reliability of halogen-free substrate cores subjected to thermal aging at 85 °C and 125 °C.

The modulus values of these substrates were then compared with those of substrates that were exposed to air at the same temperatures. Consistent with expectations, the modulus exhibited a negative correlation with rising temperature. A minor deviation in modulus values is detected between air-exposed samples and those immersed in EC100. When the substrates were subjected to 125 °C, the modulus of samples immersed in PAO 6 was found to align closely with those exposed to air. However, for specimens subjected to a temperature of 85 °C in air, PAO 6, and EC100 fluid environments, an irregular pattern was detected. Further analysis is necessary to gain a better comprehension of this phenomenon in the halogen-free substrates aged at 85 °C. The study also explored the impact of immersion fluids on the Tg of the samples. The results showed consistent Tg values across various fluid conditions, indicating minimal influence of immersion fluids on the substrate's Tg. This consistency highlights the resilience of halogen-free substrates across different environmental conditions. Further research is needed to explore the long-term effects of different immersion fluids and environments on these substrates for more comprehensive insights.

Data Availability Statement

The authors attest that all data for this study are included in the paper.

Nomenclature

E* =

elastic modulus

E′ =

storage modulus

E″ =

loss modulus

Tg =

glass transition temperature

tan δ =

damping ratio

References

1.
Garimella
,
S. V.
,
Persoons
,
T.
,
Weibel
,
J.
, and
Yeh
,
L. T.
,
2013
, “
Technological Drivers in Data Centers and Telecom Systems: Multiscale Thermal, Electrical, and Energy Management
,”
Appl. Energy
,
107
, pp.
66
80
.10.1016/j.apenergy.2013.02.047
2.
Masanet
,
E.
,
Shehabi
,
A.
,
Lei
,
N.
,
Smith
,
S.
, and
Koomey
,
J.
,
2020
, “
Recalibrating Global Data Center Energy-Use Estimates
,”
Science
,
367
(
6481
), pp.
984
986
.10.1126/science.aba3758
3.
Luo
,
Q.
,
Wang
,
C.
, and
Wu
,
C.
,
2023
, “
Study on Heat Transfer Performance of Immersion System Based on SiC/White Mineral Oil Composite Nanofluids
,”
Int. J. Therm. Sci.
,
187
, p.
108203
.10.1016/j.ijthermalsci.2023.108203
4.
Liu
,
Y. N.
,
Wei
,
X. X.
,
Xiao
,
J. Y.
,
Liu
,
Z. J.
,
Xu
,
Y.
, and
Tian
,
Y.
,
2020
, “
Energy Consumption and Emission Mitigation Prediction Based on Data Center Traffic and PUE for Global Data Centers
,”
Global Energy Interconnect.
,
3
(
3
), pp.
272
282
.10.1016/j.gloei.2020.07.008
5.
Abreu
,
V.
,
Harrison
,
M.
,
Gess
,
J.
, and
Moita
,
A. S.
,
2018
, “
Two-Phase Thermosiphon Cooling Using Integrated Heat Spreaders With Copper Microstructures
,” Proceedings of the 17th InterSociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (
ITherm 2018
),
San Diego, CA
,
May 29–June 1
, pp.
645
652
.10.1109/ITHERM.2018.8419644
6.
Sohel Murshed
,
S. M.
, and
Nieto de Castro
,
C. A.
,
2017
, “
A Critical Review of Traditional and Emerging Techniques and Fluids for Electronics Cooling
,”
Renewable Sustainable Energy Rev.
,
78
, pp.
821
833
.10.1016/j.rser.2017.04.112
7.
Bansode
,
P. V.
,
Shah
,
J. M.
,
Gupta
,
G.
,
Agonafer
,
D.
,
Patel
,
H.
,
Roe
,
D.
, and
Tufty
,
R.
,
2020
, “
Measurement of the Thermal Performance of a Custom-Build Single-Phase Immersion Cooled Server at Various High and Low Temperatures for Prolonged Time
,”
ASME J. Electron. Packag.
,
142
(
1
), p.
011010
.10.1115/1.4045156
8.
Shah
,
J.
,
Padmanaban
,
K.
,
Singh
,
H.
,
Duraisamy
,
S.
,
Saini
,
S.
, and
Agonafer
,
D.
,
2021
, “
Evaluating the Reliability of Passive Server Components for Single-Phase Immersion Cooling
,”
ASME J. Microelectron. Electron. Packag.
,
18
(
1
), p. 021109.10.1115/1.4052536
9.
Shivaprasad
,
P.
,
Bean
,
J.
,
Shah
,
J.
,
Azevedo
,
E.
,
Brink
,
R.
,
Sengupta
,
S.
,
Wirtz
,
K.
, et al.,
2022
, “
Material Compatibility in Immersion Cooling
,” Open Compute Project.
10.
Chauhan
,
T.
,
Bhandari
,
R.
,
Sivaraju
,
K.
,
Chowdhury
,
R.
, and
Agonafer
,
D.
,
2021
, “
Impact of Immersion Cooling on Thermomechanical Properties of Low-Loss Material Printed Circuit Boards
,”
J. Enhanced Heat Transfer
,
28
(
7
), pp.
73
90
.10.1615/JEnhHeatTransf.2021039486
11.
Ramdas
,
S.
,
Rajmane
,
P.
,
Chauhan
,
T.
,
Misrak
,
A.
, and
Agonafer
,
D.
,
2019
, “
Impact of Immersion Cooling on Thermo-Mechanical Properties of PCB's and Reliability of Electronic Packages
,”
ASME
Paper No. IPACK2019-6568.10.1115/IPACK2019-6568
12.
Bhandari
,
R.
,
Lakshminarayana
,
A. B.
,
Sivaraju
,
K. B.
,
Bansode
,
P.
,
Kejela
,
E.
, and
Agonafer
,
D.
,
2022
, “
Impact of Immersion Cooling on Thermomechanical Properties of Non-Halogenated Substrate
,”
ASME
Paper No. IPACK2022-97423.10.1115/IPACK2022-97423
13.
Hitachi
,
2013
,
TA7000 Series, Dynamic Mechanical Analyzer, Operation Manual
,
Hitachi Hi-Tech Science Corp., Tokyo, Japan
.