Abstract

Increasing power densities in data centers due to the rise of artificial intelligence, high-performance computing, and machine learning compel engineers to develop new cooling strategies and designs for high-performance information technology (IT) equipment. Two-phase cooling is a promising technology that exploits the latent heat of the coolant which is significantly more effective in removing high heat fluxes than when using the sensible heat of the fluid. Also, utilizing the latent heat allows operating at lower coolant flow rates and implies more uniformity in the temperature of heated surfaces. Despite the benefits of two-phase cooling, the phase change adds complexities to a system when multiple evaporators (exposed to different heat fluxes potentially) are connected to a single coolant distribution unit. In this article, a commercial coolant distribution unit is used to investigate pumped two-phase cooling in rack scale. Seventeen two-rack unit servers from two distinct models are retrofitted with 34 impinging jet evaporators and deployed in a rack. Four case studies are presented to provide insights into the complex behavior of a pumped two-phase cooling system with several evaporators. The flow rates and pressure distribution across the rack are studied in various filling ratios. Also, investigated is the transient behavior of the cooling system due to a step change in the IT workload. Finally, a control system is designed to regulate the temperature of the supplied coolant in response to the step change in the IT workload and is tested.

References

References
1.
Shehabi
,
A.
,
Smith
,
S.
,
Sartor
,
D.
,
Brown
,
R.
,
Herrlin
,
M.
,
Koomey
,
J.
,
Masanet
,
E.
, and
Lintner
,
W.
,
2016
, “
United States Data Center Energy Usage Report
,” Lawrence Berkeley National Lab. (LBNL), Berkeley, CA, Report No.
LBNL-1005775
.https://www.osti.gov/servlets/purl/1372902/
2.
Shehabi
,
A.
,
Smith
,
S. J.
,
Masanet
,
E.
, and
Koomey
,
J.
,
2018
, “
Data Center Growth in the United States: Decoupling the Demand for Services from Electricity Use
,”
Environ. Res. Lett.
,
13
(
12
), p.
124030
.10.1088/1748-9326/aaec9c
3.
Kadam
,
S. T.
, and
Kumar
,
R.
,
2014
, “
Twenty First Century Cooling Solution: Microchannel Heat Sinks
,”
Int. J. Therm. Sci.
,
85
, pp.
73
92
.10.1016/j.ijthermalsci.2014.06.013
4.
Schmidt
,
R.
,
2009
, “
Packaging of New Servers—Energy Efficiency Aspects
,” First Berkeley Symposium on Energy Efficient Electronics, Berkeley, CA, June 11–12.https://e3s-center.berkeley.edu/wp-content/uploads/2017/07/RogerSchmidt.pdf
5.
Kulkarni
,
D.
,
Tang
,
X.
,
Ahuja
,
S.
,
Dischler
,
R.
, and
Mahajan
,
R.
,
2018
, “
Experimental Study of Two-Phase Cooling to Enable Large-Scale System Computing Performance
,”
Proceedings of the 17th InterSociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems, ITherm 2018
, IEEE, San Diego, CA, May 29–June 1, pp.
596
601
.10.1109/ITHERM.2018.8419545
6.
Hetsroni
,
G.
,
Mosyak
,
A.
,
Segal
,
Z.
, and
Ziskind
,
G.
,
2002
, “
A Uniform Temperature Heat Sink for Cooling of Electronic Devices
,”
Int. J. Heat Mass Transfer
,
45
(
16
), pp.
3275
3286
.10.1016/S0017-9310(02)00048-0
7.
Mudawar
,
I.
, and
Wadsworth
,
D. C.
,
1991
, “
Critical Heat Flux From a Simulated Chip to a Confined Rectangular Impinging Jet of Dielectric Liquid
,”
Int. J. Heat Mass Transfer
,
34
(
6
), pp. 1
465
1479
.10.1016/0017-9310(91)90289-Q
8.
Capozzoli
,
A.
, and
Primiceri
,
G.
,
2015
, “
Cooling Systems in Data Centers: State of Art and Emerging Technologies
,”
Energy Procedia
,
83
, pp.
484
493
.10.1016/j.egypro.2015.12.168
9.
Saums
,
D. L.
,
2010
, “
Vaporizable Dielectric Fluid Cooling for IGBT Power Semiconductors
,”
Sixth International Conference on Integrated Power Electronics Systems
, Nuremberg, Germany, Mar. 16–18, pp.
1
7
.https://ieeexplore.ieee.org/document/5730673
10.
Chainer
,
T. J.
,
Schultz
,
M. D.
,
Parida
,
P. R.
, and
Gaynes
,
M. A.
,
2017
, “
Improving Data Center Energy Efficiency With Advanced Thermal Management
,”
IEEE Trans. Compon., Packag. Manuf. Technol.
,
7
(
8
), pp.
1228
1239
.10.1109/TCPMT.2017.2661700
11.
Madhour
,
Y.
,
Olivier
,
J.
,
Costa-Patry
,
E.
,
Paredes
,
S.
,
Michel
,
B.
, and
Thome
,
J. R.
,
2011
, “
Flow Boiling of R134a in a Multi-Microchannel Heat Sink With Hotspot Heaters for Energy-Efficient Microelectronic CPU Cooling Applications
,”
IEEE Trans. Compon., Packag. Manuf. Technol.
,
1
(
6
), pp.
873
883
.10.1109/TCPMT.2011.2123895
12.
Kheirabadi
,
A. C.
, and
Groulx
,
D.
,
2016
, “
Cooling of Server Electronics: A Design Review of Existing Technology
,”
Appl. Therm. Eng.
,
105
, pp.
622
638
.10.1016/j.applthermaleng.2016.03.056
13.
Shedd
,
T. A.
, and
Pautsch
,
A. G.
,
2005
, “
Spray Impingement Cooling With Single- and Multiple-Nozzle Arrays: Part II—Visualization and Empirical Models
,”
Int. J. Heat Mass Transfer
,
48
(
15
), pp.
3176
3184
.10.1016/j.ijheatmasstransfer.2005.02.013
14.
Lee
,
J.
, and
Mudawar
,
I.
,
2005
, “
Two-Phase Flow in High-Heat-Flux Micro-Channel Heat Sink for Refrigeration Cooling Applications: Part II—Heat Transfer Characteristics
,”
Int. J. Heat Mass Transfer
,
48
(
5
), pp.
941
955
.10.1016/j.ijheatmasstransfer.2004.09.019
15.
Bowers
,
M. B.
, and
Mudawar
,
I.
,
1994
, “
Two-Phase Electronic Cooling Using Mini-Channel and Micro-Channel Heat Sinks: Part 1—Design Criteria and Heat Diffusion Constraints
,”
ASME J. Electron. Packag.
,
116
(
4
), pp.
290
297
.10.1115/1.2905700
16.
Valenzuela
,
F.
,
Ortega
,
A.
,
Jones
,
G.
,
Fleischer
,
A.
,
Schon
,
S.
, and
Tipton
,
R.
,
2017
, “
Experiments on the Simultaneous Two-Phase Liquid Cooling of Multiple Simulated Servers at Differing Vertical Rack Positions in Steady State
,”
16th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems
, ITherm 2017, Orlando, FL, May 30–June 2, pp.
871
877.
10.1109/ITHERM.2017.7992577
17.
3M, 2019,
Product Information: 3MTM NovecTM 7000 Engineered Fluid
,” 3M, Saint Paul, MN, accessed Apr. 3, 2019, https://multimedia.3m.com/mws/media/121372O/3m-novec-7000-engineered-fluid-tds.pdf
18.
Stachecki
,
T. J.
, and
Ghose
,
K.
,
2015
, “
Short-Term Load Prediction and Energy-Aware Load Balancing for Data Centers Serving Online Requests
,” 42nd International Symposium on Computer Architecture (
ISCA
), Portland, OR, June 13–17, pp.
13
17
.https://pdfs.semanticscholar.org/6fa8/1c40ce225bc15a88804d02549711fd178d64.pdf
19.
Kayahan
,
E.
,
Eroglu
,
I.
, and
Koku
,
H.
,
2016
, “
Design of an Outdoor Stacked—Tubular Reactor for Biological Hydrogen Production
,”
Int. J. Hydrogen Energy
,
41
(
42
), pp.
19357
19366
.10.1016/j.ijhydene.2016.04.086
20.
Wang
,
J.
,
2011
, “
Theory of Flow Distribution in Manifolds
,”
Chem. Eng. J.
,
168
(
3
), pp.
1331
1345
.10.1016/j.cej.2011.02.050
21.
Gmach
,
D.
,
Rolia
,
J.
,
Cherkasova
,
L.
, and
Kemper
,
A.
,
2009
, “
Resource Pool Management: Reactive Versus Proactive or Let's Be Friends
,”
Comput. Networks
,
53
(
17
), pp.
2905
2922
.10.1016/j.comnet.2009.08.011
22.
Arlitt
,
M.
, and
Jin
,
T.
,
2000
, “
A Workload Characterization Study of the 1998 World Cup Web Site
,”
IEEE Networks
,
14
(
3
), pp.
30
37
.10.1109/65.844498
23.
Gandhi
,
A.
,
2013
, “
Dynamic Server Provisioning for Data Center Power Management
,”
Ph.D. thesis
, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA.http://reports-archive.adm.cs.cmu.edu/anon/2013/CMU-CS-13-110.pdf
24.
Atikoglu
,
B.
,
Xu
,
Y.
,
Frachtenberg
,
E.
,
Jiang
,
S.
, and
Paleczny
,
M.
,
2012
, “
Workload Analysis of a Large-Scale Key-Value Store
,”
ACM SIGMETRICS Performance Evaluation Review
,
ACM
, London, UK, June 11–15, pp.
53
64
.https://research.fb.com/publications/workload-analysis-of-a-large-scale-key-value-store/#:~:text=Key%2Dvalue%20stores%20are%20a,online%20retail%2C%20and%20risk%20analysis.&text=We%20analyze%20the%20workloads%20from,patterns%3B%20and%20application%20use%20cases
25.
Khalili
,
S.
,
Mohsenian
,
G.
,
Desu
,
A.
,
Ghose
,
K.
, and
Sammakia
,
B.
,
2019
, “
Airflow Management Using Active Air Dampers in Presence of a Dynamic Workload in Data Centers
,” 35th Thermal Measurement, Modeling & Management Symposium (
SEMI-THERM
), San Jose, CA, Mar. 18–22, pp.
101
110
.https://ieeexplore.ieee.org/abstract/document/9165298
You do not currently have access to this content.