This paper presents the results of an experimentally validated computational fluid dynamics (CFD) model for a data center with fully implemented fan curves on both the servers and the computer room air conditioner (CRAC). Both open and contained cold aisle systems are considered in this study. This work is divided into sections for the baseline system (prior to installing containment) calibration and the fully contained cold aisle system calibration and leakage characterization. In the open system, the fan curve of the CRAC unit is extracted from the manufacturer data, while the fan curve of the load banks is obtained through experimental measurements. The experimental results are found to be in good agreement with the average model predictions. In the fully contained cold aisle system, a detailed containment CFD model is developed based on experimental measurements. The model is validated by comparing the flow rate through the perforated floor tiles and the rack inlet temperatures with the experimental measurements. The CFD results are found to be in good agreement with the experimental data with an average relative error between the measured and computed flow rate of approximately 6.7%. Temperature measurements are used to calibrate the sources of leakage in the containment and rack mounting rails. The temperature measurements and the CFD results agree well with an average difference of less than 1 °C. This study provides important modeling guidelines for data centers. In order to predict the performance of contained cold aisle systems flow distribution, it is crucial that physics based models of fan curves, server internal resistances, detailed rack models, and other design details are all accurate and experimentally verified.

References

1.
U.S. Environmental Protection Agency (EPA),
2007
, “
Report to Congress on Server and Data Center Energy Efficiency: Public Law 109-43
,”
U.S. Environmental Protection Agency
, Washington, D.C.
2.
Choi
,
J.
,
Kim
,
Y.
,
Sivasubramaniam
,
A.
,
Srebric
,
J.
,
Wang
,
Q.
, and
Lee
,
J.
,
2008
, “
A CFD-Based Tool for Studying Temperature in Rack-Mounted Servers Computers
,”
IEEE Trans.
57
(
8
), pp.
1129
1142
.10.1109/TC.2008.52
3.
Iyengar
,
M.
,
Schmidt
,
R.
,
Hamann
,
H.
, and
VanGilder
,
J.
,
2007
, “
Comparison Between Numerical and Experimental Temperature Distributions in a Small Data Center Test Cell
,”
ASME
Paper No. IPACK2007-33508.10.1115/IPACK2007-33508
4.
Shrivastava
,
S.
,
Iyengar
,
M.
,
Sammakia
,
B.
,
Schmidt
,
R.
, and
VanGilder
,
J.
,
2006
, “
Experimental-Numerical Comparison for a High-Density Data Center: Hot Spot Fluxes in Excess of 500 W/ft2
,”
Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronics Systems (ITHERM)
, San Diego, CA, May 30–June 2, pp.
402
411
.
5.
Kang
,
S.
,
Schmidt
,
R.
,
Kelkar
,
K. M.
,
Radmehr
,
A.
, and
Patankar
,
S. V.
,
2001
, “
A Methodology for the Design of Perforated Tiles in Raised Floor Data Centers Using Computational Flow Analysis
,”
IEEE Trans. Compon., Packag. Manuf. Technol.
,
24
(
2
), pp.
177
183
.10.1109/6144.926380
6.
Shrivastava
,
S.
,
Sammakia
,
B.
,
Schmidt
,
R.
, and
Iyengar
,
M.
,
2005
, “
Comparative Analysis of Different Data Center Airflow Management Configurations
,”
ASME
Paper No. IPACK2005-73234.10.1115/IPACK2005-73234
7.
Abdelmaksoud
,
W.
,
2012
, “
Experimental and Numerical Investigations of the Thermal Environment in Air-Cooled Data Centers
,” Ph.D. thesis, Syracuse University, New York.
8.
Abdelmaksoud
,
W. A.
,
Dang
,
T. Q.
,
Khalifa
,
H. E.
, and
Schmidt
,
R. R.
,
2013
, “
Improved Computational Fluid Dynamics Model for Open-Aisle Air-Cooled Data Center Simulations
,”
ASME J. Electron. Packag.
,
135
(
3
), p.
030901
.10.1115/1.4024766
9.
Bhopte
,
S.
,
Sammakia
,
B.
,
Schmidt
,
R.
,
Iyengar
,
M.
, and
Agonafer
,
D.
,
2006
, “
Effect of Under Floor Blockages on Data Center Performance
,”
Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronics Systems (ITHERM)
, San Diego, CA, May 30–June 2, pp.
426
433
.
10.
Patankar
,
S. V.
,
2010
, “
Airflow and Cooling in a Data Center
,”
ASME J. Heat Transfer
,
132
(
7
), p.
073001
.10.1115/1.4000703
11.
Samadiani
,
E.
,
Rambo
,
J.
, and
Joshi
,
Y.
,
2012
, “
Numerical Modeling of Perforated Tile Flow Distribution in a Raised-Floor Data Center
,”
ASME J. Electron. Packag.
,
132
(
2
), p.
021002
.10.1115/1.4001589
12.
Demetriou
,
D. W.
, and
Khalifa
,
H. E.
,
2012
, “
Optimization of Enclosed Aisle Data Centers Using Bypass Recirculation
,”
ASME J. Electron. Packag.
,
134
(
2
), p.
020904
.10.1115/1.4005907
13.
Zhou
,
R.
,
Wang
,
Z.
,
Bash
,
C. E.
, and
McReynolds
,
A.
,
2011
, “
Modeling and Control for Cooling Management of Data Centers With Hot Aisle Containment
,”
ASME
Paper No. IMECE2011-62506.10.1115/IMECE2011-62506
14.
Shrivastava
,
S. K.
,
Calder
,
A. R.
, and
Ibrahim
,
M.
,
2012
, “
Quantitative Comparison of Air Containment Systems
,”
Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronics Systems (ITHERM)
, San Diego, CA, May 30–June 1, pp.
68
77
.
15.
Muralidharan
,
B.
,
Ibrahim
,
M.
,
Shrivistava
,
S.
,
Alkharabsheh
,
S.
, and
Sammakia
,
B.
,
2013
, “Impact of Cold Aisle Containment on Thermal Performance of Data Center,”
ASME
Paper No. IPACK2013-73201.10.1115/IPACK2013-73201
16.
Arghode
,
V. K.
,
Sundaralingam
,
V.
,
Joshi
,
Y.
, and
Phelps
,
W.
,
2013
, “
Thermal Characteristics of Open and Contained Data Center Cold Aisle
,”
ASME J. Heat Transfer
,
135
(
6
), p.
061901
.10.1115/1.4023597
17.
Arghode
,
V. K.
, and
Joshi
,
Y.
,
2014
, “
Room Level Modeling of Air Flow in a Contained Data Center Aisle
,”
ASME J. Electron. Packag.
,
136
(
1
), p.
011011
.10.1115/1.4026540
18.
Arghode
,
V. K.
,
Sundaralingam
,
V.
,
Joshi
,
Y.
, and
Phelps
,
W.
,
2015
, “
Experimental Characterization of Various Cold Aisle Containment Configurations for Data Centers
,”
ASME J. Electron. Packag.
,
137
(
1
), p. 011007.10.1115/1.4028520
19.
Alkharabsheh
,
S. A.
,
Sammakia
,
B. G.
,
Shrivastava
,
S. K.
, and
Schmidt
,
R.
,
2013
, “
Utilizing Practical Fan Curves in CFD Modeling of a Data Center
,” Semiconductor Thermal Measurement and Management Symposium (
SEMI-THERM
), San Jose, CA, Mar. 17–21, pp.
211
215
.10.1109/SEMI-THERM.2013.6526831
20.
Alkharabsheh
,
S. A.
,
Sammakia
,
B. G.
,
Shrivastava
,
S. K.
,
Ellsworth
,
M.
,
David
,
M.
, and
Schmidt
,
R.
, 2013 “
A Numerical Steady State and Dynamic Study in a Data Center Using Calibrated Fan Curves for CRACs and Servers
,”
ASME
Paper No. IPACK2013-73217.10.1115/IPACK2013-73217
21.
Alkharabsheh
,
S. A.
,
Sammakia
,
B. G.
,
Shrivastava
,
S. K.
, and
Schmidt
,
R.
,
2014
, “
Dynamic Models of Servers Racks and CRAHs in a Room Level CFD Model of a Data Center
,”
Inter Society Conference on Thermal Phenomena (ITHERM)
, Orlando, FL, May 27–30.
22.
Mentor
,
2012
,
FloTHERM V9.3 User Reference Guide
, Mentor Graphics Corp., Wilsonville, OR.
23.
Alkharabsheh
,
S. A.
,
Muralidharan
,
B.
,
Ibrahim
,
M.
,
Shrivastava
,
S.
, and
Sammakia
,
B.
,
2013
, “
Open and Contained Cold Aisle Experimentally Validated CFD Model Implementing CRAC and Server Fan Curves for a Data Center Testing Laboratory
,”
ASME
Paper No. IPACK2013-73214.10.1115/IPACK2013-73214
24.
Radmehr
,
A.
,
Schmidt
,
R.
,
Karki
,
K.
, and
Patankar
,
S.
,
2005
, “
Distributed Leakage Flow in Raised-Floor Data Centers
,”
ASME
Paper No. IPACK2005-73273.10.1115/IPACK2005-73273
25.
Zhang
,
X.
,
VanGilder
,
J.
,
Iyengar
,
M.
, and
Schmidt
,
R.
,
2008
, “
Effect of Rack Modeling Detail on the Numerical Results of a Data Center Test Cell
,”
Inter Society Conference on Thermal Phenomena (ITHERM)
, Orlando, FL, May 28–31, pp.
1183
1190
.
26.
Patel
,
C.
,
Bash
,
C.
, and
Belady
,
C.
,
2001
, “
Computational Fluid Dynamics Modeling of High Compute Density Data Centers to Assure System Inlet Air Specifications
,”
ASME of the Pacific Rim Technical Conference and Exposition of Packaging and Integration of Electronic and Photonic Systems (InterPACK)
, Kauai, Hawaii, July 8–13,
ASME
Paper No. IPACK2001-15622.
27.
Patankar
,
S. V.
,
1980
,
Numerical Heat Transfer and Fluid Flow
,
Taylor and Francis Publishing Corporation
,
New York
.
28.
Mentor, 2012, FloTHERM V9.3, Mentor Graphics Corp., Wilsonville, OR.
29.
Herrlin
,
M. K.
,
2005
, “
Rack Cooling Effectiveness in Data Centers and Telecom Central Offices: The Rack Cooling Index (RCI)
,”
ASHRAE Trans.
,
111
(
2
), pp.
725
731
.
30.
Herrlin
,
M. K.
, and
Belady
,
C.
,
2006
, “
Gravity-Aassisted Air Mixing in Data Centers and How It Affects the Rack Cooling Effectiveness
,”
Inter Society Conference on Thermal Phenomena (ITHERM)
, San Diego, CA, May 30–June 2, p.
438
.
31.
Rambo
,
J.
, and
Joshi
,
Y.
,
2007
, “
Modeling of Data Center Airflow and Heat Transfer: State of the Art and Future Trends
,”
Distributed Parallel Databases
,
21
(
2–3
), pp.
193
225
.10.1007/s10619-006-7007-3
32.
Cruz
,
E.
,
Joshi
,
Y.
,
Iyengar
,
M.
, and
Schmidt
,
R.
,
2009
, “
Comparison of Numerical Modeling to Experimental Data in a Small Data Center Test Cell
,”
ASME
Paper No. InterPACK2009-89306.10.1115/InterPACK2009-89306
33.
Demetriou
,
D.
, and
Khalifa
,
H. E.
,
2011
, “
Evaluation of a Data Center Recirculation Non-Uniformity Metric Using Computational Fluid Dynamics
,”
ASME
Paper No. IPACK2011-52005.10.1115/IPACK2011-52005
34.
Alkharsbsheh
,
S. A.
,
Sammakia
,
B. G.
, and
Murray
,
B. T.
,
2014
, “
Experimental Characterization of Pressure Drop in a Server Rack
,”
Inter Society Conference on Thermal Phenomena (ITHERM)
, Orlando, FL, May 31–June 4, pp.
547
556
.
35.
Fried
,
E.
, and
Idelchik
,
I. E.
,
1989
,
Flow Resistance: A Design Guide for Engineers
,
Hemisphere
,
New York
.
You do not currently have access to this content.