The percentage of the energy used by data centers for cooling their equipment has been on the rise. With that, there has been a necessity for exploring new and more efficient methods like airside economization, both from an engineering as well as business point of view, to contain this energy demand. Air cooling especially, free air cooling has always been the first choice for IT companies to cool their equipment. But, it has its downside as well. As per ASHRAE standard (2009b), the air which is entering the data center should be continuously filtered with MERV 11 or preferably MERV 13 filters and the air which is inside the data center should be clean as per ISO class 8. The objective of this study is to design a model data center and simulate the flow path with the help of 6sigma room analysis software. A high-density data center was modelled for both hot aisle and cold aisle containment configurations. The particles taken into consideration for modelling were spherical in shape and of diameters 0.05, 0.1 and 1 micron. The physical properties of the submicron particles have been assumed to be same as that of air. For heavier particles of 1 micron in size, the properties of dense carbon particle are chosen for simulating particulate contamination in a data center. The Computer Room Air Conditioning unit is modelled as the source for the particulate contaminants which represents contaminants entering along with free air through an air-side economizer. The data obtained from this analysis can be helpful in predicting which type of particles will be deposited at what location based on its distance from the source and weight of the particles. This can further help in reinforcing the regions with a potential to fail under particulate contamination.

The study of solder joint reliability is one of the priority issues in electronic packaging. Solder alloys experience a highly nonlinear material behavior when subject to thermal cycling. It is a time consuming and difficult task to study the behavior of solder joints using experimental approaches. Finite element analysis provides a more efficient way to better understand the behavior of solder joints when accurate material models are available. With the developments of FEA algorithms and computer resources, the analysis approaches used for electronic packaging assemblies have evolved from 2-dimensional to 3-dimensional analyses, with far fewer assumptions needed in the fully 3D case. In this paper, we compare different FEA approaches covering various 2D and 3D modeling techniques to understand their advantages and drawbacks, especially as related to simulation accuracy and efficiency. Several models for a typical BGA assembly were prepared and analyzed including traditional mesh continuity models (2D slice model, 3D slice model, and 3D quarter model), as well as advanced models that employ Multi-Point Constraints (MPCs) and submodeling (global/local models). The Anand viscoplastic model was used for the solder joint material behavior in all of the FEA approaches. For the 3D mesh continuity models, an optimal analysis approach has been proposed to achieve the best balance between the accuracy of the simulation result and numerical efficiency of the simulation. Mesh transitions were used to maintain mesh continuity between regions of different mesh densities. A best choice of load step size was also found to reduce overall simulation time. For the analysis using MPCs to to bond different meshes, two improved modeling strategies have been proposed including a suggested ratio of contacting elements and the use of multiple-MPC contact pairs to reduce overall mesh density of the FE model. An improved simulation simulation strategy using submodeling has also been developed to obtain the best compromise in the global and local models between the mesh quality and load step size. An improved geometric simplification of the solder joint for use with energy based fatigue criteria was developed. Finally, comparisons and suggestions were made for the best analysis approach when using FEA techniques to predict the behavior of solder joints in PBGA packages.

The study investigated how the material roughness of a tablet computer surface can affect thermal sensation and comfort of users fingers and palms at different surface temperatures. Three levels of pattern spacing were tested, and it was shown that rough material surface provided higher thermal comfort comparing to a smooth surface. In addition, the surface temperature of the material also moderates participants′ physical sensation of the roughness of the materials. The results of the study have shown evidences of the potentials to use materials with spatial patterns to improve thermal comfort while dissipating heat from electronic devices.

This paper presents an experimentally validated room-level computational fluid dynamics (CFD) model for raised-floor data center configurations employing active tiles. Active tiles are perforated floor tiles with integrated fans, which increase the local volume flowrate by redistributing the cold air supplied by the computer room air conditioning (CRAC) unit to the under-floor plenum. In a previous study [1], experiments were conducted to explore the potential of active tiles for economically and efficiently eliminating hot spots in data center. Our results indicated that active tiles, as the actuators closest to the racks, can significantly and quickly impact the local distribution of cooling resources. They could therefore be used in an appropriate control framework to rapidly mitigate hot spots, and maintain local conditions in an energy-efficient manner. The numerical model of the data center room operates in an under-floor supply and ceiling return cooling configuration and consists of one cold aisle with 12 racks arranged on both sides and three CRAC units sited around the periphery of the room. The commercial computational fluid dynamics (CFD) software package Future Facilities 6SigmaDCX [2], which is specifically designed for data center simulation, is used to develop the model. First, a baseline model using only passive tiles was developed and experimental data were used to verify and calibrate plenum leakage for the room. Then a CFD model incorporating active tiles was developed for two configurations: (a) a single active tile and 9 passive tiles in the cold aisle; and (b) an aisle populated with 10 ( i.e. , all) active tiles. The active tiles are modeled as a combination of a grill, fan elements and flow blockages to closely mimic the actual active tile used in the experimental studies. The fan curve for the active tile fans is included in the model to account for changes in flow rate through the tiles in response to changes in plenum pressure. The model with active tiles is validated by comparing the flow rate through the floor tiles, relative plenum pressure and rack inlet temperatures for selected racks with the experimental measurements. The predictions from the CFD model are found to be in good agreement with the experimental data, with an average discrepancy between the measured and computed values for total flow rate and rack inlet temperature less than 4% and 1.7 °C, respectively. These validated models were then used to simulate steady state and transient scenarios following cooling failure. This physics-based and experimentally validated room-level model can be used to predict temperature and flow distributions in a data center using active tiles. These predictions can then be used to identify the optimal number and locations of active tiles to mitigate hot spots, without adversely affecting other parts of the data center.

High performance computing server racks are being engineered to contain significantly more processing capability within the same computer room footprint year after year. The processor density within a single rack is becoming high enough that traditional, inefficient air-cooling of servers is inadequate to sustain HPC workloads. Experiments that characterize the performance of a direct water-cooled server rack in an operating HPC facility are described in this paper. Performance of the rack is reported for a range of cooling water inlet temperatures, flow rates and workloads that include actual and worst-case synthetic benchmarks. Power and temperature measurements of all processors and memory components in the rack were made while extended benchmark tests were conducted throughout the range of cooling variables allowed within an operational HPC facility. Synthetic benchmark results were compared with those obtained on a single server of the same design that had been characterized thermodynamically. Neither actual nor synthetic benchmark performances were affected during the course of the experiments, varying less than 0.13 percent. Power consumption change in the rack was minimal for the entire excursion of coolant temperatures and flow rates. Establishing the characteristics of such a highly energy efficient server rack in situ is critical to determine how the technology might be integrated into an existing heterogeneous, hybrid cooled computing facility — i.e., a facility that includes some servers that are air cooled as well as some that are direct water cooled.

One way to model the thermodynamic efficiency of a data center is to perform a second-law analysis using exergy calculations of the individual components. The in-house data center modeling tool Villanova Thermodynamic Analysis of Systems (VTAS) was applied to determine the effect of direct liquid cooling with cold plates on data center efficiency. The effectiveness of the cold plate as a function of the heat output of the servers was also examined. VTAS was used to study a configuration of two rows with eight racks each. Each rack was assumed to have twelve servers, each producing 200 W of heat. In addition to the cold plates and servers, this configuration also included a computer room air handling (CRAH) unit, a chiller, and a cooling tower. Preliminary results show that data center second-law efficiency increases as the cold plate removes more heat than the CRAH unit. Specifically, when using cold plates to remove all of the heat from the servers, the overall data center exergy destruction was reduced by over 30% compared to a configuration with only a CRAH unit and between 12–18% compared to a configuration with hybrid liquid-air technologies, thus showing that configurations with direct liquid cooling are thermodynamically more favorable. Furthermore, the effectiveness of the cold plate increases as the heat output of each server increases, suggesting that cold plates are most effective when removing a greater amount of heat.

In recent years, the internet services industry has been developing rapidly. Accordingly, the demands for compute and storage capacity continue to increase and internet data centers are consuming more power than ever before to provide this capacity. Based on the Forest & Sullivan market survey, data centers across the globe now consume around 100GWh of power and this consumption is expected to increase 30% by 2016. With development expanding, IDC (Internet Data Center) owners realize that small improvements in efficiency, from architecture design to daily operations, will yield large cost reduction benefits over time. Cooling energy is a significant part of the daily operational expense of an IDC. One trend in this industry is to raise the operational temperature of an IDC, which also means running IT equipment at HTA (Higher Ambient Temperature) environment. This might also include cooling improvements such as water-side or air-side economizers which can be used in place of traditional closed loop CRAC (Computer room air conditioner) systems. But just raising the ambient inlet air temperature cannot be done by itself without looking at more effective ways of managing cooling control and considering the thermal safety. An important trend seen in industry today is customized design for IT (Information Technology) equipment and IDC infrastructure from the cloud service provider. This trend brings an opportunity to consider IT and IDC together when designing and IDC, from the early design phase to the daily operation phase, when facing the challenge of improving efficiency. This trend also provides a chance to get more potential benefit out of higher operational temperature. The advantages and key components that make up a customized rack server design include reduced power consumption, more thermal margin with less fan power, and accurate thermal monitoring, etc. Accordingly, the specific IDC infrastructure can be re-designed to meet high temperature operations. To raise the supply air temperature always means less thermal headroom for IT equipment. IDC operators will have less responses time with large power variations or any IDC failures happen. This paper introduces a new solution called ODC (on-demand cooling) with PTAS (Power Thermal Aware Solution) technology to deal with these challenges. ODC solution use the real time thermal data of IT equipment as the key input data for the cooling controls versus traditional ceiling installed sensors. It helps to improve the cooling control accuracy, decrease the response time and reduce temperature variation. By establishing a smart thermal operation with characteristics like direct feedback, accurate control and quick response, HTA can safely be achieved with confidence. The results of real demo testing show that, with real time thermal information, temperature oscillation and response time can be reduced effectively.

Data center energy consumption can be divided into three broad categories: Information Technology (IT), Electrical, and Mechanical. An efficient data center uses the least amount of non-IT energy, which is typically divided between the mechanical and electrical systems. Mechanical systems generally contribute a large portion of the non-IT energy use by providing cooling from compressor-based equipment [1,2] and because of this, strategies to reduce compressor energy consumption can lead to significant mechanical system energy savings. The most efficient way to reduce compressor energy is through elimination or significant reduction in annual runtime. This is possible with the use of integrated airside or waterside economizers. This paper demonstrates the impacts of economization in data centers through data collected from four operating facilities over the course of implementing various economizer improvement projects. System architectures include water-cooled centrifugal chiller plant with waterside economization, direct expansion air handling units (AHU) with airside economization, air-cooled centrifugal chillers with integrated waterside economization, and direct expansion computer room air conditioners (CRAC) with evaporative cooling and waterside economization. A systematic and methodical comparison of the baseline and post-conditions is discussed, comparing expected to observed economizer operating conditions. The comparison of multiple real-world scenarios revealed a range of variances in expected operation of economizer sequences to actual observations, indicating a need for close monitoring of system performance by data center operators to fully realize economizer benefits within facilities.

Energy efficiency is an essential element of server design for high performance computers. Traditional HPC servers or nodes that are air cooled enable efficiency by using optimized system design elements that include efficient heat sink design for critical components such as CPUs, Memory, Networking and Disk Subsystems. In addition, airflow optimization is enabled via critical component placement decisions as well as fan and cooling algorithms that have an objective to optimize airflow and maximize system performance. Critical elements that cannot be avoided in traditional air cooled servers are computer center level management of both the airflow requirements and the exhaust heat flux of the servers. An alternative approach shown in this paper uses a novel water cooled design that enables both extreme energy efficiency for heat extraction of the server heat load and allows for lower device operating temperatures for the critical components. Experimental data documented in this paper illustrates the advantages of using non-chilled water to cool the server, allowing 85 to 90 percent of the server heat load to be extracted by water while allowing inlet water temperatures up to 45 degrees Celsius. A comparison is made of the energy consumption needed to cool the server components for both the air cooled and water cooled systems. The base system used for the comparison uses identical system electronics and firmware. The server thermal data shown in the paper include thermal behavior at idle, typical and maximum power consumption states for the server. The data documents the range of boundary conditions that can be tolerated for water cooled server solutions and the comparative advantages of using this technology.

Lumped capacitance models have been introduced to study transient thermal response of data centers. Chilled water interruption of a Computer Room Air Handling (CRAH) unit is one of several failure scenarios of data center cooling infrastructures. In such a scenario, predicting the transient thermal response of the CRAH unit depends requires the determination of the CRAH lumped capacitance model parameters: the thermal capacitance (thermal mass) and the time constant. In this paper, we propose an experimental methodology to extract sufficient information for the lumped capacitance modeling of CRAH units. The method requires measurements of inlet and exit air temperature, air flow rate and CRAH fan power. If the chilled water supply to a CRAH unit is intentionally interrupted in a data center with multiple redundant CRAH units, sufficient information to estimate the CRAH lumped capacitance parameters can be obtained without disturbing the data center operation.

It has been reported that tablet computer surface temperatures can rise from room temperature up to 47°C. Holding a warm or hot computer surface might cause user’s thermal discomfort and possibly skin burns. The use of a tablet often requires holding the device for prolonged time with multiple fingers and palm areas in contact with the tablet lower surface. Previous research has not tested whole finger/palm thermal sensation at a specific surface temperature in a moderate environmental heat range. The current research investigates user’s thermal sensations on the palm and fingers, in response to warm/heat stimuli in a tablet size device with a longer contact duration than used in previous studies, to provide ergonomic design guidelines for electronic device designers and manufacturers. A tablet-size heating surface was developed comprising of nine rectangular aluminum heating pads connected with computer-controlled heaters and thermal sensors. Participants were asked to report their finger/palm thermal sensation and comfort every 45 seconds when they held the prototype for 90 seconds. Results showed a positive linear relationship between surface temperature and user’s thermal sensation and thermal discomfort. Duration of holding the prototype had no significant effect on user’s thermal comfort, but it did significantly affect thermal sensation ratings.

Computer processor speeds have increased in recent years to the extent that water cooling is becoming an attractive and sometimes a necessary way of cooling the processors and their associated electronics. Water can conduct heat much faster than air, allowing processors to run at higher speeds at lower acoustic levels. The ability of water to cool computer electronics can be diminished, or even lost, if there is excessive loss of water by permeation through, or leaks past, the various materials and/or connections between materials. Knowing the rate of loss of coolant from cooling systems can help designers determine the maintenance procedures and schedules for their cooling systems. The paper describes a novel, accurate and convenient method for measuring the moisture leakage rates out of water-carrying hardware. The water-filled hardware under test is placed in a chamber that is purged dry with flowing nitrogen gas and the chamber is then sealed. The rate of rise of relative humidity in the chamber is used to determine the rate of moisture leakage out of the water-filled hardware. The errors arising from the hose terminations and the adsorption of moisture by the metal chamber walls and the plastic fittings can be accounted for and corrected. The test duration is typically less than 10 days. The paper presents examples of water leakage out of hoses and tubes and their terminations and out of quick connects.

For the next generation of high performance computers, the new challenges are to shorten the distance for transporting data (to accelerate the transfer of information) between multi-microprocessors and memories, and to cool these electronic components despite the increased heat flux that results from increased transistor density. Recent technological advances show a tendency for the development of 3D integrated circuit stacked architectures with interlayer cooling (multi-microchannels in the silicon layers). However, huge challenges exist in such design/concept, i.e. flow distribution to hundreds microchannels distributed in the different interlayers, thermo-hydrodynamic and geometrical limitations, manufacturing etc. 3D-ICs with interlayer cooling are still about a decade away, so a viable shorter term goal is 3D stacks with backside cooling, taking advantage of Si layers now able to be thineer down to only 50 μm thickness. Thus, the present work presents thermo-hydrodynamic simulations for 3D stacks considering only a backside cooler, which simplifies considerably the assembly and guarantees a high level of reliability. In summary, the results showed that this concept is thermally feasible and potentially that interlayer microchannels (between stacks) will not be necessary.

Until recently, effective cooling solutions with high performance were required especially in data-centers and super computers because of the huge and ever-increasing power consumption in these applications. Water cooling systems have been considered for use in the cooling of large scale data-centers and super computers. For the cooling of super computer CPUs, a water cooling system using advanced cold plate technology is reconsidered. The thermal resistance of a cold plate for cooling the CPU is required to dissipate 80 to 100W of heat at 0.05 K/W. Also, in this application, the cold plate is required to be mechanically reliable in withstanding a cooling water pressure of 1MPa. We adopted a micro-channel structure as a heat transfer surface of this cold plate and developed a new brazing method so that the tips of the micro-channel fins are bonded to the inside of cover plate of the cold plate. In collaboration with a customer in charge of the design, we completed the water cooling unit consisting of cold plates, pipes and coupler manifold, assembled by brazing. Finally, the high volume products were manufactured with reliability inspection (pressure test and helium leakage test) and used to effectively cool the CPUs of an advanced super computer, which was awarded the fastest super computer record. Water cooling technology provides effective high capacity cooling in compact space limits, and has been widely used in applications like fiber laser machines and others. This paper describes the development of cold plate with micro-channels and its applications.

Data center energy consumption makes up a significant and rapidly growing portion of the national energy use. A large fraction of this energy consumption, in the range of 25%–40%, is used by the data center cooling infrastructure, particularly by the computer room air handlers (CRAH) and by the chiller plant. A joint US Department of Energy and IBM effort investigated the use of warm water server cooling and water side economization to help improve energy efficiency in a volume server by minimizing the use of CRAH and chiller hours with the goal of reducing the ratio of cooling energy to IT energy from the conventional value of approximately 50% to less than 10%. Previous work focusing on system characterization and one day runs found significant energy savings as compared to a traditional chiller based data center. In the work reported in this paper we discuss the development and implementation of a dynamic control approach followed by details and results from a 60+ day run of the data center test facility starting May 11 th , 2012. Results from this long term study found the average cooling energy consumption to be 3.5% of the IT energy, which is consistent with previous short one day runs. The implemented control scheme was also able to reliably control the water temperature to within 0.5°C of the set-point under most conditions and could react and stabilize to a new set-point in 30 minutes. The work presented demonstrates the high energy efficiency of a warm water cooled, water-side economized, chiller-less data center over a long sustained period of time and highlights the benefits of energy aware control to help limit cooling energy.

Thermal management of high power data centers poses challenges due to the high operational cost which is made worse due to the many inefficiencies that arise in them. Additional challenges arise due to the dynamic behaviors that occur during normal operation, and also during emergencies such as power outages or failure of some or all of the cooling equipment. Water and hybrid air plus water cooled data centers are an alternate cooling solution combining liquid cooling systems, such as rear door heat exchangers located within the racks themselves, in addition to the traditional raised floor cold aisle air cooling system. Such a solution may be used when some of the equipment in a data center is upgraded to higher end and higher power equipment which may not be manageable with the existing air cooling system. For a data center with a hybrid cooling system, the cold air supply and the cold water supply should increase in case of an emergency, such as a CRAC (Computer Room Air Conditioner) units’ failure. In this paper, a detailed computational study is conducted to investigate the dynamic response of the impact of a CRAC failure on both water side and air side in a representative hybrid cooling room. The room studied is an air cooled data center using the common cold aisle approach, with rear door heat exchangers installed on all of the racks. CRAC failure is investigated in a hybrid cooling room. The variation and fluctuation in an average rack inlet temperature, and inlet temperatures at different detail locations are presented in plots, showing the dynamic performance of a hybrid cooling data center subjected to the different CRAC failure scenarios. Different response time studies are also presented in this paper.

An information technology (IT) container needs to be supplied with cold air to cool IT equipment housed in it. The type of cooling system to be used depends on many factors including geographical location of the modular data center. Data centers located in regions where the climate is cold benefit from use of air-side economization (ASE) and those located in hot and dry climate benefit from use of direct and/or indirect evaporative cooling (DIEC) systems. In terms of energy saving, ASE, direct evaporative cooling (DEC) system, and indirect evaporative (IEC) systems are better than compressor based cooling systems such as computer room air conditioning (CRAC) units and air handling units (AHU). In this study, an existing DIEC unit which can also be operated in ASE mode is modeled in a computational fluid dynamics (CFD) tool. The cooling unit is intended to be used for supplying cold air to a containerized data center with specified volume flow rate, dry-bulb temperature and relative humidity. The CFD model is compared with published data of the cooling unit to see how well the CFD model represents the actual system and few design improvement ideas are tested by modifying the CFD model and running simulations. Results show that supplying air horizontally or as a downdraft to an IT container has negligible effect on the overall system. Results also show that orientation of dampers and placement of blanking panels inside the mixing chamber could affect the lifespan of air filters.

The use of air containment systems has been a growing trend in the data center industry and is an important energy saving strategy for data center optimization. Cold Aisle Containment (CAC) is one of the most effective passive cooling solutions for high density heat load applications. Cold Aisle Containment provides a physical separation between the cold air and the hot exhaust air by enclosing the cold aisle, preventing hot air recirculation and cold air bypass. This separation provides uniform inlet air temperatures to the servers, which can further contribute to overall data center efficiency. This paper includes the thermal test data for a data center lab with and without a CAC set up. The paper quantifies the thermal impact of implementing a CAC system over an open Hot Aisle/Cold Aisle (HA/CA) arrangement for three different cabinet heat load conditions at two different CRAC (Computer Room Air Conditioner) return air set point conditions. It studies the advantages of CAC over standard HA/CA arrangement. A case study has been presented showing a cooling energy savings of 22% with the use of a CAC system over a standard HA/CA arrangement.

Handheld devices are increasingly capable of running applications that used to require laptop and desktop computers. The requirement that these devices provide better performance with a smaller form factor or size presents significant challenges, especially with the limitations of passive cooling. The current study presents a summary of the cooling solutions of several popular, commercially available tablets. The trends in power dissipation and thermal management techniques of handheld devices are presented. The factors affecting the maximum possible power dissipation are discussed. The effects of the selection of the outer shell materials, the thermal interface materials, heat spreaders and air gaps are presented. For all considered thermal management techniques of handheld devices, a figure of merit for the cooling solutions is defined as: Figure of Merit = Maximum Power Dissipation / Surface Temperature Rise, in (W/degC). This figure of merit allows for an objective comparison of the available cooling solutions.

This paper presents the results of an experimentally validated Computational Fluid Dynamics (CFD) model for a data center with fully implemented fan curves on both the servers and the Computer Room Air Conditioner (CRAC). Open and contained cold aisle systems are considered experimentally and numerically. This work is divided into open (uncontained) cold aisle system calibration and validation, and fully contained cold aisle system calibration and leakage characterization. In the open system, the CRAC unit is calibrated using the manufacturer fan curve. Tiles flow measurements are used to calibrate the floor leakage. The fan curves of the load banks are generated experimentally. A full physics based model of the system is validated with two different CRAC fan speeds. The results showed a very good agreement with the tile flow measurements, with an approximate average error of 5%, indicating that the average model prediction of the tile flow is five percent lower that the measured values. In the fully contained cold aisle system, a detailed containment CFD model based on experimental measurements is developed. The model is validated by comparing the flow rate through the perforated floor tiles with the experimental measurements. The CFD results are in a good agreement with the experimental data. The average error is about 6.7%. Temperature measurements are used to calibrate other sources of containment and racks leaks including mounting rails and clearance between racks. The temperature measurements and the CFD results agree well with average error less than 2%. Detailed and equivalent modeling methods for the floor and containment system are investigated. It is found that the simple equivalent models are able to predict the flow rates however they did not succeed in providing detailed fluid flow information. While the detailed models succeeded in explaining the physical phenomena and predicting the flow rates with noticeable tradeoffs regarding the computational time. Important conclusions can be drawn from this study. In order to accurately model the containment system, both the CRAC and the load banks fan curves should be simulated in the numerical model. Unavoidable racks and containment leaks could cause inlet temperature increase even if the cold aisle is overprovisioned with cold air. It is also noted that heat conduction through the floor tiles causes a slight increase the inlet temperature of the cold aisles. Finally, it is noteworthy that using detailed modeling is necessary to understand the details of the thermal systems, however simpler and faster to compute equivalent models can be used in extended optimization studies that show relative rankings of different designs.