The integration of variable and intermittent renewable energy generation into the power system is a grand challenge to our efforts to achieve a sustainable future. Flexible demand is one solution to this challenge, where the demand can be controlled to follow energy supply, rather than the conventional way of controlling energy supply to follow demand. Recent research has shown that electric building climate control systems like heat pumps can provide this demand flexibility by effectively storing energy as heat in the thermal mass of the building. While some forms of heat pump demand flexibility have been implemented in the form of peak pricing and utility demand response programs, controlling heat pumps to provide ancillary services like frequency regulation, load following, and reserve have yet to be widely implemented. In this paper, we review the recent advances and remaining challenges in controlling heat pumps to provide these grid services. This analysis includes heat pump and building modeling, control methods both for isolated heat pumps and heat pumps in aggregate, and the potential implications that this concept has on the power system.
The US electrical grid has experienced a rise in renewable energy generation capacity in recent years, increasing by more than 50% in the past ten years.2 In addition, some states are beginning to adopt aggressive clean energy goals with high percentages of wind and solar energy. This large and rapid shift in electricity generation sources poses difficult new problems for the electrical grid. Conventional grid operation relies on the practice that generators can be reliably controlled to match electrical supply and demand, while ensuring grid stability. However, with the diminishing percentage of electrical capacity provided by thermal generators and the increasing percentage of variable generation sources like wind and solar, the grid becomes much more difficult to predict and control. Therefore, to maintain a reliable electrical grid in high renewable energy scenarios, the grid requires a significant addition of supporting technology such as energy storage and demand management .
A potential source of demand management is through controlling heat pumps. Heat pumps are an efficient, electric source of building heating and cooling. Instead of converting electrical energy directly to heat, e.g., an electric resistance heater, heat pumps use a compressor-driven vapor-compression cycle to move heat from a low-temperature source to a high-temperature sink, which can provide both heating and cooling through the use of a reversing valve. A heat pump’s main efficiency metric, the coefficient of performance (COP), is defined as the ratio of the amount of heat moved to the amount of electrical input. The COP is inversely related to the difference between the indoor and outdoor temperatures, and therefore, heat pumps perform poorly in extreme environments, particularly in cold climates. Despite this, recent advancements in heat pump technology have significantly increased the COPs at both extreme high and low temperatures [2,3], expanding heat pump technical feasibility to new geographical regions. However, in many regions of the United States, it is still not economically feasible to replace a natural gas heating system with a heat pump, and given the current electrical generation mix, displacing natural gas heating with a heat pump could actually increase greenhouse gas (GHG) emissions . Nevertheless, heat pump adoption is the cornerstone of many aggressive GHG emission reduction policies, such as New York City’s 80 × 50 plan . Rapid and widespread adoption of heat pumps in areas like this is likely to create significant new operational challenges for the electrical grid, and therefore, these heat pumps must be correctly managed and integrated into an increasingly renewable grid.
As long as indoor thermal comfort is maintained, heat pumps have inherent operational flexibility. This flexibility has already been harnessed by utilities in the form of thermostat-based demand response programs. These programs generally consist of utilities turning off heat pumps during extreme peak load hours, either through setpoint modification or direct load control. In addition, the use of thermal energy storage (TES) has grown in popularity particularly in Europe and allows for load shifting to accommodate high levels of renewable energy . However, new research shows the potential for heat pumps to provide more complex grid services by operating in ancillary service markets. Ancillary services, which are often provided by controllable thermal generators, are essential for power system stability and maintain the instantaneous balance of electricity supply and demand on the grid. Providing these services often involves following a specific power trajectory sent by the system operator requiring response on the order of seconds to minutes. However, controlling heat pumps to provide ancillary services can require installation of a significant amount of additional hardware. For example, building temperature, heat pump power consumption, external disturbances, and grid signals must all be collected and processed in real time. Much of these data can now be collected and transmitted using Internet of Things devices like smart thermostats and electricity meters. Smart thermostats have seen a rapid rise in adoption , and advanced metering infrastructure smart meters have now been installed for 47% of US customers as of 2016 . Harnessing the potential of these devices is a key component in widescale implementation of heat pump ancillary services.
These three driving factors—renewable energy variability, heat pump integration, and smart grid implementation—have sparked many recent studies into the capability of providing ancillary services from heat pumps. Heat pumps have been shown experimentally to have the physical capability of providing ancillary services without significant occupant discomfort [9–12]. Simulations have shown that aggregating together hundreds or thousands of variable-speed or single-stage heat pumps significantly increases their ability to provide ancillary services. However, despite many studies showing the capability and potential of heat pumps to provide ancillary services, to the best of our knowledge, there have been no experimental results for large-scale heat pump aggregation.
While a detailed review of the role of heat pumps in a smart grid was given in Ref. , this study reviews the various methods for modeling and controlling heat pumps specifically for ancillary services. Section 2 outlines the various ancillary services that heat pumps can provide. Section 3 describes various methods for modeling and controlling heat pumps both locally and in aggregate. Section 4 shows how heat pumps participate in ancillary service markets. Section 5 analyzes potential performance, capacity, and economics. Section 6 concludes the paper and presents opportunities for the future work.
2 Ancillary Services
The reliability of the electrical grid hinges on the ability of grid operators to match electricity generation and consumption on a variety of timescales and under many contingencies. Grid operators control this balance through several types of ancillary services, which are broadly defined based on their time scale, presented in Table 1. In deregulated markets, grid operators procure these services through ancillary service markets. In contrast to energy markets, where generators are only paid for the energy they produce, ancillary service markets are primarily capacity markets, where a grid operator also pays for the capacity of a generator to alter its production. A technical review of ancillary services is given in Ref. , while Ref.  gives a review of various US ancillary service market structures. While ancillary services are often provided by generators, they can also be provided through demand response. Demand response is the process of controlling the demand to respond to grid signals. Reference  describes the role of demand response in ancillary service markets and the effects of market policies on demand response participation. The following text introduces the particular ancillary services that can be provided by heat pumps.
Heat pumps can provide ancillary services in a similar way to other energy storage devices like electrochemical batteries or pumped hydroelectric storage. Heat pumps can store energy by injecting or removing heat from the building’s thermal mass. For example, in summer, a heat pump can increase its power consumption and charge its storage by removing heat and cooling the building to its lower thermal comfort limit. By doing so, the heat pump now has the flexibility to reduce its future power consumption and allow the indoor temperature to drift up to its upper thermal comfort limit. This increase or reduction in heat pump power consumption results in a net removal or injection of power onto the grid, achieving a similar result to a generator lowering or increasing its power output, respectively. The building then acts as a virtual battery, where the indoor temperature relative to the upper and lower thermal comfort limits acts as a state of charge, and the building’s thermal mass acts as a measure of the energy storage capacity . These unique attributes introduce several key control considerations that differentiate heat pumps from generators in providing ancillary services:
Controlling strategy: When a generator is required to reduce generation, the heat pump should increase load and vice versa.
Controlling limits: Heat pumps must not violate indoor temperature constraints and therefore cannot operate above or below their setpoint for an extended period of time.
Depending on the service, these differences can have both beneficial and detrimental effects on heat pumps’ ability to provide ancillary services. The reminder of Sec. 2 describes the potential services that heat pumps can provide and how their operation differs from a conventional generator.
2.1 Frequency Regulation.
A stable grid frequency is ensured by an instantaneous balance between electrical supply and demand. The frequency will drop if demand exceeds supply and will rise if supply exceeds demand. If system frequency drifts more than 1–2 Hz from normal levels (60 Hz in the United States), the equipment can be severely damaged and generators can trip, causing cascading failures . Because of this, frequency regulation requires response on the order of seconds. Generators providing frequency regulation must be equipped with telemetry and control technology to follow an automatic generator control (AGC) signal from the grid operator, which is usually sent every 2–4 s. Demand-side frequency regulation providers can also follow this AGC signal by reducing the load when it calls for an increase in generation.
Frequency regulation is the highest priced ancillary service and is operated primarily as a capacity market. A service provider must bid a certain capacity for regulation often in the day-ahead market and, if accepted, must follow the power signal sent by the system operator. Depending on the system operator, the regulation market either has separate markets for regulation-up and regulation-down or requires symmetric regulation capacity (equal up and down regulation capacity). Currently, California Independent System Operator (CAISO) and Electric Reliability Council of Texas (ERCOT) are the only US system operators that operate separate up and down regulation markets. For generators, these two methods are equivalent . However, for energy storage and load control, significant differences in revenue can occur based on the market type.
Another challenge for demand response and energy storage systems is that the frequency regulation signal is not necessarily zero-mean. For heat pumps providing frequency regulation, a signal bias can cause the heat pump to run consistently below or above its baseline consumption, potentially violating comfort constraints. To resolve this, some system operators have introduced fast regulation signals that are designed to be zero-mean . For example, Pennsylvania, Jersey, Maryland Regional Transmission Organization (PJM) has filtered its signal into two, called RegA and RegD, which are shown in Fig. 1. The RegA signal has a slower time constant and was designed to accommodate steam generators with relatively low ramping capability. RegD consists of higher frequency fluctuations and often converges to zero-mean within 15 min . For this reason, many studies on the technical capability of providing frequency regulation with heat pumps follow the RegD signal [11,12,21,22].
2.2 Load Following.
Load following consists of generators following the slower, more predictable fluctuations in electricity demand on a time scale of several minutes to several hours. This is often procured through economic dispatch, where generators are dispatched according to their generation cost . However, as wind and solar supply an increasing percentage of electricity on the grid, this service could become much more important, particularly for ramping in the mornings and evenings . This could be a potential service provided by heat pumps, either as a reserve capacity similar to CAISO’s flexible ramping product  or through responsiveness to a real-time price disseminated by the system operator. For example, when solar and wind energy are readily available, electricity prices can drop significantly due to a surplus in supply, encouraging loads to operate during these times. In grids with high solar penetration, such as in the CAISO, there is a growing frequency of negative wholesale electricity prices , where generators must pay to produce electricity. This poses a unique opportunity for heat pumps to potentially be paid for operation.
Since heat pumps operate in the retail electricity market, they are often charged a static electricity price, giving no incentive to shift operation toward times of high energy supply. Time-of-use rates, which have predefined price tiers for peak and off-peak hours, have had some success in providing consumers’ indirect access for providing a load following service by shifting load away from peak hours . Connected thermostat demand response programs such as Austin Energy’s Power Partnersm program5 have also been widely deployed. These programs allow the utility to turn off heat pumps for short periods of time to reduce the peak load. However, these methods are simplified and therefore do not capture the full potential of heat pumps to provide a load following service.
A second challenge to provide this service is the relatively low frequency of a load following signal. If the frequency of the load following signal is on the same order of the building’s thermal response, comfort constraints can be violated . This severely limits the capacity that heat pumps can offer for load following compared with a higher frequency signal like frequency regulation. However, this time constant has the added benefit of reducing the need for fast response controllers.
Power systems are required to maintain a certain amount of reserve margin to ensure reliability in case of contingencies. For example, if a large generator unexpectedly trips, the system might need to dispatch reserves. To provide this service, a heat pump or heat pump aggregation bids a reserve capacity into the reserve capacity market. This contract requires the system to curtail its full capacity offering for a certain amount of time determined by the reserve dispatch signal. After the signal ends, the heat pump system can recover back to its baseline energy consumption.
The reserve market can be split into two main categories: spinning and nonspinning reserve. While different system operators can sometimes have different definitions , spinning (or synchronous) reserves primarily consist of online generators synchronized to the grid and capable of dispatching to full capacity within 10 min. Nonspinning reserves must respond within 10–30 min but are not necessarily connected to the grid. Providing spinning reserve is preferred over nonspinning reserves for two main reasons. First, heat pumps are already connected to the grid and have high ramping capabilities relative to thermal generators. Second, spinning reserve is priced an order of magnitude higher than nonspinning reserve. However, since reserve dispatch signals result from contingencies, the frequency and the duration can be quite unpredictable. From 2013 to 2018, PJM dispatched spinning reserve anywhere from 0 to 8 times each month with a duration anywhere from 3 to 50 min.6 For this reason, accounting for uncertainty is a vital component of providing reserve.
3 Modeling and Control
Studies on the modeling and control of heat pumps for ancillary services cover a wide range of scale and complexity. Frequency regulation requires a fast and accurate controller that can track a signal on the order of seconds. Load following controllers can be slower and simpler, while reserve controllers can be as simple as an on/off controller. However, it is important to note that for all ancillary services, the underlying goal is to track a given ancillary service signal. For this reason, many control schemes and methods of determining ancillary service capacity can work for several types of services. Sections 3.1 and 3.2 discuss how heat pumps are modeled and controlled on both local and aggregate levels.
3.1 Local Modeling and Control.
On a local level, heat pumps and their buildings can be described by high-fidelity models and directly controlled to follow an ancillary service signal. This often involves directly controlling the fan speed or compressor speed to change the power consumption. Therefore, depending on the type of the system, different models and control methods must be used.
An accurate building thermal model is also important to determine the amount of thermal energy that can be stored and to prevent violation of thermal comfort constraints. Modeling complexity varies widely based on the building type and size. Detailed reviews on various building modeling techniques are given in Refs. [41,42]. For large commercial buildings, building information modeling (BIM) is often available to provide detailed white box models based on known material properties and building dimensions. However, both accurate identification of each of these parameters and using detailed models for control can be difficult and expensive to obtain. Reference  gives a simple method for converting a more complex EnergyPlus  model to a reduced-order model usable in model predictive control. Meta-model–based optimization is used in Ref.  to identify optimal reduced-order model parameters for a building that are suitable for control.
For smaller buildings or buildings without BIM, gray-box models are often used. The most popular gray-box building modeling technique is through a thermal circuit, sometimes called equivalent thermal parameters. These thermal circuits contain resistors, which represent resistance to heat transfer, and capacitors, which represent heat storage capability. The values of each of these components can be identified from either experimental or physical data . Common circuits for small buildings include either 1R1C (1 resistor and 1 capacitor) or 3R2C. In a 1R1C model, the entire building is lumped into one thermal mass represented by the single capacitance. For a 3R2C, however, the thermal masses of the indoor air and the building material are separate, giving a more accurate prediction over longer time scales. Figure 2 shows an example of a 3R2C model. For larger buildings with many different zones, higher order models containing more capacitors and resistors can also be used .
By adding TES to a building, additional thermal capacitance is introduced, significantly increasing the potential for providing ancillary services. The most common type of TES takes the form of water tanks and has been shown to increase the power flexibility for frequency regulation , as well as allow flexibility over longer time scales . Other forms of TES technology involve phase change material, either in a tank coupled with the heat pump or directly embedded in the building construction walls . Since phase change material stays at a relatively constant temperature during operation, additional modeling considerations must be taken into account .
Based on the heat pump system, various components can be controlled to alter the power consumption. Feedback controllers are typically used for local control, but common difficulties in implementation are determining optimal controller gains and accounting for time delays.
In Ref. , a commercial VAV HVAC system was experimentally shown to be capable of following a frequency regulation signal through control of the fan. The signal was first filtered to exclude low frequencies and high amplitude oscillations. Low frequencies that are of similar order to the building’s thermal response can cause temperature constraint violations, while high amplitude oscillations can have harmful effects on the fan’s reliability, decreasing its useful life. By perturbing the existing controller’s fan speed and airflow setpoints, this controller was able to achieve PJM performance score of 0.83, exceeding PJM’s test performance requirement of 0.75. The fan speed for a commercial VAV HVAC system was also controlled to provide frequency regulation in Refs. [9,10]. In this study, the authors use a novel switched controller to maximize speed while ensuring stability. If the desired power output is within some error tolerance from the existing output, a standard proportional-integral (PI) controller is used. Otherwise, a model-based feed-forward controller is used. This controller resulted in much higher test performance scores between 0.94 and 0.98.
For a VSHP, the compressor consumes a majority of power and can be controlled to provide ancillary services. However, due to manufacturer limitations, it is usually difficult to control the compressor directly. In Ref. , the supply water temperature for an air-to-water VSHP was used to control the power consumption using a PI controller with nonlinear signal processing to ensure stability. While controlling supply water temperature setpoints was not as effective as simulations involving the direct compressor speed control, the controller was still able to achieve performance scores around 0.8. In Ref. , the VSHP compressor was directly controlled using feedback controllers and operated in a small-scale experimental microgrid, showing the feasibility of participation with other distributed energy resources.
3.2 Aggregate Modeling and Control.
By aggregating together many heat pumps, the combined capacity of ancillary services can be greatly increased. However, in aggregate heat pump control, the detailed parameters of each individual building and heat pump are difficult to obtain. Therefore, aggregate control studies often contain high-level control schemes using simplified heat pump and building models. The main objective in aggregate control is to determine which heat pumps to modulate to accurately track an ancillary service signal while maintaining thermal comfort and reliability constraints. Note that while these aggregation control studies assume that each heat pump serves a single building, district heating and cooling systems can also provide ancillary services while serving an aggregation of buildings. These systems are much larger and more complex, and a review of controlling district heating and cooling systems for grid services is given in Ref. .
The control of TCLs for ancillary services has been widely studied [53,57–67]. In Refs.  and , a feedback controller was used to control a global thermostat setpoint that turns on or off a certain number of TCLs based on statistical state predictions. This method is difficult in practice, though, as it can rely on setpoint changes down to 0.0025 °C, which is far below the measurement resolution for thermostats. In Ref. , a priority stack control method was used to directly control TCL status. This method prioritized turning on or off the TCLs that were closest to automatically turning on or off, respectively. Finally, Ref.  explored the stability of TCLs as a result of significant perturbations during control for demand response.
However, the majority of these TCL controllers use simplified, simulated models that neglect many important differences between heat pumps and other TCLs like electric heaters. For example, to avoid damaging the compressor and reducing efficiency, heat pumps have minimum on and off times, which can be the most financially and physically limiting factor for ancillary service provision . Moreover, heat pump COP can vary drastically even among the same heat pump model . Finally, there are many different types of heat pumps, including VSHPs, which do not follow the standard TCL model. Because of these additional complexities, the use of heat pump aggregations for grid services has not been commercially implemented in the same way that other TCLs like water heaters have been implemented .
A solution to the minimum off time is given in Ref. , which adds an additional “lock-out” state between the on and off states. Variable-speed heat pumps are used in Refs.  and  by dividing a frequency regulation signal equally among each heat pump. A rule-based controller is used in Ref.  to provide frequency regulation from an aggregate of ground-source heat pumps in conjunction with thermal energy storage. Finally, Ref.  shows the effect that changes in ambient temperature can have on a population of air conditioners functioning as TCLs.
For ancillary services that require fast response like frequency regulation, control and communication delays can become a serious issue. For aggregations, a reference signal must be received from the system operator and processed to determine the corresponding control decision, and then the control decision is distributed to each heat pump. Moreover, for control systems that communicate with the thermostat rather than the heat pump directly, uncertain time delays can accumulate based on internal thermostat and heat pump control systems. Without delay compensation, tracking accuracy was found to be reduced by as much as 40% for a 20-s delay in Ref. . However, a Kalman filter-based state estimation technique was used in Ref.  to mitigate this effect and produce no performance deterioration for delays up to 20 s.
While these heat pump aggregation studies are beginning to include more realistic constraints, they still require some significant assumptions, and there is little experimental validation. For example, the transient power profile of heat pumps and heat pump reliability considerations are relatively unexplored and are an avenue for further research.
4 Market Participation
While Sec. 3 describes control methods for providing ancillary services, the heat pump must establish both a baseline and flexibility capacity to bid into either the day-ahead or real-time electricity markets . A baseline is the future power trajectory that the heat pump plans to follow for the length of the ancillary service contract. A capacity, sometimes called flexibility, is the amount of power that the heat pump can go above or below its baseline without violating constraints. This is an important difference between generators and heat pumps providing ancillary services. A generator can operate indefinitely within its declared power capacity limits and thus can ignore the energy impact of the ancillary service signal, i.e., the generator can run at 10% above its baseline for an indefinite amount of time if required. A heat pump cannot do this without eventually violating temperature constraints. Therefore, the amount of capacity that a heat pump can offer for ancillary services is heavily dependent on the energy content of the ancillary service signal.
In the context of ancillary services, a baseline is analogous to a generator setpoint and must be determined ahead of time such that the contracted ancillary service capacity can be maintained. This baseline definition is slightly different from a traditional demand response counterfactual baseline, which uses the historical data to estimate what the unmodified energy consumption would have been to measure the amount of demand response provided. In contrast, an ancillary service baseline can be decided by the ancillary service provider based on market and weather conditions to optimize a user-defined objective. Model-predictive control (MPC) is among the most widely used methods to determine an ancillary service baseline. MPC is an iterative control scheme that optimizes a model-based objective function over a given time horizon. The optimal control for the first time step is then implemented, and the MPC reoptimizes with updated inputs. Possible optimization objective functions could be to maximize total profit, maximize thermal comfort, or a combination of the two.
There is a large amount of research on determining optimal power trajectories for heat pump systems . However, it is important to note that the energy optimal power trajectory does not always provide an adequate flexibility for providing ancillary services. In Ref. , a contract for declaring a baseline and flexibility capacity for ancillary services in real time is given. A robust MPC determines a baseline and flexibility determination that minimizes the energy cost minus the ancillary service revenue. One key feature in this contract is that the building owner pays only for its baseline energy consumption and not for the altered consumption based on an ancillary service signal, hedging the utility and building owner from any nonzero-mean ancillary service signal.
However, the uncertainty of disturbance predictions and the fidelity of the model can significantly degrade the performance and must be carefully considered. Common prediction methods for disturbances for heat pump control include numerical weather predictions, occupancy schedules [77,78], autoregressive regression, and neural networks [79,80]. The effect of model fidelity on MPC performance was explored in Ref. .
4.2 Capacity Determination Without Uncertainty.
As previously stated, the flexibility available at a given time step is heavily dependent on the content of the ancillary service signal in previous time steps. One way to simplify this analysis is to assume that the ancillary service signal is zero-mean over the time step, which allows for independent time-wise optimization of flexibility capacity, i.e., each time step does not depend on the ancillary service signal from the previous time step. Since this method does not consider any uncertainty of the mean of the ancillary service signal, it is the most aggressive capacity determination method and can potentially overestimate the actual capacity available. For fast frequency regulation signals such as PJM’s RegD, this assumption can be valid since it is designed to be zero-mean over a 15-min period . However, for slower frequency signals such as RegA, load following, and reserve, this method can be unfeasible.
The limitation of this assumption is often addressed by calculating the general flexibility characteristics of a heat pump or building. Reference  develops a flexibility index suitable for control on both an individual and aggregate level. Thermal energy storage is added in Ref.  to increase the flexibility of a heat pump. Finally, Ref.  determines the load reduction flexibility using behind the meter electricity data. By developing battery-like models for flexibility, these types of studies provide the basis for modeling heat pump flexibility for control.
4.3 Capacity Determination With Uncertainty.
There are two primary methods of accounting for uncertainty during capacity determination: robust and scenario based. Robust determination is the most conservative approach. This approach ensures that the flexibility offered by the heat pump can be met under the worst case ancillary service signal or disturbances. This method is of particular importance in providing reserve, since the heat pump must be able to reduce its full capacity offering for an unknown amount of time. Robust distributed optimization is used in Ref.  for day-ahead and intra-day scheduling of flexibility capacity for an aggregation of flexible loads. Reference  uses robust MPC to determine flexibility capacity for frequency regulation while considering uncertainty in both external disturbances and the frequency regulation signal. Reference  provides a robust control strategy for managing uncertain communication time delays for an aggregation.
Another way of dealing with uncertainty is scenario-based optimization. In this method, the capacity determination must not violate temperature constraints under a set of disturbance scenarios that are developed based on historical conditions. By satisfying a certain number of these scenarios, the controller can provide the flexibility it offers with a certain confidence level . While this can be computationally intensive, scenario-based optimization can provide a less conservative flexibility capacity than robust optimization while still considering uncertainty. Reference  gives a scenario-based MPC for determining optimal energy consumption of a building, while Ref.  gives a scenario-based method for determining the flexibility of a population of controllable loads. Research on accounting for uncertainty for heat pumps in both local and aggregate control are relatively limited, and this is an area for the future work.
4.4 Hierarchical Control.
Since MPC requires optimization of a sometimes complex objective function, it alone is not fast enough to ensure response to fast ancillary service signals. Many studies use a hierarchical control scheme to solve this problem [9,34,47,83,88–91]. This hierarchical control scheme combines the strategies for local and aggregate control with prediction methods used for baseline and capacity determination. For example, a three-tier hierarchical controller was used in Ref.  to control an aggregation of single-stage heat pumps consisting of (1) a load aggregator that interacts with the power system and ancillary service markets, (2) a central controller that prioritizes which heat pumps to turn on or off, and (3) a local controller that considers local constraints. Figure 3 shows a common layout for hierarchical controllers.
Level 1 is sometimes referred to as a virtual power plant (VPP) and acts as the interface to the grid. From a power system operator’s perspective, a VPP acts and is controlled similar to a conventional power plant: It bids into day-ahead ancillary service markets and its aggregate power responds to grid control signals. The VPP passes grid signals to the central controller, level 2, for real-time aggregate control. The central controller can take the form of various aggregate control schemes outlined in Sec. 3.2.1. The control signal sent from the central controller to the local controller, level 3, can take the form of setpoint change or direct load control. The local controller then responds to this control in accordance with local constraints and disturbances. Together, these controls allow an aggregation of small, distributed heat pumps to provide ancillary services to the grid as if it were a large-scale energy storage resource.
5 Performance, Capacity, and Economics
While heat pumps have the physical capability to provide ancillary services to the grid, whether there is an adequate economic incentive to do so is still an open question. With the vast amount of heat pumps already in operation, there is an enormous potential capacity available for ancillary services. However, the revenues from providing services do not always justify the accompanying capital costs and potential efficiency losses. Therefore, a holistic view of costs and performance comparison with other energy storage technologies must be considered to determine whether providing ancillary services is attractive to both heat pump owners and grid operators.
5.1 Performance and Capacity.
TCLs have been both experimentally and numerically shown to have potential capacity to provide ancillary services [9,10,57,92,93]. Reference  calculates that the ancillary service capacity provided by residential, such as refrigerators, heat pumps, and electric water heaters, can reach 10–40 GW and 8–12 GWh in California, which can easily satisfy the energy storage mandate of 1325 MW to support their renewable portfolio. This estimated capacity was heavily dependent on the climate zone: Some of the zones could only provide flexibility during either winter or summer, while those in more balanced climates could provide a higher average capacity throughout the year. While a large amount of capacity is estimated to be available, Ref.  concludes that given current technology and regulatory frameworks, widespread utilization of this flexibility is insufficient for high renewable energy portfolios.
Several studies have experimentally tested the RTE performance of single heat pumps following regulation service signals with very different results. In Ref. , an experimental study controlling a VAV HVAC system to provide a fast, symmetrical service, similar to a charge/discharge cycle in a battery, found that the extra energy consumption was significant. The RTE was only 46% for fan power and 42% for the combined power of the chiller and fans. While this RTE seems low, analysis of space conditioning data from Ref.  gave almost identical RTEs at around 46% . Relatively low RTEs were also found in Ref. , where experimentally controlled VAV HVAC systems showed RTEs ranging from 34% to 81%. Both experimental studies relied on open-loop global temperature setpoint control mechanisms, in contrast to the MPC approaches previously discussed. However, Refs. [9,10] found that the energy loss associated with following the much faster PJM frequency regulation, RegD, signal was negligible.
The causes of inefficiency were explored through physics-based modeling in Ref. , which gave RTEs of less than 100% when the power is increased and then decreased, but greater than 100% when the power is decreased and then increased. This effect can be explained by differences in efficiency due to indoor air temperature variation. Furthermore, Ref.  found that when the HVAC system is repeatedly used, the RTE converges to 100%. They attributed the low RTE values reported from experiments  to the fact that the experiment ran only one cycle. Therefore, more experimental results are required to accurately define the RTE for a heat pump.
In addition to RTE, there are efficiency losses associated with providing flexibility capacity. To provide flexibility, the heat pump might need to deviate from the energy optimal control schedule. The amount of energy increase compared with an energy optimal controller in Refs. [9,10] was 68 % for the fan and 11% for the chiller. However, by including payments for providing ancillary services, this controller provided the cost optimal solution despite increases in energy. Moreover, Ref.  found that the ratio of reserve payment to electricity cost must be above a threshold to incentivize deviating from the energy optimal control to provide flexibility for ancillary services.
This wide variety of results show that there is still no consensus on the total efficiency of a heat pump providing ancillary services. They reveal that the 100% efficient assumption may not be justified in control simulations, and flexibility capacity could be significantly overestimated. In addition, the ancillary service efficiency of an aggregation of heat pumps, as well as variable-speed and single-stage heat pumps, is relatively unstudied. Therefore, more experimental work is needed to determine how potential efficiency losses affect the actual performance of heat pumps providing ancillary services.
5.2 Economical Potential.
By receiving payments for providing ancillary services, heat pump owners can have additional revenue streams, reducing the net present cost of heat pump installations. These revenue streams are modest but not negligible. Table 2 presents a summary of potential revenues for a variety of heat pump types, locations, and markets. Revenue varies significantly depending on type of load, climate zone, and regional ancillary service prices. In Ref. , residential heat pumps providing frequency regulation in a TCL model were estimated to earn $1–52/unit/year for cooling and $11–46/unit/year for heating under the pay-for-performance pricing structure. The wide range of variation is primarily due to the difference in the climate zone. For example, heat pumps in more extreme climates like Bakersfield and Sacramento, CA, could earn significantly more than those in mild climates like San Francisco, CA.
Spinning reserve revenues are significantly lower due to the much lower spinning reserve capacity prices. Spinning reserve revenues were estimated to be less than $5/unit/year in Ref.  and therefore is not attractive under current market policies. There are relatively few revenue studies specifically for load following, but significant energy costs savings are possible by indirect participation though dynamic energy pricing and thermostat-based utility demand response programs. For example, electricity costs were reduced by up to 30% using a price-based controller in a real-time retail electricity market . Utility demand response programs primarily used for reducing peak load also give monetary incentives. The SmartAC™ program of PG&E (Pacific Gas and Electric Company) provided one-time signup bonus of $50 to each participating unit.7 The OnCall™ program of Florida Power and Light Company provides a monthly credit on bill, totaling up to $83 annually for each participating unit.8
However, these revenues must be compared with both instrumentation costs and opportunity costs for providing services. Basic telemetry devices are needed to connect the heat pump to the grid or aggregator, including a real-time electricity meter and controllable thermostat. Reference  estimated that this instrumentation could cost between $100 and $250. In addition, heat pumps could be incentivized to consume more energy during times of high ancillary service prices to provide more service despite the possibility of high energy prices or less-efficient operating conditions. A opportunity cost model was given in Ref.  that provides a rational goal for optimizing energy consumption, benefit, and ancillary service provision.
Given these revenue and cost results, providing ancillary services may not be attractive for many heat pump owners. Policy changes or price increases could have a positive impact on adoption. For example, CAISO doubled their regulation requirements in Feb. 2016 in response to increasing levels of intermittent renewable energy . This roughly tripled the regulation price, and it has continued to increase each year. Since previous studies referenced in this paper use now outdated price data, future price trends should be taken into account when assessing economic feasibility. Other policy changes that provide energy storage or demand response specific ancillary services such as PJM’s RegD and the pay-for-performance market structure could also play a part in increasing heat pump participation.
Heat pumps can be controlled to provide stability to the electrical grid in the form of ancillary services. These services range from response on the order of seconds to hours, and heat pumps can be paid for this provision. Local control of VSHPs and VAV HVAC systems has been experimentally shown to track the fastest ancillary service signal, frequency regulation. Aggregations of heat pumps have been numerically shown to be able to provide a variety of ancillary services. Heat pumps also have some key advantages compared with other energy storage systems and generators providing ancillary services, such as reduced costs, increased cycle life, and higher ramp rates.
While a large amount of research has proven the capability for heat pumps to provide ancillary services, there are still significant challenges to large-scale implementation. Recommendations for future research are as follows:
Experimental results are primarily on a local scale, controlling only a single heat pump rather than an aggregation. To our knowledge, there are no experimental heat pump aggregation studies. As a result, single-stage heat pumps, which represent a majority of residential heat pumps, have not been experimentally shown to be capable of providing ancillary services.
Dealing with uncertainty is vital for accurate forecasting of flexibility capacity and is relatively unstudied. Stochastic optimization techniques like robust and scenario-based optimization should also be considered when determining flexibility.
Aggregate control models, specifically for single-stage heat pumps, are relatively simple and do not capture the full dynamics of individual heat pumps and their buildings. Better parameter identification methods and higher order models that are scalable to heat pump aggregations could significantly improve flexibility estimation and ancillary service tracking.
Efficiency losses due to both ancillary service tracking and capacity scheduling are not completely understood. Gaps still remain between experimental and simulation results, and therefore RTE is not well defined. A high RTE is an underlying assumption in many control simulations and therefore has broad implications.
Communication latency issues are a significant barrier to frequency regulation since the system must respond on the order of seconds. Predictive methods or hardware retrofits could be a potential solution.
Revenue estimates are still quite low and represent a barrier to implementation. Trends in ancillary service prices should be considered, as well as new policy and incentive structures.
PJM is currently the only US operator that allows aggregation for frequency regulation participants.
The authors acknowledge the support from the National Science Foundation (NSF) (Grant No. 1711546).