Stephen Nimmo

Energy Trading, Risk Management and Software Development

Category: Retail Energy

Retail Power – You Can’t Manage What You Don’t Measure

The legend W. Edwards Deming said it best – “You Can’t Manage What You Don’t Measure”. When it comes to booking retail power deals it’s always good do so with the end measurements in mind. Without the proper organization of costs and the ability to allocate costs back to contracts on a per-kWh basis (or per kW if the cost is demand-driven), then the business can’t answer the fundamental questions it needs to be profitable.

  • When pricing a contract, how do we make sure the pricing estimates are both complete and accurate?
  • When servicing a contract, how do we identify events which could have a negative financial impact on the actuals and make adjustments to reduce or eliminate that risks?
  • How can we use past performance to help us identify ways to improve?

Let’s dive into a few particulars of the retail power industry to better understand how to organize the data. As with any industry, there are two types of associated costs: fixed and variable costs. The fixed costs are the obvious ones, such as rent or hardware. However, the variable costs have some different flavors that have correlations to how the power physically works. First, some assumptions needs to be verified. Energy delivery is componentized, and the components are ISO specific. There are themes across most ISOs, such as Energy or Transportation, but there are also ISO-specific components, such as Capacity, which need to be accounted for in some ISOs, but not others. For the sake of simplicity, we are going to assume that each ISO will be modeled independently, as trying to merge all ISOs into a single model doesn’t provide much benefit, it terms of data usefulness.

Next, whenever costs are discussed, it’s rarely a simple cost, like $.03 per kWh. Most usage charge estimates are modeled as an hourly price curve meaning the costs associated with a particular component of the energy are unique to the hour in which the costs happened, and can be different for every hour. Modeling a single usage cost for an entire year, with hourly granularity, creates 8,760 possible values. Some shops summarize at higher levels, such as on-peak/off-peak values per day, but that can lead to issues as spikes in a particular hour can be lost. When data is summarized, data granularity is compromised. However, keeping hourly granularity also has its costs due to the data management required. If a power product is broken down into 30 different components with hourly granularity, there is now 262,800 individual price points to deal with and doing multiplication for 8,760 estimated volumes across the 262,800 price points may have a performance impact.

The complications really begin when the model starts to enter into the actual power delivery. When pricing a contract, we are taking each one of the pricing components, multiplying those price estimates by the associated hourly measure (usage or demand), and coming out with the total estimated costs for delivery 2016-02-22(adding in fixed costs, expected profit, etc). The problem happens when the power is actually delivered because the granularity is lost. The load scheduled and subsequent invoice amounts for each of these components can come back in a aggregated fashion, which means I will need to break them down to get back to the same level of granularity used during the estimations to get actual feedback on how accurate the estimates were. If I have 12,000 contracts that represented 4 mWh of usage at 9:00 AM on Feb 22, 2016, I will want to break those invoice amounts to a per-kWh cost, and apply those costs back to each contract based on the individual meters actual usage (load ratio allocation). For simplicity sake, we won’t discuss specific schedules for individual large C&I contracts muddled into the measurements, or how some of the costs coming back may not be applicable to all contracts.

Now that some of the complications are known, let’s talk about some of the best practices to keep in mind when modeling retail power.

  • Loss of Granularity – whenever there is an aggregation, you are making a choice to possibly miss the ability to make adjustments in operations. To provide another example of aggregation, some shops will try to shrink the model by aggregating multiple components together in a single representation, such as a blanket “Ancillary Services” bucket or “Energy” category. If you price at a bucket level, the only thing you can compare is the buckets so if the “Ancillary Services” charge all of the sudden comes back double what the estimate is, there may be a lot of work in store for you if you want to determine what caused the issue.
  • Separation of Load – When possible, try to tag and categorize load as much as possible. At a bare minimum, every contract/meter can be tagged with its corresponding ISO, Utility, State, LoadZone and CapacityZone (if NYISO). From there, shops can continue to break down the load into smaller groupings but it’s usually unique to the organization. There are usually some breakdown between Residential/SMB contracts and Large contracts, because Resi/SMB tend to be a fixed price deal where larger contracts might fix or passthrough associated charges. The last tagging possibility is at the scheduling entity level, where REPs can schedule their loads by different scheduling entities or legal entities, resulting in multiple ISO invoices coming back related to each separate entity.
  • Flexibility – the model and all systems using the model need to be able to make adjustments without little cycle time. There are some cases where new, significant charges come up (we didn’t model that because it’s usually not relevant) or changes to how the charges are allocated. If a new customer wants to be able to price at a component level that you currently don’t support but your competitors do, then you will be forced to pass on the deal or make it work, making flexible models very important to larger deals.

Power To Choose needs an Upgrade

If you live in Texas, you know how important a good price is for your power. August temperatures can send your bills through the roof. A few years ago, the Public Utility Commission of Texas launched a website called PowerToChoose to serve as a centralized hub for all retail energy providers to post their offerings and give consumers the ability to shop around for their electricity. What started out as a great idea has now morphed into a pricing comparison site based on fine print tricks, giving the average customer little or no value at all when trying to actually to compare on price.

500, 1000, 2000

The original intent of the 500/1000/2000 metrics was to allow customers a quick glance into what the average price would be if they chose that particular deal. Simply, the 500/1000/2000 metric breaks down to apartment/small house/large house, and most consumers can quickly put themselves in the right bucket based on these metrics, pool owners notwithstanding. The real reason behind the 500/1000/2000 metric was due to normal pricing policies at the time, most retail energy providers charged a flat fee for consumers who used less than a certain amount of electricity. The retailers profits are normally built into the energy charge, but economic realities of the retailers fixed costs (people, office space, IT systems, etc) requires that for each contract processed, if a threshold of volume isn’t meant, they could literally lose money on the deal.

The flat services fees were nominal, but they had an impact on the 500/1000/2000 prices, as the flat fees were spread across the volumes in these prices. For example, if you got charged $10 as a flat fee and had 500 kWh of usage, then the $10 would look like $0.02 per kWh. However, if you use 1000 kWh, then it looks like $0.01 per kWh. If the energy charge was $0.05, then the 500/1000/2000 would be $0.07/0.06/0.065 respectively. However, the 500/1000/2000 prices don’t really give the consumer a good picture of what the costs of the entire contract will look like.

Reality of Energy Use2016-02-16 (2)

If you have a smart meter, you have access to all of your usage data, down to 15 minute increments (which no one uses, but that’s a story for another day). If you go to and register, you can access the data and print nice excel reports of your monthly usage, which is actually needed for a consumer to determine what the best deal is. Let’s take a look at my families’ usage last year.

Quite a range, from 508 kWh in Feb/Mar to a whopping 1824 in July/Aug. No pool, smallish home, built in 2005. The problem is the pricing – the 500/1000/2000 kWh prices are useless to me! Some months, the 500 kWh price is close to what I am paying, but for 5 months out of the year, I am using more than 1000 kWh. Without pulling my actual usage, and sitting down to do the math, the 500/1000/2000 representative prices don’t truly reflect what I am going to be charged.

1000 kWh Manipulation

The problem comes from the way the products are organized. When you put in your zip code and hit enter, the deals come up sorted by the 1000 kWh prices, lowest to highest. The retailer with the lowest 1000 kWh price is the winner, so to speak. They are the “top result” which is commonly gold when it comes to marketing. Using a parallel from Google, the result positioning for search can have a dramatic effect on “click through”, meaning the users will click on the top result 32.5% of the time and the top three get a whopping 61.5% of traffic share on Google. As a retailer, the PowerToChoose results that pop up after the user enters their zip code should hold the same value, where the top result is highly coveted, the top three a must and the first page essential. As a retailer, how do I go about getting these positions? Having the lowest 1000 kWh value wins and that’s where bill credits rear their ugly head.

Let’s look at the top three spots on Feb 16, 2016 at 7:00 AM for a downtown Houston zip code, 77002. The first result from Infuse Energy has a 3 month contract with the 1000 kWh price of $0.013 per kWh. Less than 2 cents for my power, let’s sign up immediately! Slow down, buddy. Let’s look at the reality of the contract otherwise known as the fact sheet. The fact sheet gives you what you really need to know about the contract you are signing. The fact sheet, in combination with your historical usage, can give you a very nice picture into what you’ll actually be paying. First, let’s assume you are free and clear of your previous contract and can sign with whoever you want. If I signed up for this three month contract on Feb 19 and used last year’s usage as the estimate, I would assume I would use 508, 725, and 909 kWh respectively over the next three months. But looking at the fact sheet, the price has a full paragraph with the use of bill credits – “$90 Usage Credit per billing cycle if your usage is greater than 999 and less than 1,501 kWh; $40 Usage Credit per billing cycle if your usage is greater than 1,500 and less than 2,001 kWh”. So according to their bill credit structure and assuming my usage holds true based on historical, then I wouldn’t get any bill credits and would end up paying closer to $0.11 cents per kWh – not anywhere close to $0.013 advertised by the 1000 kWh price. If over the next three months, you are using a little more than 1000 kWh every month, then this deal is awesome for you and you’ll get a rate closer to the 1.3 cent advertised price.

I hear what you are saying and I agree with you – caveat emptor. I should know what I am buying and ultimately, it’s up to me to understand the contract but for me to properly gauge the pricing and impact of the contract I am purchasing requires me to know my monthly usage for the past year and to take each and every prospective deal and map out the bill credits, minimum usage fees, service fees and other fine print items based on my usage patterns to determine what the best price is. If this is reality, then the only value PowerToChoose is to me is an aggregated place to find all the fact sheets.

What needs to change

The 500/1000/2000 values used for the ranking are ineffective and can be manipulated by retailers. They need to be replaced with a value in which consumers can do apples-to-apples comparisons across all the retailers in their service area. Assuming we are leaving out the one-off scenarios, such as contract termination fees or service cut fees, there are two different ways consumers can compare deals in an effective manner.

The more complicated but most effective way to accomplish this is to get PowerToChoose and SmartMeterTexas to create an interface to share usage data. As a retail customer, I would love to be able to log into PowerToChoose, give my ESIID or my service address and authorize PowerToChoose to pull my historical usage and apply that usage to the contracts available to show a total cost of the contract. If this is possible, then consumers would be able to compare contracts based on term (3, 6, 9, 12, 24 months) and do a true comparison between retailers. However, this is easier said than done. First, the interface to pull data from SmartMeterTexas would be difficult based on the current processes in place as SmartMeterTexas has some tight and unwieldy security requirements on sharing data outside of retailers themselves. Additionally, all of the fine print in the contracts would have to be standardized in a way to allow the software to do the math without having to parse the text of the contracts, which would be impossible using the current structures. Both would create fairly large changes to the ecosystem and without financial incentives to do so.

The second way would be to produce a standardized set of contracts for 3, 6, 12 and 24 months, where all of the baseline information is exactly the same across retailers except for the energy charge component of the price. Doing this would allow the multiple variable spreadsheet math needed by consumers today to be reduced down to comparing a single number between retailers, but to get a monthly estimate of the bill would still require some manual math using your historical usage.

Here’s an example of a standardized contract. Every retailer can offer the exact same contract terms and compete only on the highlighted energy charge.


2016-02-16 (3)


I don’t blame the retailers for the prices they publish. The residential and small business retail energy market is highly competitive and the ability to get a consumer to even click is highly sought after. As a retailer, you are almost forced to play the game, even if you don’t agree with the practice of using bill credits to manipulate the 1000 kWh price because of the value of being the “top 3” or “first page”. In a sometimes cutthroat and non-differentiable market, such as retail power, airlines, cars and others, companies are forced down to the lowest common denominator of marketing because not doing so, holding the proverbial moral high ground, brings the possibility of loss of market share, revenue and profitability – something that gets marketing people updating their resumes. Until the PUC decides to make some changes, the PowerToChoose website will continue to be a land of caveat emptor. It’s up to you to do the math.

Overview of the Retail Energy Broker Business

Most people in Texas buy their power through the unregulated market. This means there are two entities involved in the delivery of your house’s electricity: the retail electric provider (REP) and the Transmission Distribution Service Provider (TDSP).  The REP sources the power from the market, either purchasing the power directly from specific generators or by bidding in the open market for the power to be provided by a pool of available generators. They handle your contract trends, your billing based on the usage and all the other stuff related to payments or customer services. The TDSP and utility are responsible for the physical delivery of the power purchased by the REP across the grid through transmission lines and other physical assets shared by the market.

Why Energy Brokers?  

For residential customers, the process of choosing a REP can be fairly simple, usually handled by going to a REP’s website and signing up, or using one of the tools available like Residential customers power contracts are relatively simple, usually a fixed price per KWh – maybe with a few twists like free nights but overall, residential load is consistent and straightforward.  However, for some small businesses and larger commercial and industrial entities, their power usage is not as simple. They may have very particular needs for power management related to issues like large HVAC usage or load related to running large machinery. Because their needs are much more complex, they have more complicated contracts to give the ability to tailor their needs. In these cases, energy brokers come into the picture to help navigate these complexities.

Energy brokers can play many different roles for a customer, ranging from simply finding the best available prices from a variety of REPs to performing in depth consulting and analysis on the load characteristics of the business and translating those needs into an optimized contract for their customer, often negotiating with the REP in a detailed manner.

There are also different types of brokers, such as aggregators. These types of organizations consolidate load by gathering many similar entities, such as residences, together to bring a higher load to a particular REP. As with any business, the more potential revenue in a single transaction a company can bring, the more optimized it’s cost to serve can be, greatly reducing costs like cost to acquire. Aggregators use this to their advantage by creating a position of negotiation for the group and (hopefully) getting a better rate for everyone.


The compensation for the third parties is paid by the REP directly and is calculated as part of the cost to serve. The residential customer rarely sees any impact on their prices, however the larger C&I customers will see an impact on their bill, as the REP will usually pass through or incorporate the cost of the broker commissions into the fixed price portion of the billing statement.

Third parties are compensated using two main types of compensation: the flat fee and residual fees. The flat fee, usually associated with residential and small business contracts, is paid as a fixed amount after the contract is executed. These amounts are usually in the range of $25 to $100. These amounts are paid with the understanding the customer needs to remain a customer until the REP delivers enough power to break even on the costs. If a customer cancels or drops before this can happen, it may result in a “clawback” by the REP, where the commission paid will be returned due to the cancellation, perhaps on a pro-rata basis.  The residual fee is paid to the third party as a commission based on the customers actual usage. Designated in a Mill, which is 1/1000 of a cent, the commission rate is paid per KWh of usage. Let’s say a broker is contracted to receive 5 mills ($0.005) per KWh and the customer uses 10,000 KWh in the previous month, the broker would be then due a commission of $50, usually payable to the broker after the invoice is paid to ensure the REP can recoup the commission from the customer.

The Complications for REPs 

So far, the scenarios presented have been fairly straight forward and easy to manage. However, there are two other aspects to broker commission management that can make it very hard for REPs to make sure they are paying out the correct commissions: the prepayment and the Cancel-Rebill.  Let’s start with the prepayment. When signing up a new customer, the REP will request a usage history from the utility to determine the expected load. At this point, the REP has a good idea for existing customers of how much load to expect over the year and therefore, does have the ability to “prepay” the broker – usually at a discount” – for the value of the contract. Using our previous example of a 5 mill rate and 10,000 KWh per month, the REP and broker can estimate the value of the year’s commissions at $600 (10,000 KWh x 12 months x .005). The REP and broker can negotiate the commission and pay the entire sum at the beginning of the year at a discount, say 80% – meaning the REP will pay the broker $480.

But the prepayment comes with a true-up, meaning at the end of the year, the REP and broker will calculate the actual usage and what the actual commission payment would have been and make each other whole. Continuing on the example, let’s say the customer used double their expected volumes and the end commission payment would have been $1200. The REP would then pay the balance to the broker based on the non-discounted expected fees ($1200 – $600) resulting in a $600 payment to the broker for the true up. But remember, this can also work in reverse – if the customer only used half the expected amount then the broker may have to return some to the REP.

The other main complication with broker payments is the Cancel Rebill. Upon receipt of an invoice, if the customer does not believe the usage is correct, they can request that the utility re-reads the meter and recalculates the bill. In the REP billing system, this could result in multiple retracted invoices or invoice line items for a particular month, and new line items in a later month (after the re-read happens) that correspond to the flow from a previous month based on the new values. These processes take the processing cycles for residuals paid per invoice out of its customary cycle and can result in delayed payments to brokers while waiting for the new meter reads. Combine a Cancel Rebill at the end of the prepayment when a true up is due, and logic can get complicated.

Recommendations for REPs

Because broker related business can represent sometimes more than half of a REPs load, there are a few recommendations for best practices for REPs in handling the broker related business. First, give the brokers a portal to manage their data. Give them an easy way to search for customers data, for invoices and even export the data into spreadsheets for them to incorporate into their own systems for commission tracking. Second, find a standard set of commissions and resist any customization under the pressure of closing the sale. Most brokers are fine with the standard set of flat fees, residuals and prepayments and introducing new compensation models for individual brokers will result into complicate manual operations as the broker’s deals will have to be handled in one-offs. Last, create automated systems to pull contract events and invoicing events directly from the CRM and billing systems, and create commissions through software, not by spreadsheet. Spending development time now will pay dividends for years to come.

© 2017 Stephen Nimmo

Theme by Anders NorenUp ↑