Windows Power Plans and CPU Performance

Back in February, 2010, I did some preliminary testing on the effects of various BIOS settings and Windows Power Plans on processor performance in Windows Server 2008 R2 using the cross-platform Geekbench benchmark. That round of testing used an 2.8GHz Intel Core i7 860 processor in an ASUS desktop board. Back then, I measured a 22% increase in the Geekbench score when switching from the default “Balanced” power plan to the “High Performance”plan, which seemed pretty significant.

Now, I wanted to do some slightly more detailed testing on some actual server hardware, observing the Geekbench performance and measuring the actual power usage of the entire system with various combinations of BIOS settings and Windows Power Plans. My goal is to observe the performance effects of these settings vs. the effect on the actual power consumption of the entire server in order to be able to give better guidance on which settings you should choose depending on your performance and power objectives. I also wanted to measure the difference in terms of performance and power usage between having one physical CPU and two physical CPUs in an otherwise identical two socket server.

We just got in a couple of nearly identical Dell PowerEdge R410 1U servers that will eventually be used as middle-tier servers, so I took the opportunity to run some benchmark tests on them over the last couple of days as part of the burn-in process. Both of these servers have 16GB of DDR3 RAM, one 15K 146GB 3.5” SAS drive and dual power supplies. The only difference is that one server has two Intel Xeon E5620 processors, while the other server only has one Intel Xeon E5620 processor. The 32nm Intel Xeon E5620 Westmere-EP has a rated speed of 2.4GHz, with the ability to go to 2.66GHz with Turbo Boost. It is a quad-core processor that also supports hyper-threading, so the operating system can see eight logical processors when hyper-threading is enabled.

Below is a screenshot of CPU-Z, showing an Intel Xeon X5670 running with the Windows High Performance Power plan, which allows Intel Turbo Boost to kick in and raise the clock speed of individual cores from 2.93GHz to 3.32GHz.

 

This is the much more humble Intel Xeon E5620, showing the effects of Turbo Boost raising the clock speed from 2.40GHz to 2.66GHz.

In the main BIOS setup screen of the Dell R410, you can go into the Power Management section and change the setting to OS Control, which lets Windows manage the various power settings more completely. In the Processor Settings section, you can change whether Turbo Mode is enabled (the default is enabled) and whether Logical Processor is enabled (the default is enabled). You can also change whether the C1E and C States settings are enabled or disabled. These settings relate to the power saving settings that are built-in to all recent Intel processors. As energy efficiency becomes more of a mainstream concern, each new generation of processors from both Intel and AMD have become more energy efficient and have more power management features included.

In Windows Server 2008 and 2008 R2, you can get to the Power Options screen by typing powercfg.cpl in a Run prompt. There you will see three default power plans, called Balanced, Power saver, and High performance. By default, it is set to Balanced, which is what Microsoft recommends for general use. Unfortunately, the default Balanced power plan is not the best choice for database server performance.

If you run the CPU-Z utility, you can see the core speed and multiplier of Core #0 of any of your physical processors in a nearly real-time basis as you change Windows Power plans and run benchmarks. This is very useful in understanding what is going on with the processor(s) based on the power settings and the processor load.

I also used a Kill-A-Watt meter to measure the actual power usage (in watts) of the entire server during these tests. It is interesting what happens to the power usage of a modern server as you turn it on, watch it go though the POST process and boot into Windows. There are a lot of other things using power in the server besides the processors, so you have a baseline usage that exists despite anything that you do with processor power management.

The single processor Dell PowerEdge R410 peaked at 144 watts during boot and then settled down to an average usage of about 95 watts after Windows finished loading and became idle, regardless of the Windows Power Plan that was being used and regardless of whether hyper-threading or turbo boost were enabled. When the processor usage peaked during the Geekbench test runs, I saw a peak power usage of 147 watts with the Power saver power plan, 169 watts with the Balanced power plan, and 180 watts with the High Performance power plan (with both hyper-threading and Turbo Boost enabled). This means that you are only saving 33 watts of power (which is 18%) when the processor is pegged and none at idle, and taking a 32% decrease in performance when you go from the High Performance power plan to the Power saver power plan.

The two processor Dell PowerEdge R410 peaked at 174 watts during boot and then settled down to an average usage of about 110 watts after Windows finished loading and became idle, regardless of the Windows Power Plan that was being used and regardless of whether hyper-threading or turbo boost were enabled. When the processor usage peaked during the Geekbench test runs, I saw a peak power usage of 200 watts with the Power saver power plan, 217 watts with the Balanced power plan, and 244 watts with the High Performance power plan (with both hyper-threading and Turbo Boost enabled). This means that you are only saving 44 watts of power (which is 18%) when the processor is pegged and none at idle, and taking a 32% decrease in performance when you go from the High Performance power plan to the Power saver power plan.

Below are the detailed results from four runs of the Geekbench benchmark on each system, with various combinations of hyper-threading, Turbo Boost, and Windows Power Plans. You can see that both hyper-threading and Turbo Boost are quite effective in this benchmark, and that your choice of which Windows Power Plan to use is also very significant.

Windows Power Plan Effect on Geekbench Performance (Details)

One Physical CPU

Hyper-threading disabled, Turbo Boost disabled (One CPU)

Power Plan	Run 1	Run 2	Run 3	Run 4	Average
Power saver	3883	3868	3882	3877	3877.5
Balanced	4871	4831	4816	4841	4839.7
High Performance	5319	5312	5323	5321	5318.7

 

Hyper-threading disabled, Turbo Boost enabled (One CPU)

Power Plan	Run 1	Run 2	Run 3	Run 4	Average
Power saver	3878	3878	3876	3879	3877.8
Balanced	4850	4807	4789	4795	4810.3
High Performance	5704	5706	5706	5711	5606.8

 

Hyper-threading enabled, Turbo Boost disabled (One CPU)

Power Plan	Run 1	Run 2	Run 3	Run 4	Average
Power saver	4739	4742	4692	4743	4729
Balanced	6038	6034	6002	6040	6028.5
High Performance	6533	6521	6519	6546	6529.8

 

Hyper-threading enabled, Turbo Boost enabled (One CPU)

Power Plan	Run 1	Run 2	Run 3	Run 4	Average
Power saver	4742	4743	4751	4750	4746.5
Balanced	6080	6076	6062	6104	6080.5
High Performance	6969	6980	6966	6970	6971.3

 

Two Physical CPUs

Hyper-threading disabled, Turbo Boost disabled (Two CPUs)

Power Plan	Run 1	Run 2	Run 3	Run 4	Average
Power saver	5907	5897	5965	5951	5930
Balanced	6521	6711	6758	6513	6625.8
High Performance	8115	8159	8167	7823	8066

 

Hyper-threading disabled, Turbo Boost enabled (Two CPUs)

Power Plan	Run 1	Run 2	Run 3	Run 4	Average
Power saver	6007	5889	5965	5997	4729
Balanced	6872	6477	6671	6726	6686.5
High Performance	8676	8882	8871	8898	8831.8

 

Hyper-threading enabled, Turbo Boost disabled (Two CPUs)

Power Plan	Run 1	Run 2	Run 3	Run 4	Average
Power saver	7737	7697	7519	7728	7670.5
Balanced	9000	8917	9372	8944	9058.3
High Performance	10562	10541	10578	10552	10558.3

 

Hyper-threading enabled, Turbo Boost enabled (Two CPUs)

Power Plan	Run 1	Run 2	Run 3	Run 4	Average
Power saver	7718	7350	7717	7524	7577.3
Balanced	9014	9374	8909	9140	9109.3
High Performance	11147	11161	11146	11175	11157.3

 

The numbers in the tables are the Geekbench scores.

 

Summary of Hyper-threading and Turbo Boost Effects on Geekbench Performance

One Physical CPU
With hyper-threading disabled and turbo boost disabled, enabling just turbo boost increases score by 7.3%
With hyper-threading disabled and turbo boost disabled, enabling just hyper-threading increases score by 22.8%
With hyper-threading disabled, and turbo boost disabled, enabling both hyper-threading and turbo boost increases score by 31.1%

Two Physical CPUs
With hyper-threading disabled and turbo boost disabled, enabling just turbo boost increases score by 9.5%
With hyper-threading disabled and turbo boost disabled, enabling just hyper-threading increases score by 30.9%
With hyper-threading disabled, and turbo boost disabled, enabling both hyper-threading and turbo boost increases score by 38.3%

Summary of Windows Power Plan Effect on Geekbench Performance

One Physical CPU
With hyper-threading enabled and turbo boost enabled, using the Power saver power plan decreases score by 21.9%
With hyper-threading enabled and turbo boost enabled, using the High Performance power plan increases score by 14.6%

Two Physical CPUs
With hyper-threading enabled and turbo boost enabled, using the Power saver power plan decreases score by 16.8%
With hyper-threading enabled and turbo boost enabled, using the High Performance power plan increases score by 22.4%

The differences in the Geekbench scores due to the different Windows power plans are compared to the default Balanced power plan.

My conclusion based on these tests is that you should always enable hyper-threading and Turbo Boost, and that you should make sure the BIOS is set to use OS power management. You should also make sure to use the High Performance Windows Power Plan instead of the default Balanced Power Plan. The power savings you get from using a more “green” power plan are pretty negligible, especially compared to the performance loss you incur. This would be more significant for an OLTP workload, which is usually more CPU dependent (assuming you have enough I/O capacity to support your workload).

In really pains me to make that recommendation, since I think reducing electrical usage in data centers is a worthy goal, for both economic and environmental reasons. Using power saving settings makes a lot more sense for middle-tier servers than it does for a database server. Upgrading to new hardware, and consolidating multiple older servers into fewer new servers is a good strategey for reducing the power usage of your database servers.

AnandTech did some testing back in July that showed that using one of the special low power usage processors instead of a performance processor did not save very much power and caused a very significant decrease in performance.