In user test, Nest thermostat -- and its 'smart' competitors -- disappoints

A comprehensive study of how people use thermostats has found that the one by Nest Labs Inc. -- the sleek unit that is supposed to be drop-dead easy to use -- fell in the middle of the pack on many criteria and has a design flaw that gave it a near-bottom rating for usability.

Another important conclusion in the study of "smart" thermostats is that none was all that intuitive or helped users easily adjust settings in ways that save energy.

The study was carried out last year by the Sacramento Municipal Utility District (SMUD) in California, where the thermostat plays an especially important role in energy use. Residential thermostats control one-quarter of all energy consumption, far more than any California utility. On a hot summer day in SMUD's service area in the Central Valley, home air conditioning accounts for about a third of the utility's 300-gigawatt peak demand.

"What we found is that customers really do have a challenge using thermostats. They aren't able to use them very well, they find the interface nonintuitive," said Lupe Jimenez, a senior project manager at SMUD who managed the study. "You'd think by now the industry would have gotten around to it."

A new generation of "smart" thermostats wirelessly connect to other devices in the home. They are being sold by alarm companies, big-box stores and cable providers with the promise of managing energy use without much input from the occupant (EnergyWire, April 14). However, the study makes clear that the new connected thermostats aren't a lot easier to manually program than the old "dumb" thermostats.

The results for Nest were surprising, especially given that the company's founder and CEO, Tony Fadell, led the design of the original iPod and brought Apple's design aesthetic to Nest. In February, Google acquired Nest for $3.2 billion.

The survey was nothing if not comprehensive. A total of 163 people in SMUD's service area participated, selected to represent a balanced sampling of gender, age, race, income, education and home ownership. Participants were videotaped to measure how long it took to perform tasks with a thermostat, filled out questionnaires and participated in follow-up interviews.

The study looked at the "walk-up usability" of a thermostat -- in other words, without any guidance or a manual. Ten "smart" thermostats were in the study, along with two "dumb" thermostats. Each participant compared two thermostats side by side, and each thermostat was evaluated by at least 26 people.

The overall winner was the Carrier ComfortChoice Touch, which came in first in both overall ease of use and overall feel and sound. Ninety-one percent of those who tried it called it their favorite thermostat. Runners-up were the Ecobee Smart Si and Emerson Smart Energy.

Pitfalls for the Nest

In overall ease of use, Nest came in 11th, second from the bottom, behind even the two "dumb" thermostats in the competition. In overall feel and sound, it came in seventh. In overall appearance, Nest came in fourth. On the rating of "task efficiency" -- how long it takes to complete a certain function -- Nest came in second from last.

The study found that most of the dissatisfaction with Nest came down to one feature: the dial. Unlike all other thermostats tested that are boxes controlled by buttons, the Nest is essentially a circle that is manipulated by pushing and twisting the entire unit.

"More than half of the participants that tested the Nest -- 16 of 28 -- were unable to figure out the input mechanism at all or until the very end. Because of this, the Nest garnered a very low 38% Task Efficiency score," the study noted.

It added, "Removing those frustrated users who couldn't figure out the dial, the efficiency score changed to put Nest in first for efficiency and fourth for preference."

The ability to use a thermostat without a manual is crucial in managing the energy use of the many homes where the thermostat is already on the wall when a new renter or homeowner moves in.

"A customer who buys a Nest and brings it home is probably not going to have the problem. But if someone moves into that home, that customer might not be able to figure it out," Jimenez said.

Participants in the study said they liked that the Nest was sleek and modern-looking and thought the dial and the app were easy to use. On the other hand, they thought that the screen was too small, that the menu was hard to decipher and that it was hard to get started.


Jimenez said that SMUD would base its future acquisitions of thermostats on the results of this study. Moreover, she hoped that it would nudge the entire thermostat industry to make user experience a priority.

"The question remains ... whether the new thermostats will be used in a way that actually helps customers use less energy," the study said. "While it's too soon [to] pass judgment on the far end of the communications path, we can say with some certainty that these new standards will not effect energy savings if customers don't like or can't figure out how to use the new thermostats."

Twitter: @DavidFerris | Email:

Like what you see?

We thought you might.

Start a free trial now.

Get access to our comprehensive, daily coverage of energy and environmental politics and policy.



Latest Selected Headlines

More headlinesMore headlines

More headlinesMore headlines

More headlinesMore headlines

More headlinesMore headlines