The point of the thermostat is to control your AC/HEAT automatically to keep the temperature constant inside. So when it's nice inside, the thermostat will turn the heat off and when it's chilly inside, it will turn the heat on.
In this case, you're playing the role of the thermostat. Maybe that's what you call geek factor these days.
1) The point of every thermostat is to control the AC/heat. Turn the heat on when it gets too cold and turn the AC on when it gets too hot. The Nest doesn't break any new ground here.
2) The Nest is NOT designed to keep the house at a constant temperature. Most people either don't have or don't use a programmable thermostat, and do set what they have to a constant temp. This results in a lot of extra usage, such as the AC/heat running when nobody is home. The point of the Nest is to run only when necessary. Personally I used to keep the house at a constant 69F. Now that I have a Nest, I let the temp drop to 64F at night or when I'm not a home. Saves quite a bit of money.
3) The remote control feature of the Nest is one of it's biggest features and main selling points. Suppose your Nest has learned to heat your house to 70F by 5:30p when you get home from work. Suppose you leave early one day and would rather come home to a warm house rather than waiting until 5:30p for the heat to kick on. This is where the remote control feature is used.
4) You're SUPPOSED to adjust your Nest, either in person or remotely. The entire point of the Nest is that it learns your schedule and will learn to adjust itself accordingly. During the first two weeks or so, you adjust the temperature whenever you need to, such as leaving for work, coming home, night time, etc. The Nest will learn your behavior and begin to set it's own schedule based on that. Otherwise, you can just program a schedule into the Nest and never touch it again. However, this would be missing the point of a learning thermostat.
I've had my Nest (2nd gen) for about four months now and LOVE it. It's really made me conscious of how much I'm running my furnace. Also, the first two gas bills I've had since installing the Nest have been lower than the previous year, and not just by price, but by MCFs used. This despite the fact the average temperature has been lower this year than last. I expect to see similar results in the summer.
At this price, even for a 1st gen, I'd recommend one.