Friday, May 27, 2016
Most engineers obsess over measurement accuracy. In my line of work as a research engineer, we often push against the limits of what's doable. Sometimes independent verification of our results is required. It should not be surprising then, if some of this rubs off into tracking recreational pursuits.
I used a Garmin Edge 705 GPS unit for many years. I loved that unit. It produced impeccably accurate tracks in even adverse conditions. Well sealed, the unit was not. After one too many deluges in Colorado, the 705 gave up the ghost.
I transitioned to a Garmin 510, which added Russian GLONASS reception to the US-based GPS constellation. I was pleased with the 510's track accuracy too, perhaps comparable to the 705. Under tree cover or heavy overcast, the 510 consistently produced smooth, tight tracks exactly where I went.
Garmin does have a reputation, though, and not all good. They are notorious for releasing buggy products to the market. Basically, they make consumers like me their beta test group even though we did not agree to provide this service. Software updates often come fast and furious for newly released Garmin products.
Some bugs (or maybe Garmin thinks they're features?) never seem to get fixed. I have a major peeve with my 510. It does everything I want it to for cycling, the activity the unit was designed for. But I also use the 510 for running, hiking and Nordic skiing. During hiking, I've notice very large variation in my recorded distances, even for same hikes. One time I'll log 9mi, another time 11mi for exactly the same route. What gives?
I learned to turn off auto-pause when hiking. Sometimes you go so slow in technical sections that the unit will pause and stop logging data. However, upon probing into suspect track files, I noticed something peculiar. The 510 appears to stop distance tagging when speed drops below a certain value, say 2mph, but it continues logging lat/lon coordinates, barometric altimeter and time stamps. Only distance tag stays stuck. It appears turning off auto-pause only partially turns it off. The result is the distance tally displayed ends up way short for most hikes. Further, when posting to websites like Strava, Strava blindly follows the distance tagging and reports incorrect distance - most of the time. I've had a few instances where GPS distance was so short that Strava recognized a discrepancy, threw out the distance tagging, and recalculated actual distance from lat/lon coordinates.
So then I got smart and found a way to strip distance tags from my 510 hiking track files before posting to Strava, forcing Strava to compute distance from raw data. But this also stripped barometric altimeter data, which is far more accurate when hiking rugged terrain than what can be calculated from a digital elevation map that uses 30 meter or 100 meter grid. This situation sucked! I could either get accurate climbing or accurate distance, but never both!
Faced with this frustration, I decided to buy a running/hiking specific GPS. I picked up the recently introduced Suunto Traverse. It comes with barometric altimeter and tracks both GPS and GLONASS satellites. This sounded really good, although reviews were pretty negative on battery life. Suunto has a much better reputation for releasing new products with few bugs, unlike Garmin.
My first test of the Suunto Traverse was on Mt Pack Monadnock, running repeats on the autoroad to the summit. I liked the less bulk on my wrist. The user interface is vastly different from Garmin and requires a learning curve. I couldn't wait to get home to see the results, of both GPS performance and my performance. I expected to have run a PR lap, and Strava was going to tell me this after uploading to their site.
When I first saw the track, I thought this can't be right. It is so awful there must have been an error uploading. But no. I looked at it multiple ways. Track accuracy sucked donkey balls! Because of such poor positional accuracy, the climbing segment I was tracking my performance on was either missed completely or didn't start/stop in the right places. None of the three repeats I did had useful data. And I buried myself on the second one to see where my fitness was at! The Suunto didn't nuisance auto-pause on me though. Fix one problem, end up with an even bigger one.
So what should this track look like? Image below is from my Garmin 510 on the same course. It is almost perfect. Segments started and stopped where they were supposed to.
So maybe this was an anomaly. If a GPS gets a bad initial fix, it's accuracy can be FUBAR for the entire use cycle. So I gave it another chance on a lunch ride, off-road in the woods. This time I ran both my 510 and the Traverse. The results were revealing. This first image shows start and finish of my ride at the office. Blue is the Garmin 510, red the Suunto Traverse. I started both units at my car, and the fixes are within 5ft of each other. I was rolling when I stopped the two units coming back, so the 6 car spots difference is real.
This next photo shows open road tracking heading out and returning from my woods ride. Tracks are literally on top of each other and on the shoulder of the road exactly as I rode. So far, so good.
In the woods is where things got interesting. This first is a trail called Pipeline, which is mostly canopied in hardwoods. The leaves were just starting to bud out. Note there are many discrepancy areas. I've ridden this trail many, many times and can attest to the accuracy of the blue Garmin track, especially in upper left area of image. The trail comes out along a fence line. The red Suunto track comes up way short. Many other wiggles in the blue that are real, mysteriously get straight-lined out in the red Suunto. Why?
Here's another woods comparison. This is the Loop Trail in Horse Hill Preserve. Again, error on order of 100ft and smoothed out ripples.
And a final woods difference. Notice the S-curve near the tip of the lake on Duster Trail. It is accurately recorded by the blue Garmin track but missed completely by the red Suunto trace. The discrepancy grows very large later in the track as I traveled toward the upper right of the image. I had recording interval set to 1sec on the Suunto, the max rate. This is not at all what I expect from an expensive, latest generation GPS technology device.
As a radio-frequency engineer, I understand trees can attenuate GPS signals. I've always been selective on buying GPS units with high sensitivity, units with a lot of performance margin so they work well in less than ideal conditions. Antennas are an extremely important part of a GPS receiver. When you have a unit the size of the Garmin 510, or especially the 705 I used to have, you have some area to work in to make a decent antenna. These plots suggest the Suunto Traverse has marginal sensitivity. Get under trees where the signal becomes weaker, performance goes to crap. It was perfect on the road.
After poking around on Suunto sensitivity, I made some discoveries. The predecessor to the Traverse, the Suunto Ambit3, used an external nub to house the antenna. It protruded into the wrist band area. Apparently consumers thought this looked stupid. So Suunto moved the antenna into the bezel of the unit and TRASHED the receiver sensitivity. I'm an engineer. I want performance, not something that looks pretty and doesn't fucking work! I guess you could say consumers did it to themselves on this one. If you research this, you will find that Garmin wrist-watch products suffer similar poor performance.
This shouldn't be difficult to get right. If Garmin products weren't so buggy, I'd be happy continuing to wear the bulky 510 on my wrist. It offers so many benefits over a watch-sized unit anyway, like useful screen size for navigation. I could move away from wrist unit altogether. This would likely get me the sensitivity I desire for accurate tracks. There'd be more product lines to chose from, maybe some less buggy. But where do you put it, say when you are running? XC skiing? The wrist is really convenient. The Traverse is going back. I may exchange it for a Suunto Ambit3. Track files for this unit look pretty clean, and the Ambit doesn't even use the Russian GLONASS constellation. Much longer battery life too. Maybe there is no such thing as a perfect GPS unit out there. Or maybe I'm just too picky...