Friday, May 27, 2016

GPS Frustrations

Most engineers obsess over measurement accuracy. In my line of work as a research engineer, we often push against the limits of what's doable. Sometimes independent verification of our results is required. It should not be surprising then, if some of this rubs off into tracking recreational pursuits.

I used a Garmin Edge 705 GPS unit for many years. I loved that unit. It produced impeccably accurate tracks in even adverse conditions.  Well sealed, the unit was not. After one too many deluges in Colorado, the 705 gave up the ghost.

I transitioned to a Garmin 510, which added Russian GLONASS reception to the US-based GPS constellation. I was pleased with the 510's track accuracy too, perhaps comparable to the 705. Under tree cover or heavy overcast, the 510 consistently produced smooth, tight tracks exactly where I went.

Garmin does have a reputation, though, and not all good. They are notorious for releasing buggy products to the market. Basically, they make consumers like me their beta test group even though we did not agree to provide this service. Software updates often come fast and furious for newly released Garmin products.

Some bugs (or maybe Garmin thinks they're features?) never seem to get fixed. I have a major peeve with my 510. It does everything I want it to for cycling, the activity the unit was designed for. But I also use the 510 for running, hiking and Nordic skiing.  During hiking, I've notice very large variation in my recorded distances, even for same hikes. One time I'll log 9mi, another time 11mi for exactly the same route. What gives?

I learned to turn off auto-pause when hiking. Sometimes you go so slow in technical sections that the unit will pause and stop logging data. However, upon probing into suspect track files, I noticed something peculiar. The 510 appears to stop distance tagging when speed drops below a certain value, say 2mph, but it continues logging lat/lon coordinates, barometric altimeter and time stamps. Only distance tag stays stuck. It appears turning off auto-pause only partially turns it off. The result is the distance tally displayed ends up way short for most hikes. Further, when posting to websites like Strava, Strava blindly follows the distance tagging and reports incorrect distance - most of the time. I've had a few instances where GPS distance was so short that Strava recognized a discrepancy, threw out the distance tagging, and recalculated actual distance from lat/lon coordinates.

So then I got smart and found a way to strip distance tags from my 510 hiking track files before posting to Strava, forcing Strava to compute distance from raw data. But this also stripped barometric altimeter data, which is far more accurate when hiking rugged terrain than what can be calculated from a digital elevation map that uses 30 meter or 100 meter grid. This situation sucked! I could either get accurate climbing or accurate distance, but never both!

Faced with this frustration, I decided to buy a running/hiking specific GPS. I picked up the recently introduced Suunto Traverse. It comes with barometric altimeter and tracks both GPS and GLONASS satellites. This sounded really good, although reviews were pretty negative on battery life. Suunto has a much better reputation for releasing new products with few bugs, unlike Garmin.

Suunto Traverse vs Garmin 510 on the wrist

My first test of the Suunto Traverse was on Mt Pack Monadnock, running repeats on the autoroad to the summit. I liked the less bulk on my wrist. The user interface is vastly different from Garmin and requires a learning curve. I couldn't wait to get home to see the results, of both GPS performance and my performance. I expected to have run a PR lap, and Strava was going to tell me this after uploading to their site.

First Suunto Traverse track. Only lap missed summit by hundreds of feet.

When I first saw the track, I thought this can't be right. It is so awful there must have been an error uploading. But no. I looked at it multiple ways. Track accuracy sucked donkey balls! Because of such poor positional accuracy, the climbing segment I was tracking my performance on was either missed completely or didn't start/stop in the right places. None of the three repeats I did had useful data. And I buried myself on the second one to see where my fitness was at! The Suunto didn't nuisance auto-pause on me though. Fix one problem, end up with an even bigger one.

So what should this track look like? Image below is from my Garmin 510 on the same course. It is almost perfect. Segments started and stopped where they were supposed to.


So maybe this was an anomaly. If a GPS gets a bad initial fix, it's accuracy can be FUBAR for the entire use cycle. So I gave it another chance on a lunch ride, off-road in the woods. This time I ran both my 510 and the Traverse. The results were revealing. This first image shows start and finish of my ride at the office. Blue is the Garmin 510, red the Suunto Traverse. I started both units at my car, and the fixes are within 5ft of each other. I was rolling when I stopped the two units coming back, so the 6 car spots difference is real.

This next photo shows open road tracking heading out and returning from my woods ride. Tracks are literally on top of each other and on the shoulder of the road exactly as I rode. So far, so good.

In the woods is where things got interesting. This first is a trail called Pipeline, which is mostly canopied in hardwoods. The leaves were just starting to bud out. Note there are many discrepancy areas. I've ridden this trail many, many times and can attest to the accuracy of the blue Garmin track, especially in upper left area of image. The trail comes out along a fence line. The red Suunto track comes up way short. Many other wiggles in the blue that are real, mysteriously get straight-lined out in the red Suunto. Why?

Here's another woods comparison. This is the Loop Trail in Horse Hill Preserve. Again, error on order of 100ft and smoothed out ripples.

And a final woods difference. Notice the S-curve near the tip of the lake on Duster Trail. It is accurately recorded by the blue Garmin track but missed completely by the red Suunto trace. The discrepancy grows very large later in the track as I traveled toward the upper right of the image. I had recording interval set to 1sec on the Suunto, the max rate. This is not at all what I expect from an expensive, latest generation GPS technology device.

As a radio-frequency engineer, I understand trees can attenuate GPS signals. I've always been selective on buying GPS units with high sensitivity, units with a lot of performance margin so they work well in less than ideal conditions. Antennas are an extremely important part of a GPS receiver. When you have a unit the size of the Garmin 510, or especially the 705 I used to have, you have some area to work in to make a decent antenna. These plots suggest the Suunto Traverse has marginal sensitivity. Get under trees where the signal becomes weaker, performance goes to crap. It was perfect on the road.

After poking around on Suunto sensitivity, I made some discoveries. The predecessor to the Traverse, the Suunto Ambit3, used an external nub to house the antenna. It protruded into the wrist band area. Apparently consumers thought this looked stupid. So Suunto moved the antenna into the bezel of the unit and TRASHED the receiver sensitivity. I'm an engineer. I want performance, not something that looks pretty and doesn't fucking work! I guess you could say consumers did it to themselves on this one. If you research this, you will find that Garmin wrist-watch products suffer similar poor performance.

This shouldn't be difficult to get right. If Garmin products weren't so buggy, I'd be happy continuing to wear the bulky 510 on my wrist. It offers so many benefits over a watch-sized unit anyway, like useful screen size for navigation.  I could move away from wrist unit altogether. This would likely get me the sensitivity I desire for accurate tracks. There'd be more product lines to chose from, maybe some less buggy. But where do you put it, say when you are running? XC skiing? The wrist is really convenient. The Traverse is going back. I may exchange it for a Suunto Ambit3. Track files for this unit look pretty clean, and the Ambit doesn't even use the Russian GLONASS constellation. Much longer battery life too. Maybe there is no such thing as a perfect GPS unit out there. Or maybe I'm just too picky...

9 comments:

Anonymous said...

There was a perfect GPS once, they just stopped making it. Tears will be shed when my 705 finally dies. Seems to me that GPS products, like most products, have become so overloaded with non-essential features they ruin the functionality of the devices real purpose, while simultaneously driving the price way up. I cringe every time something I own needs replacing, because I know that its replacement will have way too many features I don't want, will cost way more, and most definitely will not work as good as the old one I already have.

Scott Powell said...

Doug - Have you done any comparisons with an iphone or android phone running GPS? I'm curious how they stand up to a dedicated GPS unit.

Hill Junkie said...

Scott - the early smart phones were horrendously inaccurate, especially in mountains, under tree cover, or urban environments. I've never done a direct comparison on the same ride with my own iPhone. I ride with a lot of people that use the Strava app on smart phone. The Strava website tracks your performance on user community defined segments. Too often, smart phone positional accuracy is so poor that riders move to the top of a segment leaderboard even though they did not ride the segment nearly fast enough for it. What happens is your phone might position you behind where you really are, then catch up by the time you get to the end of a timed segment. The phone gives you a head start. This happens much less often with dedicated GPS units.

I recall seeing a run track along the Charles River recorded by smart phone. The track zig-zagged across the river! Fairly open there, but there are buildings that cause multi-path. I think GPS chips for phones are getting a little better now, but still not as good as low-end dedicated GPS units. Newer dedicated units use both GPS and GLONASS and can track a large number of satellites simultaneously.

Hill Junkie said...

I ran my iPhone 6 today with my Garmin Edge 510 bike GPS. The iPhone actually didn't do too terribly. iPhone distance was about 2% less than Garmin, and total climbing about 14% less. iPhones do not have barometric altimeters, so the coarse resolution digital elevation models are used to look up vertical from lat/lon coordinates. There are some pretty big deviations between the two tracks though. The route was fairly benign, non-mountainous, no canyons, but mostly tree covered.

Nancy said...

what type of devices people that do adventure racing (serious) or rescue or military are using? Maybe it is something like that you want? the orientation part is really important in adventure racing and that's what I read in Rebecca Rusch's biography. I don't know if the device will be like a watch.

I found that the wheel sensor is the most accurate for distance in MTB single track and not running auto-stop helps.

Hill Junkie said...

I don't think rescue or military are using small wrist devices. They are much bigger, bigger displays with more info, bigger antennas for robust performance in adverse conditions. These are handheld units. Outdoor adventure types tend to gravitate toward the slightly larger waterproof wrist units. Some of these work ok, but many are plagued by the same track accuracy problems as the Suunto Traverse I tried.

I do use wheel sensors on all my MTB and cross bikes. I do average of weighted roll-out tests to calibrate them. I get truth in distance, to within a percent or so. Doesn't help hiking or training on foot when you want accurate time for hill repeats. That is specific problem I'd like better performance on. Just ordered a Suunto Ambit3 Peak. There are several excellent track accuracy reviews on the web that show this one to be pretty good. We'll see.

R Evey said...

I own the traverse and Ambit 3 Sport. I pretty much run the same trails, and alternate with the watches. I would rather use the traverse because of the barometric altimeter, as the Sport records altitude by GPS. My last two runs at appx 6-730 pm have tracked like crap with the traverse. Freaking awful. I went back to the Sport and it was dead on. Of course altitude ascent and descent was off a couple hundred of feet. Not sure if it's the time of day with the traverse or what. Maybe I should shut off glonass.

Hill Junkie said...

Interesting. I've used the Ambit3 Peak several times now and have been very pleased with it. Finally get accurate distance measurements. Unit does not have GLONASS. The baro altimeter does a good job. I've seen comparison reports with GLONASS on and off, and in most cases, it makes almost no difference except in how much faster the battery dies with it on.

Anonymous said...

FYI, the GPS antenna is directly beneath the LCD screen, right beneath the bottom (South) data field.