Monday, July 30, 2012

On GPS Units and MP3 Players

I took my son, who was on leave from the Navy, over to Great Brook for a little trail riding Sunday. Some overnight rain and 100% humidity were sure to make things interesting, as Aaron hadn't even ridden a bike in a long time. Clipless pedals, slicker than snot roots and rocks? Apparently no problem for Aaron. I'm just getting old, frail and worry too much.

Beaver Loop

Anyway, I took the opportunity to run an experiment. I have two GPSs, an Edge 500 and an Edge 705. After owning the 705 for several years now, I finally got around to mounting this cadence/speed sensor doohickey that came with it. It works wirelessly via Ant+ Sport. In theory, this should greatly improve distance measuring when riding off-road, plus keep better track of elapsed moving time. Also on the bike with the 705, I still had a wired Cateye cyclocomputer. Thus if Aaron and I stayed together, we should measure the same distance on three different devices, two GPSs and one wired computer.

We pretty much rode all of the singletrack GB has to offer, including Stone Rowe towards the end.  Aaron bombed right down the rock chute no problem, although the numerous rock step-ups on the way there gave him grief.

Stone Rowe

The experiment came out as expected. The 705 with wheel pick-up sensor and wired computer (also with magnet sensor) were within 0.6% of each other for time and distance. The 500, on the bike I rode, fell nearly 12% behind in distance. The GB trails aren't super dense either. The 500 was relying purely on satellite signal to measure traveled distance.

GPSs are really good at only one thing: placing your latitude and longitude position on planet Earth. A few meters out of 40,000km is 0.00001% accuracy.  They are much less accurate in determining elevation. You'd think distance measuring would be pretty accurate, but GPS vendors use algorithms to reduce data storage and smooth out noise that may be present in the data. This can result in severely undermeasuring travelled distance in certain cases.

When traveling on mostly straight roads, GPSs can reach 1% accuracy, on par with car odometers and wired cycle computers. In tight, twisty terrain, I've seen errors as high as 30%, where the GPS always comes up short.

That is why GPS companies offer auxiliary wheel sensors, as well as optional barometric altimeters to improve elevation measurement. The wheel sensor replaces rounded off, noisy motion data from satellite signals with counted tire revolutions on dirt.

You'd think with all our technological marvels, that things like measuring distance would monotonically improve with time. But no. Many things in the realm of humans don't behave that way. Take, for example, music reproduction. We started with wax cylinders, then vinyl records. 8-track and cassette tapes were a step backwards, but they did not displace vinyl. Then came along audio CDs. They replaced everything up until that point. Some argued sound reproduced from a compact disk was inferior to a quality pressed vinyl record. I would tend to agree, although vinyl records quickly degraded with playings. In most play-back scenarios, CDs were superior. Then came the real digital revolution: MP3 players. Music reproduction quality took a huge setback. You see, people's attitudes changed. They wanted quantity over quality. 1000's of songs on a tiny, cheap device? You betcha. It didn't matter if the music was compressed beyond recognizability. That is the primary reason I still do not own a MP3 player. Memory is getting cheap enough now, and there are decent lossless compression algorithms out there, so I may jump in at some point, some 10-15 years after these things became hot.

And so it goes with measuring sporting endeavors. The chronograph was invented in the 1800's. It had accuracy to about 5 seconds per day, or 0.006%, and resolution to 1/100th of a second. Say you were running a marked mile on a track. This chronograph from nearly a century and a half ago could time your mile to within a tiny fraction of a percent. It was a purely mechanical device.

Growing up in the early 70's, I recall having a mechanical odometer on my bike. It mounted on the fork, and a spoke stud struck a little cog on the odometer. It was pre-calibrated for a 27" wheel. I bet it was accurate to within 1-2%. Very simple, very reliable. These device actually came into existence about the same time the bicycle did in the late 1800's, like this example below. Each tick is a hundredth of a mile!

In my generation, miniature electronic timing devices arrived on the scene. Timing accuracy improved by a couple orders of magnitude over mechanical movement chronographs. In addition, you could record distance and time in a single device and have it compute average speed. Carefully calibrating these devices to your tire size could net distance measurement accuracies to about 0.5%.

Now we're in the world of GPS receivers. Wired and wireless cyclocomputers are going bye-bye. Most riders who use metrics now use GPS units. Distance and moving time are accepted as accurate, which is sometimes true on the road. Those who ride or run off-road, have learned just how inaccurate these devices can be.

We've entered an era where metric gathering accuracy on average has degraded. It seems the other features of these new techno-marvels outweigh the loss in accuracy. We can record our tracks and upload them to Strava, where you are ranked against other riders' equally suspect tracks. Music fans wanted selection of tunes over quality of reproduction in MP3 players, and now cyclists prefer suspect GPS data for Strava entertainment over benchmark quality data. Maybe it is the case that music fans never really cared about "hi-fi" in the first place, and cleaver marketing tricked them into thinking it was important for a while. Perhaps cyclists never really cared about accurate average speeds or distances they rode. They were the only metrics we could gather and talk about at the time. There was no easy way to automatically capture other metrics like VAM unil GPSs and the web came along. That created a bigger sparkle in our eyes even though accuracy became suspect.

Not sure where things will go from here. Maybe memory will become so ubiquitous that music will no longer have to be compressed. Or maybe compression algorithms will become so good that even snooty music fans like myself will not be able to discern imperfections.  Will music fans start pushing for quality sounding music over quantity? This will need to happen to drive the market.

Likewise with GPS technology. What will it take to improve accuracy for applications the system was never intended (original and still primary use is military)? A new, commercial generation of satellites with much tighter timing accuracy, and more of them so there are always a few satellites almost directly overhead?  This would require a hefty subscription fee from millions of users to fund it.   GPS receiver manufacturers would have to improve their algorithms too. You can bet the US DoD will not improve their Global Positioning Satellite system for spandex clad whining cyclists, or even let the commercial world tap into the higher levels of accuracy the current system offers.  We are lucky enough the military makes an unclassified signal available at all.  For all we know, the next generation system may already be up there.


Rami said...

Doug, I believe the restriction on GPS accuracy is intentional per (take a guess) government regulations. Civilian GPS units only have access to Coarse/Acquisition (C/A) code. This may have up to 20m error. Meanwhile, the Precision (P) code, reserved for military use, can get very high accuracy, close to within 1m.

Anonymous said...

I'd bet if you did another test you'd find your 500 GPS is far less accurate then your 705 GPS. Using GPS measurement for both (no wheel sensors) I have both, and the 500 is useless for singletrack distance measurement. My 705 usually measures about 10% less distance than a wired Cateye I have. The 500 is more like 20-25% Not sure why, from my research both units supposedly use the same GPS receiver...

Hill Junkie said...

Rami - that doesn't account for why commercial GPSs so undermeasure distance. My GPS knows I'm walking 2.5mph, but if I walk in a small circle, it just doesn't measure much distance. You'd think Garmin and others would be able to do a better job integrating that velocity vector, which can be determined very accurately, to get better distance. These things never over measure. There is a clear bias. BTW, looks like the next gen system starts launching in 2014.

Anon - There are references out there that suggest the 500 uses the newer SiRFStar IV GPS chipset. The 705 was touted as having a super sensitive antenna, and with its bigger volume, there is more area to work with. I suspect the differences in tracking performance are due to difference in antenna sensitivities, although the new chipset is supposed to be better at resolving multipath. I've never run the 500 and 705 together on the FOMBA trails, but I do know my 705 undermeasured 20-30% there relative to wired.

CB2 said...

I've run my 500 and 305 simultaneously; the 305 recorded a higher mileage despite being in my pocket (the 500 was on my stem).

Mookie said...

Ah, now I see why CB2 is hooking up the GSC.. Problem is when I upload off-road track to Strava while using speed sensor, it ignores it and uses GPS. This is also why I question the credibility of data uploaded via phone apps.

AK said...

Doug, I think this is why we should all take the Strava stuff with a grain of salt. I regularly ride with people who have various degrees of GPS instrumentation and we always have conflicting results, on the magnitudes of which you write about. Cycling is a sport where people get scientifically meticulous about counting grams and measuring this and that, but we shouldn't loose sight of the real goal which is to have fun and stay healthy in doing so. Really we are all amateurs, and the real prize is the going, not the getting.

CB2 said...

Right on AK!

Mark said...

If you want to see a good demonstration of the inaccuracies of GPS, pick a small (< 5mi) single-track loop and do multiple laps. It can vary to a very surprising degree. The forest canopy is known to create problems with the GPS signal. On top of that, if you add different GPS units to the mix it can become even more variable. Without post-processing of the GPS data against values from a known, fixed location, significant errors are almost a given. Unfortunately, elevation "accuracy" is even worse but that is inherent in GPS data. All this was even worse before the military turned off their "Selective Availability" ie. the intentional errors that used to be introduced in the GPS signal. I trust my properly calibrated wired cycle computer way more than any GPS data.

That said, I find the ability to store the approximate route I ride to be very useful at times. It's also quite fun to compete with others on Strava but I'm not relying on that to be truly accurate.

Being a little bit of music snob I have some of the same concerns with audio files. I've found that mp3s can actually provide acceptable quality audio as long as they are encoded at a reasonable bit-rate - even for classical and jazz. Below 192k the compression artifacts are evident and can be very distracting. Above that level I am not able to discern the difference between the compressed file and the original CD. It's not all bad, although I feel that your assessment of the trends is pretty accurate overall. The fact that many of recent music releases have had their dynamic range seriously reduced is more concerning but that's another rant.

BTW, I used to work as a GIS/GPS tech.

mkr said...

We just started using Ant+ sensors for the GPS on the MTBs as well. Sick of running redundant devices and having Strava devalue your efforts :)

MJC said...

For what it's worth, some types of GPS error (notably undermeasurement on trails) doesn't really affect Strava segment competition since leaderboards are ranked by time and not average speed. With segments, the GPS data is really only used for 2 things:

(1) recognizing a segment
(2) determining the elapsed time

You can have a track with tons of error, but so long as the segment timer starts and stops at the right points, you'll end up in the right place on the leaderboard. Track error in between doesn't matter as long as the data is good enough for Strava to recognize that you've actually ridden the segment.

Hill Junkie said...

MKR - I hear ya on not getting credit for your efforts. Only deal is, in two rides, my sensor stopped working! Argh! I'm borderline removing all metrics devices as AK hinted.

MJC - I've seen some pretty bizarre Strava behavior, as many have. You can actually deviate a long way off a segment and come back to it, and still get credit as doing it. I assume this works both ways, adding and subtracting distance from a segment by cutting it. This is a Strava problem, not GPS. Also, Strava fails to detect starts or stops of segments frequently, especially those with iPhone apps. Thus no credit is given. Hard to say if this is Strava or GPS problem. But point is holds in most cases I think. Must segments are up a hill say, and if there was a shorter, steeper way to do it, that would be how someone would define it. There are not many segments that can easily be cut.

jason_ssc1 said...

I like precise data and therefore continue to use wired cycle computers. I doubt they will ever completely go away.