How far is it from Los Angeles to Boston?

It is a simple question, but finding the correct answer can be a challenging proposition.

A quick spin around some of the more popular online flight planning websites yields some surprisingly different results. Here's a sampling of some of the results we found. If your preferred flight planner is not listed here, please post your results in the comments below.

### KLAX to KBOS

SkyVector.com | 2269 |
---|---|

FltPlan.com | 2262 |

AirNav.com | 2262.5 |

RunwayFinder.com | 2262 |

FlightAware.com | 2268 |

NavMonster.com | 2262 |

AOPA Internet Flight Planner | 2266 |

FlightPrep.com | 2262.5 |

DUATS.com | 2262.6 |

GCMap.com | 2269 |

To be fair, all are within half a percent, so it would be wrong to claim any kind of safety-of-flight issue here. But we like accuracy, and shouldn't computers all agree on the basic math?

Why the difference? Basically, the results can be grouped into several groups.

## Good: Salty Dogs

In the olde sailing days, when the "nautical mile" was invented, it was defined as a 60th of a degree. So if you think of the world as a sphere and do great circle trigonometry, you end up with distance defined in radians. Multiply by 180/pi and then by 60 and call it a nautical mile. This results in an answer near 2262.5, depending on your precision of pi.

## Better: More Perfect Spheres

A more sophisticated approach still uses spherical trigonometry but a better definition of the size of the earth. Fédération Aéronautique Internationale has declared that the radius of the earth is 6371km for the purposes of measuring air records. If you programmed using the GEOS library, and used its "distance_sphere" function it probably used a radius of 6370.986km. In any case, you take your distance in radians and multiply it by the radius and you get your distance in kilometers. Then you just have to agree that a nautical mile is 1852 meters and you get a better answer.

## Best: Ellipsoidal Models and Geodesic Formulae

For the last hundred years or so, cartographers have been using ellipsoidal models of the earth. The FAA uses GRS80, which was adopted into a worldwide standard in 1984 as WGS84. It defines the semi-major axis of the earth as 6,378,137.0 meters and the semi-minor axis as 6,356,752.314...meters (WGS84 and GRS80 differ by about a 10th of a millimeter on the semi-minor axis) Thanks to the late Thaddeus Vincenty, we have a very accurate way to calculate geodesic distances, Vincenty's Formulae.

## The correct answer is 2268.97nm

We used COMPSYS 21, a free program published by the FAA. For inputs we used the 1111 edition of the National Flight Database, which is the only database published by the FAA that they certify as "fit-for-flight". The relevant two lines from the NFD are:

SUSAP KLAXK2ALAX 0 120YHN33563298W118242905E014000125 1800018000C MNAR LOS ANGELES INTL 034931111 SUSAP KBOSK6ABOS 0 100YHN42214670W071002310W016000020 1800018000C MNAR GENERAL EDWARD LAWRENCE LOGAN 070890804

We've highlighted the "Airport Reference Point." We believe that this is the correct point to use for the purposes of answering the question "How far is it from KLAX to KBOS"

Entering the values into COMPSYS 21 gives the following result:

Of the flight planners we tested today, only SkyVector and FlightAware are accurate within a nautical mile. The venerable old Great Circle Mapper by Karl Swartz is thrown in there because he's been gettin' it right for years.

Did we leave out your planner? Ask it the same question and post the results in the comments below.

Hello,

I am new here, and I saw this thread...

I recently wrote two blog posts on Flight Math...

http://thestandardpilotlog.wordpress.com/2013/01/12/flight-math-part-1/

http://thestandardpilotlog.wordpress.com/2013/01/24/weight-and-balance-problem/

If you have questions, email me at tspl@outlook.com.

-Mike

Michael Teninty, CFI

www.TenintyAeronautics.com

TenintyAeronautics@outlook.com

I assume the calculations are done along the surface of the globe - but flying from LAX to BOS requires you climb over mountains and descend from altitude (which would change the total).

Then, of course, we have to ask where the start and end points are. Center of the airports? Nearest points of the airport environment to each other? And how were they defined? Fifth order survey, Google maps, Mark 1 eyeball? Again, this will change the total.

Just a little minutiae for your Saturday morning.

The effect of mountains and cruising altitude is not taken into account. The point should be the "reference point" as defined by the FAA for both airports.

LAX and BOS were chosen because LAX recently "moved." That is to say, shortly before this article was posted one of the runway thresholds was changed which caused the airport reference point to be recalculated. So this test was doubly challenging because the computing the right answer required both good data and good math.

Ultimately, the large differences came from using a Spherical instead of the more correct Ellipsoidal model of the earth. Among the Spherical models, some use the FAI sphere which is defines a earth with a radius of 6371000 meters. The worst of all models use the old "Sea Mile" rule which says that a nautical mile is a 60th of a degree. This is the equivalent of a spherical earth with a radius of 6366707 meters.

We admit we were having a little fun with this post, as the differences are in fact small and well within the recommended reserves of fuel that a prudent pilot will carry.

But should we, as pilots, be using tools that seem so unconcerned with precision that they are using mathematical models dating back hundreds of years that have been improved and refined many times since?

Even if you don't want to use the more complicated Vincenty formulae for ellipsoidal geodesy, shouldn't you at least use the more correct radius of the earth as adopted by the FAI? Surely any tool using "Sea Miles" is deserving of some public shame.

Discontinuation of COMPSYS 21 Utility

The Federal Aviation Administration (FAA) is no longer providing or supporting COMPSYS 21 Geodetic Calculations software. The National Geodetic Survey (NGS) provides computation utilities that formed the basis for FAA's COMPSYS 21. For more information, please see to NGS Geodetic Computation Utilities .