United States Alaska VFR Sectional Charts VFR Terminal Area Charts Helicopter Route Charts IFR Enroute High Charts IFR Enroute Low Charts IFR Area Charts
Seattle Sectional Chart Great Falls Sectional Chart Billings Sectional Chart Twin Cities Sectional Chart Green Bay Sectional Chart Lake Huron Sectional Chart Montreal Sectional Chart Halifax Sectional Chart Klamath Falls Sectional Chart Salt Lake City Sectional Chart Cheyenne Sectional Chart Omaha Sectional Chart Chicago Sectional Chart Detroit Sectional Chart New York Sectional Chart San Francisco Sectional Chart Las Vegas Sectional Chart Denver Sectional Chart Wichita Sectional Chart Kansas City Sectional Chart St. Louis Sectional Chart Cincinnati Sectional Chart Washington Sectional Chart Los Angeles Sectional Chart Phoenix Sectional Chart Albuquerque Sectional Chart Dallas - Ft Worth Sectional Chart Memphis Sectional Chart Atlanta Sectional Chart Charlotte Sectional Chart El Paso Sectional Chart San Antonio Sectional Chart Houston Sectional Chart New Orleans Sectional Chart Jacksonville Sectional Chart Brownsville Sectional Chart Miami Sectional Chart Atlanta Terminal Area Chart Baltimore - Washington Terminal Area Chart Boston Terminal Area Chart Charlotte Terminal Area Chart Chicago Terminal Area Chart Cincinnati Terminal Area Chart Cleveland Terminal Area Chart Dallas - Ft Worth Terminal Area Chart Denver Terminal Area Chart Detroit Terminal Area Chart Houston Terminal Area Chart Kansas City Terminal Area Chart Las Vegas Terminal Area Chart Los Angeles Terminal Area Chart Memphis Terminal Area Chart Miami Terminal Area Chart Minneapolis - St Paul Terminal Area Chart New Orleans Terminal Area Chart New York Terminal Area Chart Philadelphia Terminal Area Chart Phoenix Terminal Area Chart Pittsburgh Terminal Area Chart St. Louis Terminal Area Chart Salt Lake City Terminal Area Chart San Diego Terminal Area Chart San Francisco Terminal Area Chart Seattle Terminal Area Chart Tampa Terminal Area Chart Orlando Terminal Area Chart Colorado Springs Terminal Area Chart Grand Canyon VFR Chart Enroute L-1 Enroute L-2 Enroute L-3 Enroute L-4 Enroute L-5 Enroute L-6 Enroute L-7 Enroute L-8 Enroute L-9 Enroute L-10 Enroute L-11 Enroute L-12 Enroute L-13 Enroute L-14 Enroute L-15 Enroute L-16 Enroute L-17 Enroute L-18 Enroute L-19 Enroute L-20 Enroute L-21 Enroute L-22 Enroute L-23 Enroute L-24 Enroute L-25 Enroute L-26 Enroute L-27 Enroute L-28 Enroute L-29 Enroute L-30 Enroute L-31 Enroute L-32 Enroute L-33 Enroute L-34 Enroute L-35 Enroute L-36 Enroute H-12 Enroute H-1 Enroute H-2 Enroute H-3 Enroute H-4 Enroute H-5 Enroute H-6 Enroute H-7 Enroute H-8 Enroute H-9 Enroute H-10 Enroute H-11 Atlanta Area Chart Chicago Area Chart Denver Area Chart Detroit Area Chart Dallas - Ft. Worth Area Chart Jacksonville Area Chart Kansas City Area Chart Los Angeles Area Chart Miami Area Chart Minneapolis - St. Paul Area Chart Phoenix Area Chart San Francisco Area Chart St. Louis Area Chart D.C. Area Chart New York Helicopter Chart Inset Boston Helicopter Chart Inset D.C. Helicopter Chart Inset Dallas Helicopter Chart Inset Salt Lake City Helicopter Chart Inset Ogden Helicopter Chart Inset New York Helicopter Chart Long Island Helicopter Chart Boston Helicopter Chart D.C. Helicopter Chart Baltimore Helicopter Chart Detroit Helicopter Chart Chicago Helicopter Chart Dallas Helicopter Chart Houston Helicopter Chart Los Angeles Helicopter Chart West Los Angeles Helicopter Chart East Salt Lake City Helicopter Chart Ogden Helicopter Chart Anchorage Sectional Chart Bethel Sectional Chart Cape Lisburne Sectional Chart Cold Bay Sectional Chart Dawson Sectional Chart Dutch Harbor Sectional Chart Fairbanks Sectional Chart Juneau Sectional Chart Ketchikan Sectional Chart Kodiak Sectional Chart McGrath Sectional Chart Nome Sectional Chart Point Barrow Sectional Chart Seward Sectional Chart Western Aleutian Islands Sectional Chart West Western Aleutian Islands Sectional Chart East Whitehorse Sectional Chart Fairbanks Terminal Area Chart Anchorage Terminal Area Chart Alaska Enroute L-3 Alaska Enroute L-4 Alaska Enroute L-1 Alaska Enroute L-2 East Alaska Enroute L-2 Central Alaska Enroute L-2 West Alaska Enroute H-1 Alaska Enroute H-2 Nome Area Chart Fairbanks Area Chart Anchorage Area Chart Juneau Area Chart
10 posts / 0 new
Last post
Dave
Flight Planning Math - Is it really this hard?

How far is it from Los Angeles to Boston?

It is a simple question, but finding the correct answer can be a challenging proposition.

A quick spin around some of the more popular online flight planning websites yields some surprisingly different results. Here's a sampling of some of the results we found. If your preferred flight planner is not listed here, please post your results in the comments below.

KLAX to KBOS

SkyVector.com 2269
FltPlan.com 2262
AirNav.com 2262.5
RunwayFinder.com 2262
FlightAware.com 2268
NavMonster.com 2262
AOPA Internet Flight Planner 2266
FlightPrep.com 2262.5
DUATS.com 2262.6
GCMap.com 2269

To be fair, all are within half a percent, so it would be wrong to claim any kind of safety-of-flight issue here. But we like accuracy, and shouldn't computers all agree on the basic math?

Why the difference? Basically, the results can be grouped into several groups.

Good: Salty Dogs

In the olde sailing days, when the "nautical mile" was invented, it was defined as a 60th of a degree. So if you think of the world as a sphere and do great circle trigonometry, you end up with distance defined in radians. Multiply by 180/pi and then by 60 and call it a nautical mile. This results in an answer near 2262.5, depending on your precision of pi.

Better: More Perfect Spheres

A more sophisticated approach still uses spherical trigonometry but a better definition of the size of the earth. Fédération Aéronautique Internationale has declared that the radius of the earth is 6371km for the purposes of measuring air records. If you programmed using the GEOS library, and used its "distance_sphere" function it probably used a radius of 6370.986km. In any case, you take your distance in radians and multiply it by the radius and you get your distance in kilometers. Then you just have to agree that a nautical mile is 1852 meters and you get a better answer.

Best: Ellipsoidal Models and Geodesic Formulae

For the last hundred years or so, cartographers have been using ellipsoidal models of the earth. The FAA uses GRS80, which was adopted into a worldwide standard in 1984 as WGS84. It defines the semi-major axis of the earth as 6,378,137.0 meters and the semi-minor axis as 6,356,752.314...meters (WGS84 and GRS80 differ by about a 10th of a millimeter on the semi-minor axis) Thanks to the late Thaddeus Vincenty, we have a very accurate way to calculate geodesic distances, Vincenty's Formulae.

The correct answer is 2268.97nm

We used COMPSYS 21, a free program published by the FAA. For inputs we used the 1111 edition of the National Flight Database, which is the only database published by the FAA that they certify as "fit-for-flight". The relevant two lines from the NFD are:

SUSAP KLAXK2ALAX     0     120YHN33563298W118242905E014000125         1800018000C    MNAR    LOS ANGELES INTL              034931111
SUSAP KBOSK6ABOS     0     100YHN42214670W071002310W016000020         1800018000C    MNAR    GENERAL EDWARD LAWRENCE LOGAN 070890804

We've highlighted the "Airport Reference Point." We believe that this is the correct point to use for the purposes of answering the question "How far is it from KLAX to KBOS"

Entering the values into COMPSYS 21 gives the following result:

Of the flight planners we tested today, only SkyVector and FlightAware are accurate within a nautical mile. The venerable old Great Circle Mapper by Karl Swartz is thrown in there because he's been gettin' it right for years.

Did we leave out your planner? Ask it the same question and post the results in the comments below.

kmhanson
WeatherMeister
weathermeister.com computes 2263.2
Dave
DTC Duat
DTC Duat (duat.com) measures it at 2263
ILSVector
Various Models
You can get an idea of what you get with the various different earth models here: http://williams.best.vwh.net/gccalc.htm
Dave
Great Link!
Thanks for posting it. I wish I had found it when I was writing the original article!
koehn
Foreflight
ForeFlight 4.2.2 comes up with 2,262nm.
Teninty
Flight Math

Hello,

 

I am new here, and I saw this thread...

 

I recently wrote two blog posts on Flight Math...

 

http://thestandardpilotlog.wordpress.com/2013/01/12/flight-math-part-1/

http://thestandardpilotlog.wordpress.com/2013/01/24/weight-and-balance-problem/

 

If you have questions, email me at tspl@outlook.com.

 

-Mike

Michael Teninty, CFI

www.TenintyAeronautics.com

TenintyAeronautics@outlook.com

EdLarkin
Just to muddy the water

I assume the calculations are done along the surface of the globe - but flying from LAX to BOS requires you climb over mountains and descend from altitude (which would change the total).

Then, of course, we have to ask where the start and end points are. Center of the airports? Nearest points of the airport environment to each other? And how were they defined? Fifth order survey, Google maps, Mark 1 eyeball? Again, this will change the total.

Just a little minutiae for your Saturday morning.

Dave
Excellent point!

The effect of mountains and cruising altitude is not taken into account. The point should be the "reference point" as defined by the FAA for both airports. 

LAX and BOS were chosen because LAX recently "moved." That is to say, shortly before this article was posted one of the runway thresholds was changed which caused the airport reference point to be recalculated. So this test was doubly challenging because the computing the right answer required both good data and good math.

Ultimately, the large differences came from using a Spherical instead of the more correct Ellipsoidal model of the earth. Among the Spherical models, some use the FAI sphere which is defines a earth with a radius of 6371000 meters. The worst of all models use the old "Sea Mile" rule which says that a nautical mile is a 60th of a degree. This is the equivalent of a spherical earth with a radius of 6366707 meters.

We admit we were having a little fun with this post, as the differences are in fact small and well within the recommended reserves of fuel that a prudent pilot will carry.

But should we, as pilots, be using tools that seem so unconcerned with precision that they are using mathematical models dating back hundreds of years that have been improved and refined many times since?

Even if you don't want to use the more complicated Vincenty formulae for ellipsoidal geodesy, shouldn't you at least use the more correct radius of the earth as adopted by the FAI? Surely any tool using "Sea Miles" is deserving of some public shame.

 

wingpilot66@gma...
Discontinuation of COMPSYS 21

Discontinuation of COMPSYS 21 Utility 
 

The Federal Aviation Administration (FAA) is no longer providing or supporting COMPSYS 21 Geodetic Calculations software. The National Geodetic Survey (NGS) provides computation utilities that formed the basis for FAA's COMPSYS 21. For more information, please see to NGS Geodetic Computation Utilities .