Help:Calculation accuracy

From Meta, a Wikimedia project coordination wiki
Jump to: navigation, search
The accuracy and format of numeric results varies with the operating system of the server. Some remarks on this page are specific for Wikimedia.

Floats[edit]

As long as no conversion to integer type is carried out by trunc or mod, intermediate results of #expr and #ifexpr (such as q*r in p+q*r, but also the factors 2.1 and 1e-3 in 2.1e-3) are stored as float, i.e. in double precision floating-point format, so they are the integers up to 253=9,007,199,254,740,992, multiplied or divided by a power of 2 (between limits), with the unit in the last place (ULP, spacing) equal to , which is times |x|, rounded down to a power of 2. Divided by |x|, between each pair of consecutive powers of 2, this gradually decreases from to , somewhat like an inverted sawtooth wave, but with increasingly wide "tooths". When rounding x to the nearest float the rounding error alternately increases and decreases linearly with x, between plus and minus one half of the local ULP. Thus between each pair of consecutive powers of 2 the local maximum relative rounding error gradually decreases from to .

To find the exact value of a float one can use Template:Hex to show the significand in hexadecimal format (apart from the sign it consists of a fixed "1.", followed by 13 hexadecimal characters) and the exponent in decimal format.

Thus we can see which numbers and subexpressions are equivalent and which are not:

  • {{hex|1+2^-52}}1.0000000000001hex*2^0
  • {{hex|1.0000000000000002}}1.0000000000001hex*2^0
  • {{hex|1+2e-16}}1.0000000000001hex*2^0
  • {{hex|99999999999.99999e-11}}1.ffffffffffffehex*2^-1
  • {{hex|999999999999.9999e-12}}1.fffffffffffffhex*2^-1
  • {{hex|999999999999999.9e-15}}1.0000000000000hex*2^0

Such internal results can also be shown indirectly with high precision by subtracting a nearby representable number:

  • {{#expr:(1+2e-16)-1}} → 2.2204460492503E-16

After producing an internally representable result the system does (while converting from binary to decimal format) a further rounding to a total of 14 digits, in regular or scientific notation, to produce the expanded wikitext.

The maximum relative rounding error due to this is 5e-14:

  • {{#expr:1+5e-14}} → 1

This is 450 times as much as for the rounding to an internally representable result (225 times the spacing):

  • {{hex|1+5e-14}}1.00000000000e1hex*2^0

In terms of number of units of the 17th decimal digit, the spacing of internal numbers is 2.22 units between 1 and 2:

  • {{numfh|1.5}} → 1.5 (ca. 1.5e0) 1.8000000000000hex*2^0

increasing to 17.8 between 8 and 10:

  • {{numfh|9}} → 9 (ca. 9e0) 1.2000000000000hex*2^3

1.78 between 10 and 16, increasing to 14.2 between 64 and 100, 1.42 between 100 and 128, increasing to 11.37 between 512 and 1000, 1.14 between 1000 and 1024:

  • {{numfh|1010}} → 1,010 (ca. 1e3) 1.f900000000000hex*2^9

2.27 between 1024 and 2048, etc. Altogether the minimum is 1.11 units, for numbers which are just above a power of 10 and just below a power of 2; for example, as mentioned, it is 1.14 between 1000 and 1024). The maximum is 22.2 units, for numbers which are just above a power of 2 and just below a power of 10; for example, between 9.9e27 and 1e28 it is 22.0:

  • {{numfh|9.95e27}} → 9,950,000,000,000,000,000,000,000,000 (ca. 10e27) 1.013393a67e07ehex*2^93

Therefore 17 decimal digits are always sufficient to identify a particular internal number. (This does not mean that further digits are ignored; if two consecutive 17-digit numbers are rounded to different floats, then there is somewhere a borderline between them, see Help:Calculation#Numbers as input.) See also below.

The more accurate result can be displayed with special manipulations, for example:

  • {{#expr:1501^5}} → 7.6190962725075E+15
  • {{#expr:1000*floor((1501^5)/1000)}} + {{#expr:1501^5)mod1000}} → 7.619096272507E+15 + 501

Another example:

  • {{hex|6/7}}1.b6db6db6db6dbhex*2^-1
  • {{#expr:6/7}} → 0.85714285714286
  • {{#expr:6/7-0.85714285714286}} → -2.8865798640254E-15

This gives us indirectly some more digits of 6/7; with 17 digits we get the same internal number as with 6/7 :

  • {{hex|0.85714285714285711}}1.b6db6db6db6dbhex*2^-1

The 17th digit is that in the mathematical expansion of the internal number to which 6/7 is rounded, which in this case is not equal to that in the mathematical expansion of 6/7. This deviation can occur because the spacing of internal numbers in this range is 1.11e-16, 11.1 times the spacing of the numbers with 17 digits. For example 10 internal numbers onward we have:

  • {{hex|0.85714285714285822}}1.b6db6db6db6e5hex*2^-1

Integers[edit]

In the range from 0 through 2^53 all integers are internally exactly representable as float; in the range 2^52 through 2^53 the set of floats is exactly the set of integers.

They are exactly displayed after conversion to type integer using trunc, as is done in Template:Numf:

  • {{numf|2^53}} → 9,007,199,254,740,992

Compare:

  • {{formatnum:{{#expr:2^53}}}} → 9.007199254741E+15

In the same way larger floats in the range of the numbers of type integer can be displayed exactly, the largest being:

  • {{numf|2^63-1024}} → 9,223,372,036,854,774,784

Non-negative integers internally exactly representable as float include:

  • as type integer not larger than 9,007,199,254,740,992 (as type integer: until 1024 times this number)
  • multiples of 10 not larger than 1,801,439,850,948,198e1 (as type integer: until 512 times this number)
  • multiples of 100 not larger than 360,287,970,189,639e2 (as type integer: until 256 times this number)
  • multiples of 1e3 not larger than 72,057,594,037,927e3 (as type integer: until 128 times this number)
  • multiples of 1e4 not larger than 14,411,518,807,585e4 (as type integer: until 64 times this number)
  • multiples of 1e5 not larger than 2,882,303,761,517e5 (as type integer: until 32 times this number)
  • multiples of 1e6 not larger than 576,460,752,303e6 (as type integer: until 16 times this number)
  • multiples of 1e7 not larger than 115,292,150,460e7 (as type integer: until 8 times this number)
  • multiples of 1e8 not larger than 23,058,430,092e8 (as type integer: until 4 times this number)
  • multiples of 1e9 not larger than 4,611,686,018e9 (as type integer: until 2 times this number)
  • multiples of 1e10 not larger than 922,337,203e10 (as type integer also)
  • multiples of 1e11 not larger than 184,467,440e11
  • multiples of 1e12 not larger than 36,893,488e12
  • multiples of 1e13 not larger than 7,378,697e13
  • multiples of 1e14 not larger than 1,475,739e14
  • multiples of 1e15 not larger than 295,147e15
  • multiples of 1e16 not larger than 59,029e16
  • multiples of 1e17 not larger than 11,805e17
  • multiples of 1e18 not larger than 2,361e18
  • multiples of 1e19 not larger than 472e19
  • multiples of 1e20 not larger than 94e20
  • multiples of 1e21 not larger than 18e21
  • multiples of 1e22 not larger than 3e22

To make sure that an integer that is internally exactly representable as float is entered exactly, it has to be entered in integer form, or in scientific notation with an integer significand and a non-negative exponent, not in scientific notation with a fraction as significand (see also below):

  • {{numf|90071.99254740992e11}} → 9,007,199,254,740,991
  • {{numf|9007199254740992}} → 9,007,199,254,740,992

Large integers which are not exactly representable as float can only be entered indirectly. For 19-digit integers < 2^63, the 6 most compact expressions and the ways they are produced with Template:Int are as follows:

  • {{int|9134|067893123456789}} → trunc9134e15+trunc067893123456789
  • {{int|91340|67893123456789}} → trunc91340e14+trunc67893123456789
  • {{int|913406|7893123456789}} → trunc913406e13+trunc7893123456789
  • {{int|9134067|893123456789}} → trunc9134067e12+trunc893123456789
  • {{int|91340678|93123456789}} → trunc91340678e11+trunc93123456789
  • {{int|913406789|3123456789}} → trunc913406789e10+trunc3123456789
  • {{numf|trunc9134e15+trunc067893123456789}} → 9,134,067,893,123,456,789
  • {{numf|trunc91340e14+trunc67893123456789}} → 9,134,067,893,123,456,789
  • {{numf|trunc913406e13+trunc7893123456789}} → 9,134,067,893,123,456,789
  • {{numf|trunc9134067e12+trunc893123456789}} → 9,134,067,893,123,456,789
  • {{numf|trunc91340678e11+trunc93123456789}} → 9,134,067,893,123,456,789
  • {{numf|trunc913406789e10+trunc3123456789}} → 9,134,067,893,123,456,789

This format requires the first term to be representable as float. Compare (exact with an extra trunc operator and parentheses only):

  • {{numf|trunc9134067893e9+trunc123456789}} → 9,134,067,893,123,456,277
  • {{numf|(trunc9134067893)e trunc9+trunc123456789}} → 9,134,067,893,123,456,789

For 18-digit integers there are 7 compact expressions:

  • {{int|999|999999999999999}} → trunc999e15+trunc999999999999999
  • {{int|9999|99999999999999}} → trunc9999e14+trunc99999999999999
  • {{int|99999|9999999999999}} → trunc99999e13+trunc9999999999999
  • {{int|999999|999999999999}} → trunc999999e12+trunc999999999999
  • {{int|9999999|99999999999}} → trunc9999999e11+trunc99999999999
  • {{int|99999999|9999999999}} → trunc99999999e10+trunc9999999999
  • {{int|999999999|999999999}} → trunc999999999e9+trunc999999999
  • {{numf|trunc999e15+trunc999999999999999}} → 999,999,999,999,999,999
  • {{numf|trunc9999e14+trunc99999999999999}} → 999,999,999,999,999,999
  • {{numf|trunc99999e13+trunc9999999999999}} → 999,999,999,999,999,999
  • {{numf|trunc999999e12+trunc999999999999}} → 999,999,999,999,999,999
  • {{numf|trunc9999999e11+trunc99999999999}} → 999,999,999,999,999,999
  • {{numf|trunc99999999e10+trunc9999999999}} → 999,999,999,999,999,999
  • {{numf|trunc999999999e9+trunc999999999}} → 999,999,999,999,999,999

For 17-digit integers there are 9 compact expressions:

  • {{int|99|999999999999999}} → trunc99e15+trunc999999999999999
  • {{int|999|99999999999999}} → trunc999e14+trunc99999999999999
  • {{int|9999|9999999999999}} → trunc9999e13+trunc9999999999999
  • {{int|99999|999999999999}} → trunc99999e12+trunc999999999999
  • {{int|999999|99999999999}} → trunc999999e11+trunc99999999999
  • {{int|9999999|9999999999}} → trunc9999999e10+trunc9999999999
  • {{int|99999999|999999999}} → trunc99999999e9+trunc999999999
  • {{int|999999999|99999999}} → trunc999999999e8+trunc99999999
  • {{int|9999999999|9999999}} → trunc9999999999e7+trunc9999999
  • {{numf|trunc99e15+trunc999999999999999}} → 99,999,999,999,999,999
  • {{numf|trunc999e14+trunc99999999999999}} → 99,999,999,999,999,999
  • {{numf|trunc9999e13+trunc9999999999999}} → 99,999,999,999,999,999
  • {{numf|trunc99999e12+trunc999999999999}} → 99,999,999,999,999,999
  • {{numf|trunc999999e11+trunc99999999999}} → 99,999,999,999,999,999
  • {{numf|trunc9999999e10+trunc9999999999}} → 99,999,999,999,999,999
  • {{numf|trunc99999999e9+trunc999999999}} → 99,999,999,999,999,999
  • {{numf|trunc999999999e8+trunc99999999}} → 99,999,999,999,999,999

For 16-digit integers >= 2^53 there are 11 compact expressions:

  • {{int|9|999999999999999}} → trunc9e15+trunc999999999999999
  • {{int|99|99999999999999}} → trunc99e14+trunc99999999999999
  • {{int|999|9999999999999}} → trunc999e13+trunc9999999999999
  • {{int|9999|999999999999}} → trunc9999e12+trunc999999999999
  • {{int|99999|99999999999}} → trunc99999e11+trunc99999999999
  • {{int|999999|9999999999}} → trunc999999e10+trunc9999999999
  • {{int|9999999|999999999}} → trunc9999999e9+trunc999999999
  • {{int|99999999|99999999}} → trunc99999999e8+trunc99999999
  • {{int|999999999|9999999}} → trunc999999999e7+trunc9999999
  • {{int|9999999999|999999}} → trunc9999999999e6+trunc999999
  • {{int|99999999999|99999}} → trunc99999999999e5+trunc99999
  • {{numf|trunc9e15+trunc999999999999999}} → 9,999,999,999,999,999
  • {{numf|trunc99e14+trunc99999999999999}} → 9,999,999,999,999,999
  • {{numf|trunc999e13+trunc9999999999999}} → 9,999,999,999,999,999
  • {{numf|trunc9999e12+trunc999999999999}} → 9,999,999,999,999,999
  • {{numf|trunc99999e11+trunc99999999999}} → 9,999,999,999,999,999
  • {{numf|trunc999999e10+trunc9999999999}} → 9,999,999,999,999,999
  • {{numf|trunc9999999e9+trunc999999999}} → 9,999,999,999,999,999
  • {{numf|trunc99999999e8+trunc99999999}} → 9,999,999,999,999,999
  • {{numf|trunc999999999e7+trunc9999999}} → 9,999,999,999,999,999
  • {{numf|trunc9999999999e6+trunc999999}} → 9,999,999,999,999,999
  • {{numf|trunc99999999999e5+trunc99999}} → 9,999,999,999,999,999

Summarizing, we can keep the number of digits of the first parameter fixed at 4:

  • {{int|9199|999999999999999}} → trunc9199e15+trunc999999999999999
  • {{int|9999|99999999999999}} → trunc9999e14+trunc99999999999999
  • {{int|9999|9999999999999}} → trunc9999e13+trunc9999999999999
  • {{int|9999|999999999999}} → trunc9999e12+trunc999999999999
  • {{numf|trunc9199e15+trunc999999999999999}} → 9,199,999,999,999,999,999
  • {{numf|trunc9999e14+trunc99999999999999}} → 999,999,999,999,999,999
  • {{numf|trunc9999e13+trunc9999999999999}} → 99,999,999,999,999,999
  • {{numf|trunc9999e12+trunc999999999999}} → 9,999,999,999,999,999

Alternatively, we can keep the number of digits of the second parameter fixed at any number in the range 10 through 15 (although for manual input that is more cumbersome to count than 4):

  • {{int|919999999|9999999999}} → trunc919999999e10+trunc9999999999
  • {{int|99999999|9999999999}} → trunc99999999e10+trunc9999999999
  • {{int|9999999|9999999999}} → trunc9999999e10+trunc9999999999
  • {{int|999999|999999999}} → trunc999999e9+trunc999999999
  • {{numf|trunc919999999e10+trunc9999999999}} → 9,199,999,999,999,999,999
  • {{numf|trunc99999999e10+trunc9999999999}} → 999,999,999,999,999,999
  • {{numf|trunc9999999e10+trunc9999999999}} → 99,999,999,999,999,999
  • {{numf|trunc999999e10+trunc9999999999}} → 9,999,999,999,999,999

For smaller numbers these formats are allowed, but not needed; also int with one parameter is allowed:

  • {{int|9999|99999999999}} → trunc9999e11+trunc99999999999
  • {{int|99999|9999999999}} → trunc99999e10+trunc9999999999
  • {{int|999999999999999}} → trunc999999999999999
  • {{numf|trunc9999e11+trunc99999999999}} → 999,999,999,999,999
  • {{numf|trunc99999e10+trunc9999999999}} → 999,999,999,999,999
  • {{numf|trunc999999999999999}} → 999,999,999,999,999
  • {{numf|999999999999999}} → 999,999,999,999,999

Division[edit]

If the quotient of two integer-type numbers is an integer, the exact value is returned. Otherwise both integers are rounded to float, and the quotient is rounded to float.

  • {{numfh|1e17+12}} → 100,000,000,000,000,016 (ca. 1e17) 1.6345785d8a001hex*2^56
  • {{numfh|(trunc1e17+trunc12)/trunc10}} → 10,000,000,000,000,002 (ca. 1e16) 1.1c37937e08001hex*2^53
  • {{numfh|(trunc1e17+trunc10)/trunc10}} → 10,000,000,000,000,001 (ca. 1e16) 1.1c37937e08000hex*2^53
  • {{numfh|(trunc9199e15+trunc999999999999490)/trunc10}} → 919,999,999,999,999,949 (ca. 9.2e17) 1.988fe4052b800hex*2^59
  • {{numfh|(trunc9199e15+trunc999999999999488)/trunc10}} → 920,000,000,000,000,000 (ca. 9.2e17) 1.988fe4052b800hex*2^59
  • {{numfh|(trunc9199e15+trunc999999999999487)/trunc10}} → 919,999,999,999,999,872 (ca. 9.2e17) 1.988fe4052b7ffhex*2^59
  • {{numfh|9199999999999999488/10}} → 920,000,000,000,000,000 (ca. 9.2e17) 1.988fe4052b800hex*2^59
  • {{numfh|9199999999999999487/10}} → 919,999,999,999,999,872 (ca. 9.2e17) 1.988fe4052b7ffhex*2^59
  • {{numfh|(trunc9199e15+trunc999999999999488)/trunc1e4}} → 920,000,000,000,000 (ca. 9.2e14) 1.a25e1898c0000hex*2^49
  • {{numfh|(trunc9199e15+trunc999999999999487)/trunc1e4}} → 919,999,999,999,999.9 (ca. 9.2e14) 1.a25e1898bffffhex*2^49
  • {{numfh|(trunc9199e15+trunc999999999990001)/trunc1e4}} → 919,999,999,999,999 (ca. 9.2e14) 1.a25e1898bfff8hex*2^49
  • {{numfh|(trunc9199e15+trunc999999999990000)/trunc1e4}} → 919,999,999,999,999 (ca. 9.2e14) 1.a25e1898bfff8hex*2^49
  • {{numfh|(trunc9199e15+trunc999999999989999)/trunc1e4}} → 919,999,999,999,999 (ca. 9.2e14) 1.a25e1898bfff8hex*2^49

Mod[edit]

  • {{numfh|(trunc9199e15+trunc999999999999999)mod10}} → 9 (ca. 9e0) 1.2000000000000hex*2^3
  • {{numfh|(trunc9199e15+trunc999999999999999)mod100}} → 99 (ca. 9.9e1) 1.8c00000000000hex*2^6
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e3}} → 999 (ca. 10e2) 1.f380000000000hex*2^9
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e4}} → 9,999 (ca. 10e3) 1.3878000000000hex*2^13
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e5}} → 99,999 (ca. 10e4) 1.869f000000000hex*2^16
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e6}} → 999,999 (ca. 10e5) 1.e847e00000000hex*2^19
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e7}} → 9,999,999 (ca. 10e6) 1.312cfe0000000hex*2^23
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e8}} → 99,999,999 (ca. 10e7) 1.7d783fc000000hex*2^26
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e9}} → 999,999,999 (ca. 10e8) 1.dcd64ff800000hex*2^29
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e10}} → 9,999,999,999 (ca. 10e9) 1.2a05f1ff80000hex*2^33
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e11}} → 99,999,999,999 (ca. 10e10) 1.74876e7ff0000hex*2^36
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e12}} → 999,999,999,999 (ca. 10e11) 1.d1a94a1ffe000hex*2^39
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e13}} → 9,999,999,999,999 (ca. 10e12) 1.2309ce53ffe00hex*2^43
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e14}} → 99,999,999,999,999 (ca. 10e13) 1.6bcc41e8fffc0hex*2^46
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e15}} → 999,999,999,999,999 (ca. 10e14) 1.c6bf52633fff8hex*2^49
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e16}} → 9,999,999,999,999,999 (ca. 1e16) 1.1c37937e08000hex*2^53
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e17}} → 99,999,999,999,999,999 (ca. 1e17) 1.6345785d8a000hex*2^56
  • {{numfh|(trunc9199e15+trunc999999999999999)mod1e18}} → 199,999,999,999,999,999 (ca. 2e17) 1.6345785d8a000hex*2^57

If p is an integer > 2^53 and q is an integer <=2^53, but p/q is not an integer, regular division involves rounding p to float, and rounding the result to float, together giving a maximum relative error of 2^-52. However, if p is first rounded to a multiple of q the absolute error has a maximum of .5. For a quotient >2^51 this is less.

Examples:

  • {{numfh|(trunc9199e15+trunc999999999999994+trunc5-(trunc9199e15+trunc999999999999994+trunc5)mod10)/trunc10}} → 919,999,999,999,999,999 (ca. 9.2e17) 1.988fe4052b800hex*2^59
  • {{numfh|(trunc9199e15+trunc999999999999995+trunc5-(trunc9199e15+trunc999999999999995+trunc5)mod10)/trunc10}} → 920,000,000,000,000,000 (ca. 9.2e17) 1.988fe4052b800hex*2^59

Rounding[edit]

Rounding to float occurs when an explicit number, or the result of an operator which should be of type float, is mathematically not exactly a float, including the case where the result of an operator is of type integer in principal, but outside of the range of integers.

Rounding occurs to the nearest value:

  • {{hex|2^52+(.5-2^-53)}}1.0000000000000hex*2^52
  • {{hex|2^52+(.5+2^-53)}}1.0000000000001hex*2^52
  • {{hex|2^52+1+(.5-2^-53)}}1.0000000000001hex*2^52
  • {{hex|2^52+1+(.5+2^-53)}}1.0000000000002hex*2^52
  • {{hex|2^2+(2^-51-2^-102)}}1.0000000000000hex*2^2
  • {{hex|2^2+(2^-51+2^-102)}}1.0000000000001hex*2^2
  • {{hex|2^2+1*(2^-51-2^-102)}}1.0000000000000hex*2^2
  • {{hex|2^2+1+(2^-51+2^-102)}}1.4000000000001hex*2^2

If two floats are equally near, rounding occurs to the one with least-significant bit 0 (round half to even):

  • {{hex|2^52+2^-1}}1.0000000000000hex*2^52
  • {{hex|2^52+1+2^-1}}1.0000000000002hex*2^52
  • {{hex|2^2+2^-51}}1.0000000000000hex*2^2
  • {{hex|2^2+2^-50+2^-51}}1.0000000000002hex*2^2
  • {{hex|2^53-1.5}}1.ffffffffffffehex*2^52
  • {{hex|2^53-.5}}1.0000000000000hex*2^53

A consequence is that applying "x round 0" for an odd number x with absolute value between 2^52 and 2^53 does not give x but an even number, one more in absolute value. This is because .5 is added, the result is rounded to a float as mentioned, and truncated to an integer (not applicable in this range because it is already an integer).

Similarly, applying "x round -1" in the range where the floats are the even numbers, with x not a multiple of 4, we get x + 5, which is rounded up to x + 6, and then rounded down to a multiple of 10, for example:

  • {{numf|12345678901234574round-1}} → 12,345,678,901,234,574

Also, applying "x round -2" in the range where the floats are the multiples of 4, we get for example:

  • {{numf|22345678901234548round-2}} → 22,345,678,901,234,500

etc.

Chain of calculations[edit]

In the case of a chain of calculations, applying #expr and using the output in another expression may affect the result:

  • floats are rounded to 14 digits
  • displayed integer-type numbers used in another expression are converted to float, which for large integers involves rounding
  • for some representable round numbers, notably some multiples of 100,000, scientific notation is produced, which, if reused in an expression, is not exactly equal to the original number (see Help:Calculation#Numbers as output)

Instead of passing on a number to a template as parameter, an expression can be passed on. Similarly a template can produce an expression instead of a number.

Alternatively for a float Template:Num can be used to produce a number in almost the full precision which is readable by people as well as being allowed in an expression.

Power[edit]

For an integer raised to a non-negative integer, the result is exact up to 2^63-1, provided that we use integer type numbers, e.g.:

  • {{numf|trunc3^trunc39}} → 4,052,555,153,018,976,267

Floats:

A power of 10 is an exact float if and only if the corresponding power of 5 is, so the largest exponent is {{num|floor(53*ln2/ln5)}} = 22

For the number 1e22 the unit in the last place (ULP) is 2^21, while 1e22 is a multiple of 2^22. For the number 1e23 the ULP is 2^24, while 1e23 is an odd multiple of 2^23, so the absolute value of the error is {{#expr:2^23}} = 8388608. In fact, to determine the error we can subtract 1024e20 (which is exactly representable): {{hex|1e23-1024e20}}-1.043561a882940hex*2^71. Then we can add 24e20: {{hex|24e20}}1.043561a882930hex*2^71 (which is also exactly representable), and get {{hex|(1e23-1024e20)+24e20}}-1.0000000000000hex*2^23, and {{#expr:(1e23-1024e20)+24e20}} → -8388608. So 1e23 is "rounded" internally to 99,999,999,999,999,991,611,392. This also follows from:

  • {{numf|1e23-8e22-1e22-1e22}} → -8,388,608

or

  • {{numf|1e23-8e22-1e22-8e21-1e21-8e20-1e20-8e19-1e19-8e18-1e18}} → 999,999,999,991,611,392

taking into account that 1e22 and hence also 8e20, etc. are exactly representable, and that a subtraction of positive floats is done exactly if the smaller one is not smaller than half the larger one.

Correspondingly 5^23 is rounded to a value that is one less:

  • {{numfh|5^23}} → 11,920,928,955,078,124 (ca. 1.2e16) 1.52d02c7e14af6hex*2^53
  • {{numfh|trunc5^trunc23}} → 11,920,928,955,078,125 (ca. 1.2e16) 1.52d02c7e14af6hex*2^53

For floats as base and exponent, and a mathematical result that is exactly a float, the result of the power operator seems to be always exact (here shown for powers of powers of 10, but applicable in general):

  • {{#expr:1e22-100^11}} → 0
  • {{#expr:1e21-1000^7}} → 0
  • {{#expr:1e20-1e4^5}} → 0
  • {{#expr:1e20^.5-1e10}} → 0
  • {{#expr:1e20^.25-1e5}} → 0
  • {{#expr:1e16^.125-100}} → 0

Using exp and ln is less accurate (even if they are each also very accurate) because of the rounding of the intermediate results to floats:

  • {{#expr:exp(11*ln100)-1e22}} → 29360128
  • {{#expr:exp(5*ln1e4)-1e20}} → 786432

Example where the exponent is not exactly a float:

  • {{#expr:1e20^.2-1e4}} → 5.4569682106376E-12

If a number of type integer is raised to a power, where the exponent is also a number of type integer, and the result is too large for type integer, rounding is not always to the nearest float, i.e., the rounding error can be larger than if base, exponent, or both, are floats:

  • {{numfh|trunc257^trunc7}} → 74,051,159,531,521,793 (ca. 7.4e16) 1.0715232315070hex*2^56 (exact)
  • {{numfh|257^7}} → 74,051,159,531,521,792 (ca. 7.4e16) 1.0715232315070hex*2^56 (rounded to the nearest float; floats are multiples of 16 in this range, the result is one less than the exact result, which is hex 1.07152323150701*2^56)
  • {{numfh|trunc257^trunc8}} → 19,031,147,999,601,104,000 (ca. 1.9e19) 1.081c3846381c1hex*2^64 (rounded to another float than the nearest; floats are multiples of 4096 in this range, the result is 2049 less than the exact result, which is 1.081c3846381c0801hex*264)
  • {{numfh|257^8}} → 19,031,147,999,601,104,000 (ca. 1.9e19) 1.081c3846381c1hex*2^64 (rounded to the nearest float; the result is 2047 more than the exact result)

The difference may be due to repeated multiplication, used when both the base and the exponent are of type integer, with, beyond the range of type integer, rounding of intermediate values to float. Thus rounding errors cumulate, while when both the base and the exponent are of type real, rounding to the nearest float occurs, perhaps by the use of extended precision in the power routine:

  • 74,051,159,531,521,793 (ca. 7.4e16) 1.0715232315070hex*2^56 is rounded to 74,051,159,531,521,792 (ca. 7.4e16) 1.0715232315070hex*2^56; multiplied by 257 (ca. 2.6e2) 1.0100000000000hex*2^8 this gives 19,031,147,999,601,100,000 (ca. 1.9e19) 1.081c3846381c0hex*2^64

Template:Pow can be used to get the most accurate result in each case.

Reconstruction of a float[edit]

As mentioned, 17 decimal digits are always sufficient to identify a particular float. However, not all floats can be reconstructed by a number in scientific notation:

  • {{hex|8234567890123457}}1.d414bbdf83ac1hex*2^52
  • {{hex|8.2345678901234569e15}}1.d414bbdf83ac0hex*2^52
  • {{hex|8.234567890123457e15}}1.d414bbdf83ac2hex*2^52

This is caused by the fact that such a number is actually an expression involving a multiplication and a power, with intermediate rounding of both factors of the multiplication. In this case the rounding of the significand gives a spacing of the product (before rounding) of 1.78, while the spacing of the floats in this range is 1:

  • {{hex|8.2345678901234569}}1.0781948518b3fhex*2^3
  • {{hex|8.234567890123457}}1.0781948518b40hex*2^3
  • {{#expr:2^-49*1e15}} → 1.7763568394003

Other example: the number 4.5e15+.5 is an exact float, but written in the form (4.5e16+5)/1e1 the number 4.5e16+5 is rounded to a multiple of 8, hence to 4.5e16+8. Dividing this by 10 is rounded to 4.5e15+1 (too high), while the next lower float is 4.5e16, giving 4.5e15 (too low).

In general, if pq=r then, writing each in binary format with a significand between 1 and 2, we can distinguish two cases: the sum of the exponents of p and q is equal to that of r, or one less. If it is one less, then for fixed q the spacing of the floats r corresponds to values r/q with a spacing greater than the spacing of floats in the range of p, so each can be represented. However, if the sum of the exponents of p and q is equal to that of r, the converse is true. For example, for q=10 the relative spacing is multiplied by .8 for p between 1 and 1.6 times a power of 2, and by 1.6 for p between 1.6 and 2 times a power of 2. Correspondingly, for q=.1 the relative spacing is multiplied by .625 for p between 1 and 1.25 times a power of 2, and by 1.25 for p between 1.25 and 2 times a power of 2. On a double-logarithmic scale there is for every q on average no change in the relative spacing, so if for most numbers the spacing gets larger it is only a little larger, etc..

The conclusion is that in general, to reconstruct a float, a number in scientific notation. (at least for a fixed system of normalisation) is inadequate. We have to use the basic number notation (in the example 8234567890123457) or an expression such as here the misleading 8.234567890123457e15-1:

  • {{hex|8.234567890123457e15-1}}1.d414bbdf83ac1hex*2^52

As another example, the first number is not representable in the regular scientific notation (and adding an 18th digit does not help):

  • {{hex|.00011345678901234566}}1.dbdf4cb13a9c9hex*2^-14
  • {{hex|1.1345678901234564e-4}}1.dbdf4cb13a9c8hex*2^-14
  • {{hex|1.13456789012345649e-4}}1.dbdf4cb13a9c8hex*2^-14
  • {{hex|1.1345678901234565e-4}}1.dbdf4cb13a9cahex*2^-14
  • {{hex|1.1345678901234566e-4}}1.dbdf4cb13a9cahex*2^-14

However, the single bit can always be added:

  • {{hex|1.1345678901234564e-4+2^-66}}1.dbdf4cb13a9c9hex*2^-14
  • {{hex|1.1345678901234564e-4+1e-20}}1.dbdf4cb13a9c9hex*2^-14

Finding the exact decimal representation of a float[edit]

To find the exact decimal representation of a float in the range from 2^52 through 2^63-1 one can apply trunc, then #expr, here done combinedly with Template:Numf:

  • {{numf|1e17/3}} → 33,333,333,333,333,332
  • {{numf|1e18/3}} → 333,333,333,333,333,312
  • {{numf|1e19/3}} → 3,333,333,333,333,333,504

To find the exact decimal representation of a larger float, one can find the largest power of 2 which multiplied by 1e22 (the largest power of 10 that is exactly representable) is less or equal the given float. Subtracting that and repeating this with the difference, we get a binary representation for what remains after cutting off the rightmost 22 digits. Then we can apply the same again to the final difference, now to find the 22nd digit from the right, etc.; as seen above:

  • {{numf|1e23-8e22-1e22-8e21-1e21-8e20-1e20-8e19-1e19-8e18-1e18}} → 999,999,999,991,611,392

Hence the number is like this 18-digit number, with 5 digits 9 in front: 99,999,999,999,999,991,611,392.

For floats in the range from 2^34 to 2^52 we can take the trunc like above, and add the decimal part, as applied in Template:Num/3:

  • {{numfh|1e16/3}} → 3,333,333,333,333,333.5 (ca. 3.3e15) 1.7af4c4a80aaabhex*2^51
  • {{numfh|1e15+0.25}} → 1,000,000,000,000,000.3 (ca. 10e14) 1.c6bf526340002hex*2^49
  • {{numfh|1e15+0.1}} → 1,000,000,000,000,000.1 (ca. 10e14) 1.c6bf526340001hex*2^49
  • {{numfh|1e15/3}} → 333,333,333,333,333.3 (ca. 3.3e14) 1.2f2a36ecd5555hex*2^48
  • {{numfh|1e14/3}} → 33,333,333,333,333.332 (ca. 3.3e13) 1.e51057e155555hex*2^44
  • {{numfh|1e13/3}} → 3,333,333,333,333.3335 (ca. 3.3e12) 1.840d131aaaaabhex*2^41
  • {{numfh|1e12/3}} → 333,333,333,333.3333 (ca. 3.3e11) 1.3670dc1555555hex*2^38
  • {{numfh|1e11+.25}} → 100,000,000,000.25 (ca. 1e11) 1.74876e8004000hex*2^36
  • {{numfh|1e11/3}} → 33,333,333,333.333332 (ca. 3.3e10) 1.f0b4935555555hex*2^34
  • {{numfh|2^34+2^-18}} → 17,179,869,184.000004 (ca. 1.7e10) 1.0000000000001hex*2^34

Comparisons[edit]

Care should be taken that e.g. {{#expr:1e17=1e17+1}} → 1. This means also that if x<=1e17 gives "true", this does not imply that x<1e17+1 gives "true".

Largest and smallest number[edit]

With a significand between 1 and 2 the maximum exponent is 1023:

  • {{#expr:(2-2^-52)*2^1023}} → 1.7976931348623E+308
  • {{hex|(2-2^-52)*2^1023}}1.fffffffffffffhex*2^1023
  • {{#expr:2^1024}} → INF

If an intermediate result is INF the final result depends on the operation:

  • {{#expr:2^1024/2}} → INF
  • {{#expr:(2^1024)*0}} → NAN
  • {{#expr:(2^1024)mod3}} → 0

If an intermediate result is NAN the final result depends on the operation:

  • {{#expr:((2^1024)*0)mod3}} → -2*{{#expr:((2^1024)*0)/2}} → NAN
  • {{#expr:((2^1024)*0)*0}} → NAN
  • {{#expr:((2^1024)*0)mod3}} → -2

However, INF and NAN cannot be entered directly:

  • {{#expr:INF*0}}Expression error: Unrecognized word "inf".
  • {{#expr:(INF)mod3}}Expression error: Unrecognized word "inf".
  • {{#expr:(NAN)mod3}}Expression error: Unrecognized word "nan".

The smallest positive internal value is 2^-1074, the next lower internal value is 0:

  • {{#expr:2^-1074}} → 4.9406564584125E-324
  • {{#expr:.51*2^-1074}} → 4.9406564584125E-324
  • {{#expr:.5*2^-1074}} → 0
  • {{hex|2^-1074}}1.hex*2^-1074
  • {{hex|.51*2^-1074}}1.hex*2^-1074
  • {{hex|.5*2^-1074}}0

Note that for entering this number one can use .5e-323, but not 5e-324:

  • {{hex|.5e-323}}1.hex*2^-1074
  • {{hex|5e-324}}0

With a significand between 1 and 2 using the full 53 bits the smallest exponent is -1022:

  • {{hex|2^-1020-(2-2^-52)*2^-1021}}1.0hex*2^-1073
  • {{hex|2^-1021-(2-2^-52)*2^-1022}}1.hex*2^-1074
  • {{hex|2^-1022-(2-2^-52)*2^-1023}}0
  • {{#expr:(2^-1022-(2-2^-52)*2^-1023)}} → 0

With this exponent the smallest number is:

  • {{#expr:2^-1022}} → 2.2250738585072E-308
  • {{hex|2^-1022}}1.0000000000000hex*2^-1022

Lower numbers are denormal numbers. They use the lowest value of the exponent register, which would correspond to an exponent of -1023, but they represent values with an exponent of -1022 and a significand with 0 instead of 1 before the point (this requires a separate exponent value because the bit before the point is implicit, not an actual storage location). Thus the least significant bit for numbers below 2^-1021 is 2^-1074=.5e-323 (do not enter it as 5e-324, as this gives 0, even though {{#expr:2^-1074}} → 4.9406564584125E-324 ), the smallest positive internal value. As a result the difference of two different internal numbers is never rounded to 0.

With this lowest register value for the exponent the value 0 is represented by all bits being equal to 0.

Thus of the 2048 available values of the exponent register, 2047 are used for finite positive values and 0, leaving the largest value for INF and NAN.

BCMath[edit]

There have been temporary changes in the php code in relation to calculation accuracy ([1], [2], [3]), including the use of BCMath.

See also[edit]


Links to other help pages[edit]

Help contents
Meta · Wikinews · Wikipedia · Wikiquote · Wiktionary · Commons: · Wikidata · MediaWiki · Wikibooks · Wikisource · MediaWiki: Manual · Google
Versions of this help page (for other languages see further)
What links here on Meta or from Meta · Wikipedia · MediaWiki
Reading
Go · Search · Stop words · Namespace · Page name · Section · Backlinks · Redirect · Category · Image page · Special pages · Printable version
Tracking changes
Recent changes (enhanced) | Related changes · Watching pages · Diff · Page history · Edit summary · User contributions · Minor edit · Patrolled edit
Logging in and preferences
Logging in · Preferences · User style
Editing
Starting a new page · Advanced editing · Editing FAQ · Edit toolbar · Export · Import · Shortcuts · Edit conflict · Page size
Referencing
Links · URL · Piped links · Interwiki linking · Footnotes
Style and formatting
Wikitext examples · CSS · Reference card · HTML in wikitext · Formula · List · Table · Sorting · Colors · Images and file uploads
Fixing mistakes
Show preview · Testing · Reverting edits
Advanced functioning
Expansion · Template · Advanced templates · Parser function · Parameter default · Variable · System message · Substitution · Array · Calculation · Embed page
Others
Special characters · Renaming (moving) a page · Preparing a page for translation · Talk page · Signatures · Sandbox · Legal issues for editors
Languages: English