How much precision can you squeeze out of a table?

(johndcook.com)

31 points | by nomemory 3 hours ago

2 comments

  • PaulHoule 3 hours ago
    When I was in high school I thought book this was so much fun

    http://182.160.97.198:8080/xmlui/bitstream/handle/123456789/...

    which is all about the kind of numerical analysis you would do by hand and introduces a lot of really cool math like the calculus of differences

    https://en.wikipedia.org/wiki/Finite_difference

  • gus_massa 2 hours ago
    Nitpicking:

    > You’re never going to get error less than 10E−15 since that’s the error in the tabulated values,

    If you have like 100 (or 400) values in the table, you can squeeze one more digit.

    In the simple case, imagine you have the constant pi, with 15 digits, repeated 100 times. If the rounding was done in a random direction like

      floor(pi * 1E15 + random() - 1/2 ) / 1E15
    
    then you can use statistic to get an average with an error that is like K/sqrt(N) where K is a constant and N is the number of samples. If you were paying attention in you statistic clases, you probably know the value of K, but it's not my case. Anyway, just pick N=100 or N=400 or something like that and you get another digit for "free".

    Nobody builds a table for a constant and uses a uniform offset for rounding. Anyway, with some smooth function that has no special values in the points where it's sample, the exact value of the cut decimal is quite "random" and you get a similar randomization.