[sdiy] large numbers

mskala at ansuz.sooke.bc.ca mskala at ansuz.sooke.bc.ca
Fri Jul 13 21:22:00 CEST 2018


On Fri, 13 Jul 2018, Tim Ressel wrote:
> Turns out it is a limitation of double precision floating point numbers. I
> wrote a VB program and it is doing the same thing.

What is this supposed to be a lookup table *of*?  How will it be used?
Why does it need 64-bit precision?  Are more similar tables of the same
general kind going to be needed in the future, or is this a one-off?

Depending on the answers, I might be inclined to reach first for C and the
"long double" type, which (on my PC) offers 64-bit precision for the
significand; or simply 64-bit integers if those are the intended output.
But if it requires trig functions or something, the library routines may
not be good to that level of precision, and then I might be more willing
to look at computer algebra systems (Sage, Maple, Octave, Mathematica,
Maxima...) rather than implement high-precision versions of the functions
myself.

-- 
Matthew Skala
mskala at ansuz.sooke.bc.ca                 People before principles.
http://ansuz.sooke.bc.ca/



More information about the Synth-diy mailing list