[sdiy] large numbers

[Ben Bradley]

Windows Calculator does about twice that ("In Scientific mode,
Calculator is precise to 32 significant digits."), though of course
using it is tedious. If it helps, you can copy-paste operations into
it. Copy and paste this:
2y2r=
(two to the power of 2 reciprocal - this calculates the square root of
two) gives:
1.4142135623730950488016887242097
There's a list of keyboard shortcuts somewhere.

If you're using integers, I recall that Python, rather than
overflowing, will handle arbitrarily large integer values.


On Fri, Jul 13, 2018 at 12:42 PM, Tim Ressel <timr at circuitabbey.com> wrote:
> Hey'all,
>
> I need to generate a lookup table with 64 bit values. Usually I use Excel
> but there is a problem: It only does 14 decimal places. I don't think this
> is enough. Any suggestions?
>
> --
> --Tim Ressel
> Circuit Abbey
> timr at circuitabbey.com
>
> _______________________________________________
> Synth-diy mailing list
> Synth-diy at synth-diy.org
> http://synth-diy.org/mailman/listinfo/synth-diy