[sdiy] large numbers

[Richie Burnett]

Out of interest, what application requires table lookup of values to a resolution of better than 1 part in 10 to the 19 !?!?!? 

-Richie,

Sent from my Xperia SP on O2

---- Tim Ressel wrote ----

>Verified. I don't relish (or other condiment...) doing that calc 
>manually 4096 times. But it does tell us that a windoze machine can do 
>the deed. I wonder if its a matter of floats versus doubles?
>
>--timmers
>
>
>On 7/13/2018 9:59 AM, Ben Bradley wrote:
>> Windows Calculator does about twice that ("In Scientific mode,
>> Calculator is precise to 32 significant digits."), though of course
>> using it is tedious. If it helps, you can copy-paste operations into
>> it. Copy and paste this:
>> 2y2r=
>> (two to the power of 2 reciprocal - this calculates the square root of
>> two) gives:
>> 1.4142135623730950488016887242097
>> There's a list of keyboard shortcuts somewhere.
>>
>> If you're using integers, I recall that Python, rather than
>> overflowing, will handle arbitrarily large integer values.
>>
>>
>> On Fri, Jul 13, 2018 at 12:42 PM, Tim Ressel <timr at circuitabbey.com> wrote:
>>> Hey'all,
>>>
>>> I need to generate a lookup table with 64 bit values. Usually I use Excel
>>> but there is a problem: It only does 14 decimal places. I don't think this
>>> is enough. Any suggestions?
>>>
>>> --
>>> --Tim Ressel
>>> Circuit Abbey
>>> timr at circuitabbey.com
>>>
>>> _______________________________________________
>>> Synth-diy mailing list
>>> Synth-diy at synth-diy.org
>>> http://synth-diy.org/mailman/listinfo/synth-diy
>> _______________________________________________
>> Synth-diy mailing list
>> Synth-diy at synth-diy.org
>> http://synth-diy.org/mailman/listinfo/synth-diy
>>
>
>-- 
>--Tim Ressel
>Circuit Abbey
>timr at circuitabbey.com
>
>_______________________________________________
>Synth-diy mailing list
>Synth-diy at synth-diy.org
>http://synth-diy.org/mailman/listinfo/synth-diy