[sdiy] large numbers

[Tim Ressel]

Verified. I don't relish (or other condiment...) doing that calc 
manually 4096 times. But it does tell us that a windoze machine can do 
the deed. I wonder if its a matter of floats versus doubles?

--timmers


On 7/13/2018 9:59 AM, Ben Bradley wrote:
> Windows Calculator does about twice that ("In Scientific mode,
> Calculator is precise to 32 significant digits."), though of course
> using it is tedious. If it helps, you can copy-paste operations into
> it. Copy and paste this:
> 2y2r=
> (two to the power of 2 reciprocal - this calculates the square root of
> two) gives:
> 1.4142135623730950488016887242097
> There's a list of keyboard shortcuts somewhere.
>
> If you're using integers, I recall that Python, rather than
> overflowing, will handle arbitrarily large integer values.
>
>
> On Fri, Jul 13, 2018 at 12:42 PM, Tim Ressel <timr at circuitabbey.com> wrote:
>> Hey'all,
>>
>> I need to generate a lookup table with 64 bit values. Usually I use Excel
>> but there is a problem: It only does 14 decimal places. I don't think this
>> is enough. Any suggestions?
>>
>> --
>> --Tim Ressel
>> Circuit Abbey
>> timr at circuitabbey.com
>>
>> _______________________________________________
>> Synth-diy mailing list
>> Synth-diy at synth-diy.org
>> http://synth-diy.org/mailman/listinfo/synth-diy
> _______________________________________________
> Synth-diy mailing list
> Synth-diy at synth-diy.org
> http://synth-diy.org/mailman/listinfo/synth-diy
>

-- 
--Tim Ressel
Circuit Abbey
timr at circuitabbey.com