[sdiy] large numbers

[Tim Ressel]

Turns out it is a limitation of double precision floating point numbers. 
I wrote a VB program and it is doing the same thing.

I suspect something like python or java will have the same limitation. I 
wish I knew for sure if Sage could really do what I need. Sage looks 
really hard to setup and learn.

--tr


On 7/13/2018 10:20 AM, Veronica Merryfield wrote:
> I’ve heard good things about Sage - http://www.sagemath.org/ - on this subject since it uses the GNU Multi Precision Library. Would like to hear how well it performs.
>
>> On Jul 13, 2018, at 1:42 PM, Tim Ressel <timr at circuitabbey.com> wrote:
>>
>> Hey'all,
>>
>> I need to generate a lookup table with 64 bit values. Usually I use Excel but there is a problem: It only does 14 decimal places. I don't think this is enough. Any suggestions?
>>
>> -- 
>> --Tim Ressel
>> Circuit Abbey
>> timr at circuitabbey.com
>>
>> _______________________________________________
>> Synth-diy mailing list
>> Synth-diy at synth-diy.org
>> http://synth-diy.org/mailman/listinfo/synth-diy
>>
>
> _______________________________________________
> Synth-diy mailing list
> Synth-diy at synth-diy.org
> http://synth-diy.org/mailman/listinfo/synth-diy

-- 
--Tim Ressel
Circuit Abbey
timr at circuitabbey.com