[sdiy] large numbers

[Tim Ressel]

If you must pry...  ;-)

I am playing with using a 64 bit accumulator in a DCO (NCO?) to fix the 
top jitter problem. The notion is that increased resolution in the 
increment value will cause less top jitter.

--tr


On 7/13/2018 11:30 AM, Richie Burnett wrote:
> Out of interest, what application requires table lookup of values to a resolution of better than 1 part in 10 to the 19 !?!?!?
>
> -Richie,
>
> Sent from my Xperia SP on O2
>
> ---- Tim Ressel wrote ----
>
>> Verified. I don't relish (or other condiment...) doing that calc
>> manually 4096 times. But it does tell us that a windoze machine can do
>> the deed. I wonder if its a matter of floats versus doubles?
>>
>> --timmers
>>
>>
>> On 7/13/2018 9:59 AM, Ben Bradley wrote:
>>> Windows Calculator does about twice that ("In Scientific mode,
>>> Calculator is precise to 32 significant digits."), though of course
>>> using it is tedious. If it helps, you can copy-paste operations into
>>> it. Copy and paste this:
>>> 2y2r=
>>> (two to the power of 2 reciprocal - this calculates the square root of
>>> two) gives:
>>> 1.4142135623730950488016887242097
>>> There's a list of keyboard shortcuts somewhere.
>>>
>>> If you're using integers, I recall that Python, rather than
>>> overflowing, will handle arbitrarily large integer values.
>>>
>>>
>>> On Fri, Jul 13, 2018 at 12:42 PM, Tim Ressel <timr at circuitabbey.com> wrote:
>>>> Hey'all,
>>>>
>>>> I need to generate a lookup table with 64 bit values. Usually I use Excel
>>>> but there is a problem: It only does 14 decimal places. I don't think this
>>>> is enough. Any suggestions?
>>>>
>>>> --
>>>> --Tim Ressel
>>>> Circuit Abbey
>>>> timr at circuitabbey.com
>>>>
>>>> _______________________________________________
>>>> Synth-diy mailing list
>>>> Synth-diy at synth-diy.org
>>>> http://synth-diy.org/mailman/listinfo/synth-diy
>>> _______________________________________________
>>> Synth-diy mailing list
>>> Synth-diy at synth-diy.org
>>> http://synth-diy.org/mailman/listinfo/synth-diy
>>>
>> -- 
>> --Tim Ressel
>> Circuit Abbey
>> timr at circuitabbey.com
>>
>> _______________________________________________
>> Synth-diy mailing list
>> Synth-diy at synth-diy.org
>> http://synth-diy.org/mailman/listinfo/synth-diy

-- 
--Tim Ressel
Circuit Abbey
timr at circuitabbey.com