[sdiy] large numbers

[Jason Proctor]

ftr, Java has BigDecimal for arbitrary precision numeric stuff

i usually use Java for tools and generators and the like.



On Fri, Jul 13, 2018 at 11:44 AM, Pete Hartman <pete.hartman at gmail.com>
wrote:

> Is there a way to ruin it using fixed point math instead?
>
> On Fri, Jul 13, 2018, 1:35 PM Tim Ressel <timr at circuitabbey.com> wrote:
>
>> If you must pry...  ;-)
>>
>> I am playing with using a 64 bit accumulator in a DCO (NCO?) to fix the
>> top jitter problem. The notion is that increased resolution in the
>> increment value will cause less top jitter.
>>
>> --tr
>>
>>
>> On 7/13/2018 11:30 AM, Richie Burnett wrote:
>> > Out of interest, what application requires table lookup of values to a
>> resolution of better than 1 part in 10 to the 19 !?!?!?
>> >
>> > -Richie,
>> >
>> > Sent from my Xperia SP on O2
>> >
>> > ---- Tim Ressel wrote ----
>> >
>> >> Verified. I don't relish (or other condiment...) doing that calc
>> >> manually 4096 times. But it does tell us that a windoze machine can do
>> >> the deed. I wonder if its a matter of floats versus doubles?
>> >>
>> >> --timmers
>> >>
>> >>
>> >> On 7/13/2018 9:59 AM, Ben Bradley wrote:
>> >>> Windows Calculator does about twice that ("In Scientific mode,
>> >>> Calculator is precise to 32 significant digits."), though of course
>> >>> using it is tedious. If it helps, you can copy-paste operations into
>> >>> it. Copy and paste this:
>> >>> 2y2r=
>> >>> (two to the power of 2 reciprocal - this calculates the square root of
>> >>> two) gives:
>> >>> 1.4142135623730950488016887242097
>> >>> There's a list of keyboard shortcuts somewhere.
>> >>>
>> >>> If you're using integers, I recall that Python, rather than
>> >>> overflowing, will handle arbitrarily large integer values.
>> >>>
>> >>>
>> >>> On Fri, Jul 13, 2018 at 12:42 PM, Tim Ressel <timr at circuitabbey.com>
>> wrote:
>> >>>> Hey'all,
>> >>>>
>> >>>> I need to generate a lookup table with 64 bit values. Usually I use
>> Excel
>> >>>> but there is a problem: It only does 14 decimal places. I don't
>> think this
>> >>>> is enough. Any suggestions?
>> >>>>
>> >>>> --
>> >>>> --Tim Ressel
>> >>>> Circuit Abbey
>> >>>> timr at circuitabbey.com
>> >>>>
>> >>>> _______________________________________________
>> >>>> Synth-diy mailing list
>> >>>> Synth-diy at synth-diy.org
>> >>>> http://synth-diy.org/mailman/listinfo/synth-diy
>> >>> _______________________________________________
>> >>> Synth-diy mailing list
>> >>> Synth-diy at synth-diy.org
>> >>> http://synth-diy.org/mailman/listinfo/synth-diy
>> >>>
>> >> --
>> >> --Tim Ressel
>> >> Circuit Abbey
>> >> timr at circuitabbey.com
>> >>
>> >> _______________________________________________
>> >> Synth-diy mailing list
>> >> Synth-diy at synth-diy.org
>> >> http://synth-diy.org/mailman/listinfo/synth-diy
>>
>> --
>> --Tim Ressel
>> Circuit Abbey
>> timr at circuitabbey.com
>>
>> _______________________________________________
>> Synth-diy mailing list
>> Synth-diy at synth-diy.org
>> http://synth-diy.org/mailman/listinfo/synth-diy
>>
>
> _______________________________________________
> Synth-diy mailing list
> Synth-diy at synth-diy.org
> http://synth-diy.org/mailman/listinfo/synth-diy
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://synth-diy.org/pipermail/synth-diy/attachments/20180713/d8460483/attachment.htm>