[sdiy] large numbers

[Ian Fritz]

Also, Mathematica can work with any number of digits.  I’ve done 100 with no problems.

Ian

> On Jul 13, 2018, at 12:53 PM, Jason Proctor <jason at redfish.net> wrote:
> 
> ftr, Java has BigDecimal for arbitrary precision numeric stuff
> 
> i usually use Java for tools and generators and the like.
> 
> 
> 
>> On Fri, Jul 13, 2018 at 11:44 AM, Pete Hartman <pete.hartman at gmail.com> wrote:
>> Is there a way to ruin it using fixed point math instead?
>> 
>>> On Fri, Jul 13, 2018, 1:35 PM Tim Ressel <timr at circuitabbey.com> wrote:
>>> If you must pry...  ;-)
>>> 
>>> I am playing with using a 64 bit accumulator in a DCO (NCO?) to fix the 
>>> top jitter problem. The notion is that increased resolution in the 
>>> increment value will cause less top jitter.
>>> 
>>> --tr
>>> 
>>> 
>>> On 7/13/2018 11:30 AM, Richie Burnett wrote:
>>> > Out of interest, what application requires table lookup of values to a resolution of better than 1 part in 10 to the 19 !?!?!?
>>> >
>>> > -Richie,
>>> >
>>> > Sent from my Xperia SP on O2
>>> >
>>> > ---- Tim Ressel wrote ----
>>> >
>>> >> Verified. I don't relish (or other condiment...) doing that calc
>>> >> manually 4096 times. But it does tell us that a windoze machine can do
>>> >> the deed. I wonder if its a matter of floats versus doubles?
>>> >>
>>> >> --timmers
>>> >>
>>> >>
>>> >> On 7/13/2018 9:59 AM, Ben Bradley wrote:
>>> >>> Windows Calculator does about twice that ("In Scientific mode,
>>> >>> Calculator is precise to 32 significant digits."), though of course
>>> >>> using it is tedious. If it helps, you can copy-paste operations into
>>> >>> it. Copy and paste this:
>>> >>> 2y2r=
>>> >>> (two to the power of 2 reciprocal - this calculates the square root of
>>> >>> two) gives:
>>> >>> 1.4142135623730950488016887242097
>>> >>> There's a list of keyboard shortcuts somewhere.
>>> >>>
>>> >>> If you're using integers, I recall that Python, rather than
>>> >>> overflowing, will handle arbitrarily large integer values.
>>> >>>
>>> >>>
>>> >>> On Fri, Jul 13, 2018 at 12:42 PM, Tim Ressel <timr at circuitabbey.com> wrote:
>>> >>>> Hey'all,
>>> >>>>
>>> >>>> I need to generate a lookup table with 64 bit values. Usually I use Excel
>>> >>>> but there is a problem: It only does 14 decimal places. I don't think this
>>> >>>> is enough. Any suggestions?
>>> >>>>
>>> >>>> --
>>> >>>> --Tim Ressel
>>> >>>> Circuit Abbey
>>> >>>> timr at circuitabbey.com
>>> >>>>
>>> >>>> _______________________________________________
>>> >>>> Synth-diy mailing list
>>> >>>> Synth-diy at synth-diy.org
>>> >>>> http://synth-diy.org/mailman/listinfo/synth-diy
>>> >>> _______________________________________________
>>> >>> Synth-diy mailing list
>>> >>> Synth-diy at synth-diy.org
>>> >>> http://synth-diy.org/mailman/listinfo/synth-diy
>>> >>>
>>> >> -- 
>>> >> --Tim Ressel
>>> >> Circuit Abbey
>>> >> timr at circuitabbey.com
>>> >>
>>> >> _______________________________________________
>>> >> Synth-diy mailing list
>>> >> Synth-diy at synth-diy.org
>>> >> http://synth-diy.org/mailman/listinfo/synth-diy
>>> 
>>> -- 
>>> --Tim Ressel
>>> Circuit Abbey
>>> timr at circuitabbey.com
>>> 
>>> _______________________________________________
>>> Synth-diy mailing list
>>> Synth-diy at synth-diy.org
>>> http://synth-diy.org/mailman/listinfo/synth-diy
>> 
>> _______________________________________________
>> Synth-diy mailing list
>> Synth-diy at synth-diy.org
>> http://synth-diy.org/mailman/listinfo/synth-diy
>> 
> 
> _______________________________________________
> Synth-diy mailing list
> Synth-diy at synth-diy.org
> http://synth-diy.org/mailman/listinfo/synth-diy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://synth-diy.org/pipermail/synth-diy/attachments/20180713/6f49bc62/attachment.htm>