[sdiy] large numbers

[Tim Ressel]

I just found this add on for excel:

http://precisioncalc.com/what_is_xlprecision.html

It is cheap, a bit clumsy, but seems to work.

To answer an earlier question, I am doings logs, mults, and division.

--tr


On 7/13/2018 12:34 PM, Ian Fritz wrote:
> Also, Mathematica can work with any number of digits.  I’ve done 100 
> with no problems.
>
> Ian
>
> On Jul 13, 2018, at 12:53 PM, Jason Proctor <jason at redfish.net 
> <mailto:jason at redfish.net>> wrote:
>
>> ftr, Java has BigDecimal for arbitrary precision numeric stuff
>>
>> i usually use Java for tools and generators and the like.
>>
>>
>>
>> On Fri, Jul 13, 2018 at 11:44 AM, Pete Hartman 
>> <pete.hartman at gmail.com <mailto:pete.hartman at gmail.com>> wrote:
>>
>>     Is there a way to ruin it using fixed point math instead?
>>
>>     On Fri, Jul 13, 2018, 1:35 PM Tim Ressel <timr at circuitabbey.com
>>     <mailto:timr at circuitabbey.com>> wrote:
>>
>>         If you must pry...  ;-)
>>
>>         I am playing with using a 64 bit accumulator in a DCO (NCO?)
>>         to fix the
>>         top jitter problem. The notion is that increased resolution
>>         in the
>>         increment value will cause less top jitter.
>>
>>         --tr
>>
>>
>>         On 7/13/2018 11:30 AM, Richie Burnett wrote:
>>         > Out of interest, what application requires table lookup of
>>         values to a resolution of better than 1 part in 10 to the 19
>>         !?!?!?
>>         >
>>         > -Richie,
>>         >
>>         > Sent from my Xperia SP on O2
>>         >
>>         > ---- Tim Ressel wrote ----
>>         >
>>         >> Verified. I don't relish (or other condiment...) doing
>>         that calc
>>         >> manually 4096 times. But it does tell us that a windoze
>>         machine can do
>>         >> the deed. I wonder if its a matter of floats versus doubles?
>>         >>
>>         >> --timmers
>>         >>
>>         >>
>>         >> On 7/13/2018 9:59 AM, Ben Bradley wrote:
>>         >>> Windows Calculator does about twice that ("In Scientific
>>         mode,
>>         >>> Calculator is precise to 32 significant digits."), though
>>         of course
>>         >>> using it is tedious. If it helps, you can copy-paste
>>         operations into
>>         >>> it. Copy and paste this:
>>         >>> 2y2r=
>>         >>> (two to the power of 2 reciprocal - this calculates the
>>         square root of
>>         >>> two) gives:
>>         >>> 1.4142135623730950488016887242097
>>         >>> There's a list of keyboard shortcuts somewhere.
>>         >>>
>>         >>> If you're using integers, I recall that Python, rather than
>>         >>> overflowing, will handle arbitrarily large integer values.
>>         >>>
>>         >>>
>>         >>> On Fri, Jul 13, 2018 at 12:42 PM, Tim Ressel
>>         <timr at circuitabbey.com <mailto:timr at circuitabbey.com>> wrote:
>>         >>>> Hey'all,
>>         >>>>
>>         >>>> I need to generate a lookup table with 64 bit values.
>>         Usually I use Excel
>>         >>>> but there is a problem: It only does 14 decimal places.
>>         I don't think this
>>         >>>> is enough. Any suggestions?
>>         >>>>
>>         >>>> --
>>         >>>> --Tim Ressel
>>         >>>> Circuit Abbey
>>         >>>> timr at circuitabbey.com <mailto:timr at circuitabbey.com>
>>         >>>>
>>         >>>> _______________________________________________
>>         >>>> Synth-diy mailing list
>>         >>>> Synth-diy at synth-diy.org <mailto:Synth-diy at synth-diy.org>
>>         >>>> http://synth-diy.org/mailman/listinfo/synth-diy
>>         <http://synth-diy.org/mailman/listinfo/synth-diy>
>>         >>> _______________________________________________
>>         >>> Synth-diy mailing list
>>         >>> Synth-diy at synth-diy.org <mailto:Synth-diy at synth-diy.org>
>>         >>> http://synth-diy.org/mailman/listinfo/synth-diy
>>         <http://synth-diy.org/mailman/listinfo/synth-diy>
>>         >>>
>>         >> --
>>         >> --Tim Ressel
>>         >> Circuit Abbey
>>         >> timr at circuitabbey.com <mailto:timr at circuitabbey.com>
>>         >>
>>         >> _______________________________________________
>>         >> Synth-diy mailing list
>>         >> Synth-diy at synth-diy.org <mailto:Synth-diy at synth-diy.org>
>>         >> http://synth-diy.org/mailman/listinfo/synth-diy
>>         <http://synth-diy.org/mailman/listinfo/synth-diy>
>>
>>         -- 
>>         --Tim Ressel
>>         Circuit Abbey
>>         timr at circuitabbey.com <mailto:timr at circuitabbey.com>
>>
>>         _______________________________________________
>>         Synth-diy mailing list
>>         Synth-diy at synth-diy.org <mailto:Synth-diy at synth-diy.org>
>>         http://synth-diy.org/mailman/listinfo/synth-diy
>>         <http://synth-diy.org/mailman/listinfo/synth-diy>
>>
>>
>>     _______________________________________________
>>     Synth-diy mailing list
>>     Synth-diy at synth-diy.org <mailto:Synth-diy at synth-diy.org>
>>     http://synth-diy.org/mailman/listinfo/synth-diy
>>     <http://synth-diy.org/mailman/listinfo/synth-diy>
>>
>>
>> _______________________________________________
>> Synth-diy mailing list
>> Synth-diy at synth-diy.org <mailto:Synth-diy at synth-diy.org>
>> http://synth-diy.org/mailman/listinfo/synth-diy
>
>
> _______________________________________________
> Synth-diy mailing list
> Synth-diy at synth-diy.org
> http://synth-diy.org/mailman/listinfo/synth-diy

-- 
--Tim Ressel
Circuit Abbey
timr at circuitabbey.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://synth-diy.org/pipermail/synth-diy/attachments/20180713/705be5dd/attachment.htm>