[sdiy] large numbers

[Pete Hartman]

Is there a way to ruin it using fixed point math instead?

On Fri, Jul 13, 2018, 1:35 PM Tim Ressel <timr at circuitabbey.com> wrote:

> If you must pry...  ;-)
>
> I am playing with using a 64 bit accumulator in a DCO (NCO?) to fix the
> top jitter problem. The notion is that increased resolution in the
> increment value will cause less top jitter.
>
> --tr
>
>
> On 7/13/2018 11:30 AM, Richie Burnett wrote:
> > Out of interest, what application requires table lookup of values to a
> resolution of better than 1 part in 10 to the 19 !?!?!?
> >
> > -Richie,
> >
> > Sent from my Xperia SP on O2
> >
> > ---- Tim Ressel wrote ----
> >
> >> Verified. I don't relish (or other condiment...) doing that calc
> >> manually 4096 times. But it does tell us that a windoze machine can do
> >> the deed. I wonder if its a matter of floats versus doubles?
> >>
> >> --timmers
> >>
> >>
> >> On 7/13/2018 9:59 AM, Ben Bradley wrote:
> >>> Windows Calculator does about twice that ("In Scientific mode,
> >>> Calculator is precise to 32 significant digits."), though of course
> >>> using it is tedious. If it helps, you can copy-paste operations into
> >>> it. Copy and paste this:
> >>> 2y2r=
> >>> (two to the power of 2 reciprocal - this calculates the square root of
> >>> two) gives:
> >>> 1.4142135623730950488016887242097
> >>> There's a list of keyboard shortcuts somewhere.
> >>>
> >>> If you're using integers, I recall that Python, rather than
> >>> overflowing, will handle arbitrarily large integer values.
> >>>
> >>>
> >>> On Fri, Jul 13, 2018 at 12:42 PM, Tim Ressel <timr at circuitabbey.com>
> wrote:
> >>>> Hey'all,
> >>>>
> >>>> I need to generate a lookup table with 64 bit values. Usually I use
> Excel
> >>>> but there is a problem: It only does 14 decimal places. I don't think
> this
> >>>> is enough. Any suggestions?
> >>>>
> >>>> --
> >>>> --Tim Ressel
> >>>> Circuit Abbey
> >>>> timr at circuitabbey.com
> >>>>
> >>>> _______________________________________________
> >>>> Synth-diy mailing list
> >>>> Synth-diy at synth-diy.org
> >>>> http://synth-diy.org/mailman/listinfo/synth-diy
> >>> _______________________________________________
> >>> Synth-diy mailing list
> >>> Synth-diy at synth-diy.org
> >>> http://synth-diy.org/mailman/listinfo/synth-diy
> >>>
> >> --
> >> --Tim Ressel
> >> Circuit Abbey
> >> timr at circuitabbey.com
> >>
> >> _______________________________________________
> >> Synth-diy mailing list
> >> Synth-diy at synth-diy.org
> >> http://synth-diy.org/mailman/listinfo/synth-diy
>
> --
> --Tim Ressel
> Circuit Abbey
> timr at circuitabbey.com
>
> _______________________________________________
> Synth-diy mailing list
> Synth-diy at synth-diy.org
> http://synth-diy.org/mailman/listinfo/synth-diy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://synth-diy.org/pipermail/synth-diy/attachments/20180713/d97742cb/attachment.htm>