[sdiy] Accuracy with integer maths

[rburnett at richieburnett.co.uk]

There is a technique similar to what you describe used in digital signal 
processing called "fraction saving".  It's used to reduce the effects of 
state-quantisation in IIR digital filters.  Whenever you round the 
output from the IIR calculation up or down, you keep track of the error, 
and then you invert it and feed it back into the accumulator on the next 
iteration of the filter.  So if you rounded the value down this time, 
the error will be fed back into the accumulator and make it more likely 
that you will round the next value up.  It essentially shapes the noise 
from the truncation process so that it is kept away from DC and 
low-frequencies, and shifted up towards the Nyquist frequency where it's 
less noticeable.

It's mentioned here, but he calls it "First-order noise shaping"...

http://www.earlevel.com/main/2003/02/28/biquads/

Cheers,

-Richie,



On 2014-02-04 15:24, cheater00 . wrote:
> On 4 Feb 2014 16:14, <rburnett at richieburnett.co.uk> wrote:
>  > Tom, if you are going to truncate a 16-bit result to 8-bits
> accuracy you really should preserve the 9th bit so that you can test
> it and decide whether to round the value up or down.  Think what you
> would do if you were going to round a decimal number like "317" to the
> nearest ten.  Although you're only going to keep the top two digits
> (the '3' and the '1',) you need to test the units digit to see if the
> result should be rounded up to 320 or down to 310.
> 
> I wonder if you could noise-shape that - that is, the closer to 320
> you are, the more often you get 320.
> 
>>  (I usually implement rounding by adding in a half to produce a
> carry if the fractional part that you are going to chop off was
> greater than 0.5, then do a plain truncate.
> 
> Maybe instead one could add a random, gaussian, white noise value,
> with center at 0.5 and 2nd stddev at 1. This could dither truncation.
> Or use some other dithering noise.
> 
> Cheers,
>  D.