[sdiy] Information Content of Signals

[jhaible]

Hi Grant,

I'm no expert for thermo dynamics whatsoever, but I'll give it
a try:

If you are transmitting a "bit", this includes a terrible amount of
redundancy already. Why? Because instead of defining two states
to carry one bit of information, you could define myriads of
levels to transmit a lot more information - as many levels, as
the background noise, quantisation of electrical charge, whatever,
allows. So in a way you *are* transmitting much more information
than just one bit - but everything except this one bit which you
defined is redundant.

Of course you're using this "terrible amount of redundacy" on
purpose, simply to increase your signal to noise ratio - where
"noise" can be signals from other (artificial) sources that are
much higher than your thermal noise.
So your signal is in competition not only with the raw remains
of the big bang, but also with other *signals*, which also have
that increadible amount of redundancy if you'd compare
them with background stuff, but which of course serve as
*noise*, not signal, here.

Now if you take measures to further increase redundancy (by
oversampling, by CRC codes or whatever), you're certainly
further increasing the factor of redundancy-overkill against
the "background", but this is just an irrelevant (pun) side effect.
Because your "enemy" is elsewhere: in other high level signals.

Does this make any sense ?

JH.


PS.: You're not working on a Wiard module that runs with a quantum
        computer, are you ? (;->)



-----Ursprüngliche Nachricht-----
Von: Grant Richter <grichter at asapnet.net>
An: Magnus Danielson <cfmd at swipnet.se>; <synth-diy at dropmix.xs4all.nl>
Gesendet: Samstag, 17. Mai 2003 09:36
Betreff: [sdiy] Information Content of Signals


> As a point of interest, I have been trying to understand the theoretical
> information content of signals. Contemplation of this has led me to a
> paradox, and I wonder if anyone can shed light on this.
>
> Information and thermodynamics are linked together by the expression of an
> information "bit" in a thermal sense. The best definition I have found
uses
> Boltzman's constant (minimum energy required to create a new thermal state
> at temperature).
>
> Using room temperature and the reference of 0 dB = 1 milliwatt, the
> theoretical information content of a 1 milliwatt-second signal would be
> (approx) 2^64 bits per second. So for sample rate of 192 kHz (~2^18)
> theoretical bit depth would not exceed 2^46 (64-18).
>
> The paradox comes from the idea of over-sampling a thermally maxed out
> information channel. What information would then be gathered? Would it be
> merely redundant information, or would the waste heat of the conversion
> apparatus become a new signal generator?
>
> Note that from an information theory standpoint, thermal noise is a
> naturally occurring signal (reverb tail of the Big Bang?) mixed with the
> human generated signal.
>
> Any speculation is appreciated.
>
>