Information capacity of stochastic pooling networks is achieved by discrete inputs

Abstract

Stochastic pooling networks (SPN) are sensor networks where multiple sensors make independently noisy and compressed measurements of the same information source, which are combined via pooling. Examples of SPNs range from nanoelectronics to biological sensory neurons. Here it is shown that optimal information transmission in SPNs with nodes that quantize to a finite number of states requires the input signal distribution to be discrete. This is illustrated numerically for a simple SPN consisting of N binary-quantizing sensors. The resultant information capacity is shown to be independent of the noise distribution when the signal distribution can be freely chosen, but to imply an optimal noise distribution if the signal distribution is fixed. While larger than the best performance of previously studied continuously valued input signals, the capacity does not scale faster than the previous best result of log_{2}(sqrt[N]) bits per channel use. It is also shown that a plot of the optimal input distribution contains bifurcations as N increases, and that suprathreshold stochastic resonance occurs when the mutual information is determined for a suboptimal noise distribution.