Information Theory

Information theory is a branch of the mathematical theory of Probability and mathematical Statistics, that deals with the concepts of information and information Entropy, communication systems, data transmission and rate distortion theory, cryptography, signal-to-noise ratios, data compression, and related topics. http://www.wikipedia.org/wiki/Information_theory

Claude Shannon

Gregory Bateson wrote In fact, what we mean by information - the elementary unit of information - is a Difference Which Makes A Difference.