"...it is possible to interpret the information carried by a message as essentially the negative of its entropy, and the negative logarithm of its probability. That is, the more probable the message, the less information it gives." Norbert Wiener, "The Human use of Human Beings", p.21
That is "The Curse of Learning", a principle dual to "The Curse of Dimensionality".