Equations · equation

Shannon Entropy

Information Theory · 1948

Equation: H = −Σᵢ pᵢ·log₂ pᵢ
Field: Information Theory
Named After: Claude Shannon
Symbols: H information content in bits · pᵢ probability of each possible message · rare messages carry more information than expected ones
Year: 1948

Significance

How much surprise a message carries — the definition of the bit, and the hard floor below which no compression can squeeze data. The formula is Boltzmann's entropy in new clothes, a connection Shannon knew: information and thermodynamic disorder are the same mathematics.

Timeline

The Last Black Holes Go

Theories

Holographic Principle Quantum Information Theory Information Theory

Related

Qubit State Boltzmann Entropy Second Law of Thermodynamics Shannon–Hartley Channel Capacity Bayes' Theorem

Referenced by

equation

Bayes' Theorem Boltzmann Entropy Qubit State Second Law of Thermodynamics Shannon–Hartley Channel Capacity

event

The Last Black Holes Go

theory

Holographic Principle Information Theory Quantum Information Theory

Open on the Equations shelf — the interactive card, the filters, and everything alongside it.

From the Equations database in Notion, last edited 2026-08-15. This page is the row, rendered by the build; nothing on it is written by hand.