Founded by Claude Shannon, this theory quantifies the concept of information and forms the foundation of modern communication, data compression, and cryptography.
📖 Formal Introduction
Information Theory was founded by Claude Shannon in 1948. Its core is the mathematical quantification of information. Shannon Entropy measures information uncertainty.
H(X) = -Σ p(x) log₂ p(x)
其中 p(x) 是事件 x 发生的概率。熵越大，不确定性越高，信息量越大。
💬 Plain Language Introduction
Information content = degree of surprise. "The sun rises in the east" has little information (you already know it), while "It will snow tomorrow" has high information content (uncertain).
Examples:
- Coin flip: 50% heads/tails, entropy = 1 bit (maximum uncertainty)
- Certain event: 100% probability, entropy = 0 bit (no uncertainty)
- Weather forecast: More possibilities mean higher entropy
📡 通信理论
香农定理：信道容量 = 带宽 × log₂(1 + 信噪比)
🗜️ 数据压缩
ZIP、MP3、JPEG等压缩算法都基于信息论原理
🔐 密码学
完美保密需要密钥熵 ≥ 消息熵（一次性密码本）