
Entropy is a measure of disorder or uncertainty in a system, often related to energy dispersal and information.Entropy is a measure of disorder. Imagine a child's room: when toys are neatly arranged on shelves, entropy is low. But after a fun game, everything is scattered, and you can only restore order by making an effort. In the Universe, entropy always increases.
Rudolf Clausius introduced the term "entropy" in 1865, formulating the second law. Ludwig Boltzmann in the 1870s linked entropy with probability, proposing the formula S = k log W. The thought experiment of "Maxwell's demon" (1867) stimulated research into the connection between entropy and information, culminating in the works of Leo Szilard (1929) and Shannon (1948). In the 1970s, Jacob Bekenstein suggested that black holes have entropy proportional to the horizon area, and Stephen Hawking in 1974 confirmed this by discovering Hawking radiation and derived the exact formula.
According to the second law of thermodynamics, the total entropy of a closed system grows, approaching a maximum. This is due to the statistical nature of molecular motion: the system evolves from less probable states to more probable ones. Entropy is related to free energy: F = U - TS. In quantum theory, von Neumann entropy S(ρ) = -Tr(ρ ln ρ) generalizes the classical concept. For black holes, the Bekenstein-Hawking entropy S_BH = (k A)/(4 l_P²), where A is the horizon area, and l_P is the Planck length. This formula hints at the holographic principle.
If you shuffle a deck of cards, you will never return it to its original order just by shaking. That's entropy in action - disorder wins.
| ψ | variable | two-qubit pure state — quantum state |
| α | variable | amplitude of |00> component — probability amplitude |
| β | variable | amplitude of |01> component — probability amplitude |
| γ | variable | amplitude of |10> component — probability amplitude |
| δ | variable | amplitude of |11> component — probability amplitude |
| S | variable | entropy of the system — entropy joule per kelvin |
| Ω | variable | number of microstates corresponding to the macrostate — number of microstates |
| k_B | constant | relates temperature to energy per degree of freedom — boltzmann constant = 1.380649e-23 joule per kelvin |
| ln | operator | natural logarithm — natural logarithm |
| S | variable | Entropy — entropy joule per kelvin |
| W | variable | Number of microstates — number of states |
| k | constant | Relates temperature to energy in statistical mechanics — boltzmann constant = 1.380649e-23 joule per kelvin |
| \leq | operator | Indicates an upper bound — less than or equal |
| \ln | operator | Computes the logarithm to base e — natural logarithm |
| \eta | variable | Baryon to entropy ratio — baryon to entropy ratio |
| n_B | variable | Baryon number density — baryon number density per cubic metre |
| s | variable | Entropy density — entropy density joule per cubic metre per kelvin |
| S | variable | Shannon entropy — entropy |
| p_i | variable | Probability of the i-th outcome — probability |
| \sum_i | operator | Sum over all possible outcomes i — summation |
| \ln | operator | Natural logarithm (base e) — natural logarithm |
| \mathcal{L} | variable | Cross-entropy loss value — loss |
| y_i | variable | True probability (or one-hot label) for class i — probability |
| p_i | variable | Predicted probability for class i — probability |
| \sum_i | operator | Adds the terms for each class — summation |
| \log | operator | Computes the logarithm to base e — natural logarithm |









