lcrypto / shannon-and-polyanskiy-bound

Matlab and Python implementation of Different Information Theoretical Bounds: Shannon limit, Sphere Packaging Bound, Polyanskiy bound, Union Bound, Converse, Cramer-Rao

The GitHub repository contains MATLAB and Python (Jupyter) implementations of different information-theoretical bounds. These include the Shannon limit, sphere packaging bound, union bound, finite-length Polyanskiy bound (FER, spectral efficiency), G. Vazquez-Vilar Converse Bound, and Cramer-Rao Bound.

References:

[1] C. E. Shannon, "Probability of error for optimal codes in a Gaussian channel," in The Bell System Technical Journal, vol. 38, no. 3, pp. 611-656, May 1959, doi: 10.1002/j.1538-7305.1959.tb03905.x.

[2] G. Wiechman and I. Sason, "An Improved Sphere-Packing Bound for Finite-Length Codes Over Symmetric Memoryless Channels," in IEEE Transactions on Information Theory, vol. 54, no. 5, pp. 1962-1990, May 2008, doi: 10.1109/TIT.2008.920216.

[3] I. Sason and G. Wiechman, "Log-Domain Calculation of the 1959 Sphere-Packing Bound with Application to M-ary PSK Block Coded Modulation," 2006 IEEE 24th Convention of Electrical & Electronics Engineers in Israel, 2006, pp. 344-348, doi: 10.1109/EEEI.2006.321097.

[4] A. Valembois and M. P. C. Fossorier, "Sphere-packing bounds revisited for moderate block lengths," in IEEE Transactions on Information Theory, vol. 50, no. 12, pp. 2998-3014, Dec. 2004, doi: 10.1109/TIT.2004.838090.

[5] D. J. Costello and G. D. Forney, "Channel coding: The road to channel capacity," in Proceedings of the IEEE, vol. 95, no. 6, pp. 1150-1177, June 2007, doi: 10.1109/JPROC.2007.895188. Good review and details on union bound, power limited and bandwith limited regime of communication

[6] T. Erseghe, "On the Evaluation of the Polyanskiy-Poor–Verdú Converse Bound for Finite Block-Length Coding in AWGN," in IEEE Transactions on Information Theory, vol. 61, no. 12, pp. 6578-6590, Dec. 2015, doi: 10.1109/TIT.2015.2494061.

[7] G. Vazquez-Vilar, "Error Probability Bounds for Gaussian Channels Under Maximal and Average Power Constraints," in IEEE Transactions on Information Theory, vol. 67, no. 6, pp. 3965-3985, June 2021, doi: 10.1109/TIT.2021.3063311.

[8] Cramér, Harald (1946). Mathematical Methods of Statistics. Princeton, NJ: Princeton Univ. Press. ISBN 0-691-08004-6. OCLC 185436716.

[9] Rao, Calyampudi Radakrishna (1945). "Information and the accuracy attainable in the estimation of statistical parameters". Bulletin of the Calcutta Mathematical Society. 37: 81–89. MR 0015748.

Cramer-Rao bound is a lower limit on the variance of estimators of a deterministic parameter