Forgotten Pioneers in the History of Statistics: al-Farahidi and al-Kindi

Anwer Khurshid

Department of Mathematical and Physical Sciences,
College of Arts and Sciences,
University of Nizwa

Introduction

Statistics is reflected in every facet of our daily lives, but have we ever considered or thought about the origin of statistics? Looking through the history of modern statistics over the past millennium, one cannot help but contemplate the evolution undergone by this field of study to achieve in its present form. The pioneering influence of greats including John Graunt, Blaise Pascal, Jacob Bernoulli, Abraham De Moivre, Pierre-Simon Laplace, Carl Fredreich Gauss, Karl Pearson, Ronald Fisher and Andrey Kolmogorov to name a few, have shaped statistics into what we know it to be today. Over the course of the history of statistics, we have observed diverse factors influencing the life and work of various early ‘statisticians’, and how the society, politics and religion of their era influenced their statistical contributions.

There are different accounts regarding the development of statistics; one of which is that statistical methods date as far back as 5th century BC. It is believed that Greek philosopher and scientist Aristotle initiated a statistical method over two millennium years ago. In fact, during early times, statistics were limited to information about states, such as scrutinizing the population, resources and wealth of states in ancient Greece. Hence, the word ‘statistics’ still preserves the root word ‘state’. There were no major advances in statistics for a long span of time preceding Ancient Greece until the scientific revolution in Europe in the 17th and 18th centuries. Historians agree that statistics and probability theories started to take shape during the renaissance. The origins of statistics and probability are seemingly lost in the mists of time.

Historically, early Islamic intellectual contributions have been a part of different academic disciplines. Broemeling (2011) published an article in a highly reputed journal The American Statistician which highlights unique contributions to the history of statistics literature that have been until now ignored. A large amount of these contributions are consolidated across diverse fields, which widen the dialogue on statistics and probability as well as provide continuity that is frequently absent from the history of statistics and probability. Two critical early Islamic texts, in particular, were early contributions to the establishment of a global concept of cryptography and eventually helped shaping development of two important disciplines, namely quantitative linguistics and statistics (Singh, 2000). During the course of history these disciplines have methodically nourished each other.

Although not exactly statistical in the contemporary sense of the term, they were early attempts in the field of statistics and probability (Broemeling, 2011). In examining the subject within his paper, Broemeling discusses the emergence of cryptology and give brief accounts of four Arab cryptologists who employed statistics, which include al-Khalil, al-Kindi, Ibn Adlan and Ibn Dunainir.

Innovators were seldom members of a single discipline, although a few were recognized for their contributions toward a particular discipline, such as al Tibri (c. 839-932) for his work in the field of history, and Abu Al Qasim al Zahrawi (c. 936-1013), who is regarded as the father of modern surgery. More commonly, many Islamic intellectuals were polymaths, or scholars who were trained in the application of multiple disciplined approaches and could synthesize these disciplines in their work. Two of the earliest contributors of statistics and probability, according to Broemeling (2011) are the polymaths al-Khalil al-Farahidi and Yaqub al-Kindi.

Al-Khalil ibn Ahmad al-Farahidi (718-786)

In the 8^th century, the usage of statistics and probability were first developed by al-Farahidi, an Omani linguist, grammarian who studied phonetics, lexicology, and syntax. He was one of the first Arabic language scholars who compiled the Arabic dictionary. Although considered the greatest linguistic scholar of his time, an unpublished manuscript shows that he was also a mathematician and studied cryptology. He wrote the ‘first’ book on cryptography and cryptanalysis entitled Kitab al-Muamma (‘Book of Cryptographic Messages’) which contains the first use of permutations and combinations (combinatorics) to list all possible Arabic words with and without vowels (Broemeling, 2011). His theories are now used by people in data encoding for crypto currencies and blockchain technology. Later Arab cryptographers employed the frequency analysis of text proposed by al-Farahidi to calculate the letter frequency of words in their own works (Rashed, 1996).

Al-Khalil bin Ahmed bin Al-Farahidi. 

Regrettably, the original text of Kitab al-Muamma appears to be lost and we only know about it from references in the works of other scientists. Arguably, the first and most famous work influenced by the Kitab al-Muamma is that of Abu Yusuf Yakub ibn Ishaq ibn Sabba al-Kindi, a famous philosopher, mathematician and astronomer, which is called ‘Treatise on the decryption of cryptographic messages’.

Yakub al-Kindi (801-873)

In the 9th-century a famous philosopher, astronomer and mathematician Yakub al-Kindi wrote a book entitled Risalah fi Istikhraj al-Mu'amma (Manuscript for the Deciphering Cryptographic Messages). This book is the oldest available book on cryptology, predating any other works by at least 300 years. A manuscript was recently rediscovered in the Sulaimaniyya Ottoman Archive in Istanbul and published by the Arab Academy of Damascus in 1987 (Mrayati et al., 2003). In this book al-Kindi laid the foundation for both statistics and frequency analysis and provided a detailed description on how to use these two skills to decipher encrypted messages (al-Kadit, 1992). This work was the first treatise applying statistical studies to (Arabic) languages. In this manuscript, al-Kindi covered the statistical analysis of the letters and letter combinations in Arabic. He explained how to find letter frequencies by using ‘Samples’ of the same language and used ‘tree diagrams’ to classify cipher types. He is attributed for developing the frequency analysis method (Singh, 2000) where variations in the frequency of letters could be investigated and utilized to break ciphers (i.e. cryptanalysis by frequency analysis). This was major breakthrough and frequency analysis became an essential in any statistical analysis.

Abu Yusuf, Ya`qub Ibn Ishaq Al-Kindi (The Philosopher of the Arabs).

When al-Kindi first expounded cryptanalysis it was innovative and extremely advanced for the time and was earliest known argument of statistics. He also made the initial known use of statistical inference, while he and other Arab cryptologists exploited early statistical methods for developing encrypted messages (Broemeling, 2011).

Concluding Remarks

Much of what is labeled ‘frequency analysis’, ‘sample size’, and  ‘tree diagram’ in al-Farahidi and al-Kindi represent the first serious use of what could now be termed as the early use of statistics and probability.

It has taken more than 1200years for the rest of us to discuss the intellectual ability of al-Farahidi and al-Kindi. The spontaneous use and apparent command of combinatorics (in what may as well be their first use in history) without funfare or elaboration raise a number of questions that are worth consideration by historians of statistics which are not sufficiently recognized (Smithline, 2012).