Su Wan/Wen You are walking in the neighborhood at dusk, and suddenly you see a furry thing moving in the grass. Approaching night, the light is not good, and you intuitively judge that this is most likely the tanuki cat you often encounter. You continue to observe, you see its small, pointed ears, and you deepen the belief that this is the beaver. At this point its striped tail was exposed and curled, and you were almost certain that it was the cat. Finally the furry figure flashed by and disappeared into the bushes. You go home and tell your family, "I probably just ran into that big beaver two days ago!" ”
A typical Bayesian might tell you that this observation and speculation, which is too ordinary to be ordinary, reflects that your remarkable brain has just made a complex Bayesian calculation: you have made a judgment about the cause of a particular phenomenon based on your intuition, based on the visual information that appears sequentially.
Bayesian Games: Mathematics, Thinking, and Artificial Intelligence
Author: Huang Liyuan
Publisher: Turing | People's Post and Telecommunications Publishing House
Translator: Fang Xian
Publication date: 2021-3
One
The 18th-century English philosopher Hume pointed out in his skepticism that "we have no way of knowing the relationship between cause and effect, but only that certain things will always be related." This idea of "correlation non-causality", embodied in an article he wrote in 1748 entitled "On Miracles", in which his assertion that eyewitness testimony could never prove miracles (i.e., the resurrection of Christ) may have aroused the attention of Thomas Jefferson, who was then a priest of the Calvinical Church. Thomas Bayes' note: Can't we really deduce the real cause of it from the results we observe? If we have formed a belief in advance, how much evidence needs to be observed to determine the correctness of that belief?
In his paper, Bayes imagined himself facing a table with a white ball on it, and then asked his assistant to randomly place a black ball on the table, asking the direction of the white ball relative to the black ball each time he put one. The position of the white ball is the cause of the black ball in a certain relative position, and this process of determining the possible position of the white ball when the position of the black ball relative to the white ball is known is a typical inverse probability estimation process that can respond to Hume's question. For Bayes, as long as the number of black balls is placed, the inductive speculation on the absolute position of the white ball can be infinitely approximated accurately, so the inductive mode of thinking based on the effect is not only useful, but not as Hume said, it is not irrational.
Bayes, whose main profession was theology, would not have imagined that he himself did not have the confidence to publish a high-profile probability theory (although it is reasonable to say that his conclusions did not contradict his belief that miracles can be proved backwards by enough evidence), and that the mathematical community behind him experienced controversy and silence, and finally two centuries later, computers were reborn as soon as they appeared, and in an era when humans were increasingly dependent on and adept at processing large amounts of data, the theorems named after him were widely used in medical diagnosis, machine learning, Cognitive neuroscience and other cutting-edge fields. This originally rough theoretical prototype, revised and generalized by many geniuses, is now regarded as a doctrine, a philosophy of knowledge, and even an abstract model that can summarize the cognitive work of the human brain.
Bayes' Game: Mathematics, Thinking, and Artificial Intelligence is a book that explains Bayes' theorem "universality." The original French version of the book, titled The Formula of Knowledge: A Unified Philosophy of Knowledge Based on Bayes' Theorem (LaFormuledusavoir: Unephilosophieunifiéedusavoirfondée surle théorème de Bayes), written by Lê Nguyênhoang, a young Asian-French mathematician who graduated from the École Polytechnique, He is now a researcher at the Swiss Federal Institute of Technology in Lausanne.
Huang Liyuan has long been concerned about the ethics of artificial intelligence, and is also an active and popular popular science video blogger, and his French video channel "Sci-ence4All" covers mathematics, computer science and physics. In his book, he passionately praises the practical validity and philosophical heuristics of Bayes' formula, calling it the "wisdom equation." This book tells us that the Bayesian method and the Bayesian philosophy of knowledge derived from the Bayesian formula are like a mind pass that can travel the world, and we can even say that everything can be "Bayesian".
Bayesian formulas are used to describe the probability of an event occurring under known conditions , and its expression is P(A| B）=P（A）P（B| A）/P（B）。 We can understand bayesian formula as a way of calculating the validity of a belief (e.g., some hypothesis, claim, or argument) based on existing reliable evidence (e.g., some observation, data, information), which is simply, the original belief + new evidence = the improved new belief. where P stands for probability, A for original belief, and B for new evidence or new condition. P(A) is the probability that A is true, also known as the a priori probability, which is the "subjective bias" that Bayesians cite as the advantage, but it is also the "weakness" of Bayesian opponents to attack the scientific nature of Bayesian statistics; P(B) is the probability that B is true, also known as marginal probability or the ratiometric function, which is the most difficult to calculate in the formula, P(B| A) indicates the probability that A is true when B, also known as likelihood or thought experiment terms that "require some imagination." This formula was actually rediscovered by the French mathematician Pierre-Simon Laplace, who is considered the father of Bayesianism. Perhaps like the calculus formula, which is the full name "Newton-Leibniz formula", the Bayesian formula should at least be called the "Bayesian-Laplace formula".
Two
How is bayesian formula applied? Take the field of medicine, for example. Medical tests usually use positive or negative test results to initially determine whether a subject is ill. In the real world, tests are rarely completely reliable, and there are problems with false positives and false negatives. Suppose a 75-year-old person is tested for a cancer with a 1% incidence at age 75, and his test results are positive, the person may be very desperate and feel that he must write a will. But tests are often not entirely reliable, assuming a 99 percent accuracy rate, meaning that 99 out of 100 people with cancer tested positive and 99 out of 100 healthy people tested negative. If the test is positive, what is the real likelihood of cancer? Bayes' theorem tells you that if you only test once and get a positive result, then the probability of his cancer is only 50%.
How does Bayes' formula calculate the relatively optimistic probability of 50%? The prior probability P(A) is 1% of the incidence of cancer at age 75;P( B| A) i.e. a 99% probability of testing positive in the case of cancer. So P(A) multiplied by P(B| A) Equal to 0.01 multiplied by 0.99, i.e. 0.0099. Denominator P(B) is the probability of a positive test result, including true positives and false positives, regardless of whether you have cancer or not, and the operation is slightly more complicated, and the result is 0.0198. Then eventually P(A| B）=P（A）P（B| A)/P(B) result, i.e. the probability of cancer at the same time as testing positive P (A| B) is 0.5, which is 50%. But if the second test result is still positive, and the Bayesian formula is applied again, the probability of cancer will increase from 50% to 99%. We see that the results of the first test will affect the results of the second test, which shows that iterative Bayes' theorem can gradually produce more accurate information, which also prompts us that any medical diagnosis needs to be tested multiple times to prevent misdiagnosis.
However, it is such a formula with unlimited potential that has also experienced the ups and downs of being snubbed and squeezed out by academic authorities. In the field of statistics, frequencists have regarded Bayesianism as a fierce enemy. Frequencyism, born in the 1920s, is actually the most classic statistical framework we learn in math textbooks. Frequencyism assumes that probability is a measurement of frequency, emphasizing that errors gradually disappear when the sample size becomes large enough. The core of frequencyism is to use the p-value to statistically test the credibility of a theoretical model, and this theoretical model is only scientific if it has undergone enough new data testing.
Frequencyism was at the time excellent in genetic research, more convinced that objectivity was the only golden rule, and very disgusted with Bayesianism, which brought in a priori probability, because it was equivalent to giving subjective confidence to a theory before it was tested. They see this subjectivity (what the author calls "bias" in the book) as a flood beast, arguing that statistical methods that contain subjectivity are not science at all.
Throughout the mid-20th century, dominated by frequency-based statisticians such as Egon Pearson and Ronald Fisher, terms such as "subjective," "a priori," and "Bayesian" were expelled from the department of statistics. A medical scientist has used Bayes' theorem to prove the harm of tobacco in causing lung cancer, but Fisher, a frequency-based big man who received funding from the tobacco industry, accused the scientist of lacking the control group and repeated experiments required by the frequency-based method in the study, and then reversed the order of cause and effect, proposing that potential lung cancer would cause people to smoke.
However, frequencyism also has unavoidable weaknesses. First of all, the p-value can be manipulated by a large number of experiments, and at the same time, for the prediction of many small-probability events, such as earthquakes, we can obtain measurement data and experimental opportunities are very few. The magic of Bayesian statistics is that they can get close to the exact value when the data is scarce. Thus, in the pre-computer era, when information was more difficult to collect and process, Bayesians remained the tool people could rely on when trying to grasp the uncertainty of rare events. In addition to the two well-known examples of the anonymous authorship of the Federalist Papers through word preferences and the search for the location of the Scorpio nuclear submarine in the vast Atlantic Ocean, Bayesian calculations have been used to estimate the probability of major accidents at nuclear power plants, predict the probability of major accidents in rocket launches, and so on.
Bayesianism is a philosophy of probability, which re-asks, what is probability? The probabilities that frequency doctrine holds need to be calculated by relying on the frequency with which events are repeated. But when the amount of repetition, that is, the data is insufficient, it is difficult for us to accurately predict the future according to the previous laws. For example, if the occurrence of the previous event is regarded as a set of sequences "1, 2, 4, 8, 16", then under simple reasoning, the next occurrence of the event should be 32. But when the number represents the number of parts of the circle divided into several straight lines connected by 2, 3, 4, and 5 points on the circumference, when the number of points is 6, the number of parts that appear next, that is, the event, should be 31, not 32.
Bayesian formula
Three
At what point are we sure that the laws we know will suddenly fail in our predictions? Most of the time people are reluctant to face this problem. Humanity's desire for certainty and control is written in the genes. Pre-scientific witchcraft was the ultimate in the pursuit of certainty. For example, according to the logic of life in the pre-modern period of the Azands, the doom that occurs with a small probability is attributed to the deliberate witchcraft of the enemy, that is, a traceable deterministic external cause. The biggest difference between science, especially probability cognition, compared to witchcraft, is to gradually master a dynamic method of getting along with uncertainty under the premise of accepting the existence of uncertainty. A good prediction should be able to calculate the probability of the occurrence of all candidate values for the next term in the above sequence, and the probability here should be the degree of confidence given to these possibilities. This is what bayesian formulas are trying to achieve.
The book emphasizes that our mindset of exploring the world and accumulating knowledge can largely be summarized by Bayes' theorem. For example, if you see that all the crows are black, you deduce that all the crows in the world are black, and you make assumptions and correct them according to your observations, which either increases the probability that the reasoning is correct or cuts it. Laplace, the father of Bayes, once said that probability theory is essentially nothing more than common sense in calculations. It evaluates in an accurate way that normal minds have grasped something through a certain intuition that is often not perceived.
Ultimately, Bayes' formula points to a philosophy of knowledge, and the authors even argue that "reason" can essentially be attributed to the application of Bayes' formula, so much so that belief in this philosophy can be called Bayesianism. Bayesianism is the assumption that all models, theories, or concepts of "reality" are nothing more than some belief, fiction, or poetry, and in particular, that "all models are wrong"; then the actual data should force us to adjust the importance assigned to different models, i.e., the degree of confidence; the point is that these confidence levels should be adjusted in a way that follows bayesian formulas as rigorously as possible. Bayes defined science more accurately than Popper's falsifiable theory.
Bayesian is especially important today, due to the improvement of computer performance, data collection and processing technology has far surpassed the human brain, business, policy and other fields are more dependent on the results of big data analysis, "the evolution of technology makes us re-examine the Bayes formula and its position in the knowledge edifice."
Bayesian computing is particularly beneficial in the analysis of massive data to simplify the complexity, grasp the big and let go of the small. From astrophysics and aerospace to genome sequencing and protein research; from cancer traceability and virus detection in medicine to image recognition and information encryption in computer science; from insurance, advertising, and logistics in business to elections and resource allocation in socio-political fields. Bayesian applications are ubiquitous.
Beyond the purely mathematical framework and cutting-edge technological applications, Bayesian philosophy is well suited as a mental compass for individuals to live in this rapidly changing era. Bayesian philosophy reflects human anxiety about uncertainty and provides a way to deal with uncertainty, that is, to accept uncertainty, boldly make intuitive assumptions without overconfidence, and not forget to use new evidence to constantly update their own assumptions, putting yourself in the process of searching for answers. As the Nobel laureate in physics Feynman once said, "I can live with doubt, uncertainty, and ignorance... I have some approximate answers, and some reasonable beliefs about the various questions, either high or low, but I'm not absolutely sure of anything. ”
Bayes are not without flaws. Although it gives a place for subjectivity, if the subjective starting point is pseudoscience or rumor, then it is possible to cite dubious evidence to support or even reinforce this dubious belief. But bayesianism is most powerful in its philosophical inclusiveness, emphasizing that "the forest of incompatible models is wiser than every tree in it." ”
The book mentions, "According to Bayes' theorem, no theory is perfect. In its place is an unfinished business, which is always under scrutiny and testing. "A Bayesian state is a state of balance between conviction and doubt, in which one is not easily convinced of rumors, can feel relieved of bad luck, and can courageously defend injustice. Bayes' formula may not be perfect, but Bayesianism may be the philosophy of knowledge best suited to this age of uncertainty.