19  Probability spaces

A probability space is a very general formal-mathematical model of a random process. The central importance of this model for probability theory and probabilistic inference follows from the fact that the probability space model provides guidance for translating arbitrary random processes about which one wants to reason quantitatively into the formal-mathematical system of probability theory. At the same time, the probability space model and the concepts built on it guarantee that the mechanics of probability calculus lead to logically meaningful quantitative conclusions about random processes. In this chapter, we introduce the concept of a probability space (Section 19.1) and then use probability functions (Section 19.2) to give first examples of modeling random processes (Section 19.3).

19.1 Definition and first properties

We begin with the definition of the probability space model after Kolmogoroff (1933), which we then explain in its individual parts from a frequentist perspective.

Definition 19.1 (Probability space) A probability space is a triple \((\Omega, \mathcal{A}, \mathbb{P})\), where

  • \(\Omega\) is an arbitrary nonempty set of outcomes \(\omega\) and is called the outcome set,

  • \(\mathcal{A}\) is a set of subsets of \(\Omega\) with the properties

    • \(\Omega \in \mathcal{A}\),
    • for all \(A\in \mathcal{A}\), also \(A^c := \Omega \setminus A \in \mathcal{A}\),
    • from \(A_1,A_2,... \in \mathcal{A}\) it follows that also \(\cup_{i=1}^\infty A_i \in \mathcal{A}\)

    holds, is called a \(\sigma\)-algebra on \(\Omega\), and is called the event system,

  • \(\mathbb{P}\) is a mapping of the form \(\mathbb{P}:\mathcal{A} \to [0,1]\) with the properties

    • non-negativity \(\mathbb{P}(A) \ge 0\) for all \(A \in \mathcal{A}\),
    • normalization \(\mathbb{P}(\Omega) = 1\), and
    • \(\sigma\)-additivity \(\mathbb{P}(\cup_{i=1}^\infty A_i) = \sum\limits_{i=1}^\infty \mathbb{P}(A_i)\) for pairwise disjoint \(A_i \in \mathcal{A}\)

    holds and is called a probability measure.

Outcome set and mechanics

We begin with explanations of the concept of the outcome set \(\Omega\) and the implicit mechanics of the probability space model. To make the introduction easier, in what follows we first primarily consider finite probability spaces, in which the cardinality of \(\Omega\) is not infinitely large. Thus let \(|\Omega|<\infty\), so that \(\Omega\) has only finitely many elements. For modeling the throw of a die, for example, one could define \(\Omega := \{1,2,3,4,5,6\}\).

For the finite outcome sets considered here, we choose \(\mathcal{A}=\mathcal{P}(\Omega)\). Every singleton \(\{\omega\}\) therefore belongs to the event system.

Behind the formal definition of the probability space model are the following frequentist-influenced assumptions about its mechanics as a model of a random process. We first imagine sequential runs of a random process, for example the repeated throwing of a die. According to the assumptions of the probability space model, in each of these runs exactly one \(\omega\) from \(\Omega\) is realized, that is, selected as actually occurring. For example, if one throws a die and a two appears, one says that a two was realized. The probability with which an \(\omega\) from \(\Omega\) is realized in a run is described by the value \(\mathbb{P}(\{\omega\}) \in [0,1]\). If, for example, \(\mathbb{P}(\{\omega\}) = 1\), then this \(\omega\) is realized in every run of the random process. If \(\mathbb{P}(\{\omega\}) = 0\), then this \(\omega\) is realized in no run of the random process, and if \(\mathbb{P}(\{\omega\}) = 1/2\), then \(\omega\) is realized in about half of the runs of the random process. In the model of throwing a fair die, one usually assumes \(\mathbb{P}(\{\omega\}) = 1/6\) for all \(\omega \in \Omega\). Here, for example, a four could be realized in the first run, a one in the second run, a five in the third run, then perhaps a four again, and so on.

Events and event system

An event \(A \in \mathcal{A}\) is best understood as a conceptual grouping of one or more outcomes. When throwing a die, possible events include “An even number of pips appears”, that is, \(\omega \in \{2,4,6\}\), “A number greater than two appears”, that is, \(\omega \in \{3,4,5,6\}\), or “A one or a five appears”, that is, \(\omega \in \{1,5\}\). Under the mechanics of the probability space, the event “An even number appears” occurs precisely when the \(\omega\) realized in a run of the random process throwing a die belongs to \(\{2,4,6\}\), for example when a four appears. The event “A number greater than two appears” can likewise be read as “A run of the random process throwing a die realizes an element of \(\{3,4,5,6\}\)”, that is, a three, four, five, or six appears. With the power set chosen here as the event system, the singletons \(\{\omega\}\) for \(\omega \in \Omega\) are also events. For example, “A one appears” corresponds to the realization \(\omega \in \{1\}\), and “A six appears” to \(\omega \in \{6\}\). When considering an outcome \(\omega \in \Omega\) as an event in this context, we call it an elementary event and write it as the one-element set \(\{\omega\}\).

Overall, this procedure for describing random events corresponds to the inherent goal of the definition of the probability space. Kolmogoroff (1933) writes that the actual objects of the subsequent considerations, the random events, have been defined as sets. This has the advantage that sets are mathematical objects with which one can work mathematically, so that an aspect of reality, a “random event”, has been translated into the model domain of mathematics.

The sole purpose of the event system \(\mathcal{A}\) is now to mathematically represent all events that can arise, based on a given outcome set, when an \(\omega \in \Omega\) is selected. Thus there should be no events in reality that were not considered in advance in the probability space model of the random process. If this were the case, the model would be deficient, since it could not assign probabilities to certain events occurring in reality. The event system \(\mathcal{A}\) should therefore be the complete set of all possible events for a given \(\Omega\). The requirement that \(\mathcal{A}\) should satisfy the so-called \(\sigma\)-algebra criteria for this purpose is intuitively motivated as follows.

  • It should first be ensured that \(\omega \in \Omega\) for an arbitrary \(\omega\), that is, that some outcome is realized, is one of the possible events. This corresponds to the property \(\Omega \in \mathcal{A}\).
  • For every event \(A\), the complementary event “\(A\) does not occur” should also belong to the event system. We therefore require that \(A\in\mathcal{A}\) implies \(A^c=\Omega\setminus A\in\mathcal{A}\). In particular, this implies \(\emptyset = \Omega \setminus \Omega \in \mathcal{A}\). Thus one event is that no elementary event occurs, although this happens only with probability zero, \(\mathbb{P}(\emptyset) = 0\). At least one elementary event therefore always occurs with certainty.
  • Finally, the combination of events should always also be an event. In modeling the throw of a die, for example, in addition to the events “An even number appears” and “A number greater than two appears”, the event “An even number appears and/or this number is greater than two” should also be an event. In its most general form, this corresponds to the requirement that from \(A_1,A_2,... \in \mathcal{A}\) it should follow that also \(\cup_{i=1}^\infty A_i \in \mathcal{A}\).

Even if the concepts of the event system and the \(\sigma\)-algebra may seem somewhat abstract, their definition in the practical modeling of random processes is usually not a major challenge, since suitable event systems have long been known both for finite outcome sets and for infinite countable and uncountable outcome sets. Thus, for outcome sets with finite cardinality, the power set of the outcome set always satisfies the requirements of an event system and can always be used to formulate a model of a random process with a finite outcome set. This is the statement of the following theorem.

Theorem 19.1 (Event system for finite outcome set) Let \(\Omega := \{\omega_1,\omega_2,...,\omega_n\}\) with \(n \in \mathbb{N}\) be a finite set. Then the power set \(\mathcal{P}(\Omega)\) of \(\Omega\) is a \(\sigma\)-algebra on \(\Omega\) and thus a suitable event system in the probability space model.

Proof. The power set of \(\Omega\) is the set of all subsets of \(\Omega\). We check the \(\sigma\)-algebra properties. First, \(\Omega\) is itself a subset of \(\Omega\), so the first property holds. Now let \(A\) be a subset of \(\Omega\). Then \(A^c = \Omega \setminus A\) is also a subset of \(\Omega\), so the second property holds as well. Finally, consider the union of subsets \(A_1,A_2,\ldots\subseteq\Omega\). Then \(\cup_{i=1}^{\infty}A_i\) is the set of all \(\omega\in\Omega\) for which \(\omega\in A_1\) or \(\omega\in A_2\) or …, where or is, as always, inclusive in the sense of and/or. Every \(\omega\in\cup_{i=1}^{\infty}A_i\) therefore belongs to at least one of the subsets of \(\Omega\) under consideration. These elements themselves form a subset of \(\Omega\), so the third \(\sigma\)-algebra property also holds.

For uncountable outcome sets such as the real numbers \(\mathbb{R}\) or \(n\)-dimensional real space \(\mathbb{R}^n\), constructing a suitable event system is more complex. We therefore refer to the advanced literature for formal developments (see Meintrup & Schäffler (2005), Schmidt (2009)). However, the Borel \(\sigma\)-algebra, which goes back to Borel (1898), provides a set system satisfying the requirements of a \(\sigma\)-algebra on uncountable outcome sets. We denote the Borel \(\sigma\)-algebra on \(\mathbb{R}\) by \(\mathcal{B}(\mathbb{R})\) and that on \(\mathbb{R}^n\) by \(\mathcal{B}(\mathbb{R}^n)\). The Borel \(\sigma\)-algebra on \(\mathbb{R}\) is the smallest \(\sigma\)-algebra containing all open subsets of \(\mathbb{R}\). In particular, it contains all intervals and singletons, but not all subsets of \(\mathbb{R}\). Correspondingly, the Borel \(\sigma\)-algebra on \(\mathbb{R}^n\) is generated by its open subsets.

Overall, this leads to the following procedure for selecting event systems depending on the outcome set \(\Omega\). If \(\Omega\) is finite, one chooses as event system \(\mathcal{A}\) the power set \(\mathcal{P}(\Omega)\) of \(\Omega\). If \(\Omega\) is given by \(\mathbb{R}\), one chooses as event system \(\mathcal{A}\) the Borel \(\sigma\)-algebra \(\mathcal{B}(\mathbb{R})\). Finally, if \(\Omega\) is given by \(\mathbb{R}^n\), one chooses for \(\mathcal{A}\) the Borel \(\sigma\)-algebra \(\mathcal{B}(\mathbb{R}^n)\). In special cases and for very special outcome sets \(\Omega\), one may want to deviate from this procedure, but in general the three cases considered cover most applications.

Probability measure \(\mathbb{P}\)

With \(\Omega\) and \(\mathcal{A}\), which in tuple form \((\Omega,\mathcal{A})\) are also called a measurable space, we have so far examined more closely the structural basis of a probability space model. Many probability spaces, for example for random processes concerning real numbers, are identical with respect to their measurable space. The probability measure \(\mathbb{P}\) now represents the probabilistic characteristics of a probability space model and thus forms the functional basis of such a model. In what follows, especially after introducing the concepts of random variables and random vectors, we will encounter many different probability measures. At this point, we first want to consider only general properties of probability measures.

With the definition \[\begin{equation} \mathbb{P}: \mathcal{A} \to [0,1], A \mapsto \mathbb{P}(A) \end{equation}\] shows first that a probability measure is defined on a set of sets and assigns probabilities, that is, values in \([0,1]\), to its elements, the sets \(A \in \mathcal{A}\). When choosing \(\mathcal{A}=\mathcal{P}(\Omega)\) for finite \(\Omega\), every singleton \(\{\omega\}\) belongs to the event system. Thus \(\mathbb{P}\) assigns a probability to every elementary event. We also emphasize that, by definition, the probability \(\mathbb{P}(A)\) of an event \(A\in\mathcal{A}\) is a number in \([0,1]\), rather than a percentage or ratio. We now examine the defining properties of non-negativity, normalization, and \(\sigma\)-additivity of \(\mathbb{P}\) more closely.

The non-negativity \(\mathbb{P}(A) \ge 0\) for all \(A \in \mathcal{A}\) is of course implicit in the definition \([0,1]\) of the target set of \(\mathbb{P}\). In fact, the mapping form of \(\mathbb{P}\) is an addition we have made to the formulation of Kolmogoroff (1933) in the interest of clarity. Formally, the form of the target set of \(\mathbb{P}\) actually follows from the defining properties of non-negativity, normalization, and \(\sigma\)-additivity of \(\mathbb{P}\).

The normalization \(\mathbb{P}(\Omega) = 1\) corresponds to the fact that, in every run of a random process, the realized \(\omega\) is certainly an element of \(\Omega\). Thus in every run of a random process an elementary event occurs and, depending on the nature of the measurable space, many others as well. In the model of throwing a die, for example, the outcome/elementary event realized in a run of the random process is an element of \(\Omega := \{1,2,3,4,5,6\}\) with probability \(1\). If the realized outcome is, for example, a one, then in addition to the event “A one appears”, the events “An odd number appears”, “A number less than three appears”, “An odd number less than three appears”, and many others also occur.

The \(\sigma\)-additivity of the probability measure \(\mathbb{P}\), finally, forms the foundation of probability calculus, that is, the basis for calculating with probabilities. The \(\sigma\)-additivity of \(\mathbb{P}\) makes it possible to calculate the probabilities of other events from already known event probabilities. Based on a definition of \(\Omega, \mathcal{A}\), and \(\mathbb{P}\), one can therefore calculate probabilities for all possible events of a probability space model. Whether these probabilities actually have anything to do with real events concerning a random process in reality depends on whether the modeling is reasonably successful or not. However, the calculated probabilities are at least determined rationally, that is, according to the rules of reason, namely logic and mathematics. Overall, the probability model thus allows inferential thinking about random processes, that is, about phenomena subject to uncertainty.

Why countability matters for \(\sigma\)-algebras and \(\sigma\)-additivity

Both the definition of a \(\sigma\)-algebra and \(\sigma\)-additivity concern countable families of events. But why does the definition of a probability space not require closure under arbitrary unions and additivity for arbitrary sums, even for uncountable outcome sets? We explain this using the uniform distribution on \(\Omega=[0,1]\), where we want to assign each subinterval a probability equal to its length. For \(0\leq a\leq b\leq1\), we therefore require \[\begin{equation} \mathbb{P}([a,b]) := b-a \end{equation}\] This immediately implies \[\begin{equation} \mathbb{P}(\{a\})=\mathbb{P}([a,a])=a-a=0 \quad\text{and}\quad \mathbb{P}([0,1])=1. \end{equation}\]

Why countable unions \(\bigcup_{i=1}^{\infty}A_i\)?

Closure of a \(\sigma\)-algebra under countable unions ensures that events described by a sequence of events also belong to the event system. For example, the statement “At least one of the events \(A_1,A_2,\ldots\) occurs” should again describe an event, and each individual \(A_i\) may itself contain uncountably many outcomes. Requiring closure under arbitrary unions would be considerably stronger. For the uniform distribution on \([0,1]\), all singletons \(\{x\}\) belong to the event system. Since every subset \(A\subseteq[0,1]\) can be written as the union of its singletons, this stronger requirement would mean that every subset must be an event: \[\begin{equation} A=\bigcup_{x\in A}\{x\}. \end{equation}\]

However, there are subsets of the interval to which no length can be assigned while preserving the usual properties of a length measure, in particular translation invariance and countable additivity. Examples are Vitali sets (see Billingsley (1995), Section 3, p. 45).

This also explains why probability models on \(\mathbb{R}\) usually use the Borel \(\sigma\)-algebra \(\mathcal{B}(\mathbb{R})\) as their event system. It is the smallest \(\sigma\)-algebra containing all open subsets of \(\mathbb{R}\). In particular, it contains all intervals and singletons, together with all sets obtained from them by repeated complementation and countable unions or intersections. It therefore includes ordinary events such as “The outcome lies between \(a\) and \(b\)” without requiring all subsets of \(\mathbb{R}\) to be events. We have \[\begin{equation} \mathcal{B}(\mathbb{R})\subset\mathcal{P}(\mathbb{R}). \end{equation}\]

The power set \(\mathcal{P}(\mathbb{R})\) is not ruled out as an event system in principle. For example, one can define a probability measure on it that assigns probability one to a single outcome. However, it is too large if one wishes to assign the usual length-based probabilities to all subsets while preserving the properties above. The Borel \(\sigma\)-algebra provides a suitable starting point for this purpose. Larger event systems are also possible, such as the \(\sigma\)-algebra of Lebesgue-measurable sets, which likewise does not comprise the entire power set (see Billingsley (1995), Sections 2 and 3).

Why countable summation \(\sum_{i=1}^{\infty}\mathbb{P}(A_i)\)?

For the uniform distribution on \([0,1]\), we have \(\mathbb{P}(\{x\})=0\) for every \(x\in[0,1]\). The entire interval, however, has probability one. Requiring additivity for uncountable families of pairwise disjoint events would yield the contradiction \[\begin{equation} 1=\mathbb{P}([0,1]) =\mathbb{P}\left(\bigcup_{x\in[0,1]}\{x\}\right) =\sum_{x\in[0,1]}\mathbb{P}(\{x\}) =0. \end{equation}\]

The property of \(\sigma\)-additivity leads to no such contradiction, since the union of all singletons in the interval is uncountable. Restricting additivity to countable sums therefore allows probability models in which individual outcomes have probability zero, while their entirety has probability one (see Billingsley (1995), Sections 2 and 3).

Finally, we want to illustrate probability calculations based on the \(\sigma\)-additivity of \(\mathbb{P}\) with two basic examples. The first example states that the probability that the \(\omega\) realized in a run of a random process is not an element of the outcome set is equal to zero.

Theorem 19.2 (Probability of the impossible event) Let \((\Omega, \mathcal{A}, \mathbb{P})\) be a probability space. Then \[\begin{equation} \mathbb{P}(\emptyset) = 0. \end{equation}\]

Proof. For \(i = 1,2,...\), let \(A_i := \emptyset\). Then \(A_1,A_2,...\) is a sequence of disjoint events because \(\emptyset \cap \emptyset = \emptyset\), and \(\cup_{i=1}^\infty A_i = \emptyset\). With the \(\sigma\)-additivity of \(\mathbb{P}\) it then follows that \[\begin{equation} \mathbb{P}(\emptyset) = \mathbb{P}\left(\cup_{i=1}^\infty A_i\right) = \sum_{i=1}^\infty \mathbb{P}\left(A_i\right) = \sum_{i=1}^\infty \mathbb{P}\left(\emptyset\right). \end{equation}\] Thus the infinite addition of the number \(\mathbb{P}(\emptyset) \in [0,1]\) should again yield \(\mathbb{P}(\emptyset)\). This is only possible if \(\mathbb{P}(\emptyset) = 0\).

If the die falls out of reach under the sofa, its number of pips may remain unobserved. To represent such a trial outcome explicitly, the model must be extended with a corresponding outcome. The original model assigns no probability to this additional outcome.

As a second example, we want to show that \(\sigma\)-additivity, which in the definition of the probability space is defined only for the union of infinitely many disjoint events, implies the \(\sigma\)-additivity of finitely many disjoint events as they often occur in applications.

Theorem 19.3 (\(\sigma\)-additivity for finite sequences of disjoint events) Let \((\Omega, \mathcal{A}, \mathbb{P})\) be a probability space and let \(A_1,...,A_n\) be a finite sequence of pairwise disjoint events. Then \[\begin{equation} \mathbb{P}(\cup_{i=1}^n A_i) = \sum_{i=1}^n \mathbb{P}(A_i). \end{equation}\]

Proof. We consider an infinite sequence of pairwise disjoint events \(A_1, A_2, ...\), where for an \(n\in \mathbb{N}\) it is to hold that \(A_i := \emptyset\) for \(i>n\). Then, with the \(\sigma\)-additivity of \(\mathbb{P}\), it first follows that \[\begin{equation} \mathbb{P}\left(\cup_{i=1}^n A_i\right) = \mathbb{P}\left(\cup_{i=1}^\infty A_i\right) = \sum_{i=1}^\infty \mathbb{P}\left(A_i\right) = \sum_{i=1}^n \mathbb{P}\left(A_i\right) + \sum_{i=n+1}^\infty \mathbb{P}\left(A_i\right). \end{equation}\] With \(\mathbb{P}\left(A_i\right) = \mathbb{P}(\emptyset) = 0\) for \(i = n+1, n+2,...\), it follows directly that \[\begin{equation} \mathbb{P}\left(\cup_{i=1}^n A_i\right) = \sum_{i=1}^n \mathbb{P}\left(A_i\right) + 0 = \sum_{i=1}^n \mathbb{P}\left(A_i\right). \end{equation}\]

To prepare the concept of the joint probability of events, we finally consider one further important property of \(\sigma\)-algebras: in addition to being closed under countable unions, \(\mathcal{A}\) is also closed under countable intersections. This is a direct consequence of the closure of \(\mathcal{A}\) under complements and countable unions and is recorded in the following theorem.

Theorem 19.4 (Closure of \(\sigma\)-algebras under intersections) Let \(\mathcal{A}\) be a \(\sigma\)-algebra on a set \(\Omega\). Then from \(A,B \in \mathcal{A}\) it follows that also \(A \cap B \in \mathcal{A}\). More generally, for \(A_1,A_2,... \in \mathcal{A}\), also \(\cap_{i=1}^\infty A_i \in \mathcal{A}\).

Proof. Let \(A_1,A_2,\ldots\in\mathcal{A}\). By statement 1 of Theorem 2.8, \[\begin{equation} \bigcap_{i=1}^{\infty}A_i =\left(\bigcup_{i=1}^{\infty}A_i^c\right)^c. \end{equation}\] Since \(\mathcal{A}\) is closed under complementation, \(A_i^c\in\mathcal{A}\) for every \(i\in\mathbb{N}\). Closure under countable unions implies \[\begin{equation} \bigcup_{i=1}^{\infty}A_i^c\in\mathcal{A}. \end{equation}\] Taking the complement again gives \[\begin{equation} \bigcap_{i=1}^{\infty}A_i =\left(\bigcup_{i=1}^{\infty}A_i^c\right)^c\in\mathcal{A}. \end{equation}\] In particular, for \(A,B\in\mathcal{A}\), setting \(A_1=A\) and \(A_i=B\) for every \(i\geq2\) gives \[\begin{equation} A\cap B=\bigcap_{i=1}^{\infty}A_i\in\mathcal{A}. \end{equation}\]

According to Theorem 19.4, in addition to the probabilities \(\mathbb{P}(A), \mathbb{P}(B)\) and \(\mathbb{P}(A \cup B)\) with \(A,B \in \mathcal{A}\), the probability \(\mathbb{P}(A \cap B)\) is therefore also well-defined.

19.2 Probability functions

In this section, with probability functions, we want to learn about a way of specifying probability measures for probability spaces with finite outcome sets. In the sections on random variables and random vectors, we will encounter probability functions again under the name probability mass functions. We first define the concept of a probability function as follows.

Definition 19.2 (Probability function) Let \(\Omega\) be a finite set. Then a function \(\pi:\Omega \to [0,1]\) is called a probability function if \[\begin{equation} \sum_{\omega \in \Omega} \pi(\omega) = 1. \end{equation}\] Furthermore, let \(\mathbb{P}\) be a probability measure on \((\Omega,\mathcal{P}(\Omega))\). Then the function defined by \[\begin{equation} \pi: \Omega \to [0,1], \omega \mapsto \pi(\omega) := \mathbb{P}(\{\omega\}) \end{equation}\] is called the probability function of \(\mathbb{P}\) on \(\Omega\).

We note that because \(\mathbb{P}\) is \(\sigma\)-additive by definition, in particular \[\begin{equation} \mathbb{P}(\Omega) = \mathbb{P}(\cup_{\omega \in \Omega}\{\omega\}) = \sum_{\omega \in \Omega}\mathbb{P}(\{\omega\}) = \sum_{\omega \in \Omega}\pi(\omega) = 1. \end{equation}\] The following theorem provides the formal basis for constructing probability measures through probability functions. In particular, it states that for finite \(\Omega\), the probabilities of all possible events can be calculated from the probabilities of the elementary events \(\pi(\omega)\).

Theorem 19.5 (Construction of a probability measure) Let \(\Omega\) be a finite nonempty set, \(\mathcal{A}:=\mathcal{P}(\Omega)\), and let \(\pi:\Omega\to[0,1]\) be a probability function. Then \[\begin{equation} \mathbb{P}:\mathcal{A}\to[0,1],\quad A\mapsto\mathbb{P}(A):=\sum_{\omega\in A}\pi(\omega) \end{equation}\] defines a probability measure on \((\Omega,\mathcal{A})\) with \(\pi\) as the probability function of \(\mathbb{P}\).

Proof. We verify the three properties of a probability measure.

  1. Non-negativity. For every \(A\in\mathcal{A}\), since \(\pi(\omega)\in[0,1]\) for all \(\omega\in\Omega\), \[\begin{equation} \mathbb{P}(A)=\sum_{\omega\in A}\pi(\omega)\geq0. \end{equation}\]

  2. Normalization. Since \(\pi\) is a probability function, its values sum to one over \(\Omega\). Hence \[\begin{equation} \mathbb{P}(\Omega)=\sum_{\omega\in\Omega}\pi(\omega)=1. \end{equation}\]

  3. \(\sigma\)-additivity. Let \(A_1,A_2,\ldots\in\mathcal{A}\) be pairwise disjoint events. Then \[\begin{equation} \mathbb{P}\left(\bigcup_{i=1}^{\infty}A_i\right) =\sum_{\omega\in\bigcup_{i=1}^{\infty}A_i}\pi(\omega) =\sum_{i=1}^{\infty}\sum_{\omega\in A_i}\pi(\omega) =\sum_{i=1}^{\infty}\mathbb{P}(A_i). \end{equation}\] For the second equality, note that pairwise disjointness means \(\omega\in\bigcup_{i=1}^{\infty}A_i\) is equivalent to the existence of exactly one \(i\in\mathbb{N}\) with \(\omega\in A_i\). Every outcome in the union therefore belongs to exactly one event \(A_i\), where the corresponding index \(i\) depends on \(\omega\). For a fixed \(i\), the inner sum \(\sum_{\omega\in A_i}\pi(\omega)\) includes only the outcomes of that one event. The outer sum over \(i\) then adds the contributions of all events. The summands are merely grouped by the events \(A_i\), with each summand \(\pi(\omega)\) counted exactly once. Since \(\Omega\) is finite, only finitely many of the \(A_i\) are nonempty, so only finitely many summands occur in total.

Thus \(\mathbb{P}\) is a probability measure on \((\Omega,\mathcal{A})\).

Thus, for a given outcome set \(\Omega\), defining a function \(\pi:\Omega\to[0,1]\) whose values \(\pi(\omega)\) sum to one over all \(\omega\in\Omega\), and interpreting each value as the probability \(\mathbb{P}(\{\omega\})\) of the elementary event \(\{\omega\}\in\mathcal{A}\), constructs a probability measure. Conversely, for a finite outcome set \(\Omega\), defining a probability \(\mathbb{P}(\{\omega\})\in[0,1]\) for every elementary event and ensuring \(\mathbb{P}(\Omega)=1\) implicitly satisfies the requirements of a probability function. In this case, the \(\sigma\)-additivity of \(\mathbb{P}\) for pairwise disjoint elementary events can be used directly to calculate probabilities of compound events. We illustrate both this approach and the use of a probability function in Section 19.3.

19.3 Examples with finite outcome space

From what has been said so far, the following procedure for modeling a random process using a probability space \((\Omega, \mathcal{A}, \mathbb{P})\) can be summarized:

  1. In a first step, one considers a sensible definition of the outcome set \(\Omega\), that is, of the outcomes or elementary events that are to be realized in each run of the random process.

  2. Next, choose a suitable event system. For a finite outcome set, the power set \(\mathcal{P}(\Omega)\) is a natural choice.

  3. For a finite outcome set, first define a probability function \(\pi:\Omega\to[0,1]\) whose values sum to one over \(\Omega\). From this, construct a probability measure by \[\begin{equation} \mathbb{P}:\mathcal{A}\to[0,1],\quad A\mapsto\mathbb{P}(A):=\sum_{\omega\in A}\pi(\omega) \end{equation}\] In particular, the probabilities of the elementary events satisfy \[\begin{equation} \mathbb{P}(\{\omega\})=\pi(\omega) \quad\text{for all }\omega\in\Omega. \end{equation}\]

We now illustrate this procedure using examples of probability spaces with finite outcome sets.

Example 19.1 (Throwing one die) We model throwing one die. It is certainly sensible to define the outcome set as \(\Omega := \{1,2,3,4,5,6\}\). However, the definition

Because this is a finite outcome set, we choose the power set \(\mathcal{P}(\Omega)\) as the \(\sigma\)-algebra \(\mathcal{A}\). Then \(\mathcal{A}\) automatically contains all possible events. The cardinality of \(\mathcal{A} := \mathcal{P}(\Omega)\) is \(|\mathcal{P}(\Omega)| = 2^{|\Omega|} = 2^6 = 64\). In Table 19.1 we list six of these 64 events in their verbal description and as a subset \(A\) of \(\Omega\).

Table 19.1: Selected events in the model of throwing a die.
Description Set notation
Any number of pips appears \(\omega \in \Omega\)
No number of pips appears \(\omega \in \emptyset\)
A number greater than 4 appears \(\omega \in \{5,6\}\)
An even number appears \(\omega \in \{2,4,6\}\)
A six appears \(\omega \in \{6\}\)
A one, three, or six appears \(\omega \in \{1,3,6\}\)

This completes the definition of the measurable space \((\Omega, \mathcal{A})\) in the modeling of throwing a die.

For the model of a fair die, define the probability function \[\begin{equation} \pi:\Omega\to[0,1],\quad \omega\mapsto\pi(\omega):=\frac{1}{|\Omega|}=\frac{1}{6}. \end{equation}\] By Theorem 19.5, \[\begin{equation} \mathbb{P}:\mathcal{A}\to[0,1],\quad A\mapsto\mathbb{P}(A):=\sum_{\omega\in A}\pi(\omega) \end{equation}\] then defines a probability measure on \((\Omega,\mathcal{A})\). In particular, the probabilities of the elementary events satisfy \[\begin{equation} \mathbb{P}(\{\omega\})=\pi(\omega)=\frac{1}{6} \quad\text{for all }\omega\in\Omega. \end{equation}\]

In the case of the fair die, for example, the probability of the event “An even number of pips appears” is then obtained with the \(\sigma\)-additivity of \(\mathbb{P}\) as \[\begin{equation} \mathbb{P}(\{2,4,6\}) = \mathbb{P}(\{2\} \cup \{4\} \cup \{6\}) = \mathbb{P}(\{2\}) + \mathbb{P}(\{4\}) + \mathbb{P}(\{6\}) = \frac{1}{6} + \frac{1}{6} + \frac{1}{6} = \frac{3}{6}. \end{equation}\]

For a model of a biased die that favors a six, we could instead define the probability function \[\begin{equation} \pi:\Omega\to[0,1],\quad \omega\mapsto\pi(\omega):= \begin{cases} \frac{1}{8}, & \omega\in\{1,2,3,4,5\},\\ \frac{3}{8}, & \omega=6 \end{cases} \end{equation}\] The probability measure constructed by \(\mathbb{P}(A):=\sum_{\omega\in A}\pi(\omega)\) then satisfies \[\begin{equation} \mathbb{P}(\{\omega\})=\frac{1}{8} \quad\text{for }\omega\in\{1,2,3,4,5\} \qquad\text{and}\qquad \mathbb{P}(\{6\})=\frac{3}{8}. \end{equation}\] In the case of the above model of a biased die, by contrast, the probability of the same event is \[\begin{equation} \mathbb{P}(\{2,4,6\}) = \mathbb{P}(\{2\} \cup \{4\} \cup \{6\}) = \mathbb{P}(\{2\}) + \mathbb{P}(\{4\}) + \mathbb{P}(\{6\}) = \frac{1}{8} + \frac{1}{8} + \frac{3}{8} = \frac{5}{8}. \end{equation}\] The event considered has a higher probability in the model of the biased die than in the model of the fair die, which is intuitively sensible because six is an even number.

Example 19.2 (Simultaneously throwing a blue and a red die) We now want to model the simultaneous throwing of a blue and a red die. For this purpose it is sensible to define the outcome set as \[\begin{equation} \Omega := \{(r,b)| r \in \{1,2,3,4,5,6\}, b \in \{1,2,3,4,5,6\}\} \end{equation}\] with cardinality \(|\Omega| = 36\). Here, \(r\) denotes the number of pips on the red die and \(b\) the number on the blue die.

Again, the power set of \(\Omega\) is a natural choice of \(\sigma\)-algebra, so we define \(\mathcal{A}:=\mathcal{P}(\Omega)\). The number of possible events in this model is \(|\mathcal{A}|=2^{|\Omega|}=2^{36}=68{,}719{,}476{,}736\). Table 19.2 lists five of these events, both verbally and as subsets of \(\Omega\).

Table 19.2: Selected events in the model of throwing a red and a blue die.
Description Set notation
The red die shows a three \(\omega \in \{(3,1),(3,2),(3,3),(3,4),(3,5),(3,6)\}\)
The blue die shows a three \(\omega \in \{(1,3),(2,3),(3,3),(4,3),(5,3),(6,3)\}\)
Both dice show a three \(\omega \in \{(3,3)\}\)
Doubles are thrown \(\omega \in \{(1,1),(2,2),(3,3),(4,4),(5,5),(6,6)\}\)
The sum of the numbers is four \(\omega \in \{(1,3),(3,1),(2,2)\}\)

This completes the definition of the measurable space \((\Omega,\mathcal{A})\). For the model of two independent fair dice, define the probability function \[\begin{equation} \pi:\Omega\to[0,1],\quad \omega\mapsto\pi(\omega):=\frac{1}{|\Omega|}=\frac{1}{36}. \end{equation}\] By Theorem 19.5, \[\begin{equation} \mathbb{P}:\mathcal{A}\to[0,1],\quad A\mapsto\mathbb{P}(A):=\sum_{\omega\in A}\pi(\omega) \end{equation}\] then defines a probability measure on \((\Omega,\mathcal{A})\). In particular, the probabilities of the elementary events satisfy \[\begin{equation} \mathbb{P}(\{\omega\})=\pi(\omega)=\frac{1}{36} \quad\text{for all }\omega\in\Omega. \end{equation}\]

Under this probability measure, the probability of the event “The sum of the numbers that appear is four”, for example, is obtained with the \(\sigma\)-additivity of \(\mathbb{P}\) as \[\begin{equation} \begin{split} \mathbb{P}\left(\{(1,3),(3,1),(2,2)\}\right) & = \mathbb{P}\left(\{(1,3)\}\cup \{(3,1)\} \cup \{(2,2)\}\right) \\ & = \mathbb{P}\left(\{(1,3)\}\right) + \mathbb{P}\left(\{(3,1)\}\right) + \mathbb{P}\left(\{(2,2)\}\right) \\ & = 1/36 + 1/36 + 1/36 \\ & = 1/12. \end{split} \end{equation}\]

Example 19.3 (Tossing a coin) We model tossing a coin, one side of which shows heads and the other side tails. It is sensible to define the outcome set as \(\Omega := \{H,T\}\), where \(H\) represents “heads” and \(T\) represents “tails”. However, any other binary definition of \(\Omega\) would also be possible, e.g. \(\Omega := \{0,1\}, \Omega := \{-1,1\}\), or \(\Omega := \{1,2\}\).

The power set \(\mathcal{A} := \mathcal{P}(\Omega)\) contains all possible events. In this case, we can easily list the entire set system \(\mathcal{A}\), as shown in Table 19.3.

Table 19.3: Event system \(\mathcal{A}\) in the model of tossing a coin.
Description Set notation
Neither heads nor tails appears \(\omega \in \emptyset\)
Heads appears \(\omega \in \{H\}\)
Tails appears \(\omega \in \{T\}\)
Heads or tails appears \(\omega \in \{H,T\}\)

The definition of the measurable space \((\Omega, \mathcal{A})\) is therefore complete.

For the model of a coin, define the probability function with \(\mu\in[0,1]\) \[\begin{equation} \pi:\Omega\to[0,1],\quad \omega\mapsto\pi(\omega):= \begin{cases} \mu, & \omega=H,\\ 1-\mu, & \omega=T. \end{cases} \end{equation}\] By Theorem 19.5, \[\begin{equation} \mathbb{P}:\mathcal{A}\to[0,1],\quad A\mapsto\mathbb{P}(A):=\sum_{\omega\in A}\pi(\omega) \end{equation}\] then defines a probability measure on \((\Omega,\mathcal{A})\). In particular, the probabilities of the elementary events satisfy \[\begin{equation} \mathbb{P}(\{H\})=\pi(H)=\mu \qquad\text{and}\qquad \mathbb{P}(\{T\})=\pi(T)=1-\mu. \end{equation}\] We note that because \(\mathbb{P}\) is \(\sigma\)-additive by definition, in particular \[\begin{equation} \mathbb{P}(\Omega)=1 \Leftrightarrow \mathbb{P}(\{H\})+\mathbb{P}(\{T\})=1 \Leftrightarrow \mathbb{P}(\{T\})=1-\mathbb{P}(\{H\}). \end{equation}\] Thus, when the probability of the elementary event \(\{H\}\) is specified, the probability of the elementary event \(\{T\}\) is immediately specified as well, and conversely, of course. For the model of a fair coin, one would choose \(\mathbb{P}(\{H\}) = \mathbb{P}(\{T\}) := 1/2\). In this case, the probabilities of all possible events are

For the model of a fair coin, choose \(\mu:=1/2\), giving the probability function \[\begin{equation} \pi:\Omega\to[0,1],\quad \omega\mapsto\pi(\omega):=\frac{1}{2}. \end{equation}\] The resulting probability measure satisfies \[\begin{equation} \mathbb{P}(\{H\})=\mathbb{P}(\{T\})=\frac{1}{2}. \end{equation}\]

\[\begin{equation} \mathbb{P}(\emptyset)=0,\quad \mathbb{P}(\{H\})=1/2,\quad \mathbb{P}(\{T\})=1/2 \quad\text{and}\quad \mathbb{P}(\{H,T\})=1. \end{equation}\]

Example 19.4 (Value of a BDI-II item) We model the value of a patient for the item “Sadness” of the English version of the Beck Depression Inventory II (Hautzinger et al. (2006)),

  1. I do not feel sad. (1) I feel sad much of the time. (2) I am sad all the time. (3) I am so sad or unhappy that I can’t stand it.

The random process under consideration is therefore a patient’s response to this item. For modeling, the outcome set \(\Omega := \{0,1,2,3\}\) is then suitable. For a first patient, for example, the realization is \(3 \in \Omega\), for a second patient the realization is \(0 \in \Omega\), for a third patient \(1 \in \Omega\), and so on. As the event system \(\mathcal{A}\), the power set is again suitable. In the present case, it has cardinality \[\begin{equation} |\mathcal{P}(\Omega)| = 2^{|\Omega|} = 2^4 = 16 \end{equation}\] and can easily be written down completely as shown in Table 19.4.

Table 19.4: Selected events in the model of responding to a BDI-II item.
Response given Set notation
None \(\omega \in \emptyset\)
“I do not feel sad” \(\omega \in \{0\}\)
“I feel sad much of the time” \(\omega \in \{1\}\)
“I am sad all the time” \(\omega \in \{2\}\)
“I am so sad or unhappy that I cannot stand it” \(\omega \in \{3\}\)
“I do not feel sad” or “I feel sad much of the time” \(\omega \in \{0,1\}\)
“I do not feel sad” or “I am sad all the time” \(\omega \in \{0,2\}\)
“I do not feel sad” or “I am so sad or unhappy that I cannot stand it” \(\omega \in \{0,3\}\)
“I feel sad much of the time” or “I am sad all the time” \(\omega \in \{1,2\}\)
“I feel sad much of the time” or “I am so sad or unhappy that I cannot stand it” \(\omega \in \{1,3\}\)
“I am sad all the time” or “I am so sad or unhappy that I cannot stand it” \(\omega \in \{2,3\}\)
Not “I am so sad or unhappy that I cannot stand it” \(\omega \in \{0,1,2\}\)
Not “I am sad all the time” \(\omega \in \{0,1,3\}\)
Not “I feel sad much of the time” \(\omega \in \{0,2,3\}\)
Not “I do not feel sad” \(\omega \in \{1,2,3\}\)
Any response option \(\omega \in \{0,1,2,3\}\)

For the model of the item response, define the probability function \[\begin{equation} \pi:\Omega\to[0,1],\quad \omega\mapsto\pi(\omega):= \begin{cases} \frac{2}{10}, & \omega=0,\\ \frac{3}{10}, & \omega=1,\\ \frac{4}{10}, & \omega=2,\\ \frac{1}{10}, & \omega=3. \end{cases} \end{equation}\] By Theorem 19.5, \[\begin{equation} \mathbb{P}:\mathcal{A}\to[0,1],\quad A\mapsto\mathbb{P}(A):=\sum_{\omega\in A}\pi(\omega) \end{equation}\] then defines a probability measure on \((\Omega,\mathcal{A})\). In particular, the probabilities of the elementary events satisfy \[\begin{equation} \mathbb{P}(\{0\})=\frac{2}{10},\quad \mathbb{P}(\{1\})=\frac{3}{10},\quad \mathbb{P}(\{2\})=\frac{4}{10},\quad \mathbb{P}(\{3\})=\frac{1}{10}. \end{equation}\] This completes the definition of the probability space model \((\Omega,\mathcal{A},\mathbb{P})\) for this scenario.

Using the \(\sigma\)-additivity of \(\mathbb{P}\), we can now determine, for example, the probability of an item response greater than 1: \[\begin{equation} \mathbb{P}\left(\{2,3\}\right) = \mathbb{P}\left(\{2\} \cup \{3\}\right) = \mathbb{P}\left(\{2\}\right) + \mathbb{P}\left(\{3\}\right) = \frac{4}{10} + \frac{1}{10} = \frac{5}{10} = \frac{1}{2} \end{equation}\] Similarly, for example, the probability that a patient gives one of the two item responses “(0) I do not feel sad” or “(3) I am so sad or unhappy that I can’t stand it” is \[\begin{equation} \mathbb{P}\left(\{0,3\}\right) = \mathbb{P}\left(\{0\} \cup \{3\}\right) = \mathbb{P}\left(\{0\}\right) + \mathbb{P}\left(\{3\}\right) = \frac{2}{10} + \frac{1}{10} = \frac{3}{10}. \end{equation}\]

Example 19.5 (An agent’s action) We model an agent’s recorded choice of action. Suppose the agent has three action options \(a\), \(b\), and \(c\) and assigns a subjective value to each. Specifically, let \(w_a:=-2\), \(w_b:=1\), and \(w_c:=8\). An agent who chooses the action with the highest subjective value would therefore always choose option \(c\) if these values remained unchanged.

However, we allow disturbances to occur between the agent’s choice and its recording when choices are repeated. The choice we record may therefore differ from the agent’s actual choice. A parameter \(\tau\in]0,\infty[\) describes the extent of these measurement errors or our uncertainty about the actual choice.

A natural outcome set is \(\Omega:=\{a,b,c\}\), where \(a\), \(b\), and \(c\) denote the recorded action options. Since the outcome set is finite, we again choose the power set \(\mathcal{A}:=\mathcal{P}(\Omega)\) as the event system. Table 19.5 lists all eight possible events, both verbally and as subsets of \(\Omega\). This completes the definition of the measurable space \((\Omega,\mathcal{A})\).

Table 19.5: Event system for a model of the recording of an agent’s choice of action.
Description Set notation
None of the three action options is recorded as the choice \(\omega \in \emptyset\)
Action \(a\) is recorded as the choice \(\omega \in \{a\}\)
Action \(b\) is recorded as the choice \(\omega \in \{b\}\)
Action \(c\) is recorded as the choice \(\omega \in \{c\}\)
Action \(a\) or \(b\) is recorded as the choice \(\omega \in \{a,b\}\)
Action \(a\) or \(c\) is recorded as the choice \(\omega \in \{a,c\}\)
Action \(b\) or \(c\) is recorded as the choice \(\omega \in \{b,c\}\)
One of the three action options is recorded as the choice \(\omega \in \{a,b,c\}\)

We model the probabilities of recording each possible action using the subjective values the agent assigns to those actions. To do so, define the probability function \[\begin{equation} \pi:\Omega\to[0,1],\quad \omega\mapsto\pi(\omega) :=\frac{\exp\left(\tau^{-1}w_\omega\right)} {\sum_{\omega'\in\Omega}\exp\left(\tau^{-1}w_{\omega'}\right)} \end{equation}\] and set \(\mathbb{P}(\{\omega\}):=\pi(\omega)\). The function values are positive and sum to one. Thus \(\pi\) is a probability function and determines a probability measure \(\mathbb{P}\).

This mapping, called the softmax function in cognitive modeling and artificial intelligence, can be understood as a “softer maximization”. Instead of selecting only the action with the highest subjective value, it assigns positive probabilities to all actions. Actions with higher subjective values receive higher probabilities. In our model, this mapping describes the recording of the action, while the agent continues to choose the action with the highest subjective value.

The parameter \(\tau\) determines how strongly the probabilities concentrate on the action with the highest subjective value. For small \(\tau\), this action is recorded with high probability. For \(\tau=1\), \[\begin{equation} \mathbb{P}(\{a\})\approx0.00,\quad \mathbb{P}(\{b\})\approx0.00,\quad \mathbb{P}(\{c\})\approx1.00. \end{equation}\] Thus we record action \(c\) with near certainty.

For larger \(\tau\), the probabilities of the recorded actions become more similar. Recording becomes less reliable, and the recorded action becomes harder to predict. For \(\tau=5\), \[\begin{equation} \mathbb{P}(\{a\})\approx0.10,\quad \mathbb{P}(\{b\})\approx0.18,\quad \mathbb{P}(\{c\})\approx0.72, \end{equation}\] and for \(\tau=100\), \[\begin{equation} \mathbb{P}(\{a\})\approx0.32,\quad \mathbb{P}(\{b\})\approx0.33,\quad \mathbb{P}(\{c\})\approx0.35. \end{equation}\] As \(\tau\to0\), the probability of recording \(c\) approaches one, while the other two probabilities approach zero. As \(\tau\to\infty\), all three probabilities approach \(1/3\). The softmax function therefore allows a continuous transition between almost certain recording of the value-maximizing action and an almost uniform distribution over the recorded action options.

19.4 Bibliographic remarks

Andrey Kolmogorov’s monograph “Grundbegriffe der Wahrscheinlichkeitsrechnung” (Kolmogoroff (1933)) symbolizes the foundation of modern probability calculus. In addition to the axiomatic introduction of the probability space model discussed here, Kolmogoroff (1933) considers many other aspects of probability calculus and thus offers a readable introduction to the whole field of probability theory. Of course, the approach formulated by Kolmogoroff (1933) is only a provisional end product of the long historical development of probability theory. After all, the development of the mathematical modeling of random processes by no means came to an end with Kolmogoroff (1933). Later works in the 20th century concerned in particular the interpretation of the concept of probability (cf. De Finetti (1975)) or introduced generalized quantitative measures of subjective uncertainty (cf. Walley (1991)). A current overview of the interpretation of the concept of probability and its formal foundations is given by Hájek (2019).

Study questions

  1. Explain the concept of a random process.
  2. State the definition of the concept of a \(\sigma\)-algebra.
  3. State the definition of the concept of a probability measure.
  4. State the definition of the concept of a probability space.
  5. Explain the concept of the outcome set \(\Omega\).
  6. Explain the tacit mechanics of the probability space model.
  7. Explain the concept of an event \(A \in \mathcal{A}\).
  8. Explain the concept of the event system \(\mathcal{A}\).
  9. Which \(\sigma\)-algebra is sensibly chosen for a probability space with finite outcome set?
  10. Explain the concept of the probability measure \(\mathbb{P}\).
  11. State the definition of the concept of a probability function.
  12. Why is the concept of a probability function helpful when modeling a random process by a probability space with finite outcome set?
  13. Explain the modeling of throwing a die using a probability space.
  14. Explain the modeling of the simultaneous throwing of a red and a blue die using a probability space.

Study question answers

  1. A random process is a phenomenon of reality that cannot be predicted with absolute certainty or is associated with uncertainty for humans. See the introduction to probability theory.
  2. See Definition 19.1 regarding \(\mathcal{A}\).
  3. See Definition 19.1 regarding \(\mathbb{P}\).
  4. See Definition 19.1.
  5. The outcome set contains the elementary events possible in a run of the modeled random process. See the explanations of Outcome set and mechanics.
  6. See the explanations of Outcome set and mechanics.
  7. See the explanations of Events and event system.
  8. See the explanations of Events and event system.
  9. The power set of the outcome set. It contains all events that are possible in principle. See also Theorem 19.1.
  10. See the explanations of Probability measure \(\mathbb{P}\).
  11. See Definition 19.2.
  12. By defining the function values of a probability function, a probability measure can be specified. See also Section 19.3.
  13. See the explanations of Throwing one die.
  14. See the explanations of Simultaneous throwing of a blue and a red die.
Billingsley, P. (1995). Probability and measure (3rd ed). Wiley.
Borel, É. (1898). Leçons sur la théorie des fonctions. Paris: Gauthier Villars.
De Finetti, B. (1975). Theory of probability. John Wiley & Sons.
Hájek, A. (2019). Interpretations of probability. In E. N. Zalta (Ed.), The Stanford encyclopedia of philosophy (Fall 2019). Metaphysics Research Lab, Stanford University.
Hautzinger, M., Keller, F., & Kühner, C. (2006). BDI-II Beck Depressions-Inventar. Pearson.
Kolmogoroff, A. (1933). Grundbegriffe der Wahrscheinlichkeitsrechnung. Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-49888-6
Meintrup, D., & Schäffler, S. (2005). Stochastik: Theorie und Anwendungen. Springer.
Schmidt, K. D. (2009). Maß und Wahrscheinlichkeit. Springer.
Walley, P. (1991). Statistical reasoning with imprecise probabilities (1st ed). Chapman and Hall.