SigmaField
Mathematical Foundations for Quant Finance
0 of 256 lessons
      1. 5.1.1 Sample spaces, events and the axioms of probabilityConcept · 30 min
      2. 5.1.2 CountingConcept · 30 min
      3. 5.1.3 Conditional probability and independenceConcept · 30 min
      4. 5.1.4 Bayes' ruleConcept · 25 min
      5. 5.1.5 Practice set: probabilityPractice set · 25 min
      6. 5.1.6 Case study: default and rating-migration probabilitiesCase study · 35 min

Level 5, Probability and statistics 5.1 Probability basics Lesson 2 of 6

ConceptLesson 5.1.2 in Level 5, Probability and statistics. About 30 minutes.

Counting

The multiplication rule

When a choice is made in steps, multiply the number of ways at each step. A café offers 3 drinks and 4 cakes. Each drink goes with each cake, so there are 3×4=123 \times 4 = 12 orders.

Lesson 5.1.1's two dice work the same way: 6 faces for the red die, then 6 for the blue, 6×6=366 \times 6 = 36 outcomes. A 4-digit PIN has 10×10×10×10=10,00010 \times 10 \times 10 \times 10 = 10{,}000 possibilities.

Rule: Multiplication rule

m×nm \times n

A first step can go mm ways. If a second step can then go nn ways, whatever the first was, the two together can go m×nm \times n ways. More steps multiply on.

The options at the second step may depend on the first, but their number must not. Four runners race for gold and silver. Whoever wins gold, 3 runners are left for silver, so there are 4×3=124 \times 3 = 12 results.

Try it

product lists every drink with every cake, and len counts the list. The PIN count is 10410^4.

permutations("ABCD", 2) lists the ordered pairs of different runners: gold first, then silver.

⌘/Ctrl + Enter runs the cell

Order matters: arrangements

Put 4 books on a shelf. The first place has 4 choices, the second 3, the third 2 and the last 1: 4×3×2×1=244 \times 3 \times 2 \times 1 = 24 orders. This product is written 4!4!, read "4 factorial". In general n!=n×(n−1)×⋯×1n! = n \times (n - 1) \times \cdots \times 1 counts the orders of nn different things.

Factorials grow fast. 10!=3,628,80010! = 3{,}628{,}800, and a deck of 52 cards can be shuffled into 52!≈8.07×106752! \approx 8.07 \times 10^{67} orders. By agreement 0!=10! = 1: there is one way to arrange nothing.

Often only some of the nn things are placed. A club of 10 chooses a chair, a secretary and a treasurer, three different people: 10×9×8=72010 \times 9 \times 8 = 720 ways. That is 10!10! with the last 7!7! cancelled, 10!7!\frac{10!}{7!}.

Rule: Ordered choices

n(n−1)⋯(n−k+1)=n!(n−k)!nk if repeats are allowed\begin{gathered} n(n - 1)\cdots(n - k + 1) = \frac{n!}{(n - k)!} \\ n^k \text{ if repeats are allowed} \end{gathered}

The number of ordered lists of kk things from nn, each used at most once: a permutation of kk from nn, math.perm(n, k). If a thing may be used again, as a digit in a PIN, each place has all nn choices.

From the 5 letters A to E, 3-letter codes number 53=1255^3 = 125 when letters may repeat, and 5×4×3=605 \times 4 \times 3 = 60 when they may not. Repeats allowed is also called drawing with replacement: the letter goes back before the next draw.

Try it

math.factorial and math.perm give the counts above. Listing all the codes with product and permutations confirms 125 and 60.

⌘/Ctrl + Enter runs the cell

Order does not matter: combinations

Now the club picks a committee of 3, with no roles. The 720 ordered choices count each committee several times: once for each way to hand out the three roles, 3!=63! = 6 ways. So there are 720÷6=120720 \div 6 = 120 committees.

The picture shows the same idea with 4 runners and 2 medals. There are 12 orders, and each pair of medal winners appears twice.

Gold has 4 ways and silver 3, so 4×3=124 \times 3 = 12 orders. As pairs, {A,B}\{A, B\} comes from "A then B" and from "B then A", so there are 12÷2=612 \div 2 = 6 pairs.

Rule: Combinations

(nk)=n!k! (n−k)!\binom{n}{k} = \frac{n!}{k!\,(n - k)!}

The number of ways to choose kk things from nn when order does not matter: the subsets of size kk. Read "nn choose kk", also written C(n,k)C(n, k); math.comb(n, k). It is the ordered count divided by the k!k! orders of each choice.

Four cases are worth knowing.

  • Nothing or everything. (n0)=(nn)=1\binom{n}{0} = \binom{n}{n} = 1: one way to choose no one, one way to choose everyone.
  • Choosing who stays out. (nk)=(nn−k)\binom{n}{k} = \binom{n}{n - k}. Picking 3 of 10 to serve is picking 7 to stay home: both give 120.
  • Too many. If k>nk > n there is no way, and (nk)=0\binom{n}{k} = 0.
  • All sizes together. For n=5n = 5, the counts for k=0,…,5k = 0, \ldots, 5 are 1, 5, 10, 10, 5, 1. They add up to 32=2532 = 2^5, every subset, since each of the 5 things is either in or out.

Lesson 5.2.2 uses these numbers for the binomial distribution.

Try it

combinations("ABCDE", 2) lists the 10 pairs, each once. Then the committee count two ways, and the row for n=5n = 5 with its total.

math.comb(4, 5) is 0: five things cannot be chosen from four.

⌘/Ctrl + Enter runs the cell

Probabilities by counting

With equally likely outcomes, P(A)=∣A∣∣Ω∣P(A) = \frac{|A|}{|\Omega|} (lesson 5.1.1), and both counts come from the rules above. Count the event and the sample space the same way: both ordered, or both unordered.

Six rolls of a die. The 66=46,6566^6 = 46{,}656 ordered sequences of faces are equally likely. Of them, 56=15,6255^6 = 15{,}625 contain no six. So P(at least one six)=1−15,62546,656≈0.6651P(\text{at least one six}) = 1 - \frac{15{,}625}{46{,}656} \approx 0.6651, not the 6×16=16 \times \frac{1}{6} = 1 that adding gives.

Cards. A standard deck has 52 cards: 13 ranks (2 to 10, jack, queen, king, ace) in each of 4 suits (hearts, diamonds, clubs, spades). A 5-card hand from a well-shuffled deck is one of (525)=2,598,960\binom{52}{5} = 2{,}598{,}960 equally likely hands.

  • All hearts: (135)=1287\binom{13}{5} = 1287 hands, so P≈0.000495P \approx 0.000495. Five cards of any one suit: 4×1287=51484 \times 1287 = 5148 hands, P≈0.00198P \approx 0.00198.
  • Exactly two aces: choose the 2 aces from 4, (42)=6\binom{4}{2} = 6 ways, then the other 3 cards from the 48 that are not aces, (483)=17,296\binom{48}{3} = 17{,}296. The multiplication rule gives 103,776103{,}776 hands, P≈0.0399P \approx 0.0399.
  • At least one ace: the complement, no ace, has (485)\binom{48}{5} hands, so P=1−0.6588=0.3412P = 1 - 0.6588 = 0.3412.

A committee. A club of 10 has 6 juniors and 4 seniors, and a committee of 3 is picked at random from the 120. All juniors: (63)=20\binom{6}{3} = 20 committees, so P=16P = \frac{1}{6}. Exactly two juniors: (62)(41)=15×4=60\binom{6}{2}\binom{4}{1} = 15 \times 4 = 60, so P=12P = \frac{1}{2}.

Counting in order gives the same answer if the top is ordered too. Three juniors one after another: 6×5×410×9×8=120720=16\frac{6 \times 5 \times 4}{10 \times 9 \times 8} = \frac{120}{720} = \frac{1}{6}.

Try it

The cell works out each probability above. Fraction gives the committee answers exactly.

⌘/Ctrl + Enter runs the cell

A lottery

A lottery draws 6 different numbers from 1 to 49, and a ticket picks 6. There are (496)=13,983,816\binom{49}{6} = 13{,}983{,}816 equally likely draws, so the jackpot has probability about 7.15×10−87.15 \times 10^{-8}.

To match exactly 3 of your numbers, the draw takes 3 of your 6, in (63)=20\binom{6}{3} = 20 ways. Its other 3 come from the 43 numbers you did not pick, in (433)=12,341\binom{43}{3} = 12{,}341 ways. That is 246,820246{,}820 draws, a probability of about 0.01765. Every number of matches works the same way:

matches kk0123456
draws (6k)(436−k)\binom{6}{k}\binom{43}{6 - k}6,096,4545,775,5881,851,150246,82013,5452581
probability0.4360.4130.13240.017650.00096861.845×10−51.845 \times 10^{-5}7.151×10−87.151 \times 10^{-8}

The draws add up to all 13,983,816, so the probabilities add up to 1. At least 3 matches has probability about 0.01864.

The same pattern answers every question of this kind. Draw nn things from NN, of which KK are special. Exactly kk special ones turn up in (Kk)(N−Kn−k)\binom{K}{k}\binom{N - K}{n - k} of the (Nn)\binom{N}{n} equally likely draws.

For two aces in a hand, N=52N = 52, K=4K = 4, n=5n = 5 and k=2k = 2.

Try it

The loop builds the table and checks the total. The rest simulates, with fixed seeds: 100,000 shuffled decks, keeping the first 5 cards, and 100,000 lottery draws.

rng.random((n, 52)).argsort(axis=1) puts the 52 cards in a random order in each row. Cards 0 to 3 stand for the aces, and numbers 0 to 5 for your ticket.

⌘/Ctrl + Enter runs the cell

In this seeded run, 0.0389 of the hands had exactly two aces, against the exact 0.0399, and 0.3402 had at least one, against 0.3412. Case study 4.7.5's standard error, p(1−p)/N\sqrt{p(1 - p)/N}, is 0.0006 here: the size of a typical miss. The lottery run gives 0.0181 for exactly 3 matches, against 0.01765.

Which count to use

Two questions decide: does the order matter, and can a thing be chosen twice? Swap two of your choices: if the result changes, order matters.

exampleorder matters?repeats?count
a PIN, dice rolled in turnyesyesnkn^k
medals, officers, a shelfyesnon!(n−k)!\frac{n!}{(n - k)!}
a committee, a card hand, a lottery drawnono(nk)\binom{n}{k}

An urn holds 3 red and 2 blue balls; draw 2. Without replacement, the 2 balls form one of (52)=10\binom{5}{2} = 10 pairs, and (32)=3\binom{3}{2} = 3 are both red: P=310P = \frac{3}{10}. With replacement, the 25 ordered draws are equally likely and 9 are red twice: P=925=0.36P = \frac{9}{25} = 0.36.

Lesson 5.1.3 gets both answers from conditional probabilities.

Common mistakes

  1. Order or not. A committee of 3 from 10 has 120 choices; three officers have 720. Ask whether swapping two choices gives a different result.
  2. Count the top and the bottom the same way. An ordered count of the event over an unordered count of the sample space is wrong by a factor k!k!.
  3. "Exactly" means the rest are not of that kind. For exactly two aces the other 3 cards come from the 48 non-aces. Taking them from the 50 cards left gives 0.0452, counting hands with 3 or 4 aces.
  4. "At least one" by adding counts twice. Two-card hands with an ace number (522)−(482)=198\binom{52}{2} - \binom{48}{2} = 198, not 4×51=2044 \times 51 = 204, which counts the 6 hands with two aces twice.
  5. Unordered choices with repeats are not equally likely. Like the sums of two dice, count the ordered draws instead.
  6. Big factorials in NumPy overflow. Python's integers are exact, so math.factorial(21) is right. NumPy's np.prod(np.arange(1, 22, dtype=np.int64)) silently gives a negative number, because its 64-bit integers stop near 9.2×10189.2 \times 10^{18}.

Try it

The first line is mistake 3: the right count of two-ace hands, then the wrong one with the other cards from 50. The second is mistake 4.

The last line compares 21!21! from math with NumPy's product, and prints the largest NumPy integer.

⌘/Ctrl + Enter runs the cell

Counting

0 of 12 correct

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
How to type answers
  • Powers with ^: x^7, 2^(1/3), a^-2.
  • Multiply with * or by writing side by side: 3x^2, 2(x+1).
  • ln(x) is the natural log, log(x, 2) is log base 2, e is Euler's number, sqrt(x) is a square root.
  • sin(x), cos(x) and tan(x) take radians; write pi for π and sin(x)^2 for sin²x.
  • Your first answer to each question is saved: it counts towards your score and brings the question back for review.
  • Exact answers like ln(20)/ln(3) are best. Decimals need about 4 significant figures.

Summary card

  • Multiplication rule: steps with mm and nn ways give m×nm \times n results.
  • Ordered, no repeats: n!(n−k)!\frac{n!}{(n - k)!} (math.perm); with repeats: nkn^k. n!=n(n−1)⋯1n! = n(n - 1)\cdots 1, and 0!=10! = 1.
  • Unordered: (nk)=n!k! (n−k)!\binom{n}{k} = \frac{n!}{k!\,(n - k)!} (math.comb), the ordered count divided by k!k!. (nk)=(nn−k)\binom{n}{k} = \binom{n}{n - k}.
  • Probability by counting: count the event and the sample space the same way. "Exactly kk special" is (Kk)(N−Kn−k)\binom{K}{k}\binom{N - K}{n - k} over (Nn)\binom{N}{n}.
  • Use the complement for "at least one", and check counts by listing small cases or by a seeded simulation.

Free further reading

Optional. This lesson covers what you need; these add more examples and practice. All are free to read.