Introduction to Probability Each slide has its own narration in an audio file. For the explanation of any slide click on the audio icon to start it. Professor Friedman's Statistics Course by H & L Friedman is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. Key Terms Probability. The word probability is actually undefined, but the probability of an event can be explained as the proportion of times, under identical circumstances, that the event can be expected to occur. It is the event's long-run frequency of occurrence. For example, the probability of getting a head on a coin toss = .5. If you tossing a coin repeatedly, for a long time, you will note that a head occurs about one half of the time. Probability vs. Statistics. In probability, the population is known. In statistics you draw inferences about the population from the sample.

Probability 2 Key Terms Objective probabilities are long-run frequencies of occurrence, as above. Probability, in its classical (or, objective) meaning refers to a repetitive process, one which generates outcomes which are not identical and not individually predictable with certainty but which may be described in terms of relative frequencies. These processes are called stochastic processes (or, chance processes). The individual results of these processes are called events. Subjective probabilities measure the strength of personal beliefs. For example, the probability that a new product will succeed. Probability 3 Key Terms

Random variable. That which is observed as the result of a stochastic process. A random variable takes on (usually numerical) values. Associated with each value is a probability that the value will occur. For example, when you toss a die: P(1) = P(2) = P(3) = P(4) = P(5) = P(6) = 1/6 Simple probability. P(A). The probability that an event (say, A) will occur. Joint probability. P(A and B). P(A B). The probability of events A and B occurring together. Conditional probability. P(A|B), read "the probability of A given B." The probability that event A will occur given event B has occurred. Probability 4

Probability: Some Basic Rules 1. The probability of an event (say, event A) cannot be less than zero or greater than one. The probability that you will pass this course cannot be 150%. So, 0 < P(A) < 1 2. The sum of the probabilities of all possible outcomes (events) of a process (or, experiment) must equal one. So, Pi = 1 or P(A) + P(A') = 1 [A' means not A.] Probability 5 Probability: Some Basic Rules 3. Rules of addition. a. P(A or B) = P(A U B) = P(A) + P(B) if events A and B are mutually exclusive. Two events are mutually exclusive if they cannot occur together. E.g., male or female; heads or tails. b. In general, P(A or B) = P(A) + P(B) - P(A and B) This is the general formula for addition of probabilities, for any two events A and B. P(A and B) is the joint probability of A and B occurring together and is equal to zero if they are mutually exclusive (i.e., if they cannot occur together). Thus, the probability of getting a 1 or 2 when tossing a die: P (1 or 2) = P (1) + P (2) = 1/6 + 1/6 = 1/3; The probability of getting a 1 and 2 when

tossing a die is 0, since they are mutually exclusive. P (1 and 2) = 0. Probability 6 Probability: Some Basic Rules Mutually exclusive events, in Venn Diagrams: Events A,B are mutually exclusive. Events A, B are not mutually exclusive. A B A B P(AB)

P(AB) is the intersection of A and B. Probability 7 Probability: Some Basic Rules Also, we can see from the Venn diagrams that: P(A) = P(not A) = 1 P(A) P(A or B) = 1 P(A and B) Example: Suppose you go to a college where you are allowed to double major. 10% of students major in accounting (A); 15% major in business (B); 3% are double majors (both A and B). What is the probability of being an accounting or business major? [Hint: The events are not mutually exclusive.] Answer: P(A B) = P(A or B) = .10 + .15 - .03 = .22 Probability

8 Probability: Some Basic Rules 4. Rules of multiplication are used for determining joint probabilities, the probability that events A and B will occur together. a. P(A and B) = P(A B) = P(A)P(B) if events A and B are independent. Events A and B are independent if knowledge of the occurrence of B has no effect on the probability that A will occur. For example, P(Blue eyes | Male) = P(Blue eyes) because eye color and sex are independent. P(over 6 feet tall | Female) P(over 6 feet tall) P(getting into car accident | under 26) P(getting into car accident) P (getting head on second toss of coin/got head on first toss of coin) = ? Answer = .50 The two coin tosses are independent. What happened during the first coin toss has no effect on what will happen during the second coin toss. Probability 9 Probability: Some Basic Rules b. In general, P(A and B) = P(A|B) P(B) = P(B|A) P(A) This is the general formula for multiplying probabilities for any two events A and B, not necessarily independent. P(A|B) is a conditional probability.

If events A and B are independent, then P(A|B) = P(A), and P(A and B) reduces to P(A)P(B), as in 4.a. above. Independent events: Events A and B are independent if: P(A|B) = P(A) or if P(A and B) = P(A)P(B) 5. Conditional probability. From the formula in 4.b., we see that we can compute the conditional probability as P(A|B) = P(A and B) / P(B) Probability 10 Probability: Some Basic Rules 6. Bayes' Theorem [OPTIONAL TOPIC] Since and Therefore P(A|B) = P(A and B) / P(B) P(A and B) = P(B|A) P(A) P(A|B) = P(B|A) P(A) / P(B) [We generally need to compute P(B)]. Probability

11 Example: Readership In a small village in upstate New York, we are looking at readership of the New York Times (T) and the Wall Street Journal (W): P(T) = .25 P(W) = .20 P(T and W) = .05 Question: What is the probability of being either a New York Times reader or a Wall Street Journal reader? P (T or W) = P(T) + P(W) P (T and W) = .25 + .20 - .05 = .40 Probability 12

Example: Readership Another way to solve this problem, by Venn Diagram: 20 5 T 15 60 W N So, out of 100 people in total: 25 people read NYT (T); 20 people read WSJ (W); 5 people read both; 60 people read neither. Thus, it can be easily seen that 40 people (out of 100) read either NYT or WSJ. Probability 13

Example: Readership Other Probabilities: P(T and W) = .60 P(T and W) = .15 P(T and W) = .20 P(T or W) = 1 P(T and W) = 1 .60 = .40 P (T or W) = P(T) + P(W) P(T and W) =.75+ .80 - .60 = .95 (95% of people in this town do not read one of the two papers; In fact, only 5% read both). Note that: P(T or W) = 1- P(T and W) = 1 - .05 = .95 Probability 14 Example: Readership

Another way to do solve this problem is to construct a table of joint probabilities: T T W .05 .15 .20 W .20 .60 .80 .25 .75 1.00

Probability 15 Independence vs. Mutually Exclusive Events Important: Do not confuse mutually exclusive and independent. Mutually exclusive means that two things cannot occur at the same time [P (A and B) = 0]. You cannot get a head and tail at the same time; you cannot be dead and alive at the same time; you cannot pass and fail a course at the same time; etc. Independence has to do with the effect of, say B, on A. If knowing about B has no effect on A, then they are independent. It is very much like saying that A and B are unrelated. Are waist size and gender independent of each other? Suppose I know that someone who is an adult has a 24-inch waist, does that give me a hint as to whether that person is male or female? How many adult men have a 24-inch waist? How many women? Is P (24-inch waist/adult male) = P (24 inch waist/adult female). We suspect that the two probabilities are not the same, that there is a relationship between gender and waist size (also hand size and height for that matter). Thus, they are not independent. Probability 16

Independence vs. Mutually Exclusive Events If two events are mutually exclusive, they cannot occur together; we only examine events for independence (or, conversely, to see if they are related) if they can occur together. Researchers are always testing for relationships. Sometimes we want to know whether two variables are related. Is there a relationship between cigarette smoking and cancer or are they independent? We know the answer to that one. Is there a relationship between your occupation and how long you will live (longevity) or are they independent? Studies show that they are not independent. Librarians and professors have relatively long life spans; coal miners have the shortest life spans. Drug dealers (is that an occupation?) also have very short life spans. For women, is there a relationship between salary and how slender you are? Or, are salary and weight independent? For men, is there a relationship between how many dates you will get on a website such as eHarmony and your occupation? For women, how about number of dates and hair color? Probability

17 Example: Smoking and Cancer S (smoke r) Data in a Contingency Table: S (non-smoker) C (cancer) 100 50 150 C (no cancer) 300 550

850 400 600 1000 Are smoking and cancer independent? Joint Probabilities P(C and S) = .10 Marginal Probabilities P(C) = .15 (150/1000) P(C` and S) = .30 P(C`) = .85 (850/1000) P(C and S`) = .05 P(S) = .40 (400/1000) P(C` and S`)= .55 P(S`) = .60 (600/1000) Probability 18

Example: Smoking and Cancer Are smoking and cancer independent? To answer, check: Are the following probabilities equal or not? P(C) P(C|S) P(C|S) P(C|S) = P(C and S)/P(S) = .10/.40 = .25 P(C|S) = P(C and S)/P(S) = .05/.60 = .083 P(C) = .15 Thus, cancer and smoking are not independent. There is a relationship between cancer and smoking. Note that P(C) is a weighted average of P(C|S) and P(C|S) i.e., P(C) = P(S)P(C|S) + P(S)P(C|S) = .40(.25) + .60 (.0833) = .15 Probability 19 Example: Smoking and Cancer Are smoking and cancer independent? Alternate method of solving this problem: If C and S are independent, then P(C and S) = P(C) P(S) .10 ?= (.15)(.40) .10 .06 Since .10 is not equal to .06, we conclude that cancer and smoking are not independent. These calculations are much easier if you set up a joint probability table.

Coming up in the next two slides. Probability 20 Example: Smoking and Cancer We use the frequencies in the contingency table (above) to compute the probabilities in the joint probability table. S(smoker) S(non-smoker) C (cancer) 100/1000 50/1000 150/1000 C (no cancer) 300/1000 550/1000

850/1000 400/1000 600/1000 1000/100 0 S (smoke r) S (non-smoker) C (cancer) .10 .05 .15

C (no cancer) .30 .55 .85 Notice that the marginal probabilities are the row and totals. This1.00 .40 column .60 is not an accident. The marginal probabilities are totals of the joint probabilities and weighted averages of the conditional probabilities. Probability 21 Example: Smoking and Cancer Once we compute the table of joint probabilities, we can answer any question about a probability in this problem. The joint probabilities are in the center of the table; the marginal probabilities are in the margins, and the conditional probabilities are easily computed by dividing a joint probability by a marginal probability. P(C) = .15 P(C and S) = .10

P(C|S) = P(C and S) / P(S) = .10 / .40 = .25 Probability 22 Example: Gender and Beer M (male) Contingency Table Data: F (female) B (beer drinker) 450 350 800 B (not a beer drinker) 450 750

1200 900 1100 2000 M (male) The Joint Probability Table: F (female) B (beer drinker) .225 .175 .40 B (not a beer drinker) .225

.375 .60 .45 .55 Marginal Probabilities P(B) = .40 P(B) = .60 P(M) = .45 P(F) = .55 1.00 Joint Probabilities P(B and M) = .225 P(B and F) = .175 P(B and M) = .225 P(B and F) = .375 Probability 23 Example: Gender and Beer

Given that an individual is Male, what is the probability that that person is a beer drinker? P(B|M) = P(B and M)/P(M) = .225/.45 = .50 Given that an individual is Female, what is the probability that that person is a Beer drinker? P(B|F) = P(B and F)/P(F) = .175/.55 = .318 Are beer drinking and gender independent? P(B) = .40, therefore, beer-drinking and sex are not independent. Probability 24 Example: Gender and Dove Soap Data collected from a random sample of 1,000 adults: D (use Dove soap) D (does not use Dove soap)

M (male) F (female) 80 120 200 320 480 800 400 600 1,000 Computed probabilities: Joint Probability

P(D and M) = .08 P(D and F) = .12 P(D and M) = .32 P(D and F) = .48 Marginal Totals P(D) = .20 P(D) = .80 P(M) = .40 P(F) = .60 Probability 25 Example: Gender and Dove Soap Are the events Gender and Use of Dove soap independent? P(D)= 200/1000 = .20 P(D|M) = 800/1000 = .08/.40 = .20 P(D|F) = .12/.60 = .20 Yes, these two events are independent. Alternative method: Is P(M and D) P(M) P(D) ? P(M and D) = .08 P(M) = 400/1000 = .40 P(D) = 200/1000 = .20

Is .08 (.40)(.20)? Yes, therefore the two events M and D are independent. Probability 26 Homework Two Sample Z Test 27