Probability Theory (No comments here please)

12 Next page « Previous thread | Next thread »
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
OP - August 02, 2014, 09:33 PM

Hi guys,

I just came to the realisation that I may have jumped the gun with my "Sets, Counting and Probability" thread in the sense that it does assume some prior knowledge.
This thread will serve as an introduction to probability theory & will include worked examples.

I am still planning to work on the other thread, but I believe that this one is more needed.

Topics that will be covered:

- Set theory (basics)
- Probability
- How do we measure Probability?
- Conditional Probability
- Independence ( very quick introduction)

Could I please request that nobody posts a comment here, I have created another thread for those who wish to comment.

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #1 - August 02, 2014, 09:34 PM

Comment thread: http://www.councilofexmuslims.com/index.php?topic=26989.new#new

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #2 - August 02, 2014, 09:53 PM

Part one: Set Theory

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #3 - August 02, 2014, 09:59 PM

The starting point for studying probability is the understanding of four concepts: experiment, sample outcome, sample space and events.

Experiment: is a process that can be repeated many times and leading to at least two possible outcomes with uncertainty as to which will take place.

Sample outcome : the different possible outcomes of an experiment are called sample outcomes.

Sample space: The collection of all possible outcomes from an experiment.

Event: In probability theory, an event is a set of outcomes to which a probability is assigned. Typically,when the sample space is finite, any subset of the sample space is an event.

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #4 - August 02, 2014, 10:05 PM

Experiment: flip a coin 3 times

Question: What is the sample space S in this case?
Answer: It consists of triplets of outcomes of the first, second and third toss, altogether 8 triplets:
S = {HHH;HHT;HTH; THH;HTT; THT; TTH; TTT}

Question: Which outcomes make up the event
A = {majority of coins show head}?

Answer: an inspection of (Sample space) S gives the answer:
A = {HHH;HHT;HTH; THH}

In all of these events, the dominant outcome is "H".

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #5 - August 02, 2014, 10:08 PM

Venn Diagrams

The concept of Probability can also be expressed through Venn diagrams.

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #6 - August 02, 2014, 10:09 PM

Some notations to remember (not exhaustive):

s ∈ S means that outcome "s" belongs to the sample space S.

Ø is the empty set. This has no elements and defines the set of impossible events.

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #7 - August 02, 2014, 10:20 PM

Union(OR)

Union: the union of event A and B, denoted by A∪B, is a collection of outcomes (elements) that belong to A or B, or both.

A∪B = {s : such that s ∈ A or s ∈ B}

Let’s go back to the example concerning the flipping of a coin three times. Let’s define the two events:
A = {at least two heads} = {HHH;HHT;HTH; THH }
B = {first coin is tail} = {THH; THT; TTH; TTT}

In this case the union of the two events gives us:
A∪B = {HHH; HHT; HTH; THH; THT; TTH; TTT}

Union: Properties
A ∪ A = A: the union of the same event is the event itself.
A ∪ S = S: the union of an event A with the sample space is the sample space (since A is included in the sample space!)
A ∪ Ø = A: the union of the event A with the empty set is the event A itself.

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #8 - August 02, 2014, 10:21 PM

Intersection (AND)

Intersection: the intersection of events A and B, denoted by A ∩ B, is the set of all basic outcomes in S that belong to both A and B:
A ∩ B = {s: such that s ∈ A and s ∈ B}
Event A ∩ B happens only if events A and B occur.

Intersection:Properties

A ∩ A = A: the intersection of the same event is the event itself.
A ∩ S = A: the intersection between the event A and the sample space S is the event itself.
A ∩ Ø = Ø: the intersection between the event A and the empty set is the empty set.

Example:

Let’s go back to the example used before and concerning the flipping of a coin three times. Let’s define the two events:

A = {at least two heads} = {HHH;HHT;HTH; THH}
B = {first coin is tail} = {THH; THT; TTH; TTT}

In this case the intersection of the two events gives us:
A ∩ B = {THH}

Since there is only one basic outcome common to both events

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #9 - August 02, 2014, 10:21 PM

Complement

Complement: Let A be an event on a sample space S. The complement of A, denoted by A^(c), is the set of outcomes in S, which are not contained in A

A^(c) = {s : such that s ∉ A}

Event A^(c) happens if an event A does not occur.

Complement:Properties

(A^c)^(c) = A: the complement of a complement event is the event itself.
Ø^(c) = S: the complement of the empty set is the sample space.
A^(c) ∪ A = S: the union between the event A and its complement AC is the sample space S.
A^(c) ∩ A = Ø: the intersection between the complement AC and the event A is the empty set.

The complement of the union of A and B is equivalent to the intersection of their complements:
(A ∪ B)^(c) = A^(c) ∩ B^(c)

The complement of the intersection of A and B gives is equivalent to the union of their complements:
(A ∩ B)^(c) = A^(c) ∪ B^(c)

Example

Let’s go back to the example used before and concerning the flipping of a coin three times. Let’s define the event:
A = {at least two heads} = {HHH;HHT;HTH; THH }
In this case the complement A^(c) is given by:

A^(c) = {HTT; THT; TTH; TTT }
In other words, the complement of A is the set that contains the outcomes {less than two heads}.

* A^(c) is not the correct notation at all, but I'm using it for the sake of convenience. The correct notation can be found here: http://www.rapidtables.com/math/symbols/Set_Symbols.htm*

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #10 - August 03, 2014, 12:36 AM

Disjoint (mutually exclusive events)

Events A and B are called disjoint or mutually exclusive, if :

A ∩ B = Ø

Example

Let’s go back to the example used before and concerning the flipping of a coin three times. Let’s define the two events:
A = {at least two heads} = { HHH;HHT;HTH; THH }
B = {at least two tails} = { TTH;THT; HTT, TTT }

You can easily notice that the four outcomes in event B are not included in event A. Thus, events A and B are said to be mutually exclusive.

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #11 - August 03, 2014, 01:12 PM

Subset

Event (set) A is contained in event B, A ⊂ B, if any outcome contained in A belongs also to B:

A ⊂ B means that s ∈ A implies s ∈ B

In other words, if event B happens then event A happens too.

Properties of the subset operator

If A ⊂ B and B ⊂ A then A = B: if A is a subset of B and simultaneously B is a subset of A, then the two events are equal.

If A ⊂ B and B ⊂ C then A ⊂ C: if A is a subset of B and B is a subset of C then it follows that A must be a subset of C.

Ø ⊂ A for any event A.

If A ⊂ B then A ∩ B = A and A ∪ B = B.

The common elements of A and B are just the elements of A whilst joining B to A has no effect.

Relationships with union and intersection operators

Since A ∪ B arises by joining together A and B, both A and B are enclosed in this union:

A ⊂ (A ∪ B) and B ⊂ (A ∪ B)

Since the elements of A ∩ B belong both to A and B, this intersection is a subset of A and also a subset of B:

(A ∩ B) ⊂ A and (A ∩ B) ⊂ B

Example
Let’s go back to the example used above and concerning the flipping of a coin three times. Let’s define the two events:

A = {at least two heads} = {HHH;HHT;HTH; THH}
B = {only one tail} = {HHT;HTH; THH}

You can easily notice that the three outcomes in event B are included in event A.

Thus, event B is a subset of event A → B⊂A

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #12 - August 03, 2014, 02:32 PM

Difference of events

Let A and B be two events on a sample space S. The difference of events A and B, denoted by A\B, is the set of outcomes in S, which are contained in A and not contained in B:

A\B = {s : such that s ∈ A; s ∉ B}

Event A\B happens if an event A happens and event B does not occur.

Difference - Properties

A\A = Ø: the difference between the same event is the empty set.
A\Ø = A: subtracting the empty set from the event A leaves us with the event A.

Example

Let’s go back to the example used before to explain the intersection between two events. Let’s define the two events:

A = {at least two heads} = {HHH;HHT;HTH; THH}
B = {first coin is tail} = {THH; THT; TTH; TTT}

In this case the difference of the two events gives us:

A\B = { HHH;HHT;HTH }

The two events have one outcome in common {THH} so that the difference between the two events is given by event A minus the common outcome.

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #13 - August 03, 2014, 02:34 PM

Part two: Probability

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #14 - August 03, 2014, 07:11 PM

Probability Function: the concept of probability is intended to provide a numerical measure of the likelihood of an event’s occurrence. Probability is measured on a scale from 0 to 1

Definition: Probability of event A denoted by P(A), measures how likely is for event A to occur. It takes values from 0 to 1:

0 ≤ P(A) ≤1

A Probability function maps event A to a number:

P : {set of all possible events} →[0; 1]

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #15 - August 03, 2014, 07:15 PM

Main Axioms of Probability Functions

Axiom 1.
Let A be an event. Then P(A) ≥ 0

In words, "probability P(A) is a nonnegative number!"

Axiom 2.
P(S) = 1
In words, the "Probability of event S (sample space) including all possible outcomes is 1."

Axiom 3. (Additivity property)

If events A and B are mutually exclusive (i.e. do not intersect, A∩ B = Ø), then:

P(A ∪ B) = P(A) + P(B)

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #16 - August 03, 2014, 07:19 PM

Some basic properties of P (theorems)

Theorem 1: the probability of the complement of an event A is just 1 minus the probability of event A
P(A^c) = 1 – P(A)

Theorem 2: the probability of an empty event, which never happens, is 0
P(Ø) = 0

Theorem 3: suppose that event A is included in (is a subset of) event B (A ⊂ B). Then in this case the probability of event A is smaller than or equal to the probability of event B:

If A ⊂ B then P(A) ≤ P(B)

Theorem 4: for any event A, the probability of the event happening is less than or equal to 1

P(A) ≤ 1

Theorem 5:
The probability of the union between events A and B is given by the sum of the probabilities of the two events P(A)+P(B) minus the probability of the intersection between the two events P(A ∩ B)

P(A ∪ B) = P(A) + P(B) – P(A ∩ B)

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #17 - August 03, 2014, 07:21 PM

Part three: How do we measure Probability?

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #18 - August 03, 2014, 07:29 PM

What is probability? - There are three distinct ways of assigning probability:

Empirical : the probability is estimated from observed outcome frequency
Example: there is a 2% chance of twins in a randomly chosen birth.

Classical : the probability is known a priori by the nature of the experiment
Example: there is a 50% chance of heads on a coin flip.

Subjective: the probability is based on informed opinion or judgment
Example: there is a 100% chance that my football team will win the World Cup.

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #19 - August 03, 2014, 07:42 PM

Empirical approach

Sometimes we can collect empirical data through observations or experiments
Use the empirical or relative frequency approach to assign probabilities

Count the frequency of observed outcomes from a particular event (nA) and divide by the number of observations (N)

As the number of observations (N) increase (i.e., the number of times the experiment is performed) the estimate of the probability becomes more and more accurate.

The Law of Large Numbers

As the number of trials or observations increase, any empirical probability approaches its theoretical limit.

Alternatively, as the sample size increases, there is an increased probability that any event (even an unlikely one) will occur.

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #20 - August 03, 2014, 07:44 PM

Classical Approach

In many cases we are able to assign probabilities to events a priori or, in other words, before we actually observe the event or try an experiment.

When flipping a coin or rolling a pair of dice we do not actually have to perform an experiment to determine the probability of an event happening.

Example: the probability of getting a five when rolling a die is 1/6.

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #21 - August 03, 2014, 07:46 PM

Part four: Conditional Probability and Independence.

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #22 - August 03, 2014, 07:50 PM

Conditional Probability

In a real world situation, if we know that certain event has already occurred, such knowledge might modify the probability (our belief) of the event.

New information can used: "Probability of an event A might be adjusted if we know that some related event B has occurred"

Definition of Conditional Probability: a probability that is revised to take into account the known occurrence of other events, is called a conditional probability and it is defined as:

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #23 - August 03, 2014, 07:55 PM

Independence

It is often the case that the probability of event A remains unchanged regardless of the outcome of event B. Events sharing this property are called independent.

Two events A and B are said to be independent, if:

P(A|B) = P(A)

If events A and B are independent then by definition of conditional probability we have:

P(A ∩ B) = P(A)P(B)

Example: assume we are tossing a coin twice. The probability of getting H or T in the second toss does not depend on the outcome of the 1st toss (H or T).

Another way to see this: the result of the first toss does not provide any additional information about the result of the second toss.

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #24 - August 03, 2014, 07:55 PM

Problems

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #25 - August 03, 2014, 08:01 PM

(1) Explain what it means for two events to be mutually exclusive.

(2) If A and B are two events, define A^(c) , A∪B, A∩B, A^(c)∩B^(c).

(3) Give an example of conditional probability.

(4) Explain what it means for two events to be independent. Can two events be independent but not mutually exclusive?

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #26 - August 03, 2014, 08:03 PM

(5)

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #27 - August 03, 2014, 08:04 PM

(6)

Suppose that A and B are mutually exclusive events with P(A)=0.6 and P(B)=0.2
a) Compute P(A or B)
b) Compute P(A and B)

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #28 - August 03, 2014, 08:05 PM

(7)

Of the US population in 1980,

10% were Californian
6% were of Spanish origin
2% were from California and of Spanish origin.

If an American was drawn at random, what is the chance she would be:
a)   From California or of Spanish origin?
b)   Neither from California nor of Spanish origin?
c)   Of Spanish Origin, but not from California?

My mind runs, I can never catch it even if I get a head start.
- Qtian
- Pub Regular
- offline
Probability Theory (No comments here please)
Reply #29 - August 03, 2014, 08:05 PM

8.

In a family of 10 children, (assuming that each families have the same probabilities) what is the chance that there will be:
1) At least one boy?
2) At least one boy and one girl?

My mind runs, I can never catch it even if I get a head start.
12 Next page « Previous thread | Next thread »