A sample space is a specification of all the outcomes that might result from an action taken, a decision made, or an experiment performed. The specification may be a list, either finite or infinite, or a collection of ranges, or a combination of a list and ranges.
The outcomes in a sample space are supposed to be elemental --- they are called atomic events, indecomposable events, simple events. (We didn't used to know that elements could be broken up, or that atoms could be broken up.)
The outcomes in a sample space must be distinct and non-overlapping. Exactly one of them, and only one, must occur when the randomness is resolved.
For example, if a stranger gives you a tulip bulb and you intend to plant it in the fall, the outcome could be defined as what you will see in the spring. You could define the sample space in many ways, depending on what you intend to observe, and what you intend to ignore. For example, you could choose to observe the color of the flowers and the height of the plant. Then, you could define the following sample space:
the bulb will produce a short tulip plant with yellow flowers
the bulb will produce a tall tulip plant with red flowers
the bulb will produce a short tulip plant with red flowers
This
is a list. When you prepare a list like this for a sample space, you are
saying that the list contains everything that your experiment will produce.
If you think the flowers might be some other color than red or yellow,
you should add more items to the list.
For example, you could add the following items. (If you don't believe that any of them could happen, they don't need to be in the list.)
the bulb will produce some other kind of tulip plant
the bulb will produce a plant that is not a tulip
If you want to be more precise about the outcomes, you can define (for each possible height) ranges of red and yellow, from least red to most red and from least yellow to most yellow, with each of the two colors varying continuously within its range.
The
way you define a sample space is up to you, as long as you follow the rules,
and accept the consequences.
The outcomes in a sample space can be grouped into sets containing one or more elements. These sets are called events.
There are well-defined ways to form events:
JOINT EVENTS
An intersection is sometimes called a joint event, since if it occurs, two events will occur jointly. Joint events have joint probabilities.
SEQUENTIAL EVENTS
Joint events often consist of one event followed in time by another, such as a coin picked at random from your pocket, then thrown into a fountain to produce a splash with a random height. Sequential events should be considered sequential in terms of gaining information: first you will learn that this has happened, then you will learn that something else has happened. The order in which the events will occur might be exactly the opposite of the order that you will learn about them. This is the point of view to adopt to understand Bayes' Theorem.
MUTUALLY EXCLUSIVE EVENTS (DISJOINT EVENTS)
If two events cannot both occur, they are called mutually exclusive. The occurrence of one excludes the occurrence of the other. Three or more events can also be mutually exclusive. The occurrence of any single one of them excludes the occurrence of all the others. For example, if you have a die with four faces colored red, yellow, green, blue, and if you drop the die on the floor and then pick it up and look at the color that was next to the floor, the four events "you will see green", "you will see yellow", "you will see blue", "you will see red" are mutually exclusive.
TOTALLY EXHAUSTIVE EVENTS
If
every point in the sample space is contained in at least one member of
a family of events, then the family of events is called totally exhaustive.
At least one of the events in a totally exhaustive family will occur, because
each possible outcome is contained somewhere in the family. The reason
for the name? The family of events totally uses up --- or exhausts ---
the sample space. The Law of Total Probability
uses families of events that are both totally exhaustive and also mutually
exclusive.
Each event is assigned a number between 0 and 1 (inclusive).
This number is called the probability of the event. The purpose is to quantify the tendency, or propensity, or believability, of the event. (See Ian Hacking's book for discussion of how this idea developed, and on changes in the way the word "probability" has been used.)
The events are related in complicated ways, and the probabilities must be assigned in a way that is consistent with these relationships --- probabilities must follow rules. The basic rules are simple:
There are a couple of mathematical complications. Most mathematicians extend the second bullet to include infinite sums. This makes the math easier (and more powerful). Also, for some sample spaces, mathematicians know how to construct probabilities that don't work for all the subsets. In other words, there are sometimes subsets that can't be events with probabilities. The sample spaces and probabilities for which this is true can be quite ordinary, but the "non-probalizable" subsets are weird, and don't have any engineering significance (probably and hopefully).
Where do probabilities come from? They are chosen by people, based on experience and judgment, using theoretical models, historical data, assessment, and computer simulations (and anything else that might be handy and useful). All probabilities are chosen (the result of a decision), even those that seem objective and inevitable, such as the probaility of getting a Head when a fair coin is tossed.
Your probabilities may depend on your personal decisions. For example, the probability that you will be stranded on the highway sometime within the next year may depend on what kind of transportation you choose to use throughout the year.