Probability Statistics and Random Signals 1st Edition By Charles Boncelet ISBN 0190200510 978-0190200510

1 PROBABILITY BASICS

1.1 What Is Probability?

1.2 Experiments, Outcomes, and Events

1.3 Venn Diagrams

1.4 Random Variables

1.5 Basic Probability Rules

1.6 Probability Formalized

1.7 Simple Theorems

1.8 Compound Experiments

1.9 Independence

1.10 Example: Can S CommunicateWith D?

1.10.1 List All Outcomes

1.10.2 Probability of a Union

1.10.3 Probability of the Complement

1.11 Example: Now Can S CommunicateWith D?

1.11.1 A Big Table

1.11.2 Break Into Pieces

1.11.3 Probability of the Complement

1.12 Computational Procedures

Summary

Problems

2 CONDITIONAL PROBABILITY

2.1 Definitions of Conditional Probability

2.2 Law of Total Probability and BayesTheorem

2.3 Example: Urn Models

2.4 Example: A Binary Channel

2.5 Example: Drug Testing

2.6 Example: A Diamond Network

Summary

Problems

3 A LITTLE COMBINATORICS

3.1 Basics of Counting

3.2 Notes on Computation

3.3 Combinations and the Binomial Coefficients

3.4 The Binomial Theorem

3.5 Multinomial Coefficient andTheorem

3.6 The Birthday Paradox and Message Authentication

3.7 Hypergeometric Probabilities and Card Games

Summary

Problems

4 DISCRETE PROBABILITIES AND RANDOM VARIABLES

4.1 Probability Mass Functions

4.2 Cumulative Distribution Functions

4.3 Expected Values

4.4 Moment Generating Functions

4.5 Several Important Discrete PMFs

4.5.1 Uniform PMF

4.5.2 Geometric PMF

4.5.3 The Poisson Distribution

4.6 Gambling and Financial Decision Making

Summary

Problems

5 MULTIPLE DISCRETE RANDOM VARIABLES

5.1 Multiple Random Variables and PMFs

5.2 Independence

5.3 Moments and Expected Values

5.3.1 Expected Values for Two Random Variables

5.3.2 Moments for Two Random Variables

5.4 Example: Two Discrete Random Variables

5.4.1 Marginal PMFs and Expected Values

5.4.2 Independence

5.4.3 Joint CDF

5.4.4 TransformationsWith One Output

5.4.5 TransformationsWith Several Outputs

5.4.6 Discussion

5.5 Sums of Independent Random Variables

5.6 Sample Probabilities, Mean, and Variance

5.7 Histograms

5.8 Entropy and Data Compression

5.8.1 Entropy and InformationTheory

5.8.2 Variable Length Coding

5.8.3 Encoding Binary Sequences

5.8.4 Maximum Entropy

Summary

Problems

6 BINOMIAL PROBABILITIES

6.1 Basics of the Binomial Distribution

6.2 Computing Binomial Probabilities

6.3 Moments of the Binomial Distribution

6.4 Sums of Independent Binomial Random Variables

6.5 Distributions Related to the Binomial

6.5.1 Connections Between Binomial and Hypergeometric Probabilities

6.5.2 Multinomial Probabilities

6.5.3 The Negative Binomial Distribution

6.5.4 The Poisson Distribution

6.6 Binomial and Multinomial Estimation

6.7 Alohanet

6.8 Error Control Codes

6.8.1 Repetition-by-Three Code

6.8.2 General Linear Block Codes

6.8.3 Conclusions

Summary

Problems

7 A CONTINUOUS RANDOM VARIABLE

7.1 Basic Properties

7.2 Example Calculations for One Random Variable

7.3 Selected Continuous Distributions

7.3.1 The Uniform Distribution

7.3.2 The Exponential Distribution

7.4 Conditional Probabilities

7.5 Discrete PMFs and Delta Functions

7.6 Quantization

7.7 A FinalWord

Summary

Problems

8 MULTIPLE CONTINUOUS RANDOM VARIABLES

8.1 Joint Densities and Distribution Functions

8.2 Expected Values and Moments

8.3 Independence

8.4 Conditional Probabilities for Multiple Random Variables

8.5 Extended Example: Two Continuous Random Variables

8.6 Sums of Independent Random Variables

8.7 Random Sums

8.8 General Transformations and the Jacobian

8.9 Parameter Estimation for the Exponential Distribution

8.10 Comparison of Discrete and Continuous Distributions

Summary

Problems

9 THE GAUSSIAN AND RELATED DISTRIBUTIONS

9.1 The Gaussian Distribution and Density

9.2 Quantile Function

9.3 Moments of the Gaussian Distribution

9.4 The Central LimitTheorem

9.5 Related Distributions

9.5.1 The Laplace Distribution

9.5.2 The Rayleigh Distribution

9.5.3 The Chi-Squared and F Distributions

9.6 Multiple Gaussian Random Variables

9.6.1 Independent Gaussian Random Variables

9.6.2 Transformation to Polar Coordinates

9.6.3 Two Correlated Gaussian Random Variables

9.7 Example: Digital Communications Using QAM

9.7.1 Background

9.7.2 Discrete Time Model

9.7.3 Monte Carlo Exercise

9.7.4 QAM Recap

Summary

Problems

10 ELEMENTS OF STATISTICS

10.1 A Simple Election Poll

10.2 Estimating the Mean and Variance

10.3 Recursive Calculation of the Sample Mean

10.4 ExponentialWeighting

10.5 Order Statistics and Robust Estimates

10.6 Estimating the Distribution Function

10.7 PMF and Density Estimates

10.8 Confidence Intervals

10.9 Significance Tests and p-Values

10.10 Introduction to EstimationTheory

10.11 Minimum Mean Squared Error Estimation

10.12 Bayesian Estimation

Problems

11 GAUSSIAN RANDOM VECTORS AND LINEAR REGRESSION

11.1 Gaussian Random Vectors

11.2 Linear Operations on Gaussian Random Vectors

11.3 Linear Regression

11.3.1 Linear Regression in Detail

11.3.2 Statistics of the Linear Regression Estimates

11.3.3 Computational Issues

11.3.4 Linear Regression Examples

11.3.5 Extensions of Linear Regression

Summary

Problems

12 HYPOTHESIS TESTING

12.1 Hypothesis Testing: Basic Principles

12.2 Example: Radar Detection

12.3 Hypothesis Tests and Likelihood Ratios

12.4 MAP Tests

Summary

Problems

13 RANDOM SIGNALS AND NOISE

13.1 Introduction to Random Signals

13.2 A Simple Random Process

13.3 Fourier Transforms

13.4 WSS Random Processes

13.5 WSS Signals and Linear Filters

13.6 Noise

13.6.1 Probabilistic Properties of Noise

13.6.2 Spectral Properties of Noise

13.7 Example: Amplitude Modulation

13.8 Example: Discrete TimeWiener Filter

13.9 The Sampling Theorem forWSS Random Processes

13.9.1 Discussion

13.9.2 Example: Figure 13.4

13.9.3 Proof of the Random Sampling Theorem

Summary

Problems

14 SELECTED RANDOM PROCESSES

14.1 The Lightbulb Process

14.2 The Poisson Process

14.3 Markov Chains

14.4 Kalman Filter

14.4.1 The Optimal Filter and Example

14.4.2 QR Method Applied to the Kalman Filter

Summary

Problems

A COMPUTATION EXAMPLES

A.1 Matlab

A.2 Python

A.3 R

B ACRONYMS

C PROBABILITY TABLES

C.1 Tables of Gaussian Probabilities

D BIBLIOGRAPHY

INDEX