Srinivasan Keshav is a Professor and a Canada Research Chair at the David R. Cheriton School of Computer Science, University of Waterloo, Ontario, Canada.

Preface xv

Chapter 1: Probability 1

1.2 Joint and Conditional Probability 7

1.3 Random Variables 14

1.4 Moments and Moment Generating Functions 21

1.5 Standard Discrete Distributions 25

1.6 Standard Continuous Distributions 29

1.7 Useful Theorems 35

1.8 Jointly Distributed Random Variables 42

1.8.1 Bayesian Networks 44

1.9 Further Reading 47

1.10 Exercises 47

Chapter 2: Statistics 53

2.2 Describing a Sample Parsimoniously 57

2.3 Inferring Population Parameters from Sample Parameters 66

2.4 Testing Hypotheses about Outcomes of Experiments 70

2.5 Independence and Dependence: Regression and Correlation 86

2.6 Comparing Multiple Outcomes Simultaneously: Analysis of Variance 95

2.7 Design of Experiments 99

2.8 Dealing with Large Data Sets 100

2.9 Common Mistakes in Statistical Analysis 103

2.10 Further Reading 105

2.11 Exercises 105

Chapter 3: Linear Algebra 109

3.2 Vector and Matrix Algebra 111

3.3 Linear Combinations, Independence, Basis, and Dimension 114

3.4 Using Matrix Algebra to Solve Linear Equations 117

3.5 Linear Transformations, Eigenvalues, and Eigenvectors 125

3.6 Stochastic Matrices 138

3.7 Exercises 143

Chapter 4: Optimization 147

4.2 Introduction to Optimization 149

4.3 Optimizing Linear Systems 152

4.4 Integer Linear Programming 157

4.5 Dynamic Programming 162

4.6 Nonlinear Constrained Optimization 164

4.7 Heuristic Nonlinear Optimization 167

4.8 Exercises 170

Chapter 5: Signals, Systems, and Transforms 173

5.2 Signals 185

5.3 Systems 188

5.4 Analysis of a Linear Time-Invariant System 189

5.5 Transforms 195

5.6 The Fourier Series 196

5.7 The Fourier Transform and Its Properties 200

5.8 The Laplace Transform 209

5.9 The Discrete Fourier Transform and Fast Fourier Transform 216

5.10 The Z Transform 226

5.11 Further Reading 233

5.12 Exercises 234

Chapter 6: Stochastic Processes and Queueing Theory 237

6.2 Stochastic Processes 240

6.3 Continuous-Time Markov Chains 252

6.4 Birth-Death Processes 255

6.5 The M/M/1 Queue 262

6.6 Two Variations on the M/M/1 Queue 266

6.7 Other Queueing Systems 270

6.8 Further Reading 272

6.9 Exercises 272

Chapter 7: Game Theory 277

7.2 Solving a Game 291

7.3 Mechanism Design 301

7.4 Limitations of Game Theory 314

7.5 Further Reading 315

7.6 Exercises 316

Chapter 8: Elements of Control Theory 319

8.2 Modeling a System 323

8.3 A First-Order System 329

8.4 A Second-Order System 331

8.5 Basics of Feedback Control 336

8.6 PID Control 341

8.7 Advanced Control Concepts 346

8.8 Stability 350

8.9 State Space–Based Modeling and Control 360

8.10 Digital Control 364

8.11 Partial Fraction Expansion 367

8.12 Further Reading 370

8.13 Exercises 370

Chapter 9: Information Theory 373

9.2 A Mathematical Model for Communication 374

9.3 From Messages to Symbols 378

9.4 Source Coding 379

9.5 The Capacity of a Communication Channel 386

9.6 The Gaussian Channel 399

9.7 Further Reading 407

9.8 Exercises 407

Solutions to Exercises 411

Index 457