Probability and Random Processes: Theory and Simulation for Homework Solutions
Introductory discrete and continuous probability concepts, single and multiple random variable distributions, expectation, introductory stochastic processes, correlation and power spectral density properties of random signals, random signals through linear filters.
Probability And Random Processes Homework Solutions
Notice : Efforts on homework should be individual based on the material in class and in the book. Students that will get caught copying their solution from the solution manual, past solutions, or other students, will face disciplinary actions.
This is an advanced undergraduate+graduate course on applied stochastic processes, designed for those students who are going to need to use stochastic processes in their research but do not have the measure-theoretic background to take the Math 561-562 sequence. Measure theory is not a prerequisite for this course. However, a basic knowledge of probability theory (Math 461 or its equivalent) is expected, as well as some knowledge of linear algebra and analysis. The goal of this course is a good understanding of basic stochastic processes, in particular discrete-time and continuous-time Markov chains, and their applications. The materials covered in this course include the following:Fundamentals: background on probability, linear algebra, and set theory.
Discrete-time Markov chains: classes, hitting times, absorption probabilities, recurrence and transience, invariant distribution, limiting distribution, reversibility, ergodic theorem, mixing times;
Continuous-time Markov chains: same topics as above, holding times, explosion, forward/backward Kolmogorov equations;
Related topics: Discrete-time martingales, potential theory, Brownian motion;
Applications: Queueing theory, population biology, MCMC.
This course can be tailored to the interests of the audience.
Assignment 4: Assigned 9/25/13; Due 9/30/13 at 2 P.M.This material will be covered on the exam, submit to the GTA George MacCartney by 2pm in room 9.009 at the end of the help session or submit by uploading on blackboard a scanned PDF following the homework guidelines.View HW 4 Due 9/30/13 at 2 P.M.View HW 4 solutions
HW 10 assigned due 12/04/1312/4Mean square estimation, Interpolation, Markov sequences, Wiener Filter, Wiener Hopf equation, PredictorsPages 580-605HW 10 due12/11Approximately 1/3 of the final exam will cover material after exam 2 and 2/3 of the final exam will cover material up to and including exam 2. Three 8.5 x 11 Crib sheets may be used by students; this exam is closed book and notes. The final exam date is December 18, 2013 more details to come.Course ReviewCourse OutlineContinuous and discrete random variables and their joint Probability Distribution and density functions; Functions of one Random Variable and their distributions Independent Random Variables and conditional distributions; One Function of One and Two Random Variables and Two Functions of Two Random variables and their joint density functions; Jointly distributed Discrete Random Variables and their Functions; Characteristic Functions and Higher Order Moments; Covariance, Correlation, Orthogonality; Jointly Gaussian Random Variables; Linear functions of Gaussian Random Variables and their joint density functions. Stochastic Processes and the concept of Stationarity; Strict Sense Stationary (SSS) and Wide Sense Stationary (WSS) Processes; Auto correlation function and its properties; Poisson Processes and Wiener processes; Stochastic Inputs to Linear Time-Invariant (LTI) Systems and their input-output Autocorrelations; Input-Output Power Spectrum for Linear Systems with Stochastic Inputs; Minimum Mean Square Error Estimation(MMSE) and Orthogonality Principle; Auto Regressive Moving Average (ARMA) Processes and their power spectra.
Instructor : Alexander Barg, ProfessorDepartment of Electrical and Computer Engineering/Institute for Systems ResearchOffice: 2361 A.V.Williams Building. E-mail abarg at umd dot eduCourse Homepage: abarg/620TA: Purbesh Mitra, pmitra@umd.eduTeaching arrangements: Class times: TuTh 12:30-1:45, EGR0108 Discussion sessions: F 11:00-11:50 JMP 1202 (Sec.0101); F 2:00-2:50 (Sec.0102); TA Office hours: Mo 2:00-3:30, AVW1111 Instructor availability outside class hours: after class (preferred), or send me an email Canvas/ELMS is used for (1) submission of homework and exam papers; (2) posting of lecture notes, discussion materials, lecture recordings.Grading: Several (5-6) home assignments (20%), Max(midterm1, midterm2) 30%, Min(midterm1,midterm2) 20%, final (30%). All exams are take home.This course does not rely on a single textbook. In preparing the lectures I use multiple sources including the books listed below and web resources. References are provided in the detailed outline of the course, and lecture notes will be posted on Canvas are we progress. Lecture notes on probability theory:Lecture notes - advanced probability and measure theory (A. Dembo).
Lecture notes - elementary probability (C. Grinstead).
Main topics: Selective review of probability theory (Probability distributions, expectation) Convergence of sequences of random variables; Laws of large numbers Discrete-time Markov chains: Ergodic theorems, examples The Poisson process; Continuous-time Markov chains L2-theory of random processes; Gaussian processes and the Wiener process Discrete-time martingales; Convergence and inequalities
This course covers the basic concepts of probability theoryand random processes.Targeted at first year graduate students it introduces conceptsat an appropriately rigorous level and discusses applications through examplesand homework, such as to Digital Communication Systems. The syllabus coverselementary probability theory, random variables, limiting theorems such asthe Law of Large Numbers, the Central Limit Theorem, and Martingales,as well as Gaussian, Markovian and Renewal Processes.
Three extra topics not covered this semester are renewal processes, queueing theory, and the connection between random walks and electrical networks. If you are interested to learn more about renewal processes and queueing theory, check Chapter 3 and sections 4.5-4.6 of the textbook. For random walks and electrical networks, there is a very nice introduction in Chapter 9 of Markov chains and mixing times by Levin, Peres, and Wilmer, and much more information in Probability on trees and networks by Lyons and Peres.
I've started putting the notes for my lectures on stochastic processes(36-754) online atthe course homepage.ContentsTable of contents, which gives a running list of definitions, lemmas,theorems, etc. This will be updated with each new lecture.Lecture1 (16 January)Definition of stochastic processes, examples, random functionsLecture2 (18 January)Finite-dimensional distributions (FDDs) of a process, consistency of afamily of FDDs, theorems of Daniell and Kolmogorov on extending consistentfamilies to processesLecture 3 (20 January)Probability kernels and regular conditional probabilities, extendingsfinite-dimensional distributions defined recursively through kernels toprocesses (the Ionescu Tulcea theorem).Homework Assignment 1 (due 27 January)Exercise 1.1; Exercise3.1. Solutions.Lecture 4(23 January)One-paramater processes and their representation by shift-operatorsemi-groups.Lecture 5 (25 January)Three kinds of stationarity, the relationship between strong stationarity and measure-preserving transformations (especially shifts).Lecture 6 (27January)Reminders about filtrations and optional times, definitions of varioussorts of waiting times, and Kac's Recurrence Theorem.Homework Assigment 2 (due 6 February)Exercise 5.3; Exercise 6.1; Exercise6.2. SolutionsLecture 7(30 January)Kinds of continuity, versions of stochastic processes, difficulties ofcontinuity, the notion of a separable random function.Lecture 8(1 February)Existence of separable modifications of stochastic processes, conditionsfor the existence of measurable, cadlag and continuous modifications.Lecture 9(3 February)Markov processes and their transition-probability semi-groups.Lecture10 (6 February)Markov processes as transformed IID noise; Markov processes as operatorsemi-groups on function spaces.Lecture11 (8 February)Examples of Markov processes (Wiener process and the logistic map).Overlaps with solutions tothe second homework assignment.10 FebruaryMaterial from section 2 of lecture 10, plus an excursion into soficprocesses.Lecture12 (13 February)Generators of homogeneous Markov processes, analogy with exponentialfunctions.Lecture 13 (15 February)The strong Markov property and the martingale problem.Homework Assignment 3 (due 20 February)Exercises 10.1 and 10.2Lecture 14 (17, 20 February)Feller processes, and an example of a Markov process which isn'tstrongly Markovian.Lecture 15 (24 February, 1 March)Convergence in distribution of cadlag processes, convergence of Fellerprocesses, approximation of differential equations by Markov processes.Lecture 16 (3 March)Convergence of random walks to Wiener processes.Homework Assignment 4 (due 13 March)Exercise 16.1, 16.2 and 16.4.Lecture 17 (6 March)Diffusions, Wiener measure, non-differentiability of almost all continuous curves.Lecture 18 (8 March)Stochastic integrals: heuristic approach via Euler's method, rigorous approach.Lecture 19 (20, 21, 22 and 24 March)Examples of stochastic integrals. Ito's formula for change of variables.Stochastic differential equations, existence and uniqueness of solutions.Physical Brownian motion: the Langevin equation, Ornstein-Uhlenbeckprocesses.Lecture 20 (27 March)More on SDEs: diffusions, forward (Fokker-Planck) and backward equations.White noise.Lecture 21 (29, 31 March)Spectral analysis; how the white noise lost its color. Mean-squareergodicity.Lecture22 (3 April)Small-noise limits for SDEs: convergence in probability to ODEs, and ourfirst large-deviations calculations.Lecture23 (5 April)Introduction to ergodic properties and invariance.Lecture24 (7 April)The almost-sure (Birkhoff) ergodic theorem.Lecture25 (10 April)Metric transitivity. Examples of ergodic processes. Preliminaries onergodic decompositions.Lecture26 (12 April)Ergodic decompositions. Ergodic components as minimal sufficientstatistics.Lecture27 (14 April)Mixing. Weak convergence of distribution and decay of correlations.Central limit theorem for strongly mixing sequences.Lecture28 (17 April)Introduction to information theory. Relations between Shannon entropy,relative entropy/Kullback-Leibler divergence, expected likelihood and Fisherinformation.Lecture 29 (24 April)Entropy rate. The asymptotic equipartition property, a.k.a. theShannon-MacMillan-Breiman theorem, a.k.a. the entropy ergodic theorem.Asymptotic likelihoods.Lecture30 (26 April)General theory of large deviations. Large deviations principles and ratefunctions; Varadhan's Lemma. Breeding LDPs: contraction principle,"exponential tilting", Bryc's Theorem, projective limits.Lecture31 (28 April)IID large deviations: cumulant generating functions, Legendre's transform,the return of relative entropy. Cramer's theorem on large deviations ofempirical means. Sanov's theorem on large deviations of empirical measures.Process-level large deviations.Lecture32 (1 May)Large deviations for Markov sequences through exponential-familydensities.Lecture 33 (2 May)Large deviations in hypothesis testing and parameter estimation.Lecture 34 (3 May)Large deviations for weakly-dependent sequences (Gartner-Ellistheorem).Lecture 35 (5 May)Large deviations of stochastic differential equations in the small-noiselimit (Freidlin-Wentzell theory).ReferencesThe bibliography, currently confined to works explicitly cited.Everything todateIn the staggeringly-unlikely event that anyone wants to keep track of thecourse by RSS, this should dothe trick.Enigmas of Chance;Corrupting the Young