MALLA REDDY COLLEGE OF
ENGINEERING AND TECHNOLOGY
(AUTONOMOUS INSTITUTION: UGC, GOVT. OF INDIA)
ELECTRONICS AND COMMUNICATION ENGINEERING
II ECE I SEM
PROBABILITY THEORY
AND STOCHASTIC
PROCESSES
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community
Ask the community for help and clear up your study doubts
University Rankings
Discover the best universities in your country according to Docsity users
Free resources
Our save-the-student-ebooks!
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
Probability and Statistics: Random Variables and Stochastic Processes, Transcriptions of Probability and Stochastic Processes
A comprehensive overview of probability and statistics, focusing on random variables and stochastic processes. It covers key concepts such as expected value, variance, and characteristic functions, as well as the central limit theorem. The document also explores the properties of multiple random variables and their joint distributions, including statistical independence and conditional distributions. Additionally, it delves into the concept of stochastic processes, their classification, and their temporal characteristics, including stationarity, ergodicity, and autocorrelation. Suitable for students studying probability and statistics at the university level.
Typology: Transcriptions
2022/2023
1 / 194
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Probability and Statistics: Random Variables and Stochastic Processes and more Transcriptions Probability and Stochastic Processes in PDF only on Docsity!
MALLA REDDY COLLEGE OF
ENGINEERING AND TECHNOLOGY
(AUTONOMOUS INSTITUTION: UGC, GOVT. OF INDIA)
ELECTRONICS AND COMMUNICATION ENGINEERING
II ECE I SEM
PROBABILITY THEORY
AND STOCHASTIC
PROCESSES
SYLLABUS
UNIT-I-PROBABILITY AND RANDOM VARIABLE
UNIT-II- DISTRIBUTION AND DENSITY FUNCTIONS AND OPERATIONS ON
ONE RANDOM VARIABLE
UNIT-III-MULTIPLE RANDOM VARIABLES AND OPERATIONS
UNIT-IV-STOCHASTIC PROCESSES-TEMPORAL CHARACTERISTICS
UNIT-V- STOCHASTIC PROCESSES-SPECTRAL CHARACTERISTICS
UNITWISE IMPORTANT QUESTIONS
MODEL PAPAERS
JNTU PREVIOUS QUESTION PAPERS
CONTENTS
Ergodicity, Mean-Ergodic Processes, Correlation-Ergodic Processes Autocorrelation Function and Itsproperties, Gaussian Random Processes. Properties, Cross-Correlation Function and Its Properties, Covariance Functions and its Linear system Response: Functions. Mean and Mean-squared value, Autocorrelation, Cross-Correlation UNIT V:Stochastic Processes-Spectral Characteristics: The Power Spectrum and its Properties, Relationship between Power Spectrum and Autocorrelation Function, the Cross-Power Density Spectrum and Properties, Relationship between Cross-Power Spectrum and Cross-CorrelationFunction. Spectral characteristics of system response spectral density of input and output of a linear system: power density spectrum of response, cross power
TEXT BOOKS: 1. Probability, Random Variables & Random Signal Principles -Peyton Z. Peebles, TMH, 4th
- Edition, 2001.Probability and Random Processes-Scott Miller, Donald Childers,2Ed,Elsevier, REFERENCE BOOKS: 1. Theory of probability and Stochastic Processes-Pradip Kumar Gosh, University Press
- Probability and Random Processes with Application to Signal Processing - Henry Stark and John W. Woods, Pearson Education, 3rd Edition.
- Probability Methods of Signal and System Analysis- George R. Cooper, Clave D. MC Gillem, Oxford, 3rd Edition, 1999.
- Statistical Theory of Communication -S.P. Eugene Xavier, New Age Publications 2003Probability, Random Variables and Stochastic Processes Athanasios Papoulis and S.Unnikrishna Pillai, PHI, 4th Edition, 2002. OUTCOMES: Upon completion of the subject, students will be able to compute:
- Simple probabilities using an appropriate sample space.
- Simple probabilities and expectations from probability density functions (pdfs)Likelihood ratio tests from pdfs for statistical engineering problems.
- Least -square & maximum likelihood estimators for engineering problems.Mean and covariance functions for simple random processes.
UNIT – 1
PROBABILITY AND RANDOM VARIABLE
PROBABILITY
Introduction
should have become the most important object of human knowledge.^ It is remarkable that a science which began with the consideration of games of chance A brief history
Probability has an amazing history. A practical gambling problem faced by the French nobleman Chevalier de Méré sparked the idea of probability in the mind of Blaise Pascal (1623-1662) , the famous French mathematician. Pascal's correspondence with Pierre de Fermat (1601-1665), another French Mathematician in the form of seven letters in 1654 is regarded as the genesis of probability. Early mathematicians like Jacob Bernoulli (1654-1705), Abraham de Moivre (1667-1754), ThomasBayes (1702-1761) and Pierre Simon De Laplace (1749-1827) contributed to the development of probability. Laplace's Theory Analytique des Probabilities gave comprehensive tools to calculate probabilities based on the principles of permutations and combinations. Laplace also said, "Probability theory is nothing but common sense reduced to calculation."
Later mathematicians like Chebyshev (1821-1894), Markov (1856-1922), von Mises (1883-1953), Norbert Wiener (1894-1964) and Kolmogorov (1903-1987) contributed to new developments. Over the last four centuries and a half, probability has grown to be one of the mostessential mathematical tools applied in diverse fields like economics, commerce, physical sciences, biological sciences and engineering. It is particularly important for solving practicalelectrical-engineering problems in communication , signal processing and computers. Notwithstandingmathematicians for centuries. Kolmogorov in 1933 gave the the above developments, a precise definition axiomatic definition of probability of probability eluded the and resolved the problem. Randomness arises because of random nature of the generation mechanism Limited understanding of the signal dynamics inherent imprecision in measurement,observation, etc.
For example,electrons. We have deterministic model for weather prediction; it takes into account of the factors thermal noise appearing in an electronic device is generated due to random motion of affecting weather. We can locally predict the temperature or the rainfall of a place on the basis ofprevious data. Probabilistic models are established from observation of a random phenomenon. Whilesuch models from data. probability is concerned with analysis of a random phenomenon, statistics help in building
Applications Probability theory is applied in everyday life in risk assessment and in trade on financial markets. Governments apply probabilistic methods in environmental regulation, where it is called pathwayanalysis
Another significant application of probability theory in everyday life is reliability. Many consumer products, such as automobiles and consumer electronics, use reliability theory in product design to reduce the probability of failure. Failure probability may influence a manufacturer's decisions on a product's warranty.
THE BASIC CONCEPTS OF SET THEORY Some of the basic concepts of set theory are: Set the set. Usually uppercase letters are used to denote sets.: A set is a well defined collection of objects. These objects are called elements or members of
The set theory was developed by George Cantor in 1845-1918. Today, it is used in almost every branch of mathematics and serves as a fundamental part of present-day mathematics. In everyday life, we often talk of the collection of objects such as a bunch of keys, flock of birds, pack of cards, etc. In mathematics, we come across collections like natural numbers, whole numbers, prime and composite numbers.
We assume that, ● the word set is synonymous with the word collection, aggregate, class and comprises of elements. ● Objects, elements and members of a set are synonymous terms. ● Sets are usually denoted by capital letters A, B, C, ....., etc. ● Elements of the set are represented by small letters a, b, c, ....., etc. If ‗a‘ is an element of set A, then we say that ‗a‘ belongs to A. We denote the phrase ‗belongs to‘by the Greek symbol ‗∈‗ (epsilon). Thus, we say that a ∈ A.
If ‗b‘ is an element which does not belong to A, we represent this as b ∉ A. Examples of sets:
- Describe the set of vowels. If A is the set of vowels, then A could be described as A = {a, e, i, o, u}.
2.Describe the set of positive integers. Since it would be impossible to list all of the positive integers, we need to use a rule to describe this set. We might say A consists of all integers greater than zero.
- Set A = {1, 2, 3} and Set B = {3, 2, 1}. Is Set A equal to Set B? Yes. Two sets are equal if they have the same elements. The order in which the elements are listeddoes not matter.
- What is the set of men with four arms? Since all men have two arms at most, the set of men with four arms contains no elements. It is thenull set (or empty set).
- Set A = {1, 2, 3} and Set B = {1, 2, 4, 5, 6}. Is Set A a subset of Set B? Set A would be a subset of Set B if every element from Set A were also in Set B. However, this is not the case. The number 3 is in Set A , but not in Set B. Therefore, Set A is not a subset of Set B Some important sets used in mathematics are N: the set of all natural numbers = {1, 2, 3, 4, .....} Z: the set of all integers = {....., -3, -2, -1, 0, 1, 2, 3, .....} Q: the set of all rational numbers R: the set of all real numbers Z+: the set of all positive integers W: the set of all whole numbers The different types of sets are explained below with examples. 1. Empty Set or Null Set: A set which does not contain any element is called an empty set, or the null set or the void set and it is denoted bysince the number of elements in an empty set is finite, i.e., 0. ∅ and is read as phi. In roster form, ∅ is denoted by {}. An empty set is a finite set,
For example: (a) the set of whole numbers less than 0. (b) Clearly there is no whole number less than 0. Therefore, it is an empty set.
3. Finite Set: A set which contains a definite number of elements is called a finite set. Empty set is also called a finite set. For example: - The set of all colors in the rainbow. - N = {x : x ∈ N, x < 7} - P = {2, 3, 5, 7, 11, 13, 17, ...... 97} 4. Infinite Set: The set whose elements cannot be listed, i.e., set containing never-ending elements is called an infinite set. For example: - Set of all points in a plane - A = {x : x ∈ N, x > 1} - Set of all prime numbers - B = {x : x ∈ W, x = 2n} Note: All infinite sets cannot be expressed in roster form. For example: The set of real numbers since the elements of this set do not follow any particular pattern. 5. Cardinal Number of a Set: The number of distinct elements in a given set A is called the cardinal number of A. It is denoted by n(A). And read as ‗the number of elements of the set‘.
For example:
- A {x : x ∈ N, x < 5} A = {1, 2, 3, 4} Therefore, n(A) = 4 - B = set of letters in the word ALGEBRA B = {A, L, G, E, B, R} Therefore, n(B) = 6 6. Equivalent Sets: Two sets A and B are said to be equivalent if their cardinal number is same, i.e., n(A) = n(B). The symbol for denoting an equivalent set is ‗↔‘.
For example: A = {1, 2, 3} Here n(A) = 3 B = {p, q, r} Here n(B) = 3 Therefore, A ↔ B
7. Equal sets: Two sets A and B are said to be equal if they contain the same elements. Every element of A is anelement of B and every element of B is an element of A.
For example: A = {p, q, r, s} B = {p, s, r, q} Therefore, A = B
8. Disjoint Sets: Two sets A and B are said to be disjoint, if they do not have any element in common. For example;
Here A is a subset of B Since, all the elements of set A are contained in set B. But B is not the subset of A Since, all the elements of set B are not contained in set A. Notes: If ACB and BCA, then A = B, i.e., they are equal sets. Every set is a subset of itself. Null set or ∅ is a subset of every set.
2. The set N of natural numbers is a subset of the set Z of integers and we write N ⊂ Z. 3. Let A = {2, 4, 6} B = {x : x is an even natural number less than 8} Here A ⊂ B and B ⊂ A. Hence, we can say A = B 4. Let A = {1, 2, 3, 4} B = {4, 5, 6, 7} Here A ⊄ B and also B ⊄ C [⊄ denotes ‗not a subset of‘] 11. Super Set: Whenever a set A is a subset of set B, we say the B is a superset of A and we write, B ⊇ A. Symbol ⊇ is used to denote ‗is a super set of‘ For example;
A = {a, e, i, o, u} B = {a, b, c, ............., z} Here A ⊆ B i.e., A is a subset of B but B ⊇ A i.e., B is a super set of A
12. Proper Subset: If A and B are two sets, then A is called the proper subset of B if A ⊆ B but B ⊇ A i.e., A ≠ B. The symbol ‗⊂‘ is used to denote proper subset. Symbolically, we write A ⊂ B. For example; 1. A = {1, 2, 3, 4} Here n(A) = 4 B = {1, 2, 3, 4, 5} Here n(B) = 5 We observe that, all the elements of A are present in B but the element ‗5‘ of B is not present in A. So, we say that A is a proper subset of B. Symbolically, we write it as A ⊂ B Notes: No set is a proper subset of itself. Null set or ∅ is a proper subset of every set. 2. A = {p, q, r} B = {p, q, r, s, t} Here A is a proper subset of B as all the elements of set A are in set B and also A ≠ B. Notes: No set is a proper subset of itself. Empty set is a proper subset of every set.
The symbol for denoting union of sets is ‗∪‘. Some properties of the operation of union: (i) A∪B = B∪A (Commutative law) (ii) A∪(B∪C) = (A∪B)∪C (Associative law) (iii) A ∪ Ф = A (Law of identity element, is the identity of ∪ ) (iv) A∪A = A (Idempotent law) (v) U∪A = U (Law of ∪ ) ∪ is the universal set. Notes: A ∪ Ф = Ф∪ A = A i.e. union of any set with the empty set is always the set itself. Examples:
1. If A = {1, 3, 7, 5} and B = {3, 7, 8, 9}. Find union of two set A and B. Solution: A ∪ B = {1, 3, 5, 7, 8, 9} No element is repeated in the union of two sets. The common elements 3, 7 are taken only once. 2. Let X = {a, e, i, o, u} and Y = {ф}. Find union of two given sets X and Y. Solution: X ∪ Y = {a, e, i, o, u} Therefore, union of any set with an empty set is the set itself. 2. Definition of Intersection of Sets: Intersection of two given sets is the largest set which contains all the elements that are common toboth the sets.
To find the intersection of two given sets A and B is a set which consists of all the elements which are common to both A and B. The symbol for denoting intersection of sets is ‗ ∩ ‗. Some properties of the operation of intersection
(i) A∩B = B∩A (Commutative law) (ii) (A∩B)∩C = A∩ (B∩C) (Associative law) (iii) Ф ∩ A = Ф (Law of Ф) (iv) U∩A = A (Law of ∪) (v) A∩A = A (Idempotent law) (vi) A∩(B∪C) = (A∩B) ∪ (A∩C) (Distributive law) Here ∩ distributes over ∪ Also, A∪(B∩C) = (AUB) ∩ (AUC) (Distributive law) Here ∪ distributes over ∩ Notes: A ∩ Ф = Ф ∩ A = Ф i.e. intersection of any set with the empty set is always the empty set. Solved examples :
1. If A = {2, 4, 6, 8, 10} and B = {1, 3, 8, 4, 6}. Find intersection of two set A and B. Solution: A ∩ B = {4, 6, 8} Therefore, 4, 6 and 8 are the common elements in both the sets. 2. If X = {a, b, c} and Y = {ф}. Find intersection of two given sets X and Y. Solution: X ∩ Y = { } 3. Difference of two sets If A and B are two sets, then their difference is given by A - B or B - A.
- If A = {2, 3, 4} and B = {4, 5, 6} A - B means elements of A which are not the elements of B. i.e., in the above example A - B = {2, 3} In general, B - A = {x : x ∈ B, and x ∉ A}
Some properties of complement sets (i) A ∪ A' = A' ∪ A = ∪ (Complement law) (ii) (A ∩ B') = ϕ (Complement law) - The set and its complement are disjoint sets. (iii) (A ∪ B) = A' ∩ B' (De Morgan‘s law) (iv) (A ∩ B)' = A' ∪ B' (De Morgan‘s law) (v) (A')' = A (Law of complementation) (vi) Ф' = ∪ (Law of empty set - The complement of an empty set is a universal set. (vii) ∪' = Ф and universal set) - The complement of a universal set is an empty set. For Example; If S = {1, 2, 3, 4, 5, 6, 7} A = {1, 3, 7} find A'. Solution: We observe that 2, 4, 5, 6 are the only elements of S which do not belong to A. Therefore, A' = {2, 4, 5, 6} Algebraic laws on sets:
1. Commutative Laws: For any two finite sets A and B; (i) A U B = B U A (ii) A ∩ B = B ∩ A 2. Associative Laws: For any three finite sets A, B and C; (i) (A U B) U C = A U (B U C) (ii) (A ∩ B) ∩ C = A ∩ (B ∩ C) Thus, union and intersection are associative.
3. Idempotent Laws: For any finite set A; (i) A U A = A (ii) A ∩ A = A 4. Distributive Laws: For any three finite sets A, B and C; (i) A U (B ∩ C) = (A U B) ∩ (A U C) (ii) A ∩ (B U C) = (A ∩ B) U (A ∩ C) Thus, union and intersection are distributive over intersection and union respectively. 5. De Morgan’s Laws: For any two finite sets A and B; (i) A – (B U C) = (A – B) ∩ (A – C) (ii) A - (B ∩ C) = (A – B) U (A – C) De Morgan‘s Laws can also we written as: (i) (A U B)‘ = A' ∩ B' (ii) (A ∩ B)‘ = A' U B' More laws of algebra of sets: 6. For any two finite sets A and B; (i) A – B = A ∩ B' (ii) B – A = B ∩ A' (iii) A – B = A ⇔ A ∩ B = ∅ (iv) (A – B) U B = A U B (v) (A – B) ∩ B = ∅