Download Inverse Transform Method Supplemental Notes and more Study notes Mathematical Modeling and Simulation in PDF only on Docsity!
The inverse transform method, first introduced in
M1L8, will be encountered frequently throughout the
course. It's important to note that fully understanding
the concept requires time and practice. If you're
unsure about what it is or why it works, don't worryโ
this is completely normal, and you're not alone.
Below is an explanation of the inverse transform theorem/method provided by Angela Smiley, a teaching assistant for this course in previous terms which I have modified and expanded upon throughout my time as a teaching assistant. Angela is the best!
Preliminaries
This... is a function! (It gets more interesting from here, promise.) Or more precisely, it's a graph of a function. You render a function 's graph by putting in a bunch of x values and drawing the point for each one. So in this case, for , you get values like , and so on, all the way up to.
Among other things, this means that if we want to "apply" the function we've just graphed, we can pick a point on the x axis and follow it up to the curve and figure out where that point falls on the y axis: Voila!. In this case,. Now, here's the less-obvious corollary: if we want to find , we can do something remarkably similar. Pick some point on the y axis and follow it over to reach the curve. Then figure out where that point falls on the x axis:
The other function, the CDF or cumulative distribution function , describes the probability that any given sample from this distribution will be less than some threshold,. So for the distribution I described above, the probability of sampling a value less than is. The probability of sampling is ; the probability of sampling is ; the probability of sampling less than, say, , is. (This makes sense, because all of the possible x values , , and - qualify, right?) That's a plot of the CDF.. which is a function. In part 1 we were looking at and using a plot. No reason we can't do the same here! , which checks out - for this distribution with possible values , and , the probability is simply.
, which also checks out -. What happens if we do another one?
. Again, makes sense since.
This distribution has some high-ish probability of sampling values between and , which within that interval is very high near and drops precipitously towards ; a fairly low probability of sampling a given number between and ; a somewhat higher chance of sampling a given number between and ; and a chance of sampling anything greater than or less than This distribution has the following CDF :
Now, we can do all the same things with this CDF plot that we did with the other one; after all, they are both CDFs. For example, let's approximate the value of : It is Or, we can find the value of : Itโs
doesn't appeal, we can talk about the slope at a point on the graph - if we could somehow get the slope of every point, we could describe differences like " is twice as likely to be 'hit' as ." And by slope I mean derivative. And by somehow, I mean by recycling a graph we already saw above, because... The derivative of the CDF is the PDF.
Bringing it all together
So both types of distributions have a CDF and a PMF or PDF. The PDF describes the relative likelihoods of obtaining different values when you sample this distribution - that is, the likelihood of sampling value x from the distribution is proportional to. If we uniformly take a bunch of samples from the interval and send them through the looking glass, they will land on the -axis with a density described by. The more samples we take, the closer the final distribution will be to. If we take samples uniformly at random from - that is, if we sample - and dump them into , what happens? Well, the likelihood of sampling value will be proportional to
. But this is exactly the definition of sampling a distribution with pdf or pmf! Hence, the inverse transform theorem, which is handy for any distribution where you can obtain
- regardless of its type (or ugliness). Maybe continuous and discrete distributions aren't so different after all. The whole concept revolves around this graphical depiction. We are starting with some value between 0 and 1 and then working it backwards (inverting it) to get a value from the target distribution we wish to generate from.
Proof
represents a random variable that is a function of the random variable. It represents picking some random value and finding the value of the CDF there. I like to think of it very informally as taking my finger and putting it somewhere on the -axis when graphing the CDF. Wherever your finger goes is some random place between and each with equal probability (as long as my finger isn't biased which I hope it isn't) Let and suppose that has CDF. (We now want to find the distribution of (this is similar to what we've done in M2L8 Functions of a Random Variable) (Definition of a CDF) ( ) (Take inverse of both sides) (Definition of a CDF; i.e. we had where ) (Applying the CDF to the inverse of a CDF are inverse operations and "cancel" each other out) is the CDF of the distribution, so we have shown that
Example R Code
Discrete Distribution
This code is demonstrating inverse transform sampling for a discrete distribution. It follows these steps: โ Define the inverse cumulative distribution function (inverse CDF) for the discrete distribution. This function takes an input value y and returns the corresponding value from the discrete distribution based on the cumulative probabilities: (scroll back up to the graph with the green, blue, and yellow lines to see why this works) โ If y < 0.5, it returns 2. โ If 0.5 โคy < 0.8, it returns 3. โ If y โฅ0.8, it returns 4. โ Set a seed for random number generation to ensure reproducible results (set.seed(1)). โ Define the number of samples to generate (n_samples <- 1e5, which is equal to 100,000). โ Generate random samples from a uniform distribution between 0 and 1 (uniform_samples <- runif(n_samples)). These samples represent uniformly distributed probabilities. โ Apply the inverse CDF for the discrete distribution to the uniform samples. The sapply() function applies the inverse_cdf_discrete() function to each value in uniform_samples and stores the results in a new vector called discrete_samples. โ Create a histogram to visualize the resulting distribution of the discrete_samples. This histogram shows the frequencies of the discrete values (2, 3, and 4) in the generated samples. # Inverse CDF for the discrete distribution inverse_cdf_discrete <- function (y) { ifelse(y < 0.5, 2 , ifelse(y < 0.8, 3 , 4 )) } # Generate random samples from a uniform distribution set.seed( 1 ) n_samples <- 1e
uniform_samples <- runif(n_samples) # Apply the inverse CDFs to the uniform samples discrete_samples <- sapply(uniform_samples, inverse_cdf_discrete) # Examine the proportions of each value table(discrete_samples) / length(discrete_samples)
discrete_samples
2 3 4
0.50008 0.29951 0.
# Create histograms to visualize the resulting distributions hist(discrete_samples, main = "Discrete Distribution", xlab = "x", col = "lightblue", border = "black") Compare this histogram with the graph of the PMF of this random variable.
Continuous Distribution
This code demonstrates inverse transform sampling for a continuous distribution and examines the proportions of samples in specified intervals. The code follows these steps: โ Define the inverse cumulative distribution function (inverse CDF) for the continuous distribution. This function takes an input value y and returns the corresponding value
n_samples <- 1e uniform_samples <- runif(n_samples) # Apply the inverse CDFs to the uniform samples continuous_samples <- sapply(uniform_samples, inverse_cdf_continuous) # Examine the proportions of each interval table(cut(continuous_samples, breaks = c( 0 , 1 , 3 , 4 ))) / length(continuous_samples)
(0,1] (1,3] (3,4]
0.50008 0.24911 0.
# Create histograms to visualize the resulting distributions hist(continuous_samples, 100 , main = "Continuous Distribution", xlab = "x", col = "lightblue", border = "black") Compare this histogram with the graph of the PDF of this random variable.
F(X) is Uniform
This code demonstrates the transformation of a set of exponential random variates back to a uniform distribution using the cumulative distribution function (CDF) of the exponential distribution.
โ Set a seed for random number generation to ensure reproducible results (set.seed(1)). โ Define the parameter lambda for the exponential distribution (lambda <- 2). โ Generate a large number of exponential random variates (1 million) using the rexp() function with the specified rate parameter lambda. The generated random variates are stored in the variable X. โ Create a histogram of the generated exponential random variates with 30 bins, using the hist() function. The title of the histogram is set to "X" (main = "X"). โ Transform the exponential random variates X back to a uniform distribution using the CDF of the exponential distribution. The CDF is given by F(x) = 1 - exp(-lambda * x). The transformed values are stored in the variable FX. โ Create a histogram of the transformed uniform distribution values FX, with 30 bins, using the hist() function. The title of the histogram is set to "F(X)" (main = "F(X)"). The two histograms display the distribution of the original exponential random variates (X) and the transformed uniform distribution values (F(X)). The second histogram should resemble a uniform distribution, demonstrating that applying the CDF to the exponential random variates results in a uniform distribution. set.seed( 1 ) # Generate a bunch of exponential random variates lambda <- 2 X <- rexp(1e6, rate = lambda) hist(X, 30 , main = "X", col = "lightblue", border = "black")
U vs. 1-U
In the long run, or will generate the target distribution for most distributions but not all. Each individual result will be different, but you will still get the correct distribution. Even though an individual random variate generated via U and 1-U will be different, we are thinking in terms of many many generated random variates and the distribution as a whole. The reason why this works is that if then let a. is also Let Find the CDF of V. This is finding the CDF of a function of a random variable (See M2L Functions of a Random Variable) Therefore has the same CDF as a random variable so
Geometric Distribution
This code generates random samples from the geometric distribution with success probability p, using inverse transform sampling, and computes the mean and variance.. โ Set a seed for random number generation to ensure reproducible results (set.seed(6644)). โ Define the success probability p for the geometric distribution (p <- 0.3). โ Define the number of random samples to generate (n <- 1e5). โ Generate n uniform random samples between 0 and 1 using the runif() function and store them in the variable U. โ Generate random samples from a geometric distribution using the formula ceil((log(1-U)) / (log(1-p))). Store the generated samples in the variable one_minus. โ Create a histogram of the one_minus samples. โ Generate random samples from a geometric distribution using the formula ceil((log(U)) / (log(1-p))). Store the generated samples in the variable not_one_minus.
โ Create a histogram of the not_one_minus samples. โ Calculate the mean and variance of the one_minus and not_one_minus samples using the mean() and var() functions, respectively. set.seed( 6644 ) p <- 0. n <- 1e U <- runif(n) one_minus <- ceiling((log( 1 -U)) / (log( 1 -p))) hist(one_minus, main = "Geometric Distribution (1 - U)", col = "lightblue", border = "black") not_one_minus <- ceiling((log(U)) / (log( 1 -p))) hist(not_one_minus, main = "Geometric Distribution (U)", col = "lightgreen", border = "black")
