Monte Carlo methods are a class of methods based on the idea of sampling to study mathematical problems for which analytical solutions may be unavailable. The basic idea is to create samples through repeated simulations that can be used to derive approximations about a quantity we’re interested in, and its probability distribution.
In this post, we’ll describe how you can use AWS Batch to run Monte Carlo simulations optimally and efficiently at scale. We’ll illustrate using an example from the financial sector: the stock price prediction problem.
Some background to the Monte Carlo method
The Monte Carlo method was developed as a statistical computing tool in the mid-1940s near the advent of computing machines. John von Neumann and Stanislaw Ulam first came up with the idea of using random numbers generated by a computer in order to solve problems encountered in the development of the atomic bomb. The title of their paper “The Monte Carlo method”, which gave birth to the Monte Carlo method, is a reference to the famous casino in Monaco. You can read their paper yourself in the Journal of the American Statistical Association, 44:335–341, 1949.
Monte Carlo methods are widely used in a variety of fields like finance, physics, chemistry, and biology (and more). In these fields, many realistic models of systems of interest usually involve the assumption that at least some of the underlying variables behave in a random way. For example, in the financial domain, the prices of different assets (like stocks) are assumed to evolve in time by following a random (or stochastic) process. The Monte Carlo simulation method uses random sampling to study properties of such systems by simulating on a computer the stochastic behavior of these systems. In detail, a computer is used to randomly generate values for the stochastic variables that influence the behavior of the system. Then for each stochastic simulation, samples of the quantities of interest are computed, which are further analyzed by methods of statistical inference.
The Monte Carlo method can also be used for problems that have no inherent probabilistic structure, like computing high-dimensional multivariate integrals or solving huge systems of linear equations. While the main advantages of Monte Carlo methods over other techniques is ease of implementation and parallelization, it does have some drawbacks. One of the main drawbacks of Monte Carlo simulations is their slower rate of convergence compared to some other specialized techniques, like so-called quasi-Monte Carlo methods† When you have a large number of variables bounded to distinct constraints, this method might be computationally challenging. It can be demanding in terms of computing power and computing time to approximate a solution.
In this post, we’ll use Monte Carlo methods to value and analyze some financial assets by simulating the various sources of uncertainty affecting their value (usually done with the help of stochastic asset models), and then determine the distribution of their value over the range of resultant outcomes. We’ll walk through technical details to efficiently scale and optimize your Monte Carlo simulations, but we won’t focus on any specific data preparation or processing steps.
Read the full blog to learn more about optimizing your Monte Carlo simulations using AWS Batch.
Reminder: You can learn a lot from AWS HPC engineers by subscribing to the HPC Tech Short YouTube channel, and following the AWS HPC Blog channel.