About: Probabilistic programming languages (PPLs) unify techniques for the formal description of computation and for the representation and use of uncertain knowledge. PPLs have seen recent interest from the artificial intelligence, programming languages, cognitive science, and natural languages communities. This book explains how to implement PPLs by lightweight embedding into a host language. We illustrate this by designing and implementing WebPPL, a small PPL embedded in Javascript. We show how to implement several algorithms for universal probabilistic inference, including priority-based enumeration with caching, particle filtering, and Markov chain Monte Carlo. We use program transformations to expose the information required by these algorithms, including continuations and stack addresses. We illustrate these ideas with examples drawn from semantic parsing, natural language pragmatics, and procedural graphics.
Citing: Please cite this book as:
N. D. Goodman and A. Stuhlmüller (electronic). The Design and Implementation of Probabilistic Programming Languages. Retrieved from http://dippl.org
. [bibtex]
@misc{dippl, title = {{The Design and Implementation of Probabilistic Programming Languages}}, author = {Goodman, Noah D and Stuhlm\"{u}ller, Andreas}, year = {2014}, howpublished = {\url{http://dippl.org}}, note = {Accessed: } }
Introduction
Course overview and brief introduction.
The WebPPL language
A small probabilistic language embedded in Javascript.
Exploring the executions of a random computation
Implementing marginal inference by enumeration using continuations, coroutines, and CPS.
Early, incremental evidence
Inserting and commuting factor statements to get the right incremental sequencing.
Particle Filtering
Models with continuous variables, importance sampling, and sequential Monte Carlo.
Markov Chain Monte Carlo
Trace-based implementation of MCMC.
Pragmatics
The (simplified) Rational Speech Acts model of natural language pragmatics.
Semantic parsing
A Bayesian literal listener who conditions on the meaning of a sentence. The meaning is computed by direct composition in a categorial grammar.
Computer vision
Finding latent structure that renders to a target image.
Pragmatic inference based on compositional semantic meanings
Combining pragmatic reasoning with incremental semantic parsing.
Book content
Markdown code for the book chapters
WebPPL
A probabilistic programming language for the web
The construction of this tutorial was made possible by grants from DARPA, under agreement number FA8750-14-2-0009, and the Office of Naval Research, grant number N00014-13-1-0788. (The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DARPA or the U.S. Government.)
This book is based on notes from the ESSLLI 2014 class on Probabilistic Programming Languages, taught by the authors.