
John
Paisley
Office: Mudd 422
Email:
jpaisley@columbia.edu
Phone: (212) 8548024
Mail Address:
Columbia University
500 W. 120th St., Suite 1300
New York, NY
10027


Variational methods for approximate posterior inference

Integralfree (a.k.a. blackbox) variational inference 
I
developed a stochastic gradient
approach to variational inference that allows one to optimize the
objective function without having to calculate integrals. Later called
"blackbox variational inference," this method forms unbiased
approximations to the true gradient. To reduce variance I proposed
using control variates, which are particularly wellsuited to the variational
inference problem. This can lead to easy, automatic
variational inference in a wide variety of nonconjugate (or
conjugate) models.

J.
Paisley, D. Blei and M.I. Jordan. Variational
Bayesian inference with
stochastic search, Int. Conf. on Machine Learning (ICML), 2012. 



Scalable inference for Big Data

Another
focus of my research is on scalable model inference. These methods
build on a technique called "stochastic variational inference," which
is a general framework for scalable Bayesian modeling that allows the
algorithm to quickly converge to an approximate posterior distribution
without sacrificing any data.
Having massive data sets means that we can learn greater structure in
the data via more complex models. SVI
allows for this increase in data and model size without an equally
significant increase in computational burden. With my collaborators, I
have exploited this fact to learn greater structure from data using the
mixedmembership modeling framework.
I have studied scalable inference
for a variety of model structures, for example treestructured models,
graphbased models, dictionary learning models and matrix
factorization. I have applied these techniques to topic modeling, image processing, automatic tagging and timeevolving data.

Representative publications
M. Hoffman, D. Blei, C. Wang and J. Paisley. Stochastic variational inference, Journal of Machine Learning Research,
vol. 14, pp. 13031347, 2013.
A. Zhang, S. Gultekin and J. Paisley. Stochastic variational inference for the HDPHMM, International Conference on Artificial Intelligence and Statistics (AISTATS), 2016.
S. Gultekin and J. Paisley. A collaborative Kalman filter for timeevolving dyadic processes, IEEE International Conference on Data Mining (ICDM), 2014.
D. Liang, J. Paisley and D. Ellis. Codebookbased scalable music tagging with Poisson matrix factorization, International Society for Music Information Retrieval Conference, 2014.


Batch inference

Stochastic inference



Bayesian models and applied deep learning for text and images


A
major focus of my research is on developing Bayesian models for various
problems involving text and images. For example, I developed beta
process factor analysis (BPFA), which I've applied to
image processing problems such as denoising and compressed sensing for
MRI (toy example at right). Recently I've begun working on applied deep learning for image processing and text.
Topic
models are another area of focus. My recent work
includes developing structured models for large scale text data based
on Dirichlet processes and Markov processes. I have also recently
worked on applying Gaussian processes to manifold learning for both
unsupervised and supervised problems.



Zerofilling

Total variation (best)



BPFA (reconstructed part)

BPFA (denoised part)



Representative publications
J. Paisley and L.
Carin. Nonparametric factor
analysis with beta process priors, International
Conference on Machine Learning (ICML), 2009.
J. Yang, X. Fu, Y. Hu, Y. Huang, X. Ding and J. Paisley. PanNet: A deep network architecture for pansharpening, International Conference on Computer Vision (ICCV), 2017.
A. Zhang and J. Paisley. Markov mixed membership models, International Conference on Machine Learning (ICML), 2015.
D. Liang and J. Paisley. Landmarking manifolds with Gaussian processes, International Conference on Machine Learning (ICML), 2015.
J. Paisley, C. Wang, D. Blei and M. Jordan. Nested hierarchical Dirichlet processes, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 37, no. 2, pp. 256270, 2015.
Y. Huang, J. Paisley, Q. Lin, X. Ding, X. Fu and X.P. Zhang. Bayesian nonparametric dictionary learning for compressed sensing MRI, IEEE Transactions on Image Processing, vol. 23, no. 12, pp. 50075019, 2014.



Stochastic processes and Bayesian nonparametric theory


I
am also interested in more theoretical ideas around stochastic
processes and Bayesian nonparametrics. I developed a stickbreaking
construction for the beta process, which gives a theoretically correct
way to generate a sizebiased sample of this infinite jump process, and
made the connection to Poisson process theory.
Theoretical connections that I've made between the stickbreaking construction of
the Dirichlet process and the hierarchical Dirichlet
process have allowed for easy stochastic variational inference of
Bayesian nonparametric topic models.
I am also fascinated by the Poisson process and enjoy teaching about them and their connection to Bayesian nonparametrics.
Representative publications, preprints, course notes
J. Paisley. Course notes for Advanced Probabilistic Machine Learning, Columbia University, 2014.
J. Paisley and M. Jordan. A constructive definition of the beta process, arXiv:1604.0068, 2016.
J. Paisley, D. Blei and M.I. Jordan. Stickbreaking beta processes
and the Poisson process, International
Conference on Artificial
Intelligence and Statistics (AISTATS), 2012.


Dots in spaces making measures...
sitting at tables...
and breaking sticks!



