scott.linderman at columbia.edu
Recurrent Switching Linear Dynamical Systems
Many natural systems, such as neurons firing in the brain or basketball
teams traversing a court, give rise to time series data with complex,
nonlinear dynamics. We can gain insight into these systems by decomposing
the data into segments that are each explained by simpler dynamic units.
Building on switching linear dynamical systems (SLDS), we present a new
model class that not only discovers these dynamical units, but also
explains how their switching behavior depends on observations or continuous
latent states. These "recurrent" switching linear dynamical systems provide
further insight by discovering the conditions under which each unit is
deployed, something that traditional SLDS models fail to do. We leverage
recent algorithmic advances in approximate inference to make Bayesian
inference in these models easy, fast, and scalable.
Joint work with Matt Johnson, Andy Miller, Ryan Adams, David Blei, and Liam Paninski
Discovering latent network structure in point process data
We develop a probabilistic model combining mutually-exciting point processes with random graph models, and derive a fully-Bayesian, parallel inference algorithm based on an auxiliary variable formulation.
Joint work with Ryan Adams.
ICML '14 | arXiv:1402.0914 [stat.ML] | ICML Talk
Building on this work, we have proposed a scalable stochastic variational inference (SVI) algorithm for discovering latent networks from excitatory point process observations.
Using a discrete-time formulation and a weak spike-and-slab approximation, we have developed highly scalable algorithms.
I've also worked on developing similar ideas for modeling neural spike
trains, which have nonlinear dynamics and a combination of excitatory
and inhibitory interactions. Leveraging and extending recent work on
Pólya-gamma augmentation schemes, I have developed efficient Bayesian
inference algorithms for discovering structured patterns of
connectivity underlying spike trains.
Joint work with Ryan Adams and Jonathan Pillow.
Cosyne 2015 Abstract
Nonparametric state space models for neural spike trains
My collaborators and I have been developing scalable, nonparametric
latent state space models like hidden Markov and semi-Markov models
(HMMs and HSMMs), linear dynamical systems (LDSs), and switching
linear dynamical systems (SLDSs) to probe the low-dimensional discrete and continuous
states of neural systems. These models push the frontier of
computational modeling, allowing us to express sophisticated
hypotheses about neural dynamics in the form of hierarchical,
As the sophistication of our models and the size of our datasets grow,
inference becomes a major challenge. We have developed novel augmentation schemes
and fast Bayesian inference algorithms to fit these models at scale.
Joint work with Matt Johnson, Aaron Tucker, Matt Wilson, Zhe Chen, Bob Datta, and Ryan Adams
JNM paper | Cosyne 2016 Abstract | Cosyne 2015 Abstract
Dependent multinomial models made easy: Stick breaking with the Pólya-gamma augmentation
Many practical modeling problems involve discrete multinomial data
with underlying dependencies that cannot be captured by the
Dirichlet-multinomial formulation. We leverage a logistic
stick-breaking representation and recent innovations in Pólya-gamma
augmentation to perform efficient fully Bayesian inference for models
with latent Gaussian structure and multinomial observations like
correlated topic models, multinomial GP's, and multinomial linear dynamical systems.
Joint work with Matthew Johnson and Ryan Adams.
NIPS '15 | arXiv:1506.05843 [stat.ML]
A framework for studying synaptic plasticity with neural spike train data
Learning and memory in the brain are implemented by complex, time-varying changes in neural circuitry. We present a framework for incorporating synaptic plasticity rules into the popular generalized linear models for neural spike trains, and derive Bayesian inference algorithms based on particle MCMC.
Joint work with Chris Stock and Ryan Adams.
NIPS '14 | arXiv:1411.4077 [stat.ML]
Conference and Journal Papers
- Scott W. Linderman*, Matthew J. Johnson*, Andrew C. Miller, Ryan P. Adams, David M. Blei, and Liam Paninski. Bayesian learning and inference in recurrent switching linear dynamical systems. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 2017
- Christian A. Naesseth, Francisco J. R. Ruiz, Scott W. Linderman, and David M. Blei. Reparameterization gradients through acceptance-rejection sampling algorithms. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 2017.
- Scott W. Linderman and Samuel J Gershman. Using computational theory to constrain statistical models of neural data. bioRxiv, 2017.
- Scott W. Linderman, Ryan P. Adams, and Jonathan W. Pillow. Bayesian latent structure discovery from multi-neuron recordings. In Advances in Neural Information Processing Systems (NIPS), 2016.
- Huseyin Melih Elibol, Vincent Nguyen, Scott W. Linderman, Matthew J. Johnson, Amna Hashmi, and Finale Doshi-Velez. Cross-corpora unsupervised learning of trajectories in autism spectrum disorders. Journal of Machine Learning Research, 17(133):1–38, 2016
- Linderman, Scott W., Johnson, Matthew J., Wilson, Matthew W., and Chen, Zhe. A Bayesian nonparametric approach for uncovering rat hippocampal population codes during spatial navigation Journal of Neuroscience Methods, 2016.
- Linderman, Scott W.*, Johnson, Matthew J.*, and Adams, Ryan P.
Dependent Multinomial Models Made Easy: Stick-Breaking with the Pólya-gamma Augmentation.
Neural Information Processing Systems, 2015.
NIPS paper | arXiv:1506.05843 [stat.ML]
Linderman, Scott W., Stock, Christopher H., and Adams, Ryan P. A Framework for Studying Synaptic Plasticity with Neural Spike Train Data. Neural Information Processing Systems (NIPS), 2014.
Linderman, Scott W. and Adams, Ryan P. Discovering Latent Network Structure in Point Process Data. International Conference on Machine Learning (ICML), 2014.
ICML paper | arXiv:1402.0914 [stat.ML] | ICML Talk.
* authors contributed equally.
NIPS paper | arXiv:1411.4077 [stat.ML]
Workshop Papers, Abstracts, and Posters
Linderman, Scott W., Tucker, Aaron, and Johnson, Matthew J. Bayesian latent state space models of neural activity. Computational and Systems Neuroscience (Cosyne) Abstracts, 2016, Salt Lake City, UT USA.
Linderman, Scott W., Adams, Ryan P., and Pillow, Jonathan W. Inferring structured connectivity from spike trains under negative-binomial generalized linear models. Computational and Systems Neuroscience (Cosyne) Abstracts, 2015, Salt Lake City, UT USA.
Johnson, Matthew J., Linderman, Scott W., Datta, Sandeep R., and Adams, Ryan P. Discovering switching autoregressive dynamics in neural spike train recordings. Computational and Systems Neuroscience (Cosyne) Abstracts, 2015, Salt Lake City, UT USA.
Linderman, Scott W., Stock, Christopher H., and Adams, Ryan P. Fully-Bayesian inference of time-varying synaptic weights from neural spike trains. Annual Meeting of the Society for Neuroscience (SfN), 2014.
Linderman, Scott W. Discovering Latent States of the Hippocampus with Bayesian Hidden Markov Models. Abstracts of the 2014 Brains, Minds, and Machines Summer School, 2014.
Nemati, Shamim, Linderman, Scott W., and Chen, Zhe (2014). A Probabilistic Modeling Approach for Uncovering Neural Population Rotational Dynamics. Cosyne Abstracts 2014, Salt Lake City, UT, USA.
- Linderman, Scott W., Adams, Ryan P. (2013). Fully-Bayesian Inference of Structured Functional Networks in GLMs. Acquiring and Analyzing the Activity of Large Neural Ensembles, Workshop at Neural Information Processing Systems (NIPS), Lake Tahoe, NV, USA. poster
Linderman, Scott W., Adams, Ryan P. (2013). Discovering Structure in Spiking Data. New England Machine Learning Day 2013, Cambridge, MA USA.
Linderman, Scott W., Adams, Ryan P. (2013). Inferring functional connectivity with priors on network topology. Cosyne Abstracts 2013, Salt Lake City USA.
- Discovering Latent Network Structure in Neural Spike Trains.
Machine Learning and Friends, University of Massachusetts at Amherst. February 12th, 2015.
- Discovering Interpretable Structure in Neural Spike Trains with Negative Binomial GLMs. Applied Math Seminar, University of Washington. January 8th, 2015.
- Discovering Interpretable Structure in Neural Spike Trains with Negative Binomial GLMs. Harvard Center for Brain Science Neurolunch. December 3rd, 2014.
- Discovering Latent Network Structure in Point Process Data. Lazer Lab Meeting, Northeastern University. September 4th, 2014.
- Discovering Latent Network Structure in Point Process Data. Harvard Computer Science Colloquium. July 24th, 2014.
- Discovering Latent Network Structure in Spiking Data. Boston Data Mining Meetup. May 1st, 2014.
- Discovering Latent Network Struture in Spiking Data Applied Statistics Workshop, Harvard University, September 4th, 2013
Neuroscience is entering an exciting new age. Modern recording technologies enable simultaneous measurements of thousands of neurons in organisms performing complex behaviors. Such recordings offer an unprecedented opportunity to glean insight into the mechanistic underpinnings of intelligence, but they also present an extraordinary statistical and computational challenge: how do we make sense of these large scale recordings? This thesis develops a suite of tools that instantiate hypotheses about neural computation in the form of probabilistic models and a corresponding set of Bayesian inference algorithms that efficiently fit these models to neural spike trains. From the posterior distribution of model parameters and variables, we seek to advance our understanding of how the brain works.
Chapter 1: Introduction
Chapter 2: Background
Chapter 3: Hawkes Processes with Latent Network Structure
Chapter 4: Discrete-Time Linear Autoregressive Poisson Models
Chapter 5: Networks with Nonlinear Autoregressive Dynamics
Chapter 6: Dynamic Network Models
Chapter 7: Bayesian Nonparametric Hidden Markov Models
Chapter 8: Switching Linear Dynamical Systems with Count Observations
Chapter 9: Reverse Engineering Bayesian Computations from Spike Trains
Chapter 10: Conclusion
Or, if you prefer the full document, here is the complete thesis. The source is available at https://github.com/slinderman/thesis.