Rachit Singh
https://rachitsingh.com/
Recent content on Rachit SinghHugo -- gohugo.ioen-usSun, 22 Apr 2018 18:57:21 -0400The Langevin Equation
https://rachitsingh.com/the-langevin-equation/
Sun, 22 Apr 2018 18:57:21 -0400https://rachitsingh.com/the-langevin-equation/This post covers the Langevin equation, a stochastic differential equation that models the dynamics of particles in Brownian motion 1. This covers the ideas used in this reference due to Lennart Sjögren.
Langevin Equation In 1907 Einstein published a paper that derived a macroscopic quantity \(D\), the diffusion constant, with microscopic quantities:
\[D = \frac{k_BT}{6\pi\eta a}\]
where \(\eta\) is the viscosity of the liquid and \(a\) is the radius of the particle.Persistence Length
https://rachitsingh.com/persistence-length/
Sun, 22 Apr 2018 10:52:54 -0400https://rachitsingh.com/persistence-length/td { padding: 5px; font-family: monospace; font-size: 1.25rem; } th { text-align: center; padding: 0px 5px; } th.left_column { text-align: right; } figure { margin: 0px 20px; max-width: 50rem; } img[src*="#smaller"] { width: 65%; margin: auto; margin-bottom: 15px; } In class we recently discussed the simplified elastic rod model for polymers, which assumes that polymers can be modeled as an inextensible rod, i.e. that the length of the rod doesn't change, and that the twist of the polymer is ignorable (possibly because the polymer is joined by single bonds).A few favorite papers of 2017
https://rachitsingh.com/a-few-favorite-papers-of-2017/
Tue, 09 Jan 2018 12:48:10 -0800https://rachitsingh.com/a-few-favorite-papers-of-2017/This isn't an exhaustive list, and I will inevitably forget some papers. I'll keep updating as a remember, and will probably expand some of the background/contribution sections as I have time, so that they're more accessible.
Breaking the Softmax Bottleneck: A High-Rank RNN Language Model [link] Background: Language models and NLP tasks almost always use a softmax to compute a distribution over the vocabulary, and usually this is computed as \(\sigma(\mathbf{W}\mathbf{h})\), where \(\mathbf{h}\) is a \(d\)-dimensional context vector from a previous layer, and \(\mathbf{W} \in \mathbb{R}^{M \times d}\) is a word embedding, letting $M$ be the vocabulary size.PyTorch Internals, cuRAND, and numerical instability
https://rachitsingh.com/pytorch-internals-curand-and-numerical-instability/
Wed, 03 Jan 2018 18:44:27 -0800https://rachitsingh.com/pytorch-internals-curand-and-numerical-instability/Random sampling I've been working lately to implement random samplers from a number of distributions in PyTorch, both on CPU and CUDA. This is a topic near and dear to my heart, since it has caused me a lot of trouble multiple times. Once this PR is merged, I'll post an explanation/notebook of why this is important.
Here's a brief summary of the motivation:
We want to sample from distributions like \(\operatorname{Beta}(a, b)\).ELBO Surgery
https://rachitsingh.com/elbo_surgery/
Sat, 23 Dec 2017 12:08:00 -0800https://rachitsingh.com/elbo_surgery/td { padding: 5px; font-family: monospace; font-size: 1.25rem; } th { text-align: center; padding: 0px 5px; } th.left_column { text-align: right; } figure { margin: 0px 20px; max-width: 50rem; } img[src*="#smaller"] { width: 50%; margin: auto; } tldr: The ubiquitous isotropic Gaussian prior for generative models doesn't make sense / doesn't work, which motivates work on priors.
At NIPS, Dawen Liang mentioned Hoffman & Johnson's ELBO surgery paper offhand while talking about tuning KL divergences, and it's very interesting, so I thought I'd go over it.Links
https://rachitsingh.com/links/
Thu, 14 Dec 2017 16:55:26 -0500https://rachitsingh.com/links/Here are some useful links I’ve found:
LaTeX A tikz-cd graphical editor that I wish I’d during 55… - http://tikzcd.yichuanshen.de/ For high power Bayesian diagrams, I like tikz-bayesnet, but honestly it’s often not worth the trouble vs. using tikz-cd and adding a circle macro. ShareLaTeX is quite useful for collaborating, and open source. One day when I have time I’ll make a PR… Vim My dotfiles are here: http://github.NIPS 2017
https://rachitsingh.com/nips/
Sat, 09 Dec 2017 00:47:37 -0800https://rachitsingh.com/nips/I'm starting this blog to share research ideas that I have, and some solutions to problems I find along the way. I've been helped immensely by other people's blogs in the past, and want to do the same. Also it'll give me a chance to communicate the way I approach problems, and hopefully people will give me alternative perspectives either by email (rachitsingh@college.harvard.edu) or in the comments, once I figure out how that works.Ideas
https://rachitsingh.com/ideas/
Fri, 08 Dec 2017 15:41:37 -0800https://rachitsingh.com/ideas/Ideas for libraries (so that people can use this stuff quickly)
Implement a discrete Poisson distribution using RELAX/REBAR and/or the TRE. Reimplement the CUDA code for unbiased MCMC so that it’s PyTorch based, if that’s possible. That way we can let people define arbitrary likelihoods and still get coupled kernels. Maybe use CuPy as a glue language, though if the PyTorch story for external modules gets better then this shouldn’t be an issue.About
https://rachitsingh.com/about/
Thu, 07 Dec 2017 18:14:16 -0800https://rachitsingh.com/about/I’m Rachit Singh, and I’m a senior at Harvard, doing research in variational inference, Indian Buffet processes, and language models. I work with Alexander Rush, Finale Doshi-Velez, and I’m part of the Harvard NLP research group. I frequently work with Jeffrey Ling.
I’m very interested in probabilistic programs, especially those like Edward, and I’m hoping to help build Pyro to be just as powerful, once I understand how it works. I like to write very fast code (I have a bit of CUDA + RTOS microcontroller work under my belt) in a variety of languages.