Published on January 9th, 2014 | by Emily Corbett0
Theory for Gaussian Variational Approximation of Bayesian Generalised Linear Models
By Gemma Moran, University of Sydney
This student took part in the 2012/13 AMSI Vacation Research Scholarship program. For more information on this years program please click here
As the `Big Data’ phenomenon continues to accelerate, there is ever increasing demand to find effective and efficient ways of dealing with large and complex datasets. Such datasets are often extremely difficult to analyse, presenting a major challenge for the statisticians of today. In response to this, a number of research areas have arisen, the most prominent of which is Markov chain Monte Carlo (MCMC). Whilst MCMC can be made arbitrarily accurate, its drawback lies in the amount of time required for computation, which can become untenable for large datasets.
Variational approximation methods are a newly emerging class of deterministic techniques, providing an alternative to the conventional MCMC in the fitting of and inference for complex statistical models. Variational approximations offer vast improvements in computational speed – depending on the dataset, they can be a hundred to a thousand times faster than MCMC – however, they are more limited in the accuracy of the approximation.
Variational approximations have become a key component of inference in Computer Science, having applications in such diverse areas as speech recognition, document retrieval and genetic linkage analysis. Despite this, variational approximations have yet to attain widespread attention in statistical settings.
As an AMSI Vacation Scholar, I worked on a project involving a particular variational method, Gaussian variational approximation, to facilitate inference for parameters in Bayesian generalised linear models. Such models are often used to analyse patient data and other categorical datasets. My project was concerned with developing theory for the Gaussian variational approximation estimators for the model, proving that they possess useful frequentist properties such as consistency and can be used to calculate asymptotically valid standard errors. In this regard, the impetus of my project was to contribute to the discourse surrounding variational approximation methods.
In conclusion, variational approximations have the potential to become an important tool in statistical inference, particularly in applications where timely results are essential and the use of MCMC becomes unfeasible.
I would like to thank my supervisor, Dr John Ormerod, for his support, without which the project would not have been possible. Additionally, I would like to thank AMSI for their generous support in facilitating this valuable research experience and CSIRO for providing the opportunity to both present my findings and learn about the exciting research being undertaken by other vacation scholars across Australia at the Big Day In conference.