Optimisation and Sampling Strategies for Bayesian Mixed-Effects Models: A comparative study

Abstract

In this thesis we investigate, and compare, the efficiency of both stochastic optimisation using stochastic automatic differentiation (SAD) and Bayesian sampling using Metropolis- Hastings algorithms for parameter estimation of the Ornstein-Uhlenbeck process. We present the necessary, non-trivial, theory and methods required to complete such investigation. The results obtained would indicate that in most cases we tested the Bayesian sampling is preferable due to several reasons. Firstly it is a less time consuming procedure in our tests, secondly the estimates are often closer to the true parameters and finally that the variance of the method is in many cases smaller than the variance of the SAD procedure. However in some cases, for instance in cases of multiple mixed effects, the SAD methods yields preferable results. However we are unable to conclusively recommend any method due to the settings of our tests not being broad enough. Moreover these results can later be used to guide eventual further investigations into these methods, some avenues of which are suggested.