This paper deals with Gibbs samplers that include high dimensional conditional Gaussian distributions. It proposes an efficient algorithm that avoids the high dimensional Gaussian sampling and relies on a random excursion along a small set of directions. The algorithm is proved to converge, i.e. the drawn samples are asymptotically distributed according to the target distribution. Our main motivation is in inverse problems related to general linear observation models and their solution in a hierarchical Bayesian framework implemented through sampling algorithms. It finds direct applications in semi-blind/unsupervised methods as well as in some non-Gaussian methods. The paper provides an illustration focused on the unsupervised estimation for super-resolution methods.

Multidimensional recurrent neural network (MDRNN) has shown a remarkable performance in speech and handwriting recognition. The performance of MDRNN is improved by further increasing its depth, and the difficulty of learning the deeper network is overcome by Hessian-free (HF) optimization. Considering that connectionist temporal classification (CTC) is utilized as an objective of learning MDRNN for sequence labelling, the non-convexity of CTC poses a problem to apply HF to the network. As a solution to this, a convex approximation of CTC is formulated and its relationship with the EM algorithm and the Fisher information matrix is discussed. MDRNN up to the depth of 15 layers is successfully trained using HF, resulting in improved performance for sequence labelling.