Bayesian estimation of reduced rank regression models requires careful consideration of the well known identification problem. We demonstrate that this identification problem can be handled efficiently by using prior distributions that restrict a part of the parameter space to the Stiefel manifold and post-processing the obtained Gibbs sampler output according to an appropriately specified loss function. This extends the possibilities for Bayesian inference in reduced rank regression models. Besides inference, we also discuss model selection in terms of posterior predictive assessment. We choose this approach because computing the marginal data likelihood under the identifying restrictions implies prohibitive computational burden. We illustrate the proposed approach with a simulation study and an empirical application.