Summary: Models of the term structure of interest rates are considered for which, under the martingale measure, instantaneous forward rates are Gaussian. The possible forms of the covariance structure are characterized under appropriate formulations of the Markov property. It is demonstrated that imposing Markovian assumptions limits severely the covariances that may be obtained and that the strongest such formulation together with stationarity implies that the whole forward rate surface is necessarily a Gaussian random field described by just three parameters.