Summary: Modeling the mean and covariance simultaneously is a common strategy to efficiently estimate the mean parameters when applying generalized estimating equation techniques to longitudinal data. In this article, using generalized estimation equation techniques, we propose a new kind of regression models for parameterizing covariance structures. Using a novel Cholesky factor, the entries in this decomposition have moving average and log innovation interpretation and are modeled as linear functions of covariates. The resulting estimators for the regression coefficients in both the mean and the covariance are shown to be consistent and asymptotically normally distributed. Simulation studies and a real data analysis show that the proposed approach yields highly efficient estimators for the parameters in the mean, and provides parsimonious estimation for the covariance structure.