Summary: Generalizing a previous work concerning cosmological linear tensor perturbations, we show that the Lagrangians and Hamiltonians of cosmological linear scalar and vector perturbations can be put in simple form through the implementation of canonical transformations and redefinitions of the lapse function, without ever using the background classical equations of motion. A similar result was obtained by Langlois in the case of a scalar field, but we generalize it for any perfect fluid. In such case, i.e., when the matter content of the Universe is a perfect fluid, we can go further and show that the Hamiltonian of scalar perturbations can be reduced, as usual, to a Hamiltonian of a scalar field with variable mass depending on background functions, independently of the fact that these functions satisfy the background Einstein classical equations. These simple Lagrangians and Hamiltonians can then be used in situations where the background metric is also quantized, hence providing a substantial simplification over the direct approach originally developed by Halliwell and Hawking.