Summary: We argue that the obstacles to having a first-order formalism for odd-derivative actions presented in a pedagogical note by S. Deser [Classical Quantum Gravity 23, No. 18, 5773–5776 (2006; Zbl 1101.83040)] are based on examples which are not first-order forms of the original actions. The general derivation of an equivalent first-order form of the original second-order action is illustrated using the example of topologically massive electrodynamics (TME). The correct first-order formulations of the TME model keep intact the gauge invariance presented in its second-order form demonstrating that the gauge invariance present in the original action is not lost when one correctly employs the Ostrogradsky procedure.