Summary: We revisit a particular ghost-free bimetric model which is related to both partial masslessness (PM) and conformal gravity. Linearly, the model propagates six instead of seven degrees of freedom not only around de Sitter but also around flat spacetime. Nonlinearly, the equations of motion can be recast in the form of expansions in powers of curvatures, and exhibit a remarkable amount of structure. In this form, the equations are shown to be invariant under scalar gauge transformations, at least up to six orders in derivatives, the lowest order term being a Weyl scaling of the metrics. The terms at two-derivative order reproduce the usual PM gauge transformations on de Sitter backgrounds. At the four-derivative order, a potential obstruction that could destroy the symmetry is shown to vanish. This in turn guarantees the gauge invariance to at least six-orders in derivatives. This is equivalent to adding up to ten-derivative corrections to conformal gravity. More generally, we outline a procedure for constructing the gauge transformations order by order as an expansion in derivatives and comment on the validity and limitations of the procedure. We also discuss recent arguments against the existence of a PM gauge symmetry in bimetric theory and show that, at least in their present form, they are evaded by the model considered here. Finally, we argue that a bimetric approach to PM theory is more promising than one based on the existence of a fundamental PM field.