Summary: We continue to explore the notion of Rota-Baxter algebras in the context of the Hopf algebraic approach to renormalization theory in perturbative quantum field theory. We show in very simple algebraic terms that the solutions of the recursively defined formulae for the Birkhoff factorization of regularized Hopf algebra characters, i.e., Feynman rules, naturally give a non-commutative generalization of the well-known Spitzer’s identity. The underlying abstract algebraic structure is analysed in terms of complete filtered Rota-Baxter algebras.