Summary: Quantum Hamiltonians that are fine-tuned to their so-called Rokhsar-Kivelson (RK) points, first presented in the context of quantum dimer models, are defined by their representations in preferred bases in which their ground state wave functions are intimately related to the partition functions of combinatorial problems of classical statistical physics. We show that all the known examples of quantum Hamiltonians, when fine-tuned to their RK points, belong to a larger class of real, symmetric, and irreducible matrices that admit what we dub a Stochastic Matrix Form (SMF) decomposition. Matrices that are SMF decomposable are shown to be in one-to-one correspondence with stochastic classical systems described by a Master equation of the matrix type, hence their name. It then follows that the equilibrium partition function of the stochastic classical system partly controls the zero-temperature quantum phase diagram, while the relaxation rates of the stochastic classical system coincide with the excitation spectrum of the quantum problem. Given a generic quantum Hamiltonian construed as an abstract operator defined on some Hilbert space, we prove that there exists a continuous manifold of bases in which the representation of the quantum Hamiltonian is SMF decomposable, i.e., there is a (continuous) manifold of distinct stochastic classical systems related to the same quantum problem. Finally, we illustrate with three examples of Hamiltonians fine-tuned to their RK points, the triangular quantum dimer model, the quantum eight-vertex model, and the quantum three-coloring model on the honeycomb lattice, how they can be understood within our framework, and how this allows for immediate generalizations, e.g., by adding non-trivial interactions to these models.