Summary: This work discusses a two-step procedure, based on formal abstractions, to generate a finite-space stochastic dynamical model as an aggregation of the continuous temperature dynamics of a homogeneous population of Thermostatically Controlled Loads (TCL). The temperature of a single TCL is described by a stochastic difference equation and the TCL status (ON, OFF) by a deterministic switching mechanism. The procedure is formal as it allows the exact quantification of the error introduced by the abstraction – as such it builds and improves on a known, earlier approximation technique in the literature. Further, the contribution discusses the extension to the case of a heterogeneous population of TCL by means of two approaches resulting in the notion of approximate abstractions. It moreover investigates the problem of global (population-level) regulation and load balancing for the case of TCL that are dependent on a control input. The procedure is tested on a case study and benchmarked against the mentioned alternative approach in the literature.