Summary: We show that Weyl-invariant dilaton gravity provides a description of black holes without classical space-time singularities. Singularities appear due to the ill behaviour of gauge fixing conditions, one example being the gauge in which theory is classically equivalent to standard General Relativity. The main conclusions of our analysis are as follows: (1) singularities signal a phase transition from broken to unbroken phase of Weyl symmetry; (2) instead of a singularity, there is a “baby universe” or a white hole inside a black hole; (3) in the baby universe scenario, there is a critical mass after which reducing mass makes the black hole larger as viewed by outside observers; (4) if a black hole could be connected with white hole through the “singularity,” this would require breakdown of (classical) geometric description; (5) the singularity of Schwarzschild BH solution is nongeneric and so it is dangerous to rely on it in deriving general results. Our results may have important consequences for resolving issues related to information loss puzzle. Though quantum effects are still crucial and may change the proposed classical picture, a position of building quantum theory around essentially regular classical solutions normally provides a much better starting point.