The time-honored Allen-Feldman theory of heat transport in glasses is generally assumed to predict a finite value for the thermal conductivity, even if it neglects the anharmonic broadening of vibrational normal modes. We demonstrate that the harmonic approximation predicts that the bulk lattice thermal conductivity of harmonic solids inevitably diverges at any temperature, irrespective of configurational disorder, and that its ability to represent the heat-transport properties observed experimentally in most glasses is implicitly due to finite-size effects. Our theoretical analysis is thoroughly benchmarked against careful numerical simulations. Our findings thus reveal that a proper account of anharmonic effects is indispensable to predict a finite value for the bulk thermal conductivity in any solid material, be it crystalline or glassy.