Summary: The clustering problem has many applications in machine learning, operations research and statistics. We propose three algorithms to create starting solutions for improvement algorithms for the minimum sum of squares clustering problem. We test the algorithms on 72 instances that were investigated in the literature. We found five new best known solutions and matched the best known solution for 66 of the remaining 67 instances. Thus, we are able to demonstrate that good starting solutions combined with a simple local search get results comparable with, and sometimes even better than, more sophisticated algorithms used in the literature.