Summary: This paper is a survey on semi-Markov decision processes (SMDPs). We present the background, the significance, and the research actuality of the infinite horizon expected discounted reward criterion, the long-run expected average reward criterion, the finite horizon expected reward criterion, the expected first passage reward criterion, the probability criterion, constrained problems, and mean-variance problems in SMDPs. At the same time, some issues to be studied in the future for these criteria or problems are pointed out. We also discuss potential research directions for SMDPs.