Herein, a LeNet-based method for wave height inversion and accurate classification of the radar-measured sea clutter data under varying sea state levels is proposed. Traditionally, empirical formula or spectral integration is employed for deducing parameters from the wave spectrum. Differently, the proposed method adopts the time-distance two-dimensional sea clutter data. It combines the significant wave height information recorded synchronously during data collection to construct the training and test data sets. The neural network is trained by random gradient descent (SGD), the efficiency of which is improved by modifying/adjusting the size of convolution kernel, learning rate, regularization rate and other parameters. The neural network model able to distinguish sea clutter data with different sea state level or different effective wave heights is then formulated. Afterward, through the training by different measured data, the neural network models adapted to different radars and sea areas are obtained. Finally, the training is performed using the measured X-band sea clutter data sets IPIX (including sea state level 3 and 4) and CSIR (including level 4 and 5). The results show that the neural network model can well classify the sea clutter data into appropriate sea state level. Under the optimal parameters obtained by experiments, the average accuracy of the two data sets reaches over 93%. In addition, by constructing data set of different preset wave height interval lengths for training, the test results show that the high accuracy can be guaranteed when the scale of wave height inversion for CSIR data set is about 0.5.