4.1. Comparison with ERA5 Data
In this section, the performance of the GloWS-Net model architecture is compared with the traditional MVE method and some existing network architectures (such as FCN and CNN models). The wind speed retrieval performance test is conducted on the test data set, which is not used during training and is separated from the training set and verification set. Therefore, it is very suitable for evaluating the generalization ability of corresponding network models on blind data sets. However, this training method also has a disadvantage that any systematic errors will be invisible to the algorithm. In the following, we first compare and analyze the wind speed retrieval performance of different retrieval models, then evaluate the performance of different retrieval models in retrieving global wind speed, and finally discuss the results.
As shown in
Figure 1 above, wind speed retrieval can be divided into two stages. The first stage involves the GMF method, CNN1, CNN2, FCN1, FCN2, FCN3 and FCN4 models. The second stage includes wind speed retrieval by CNN3 model and GloWS-Net model. Among them, the CNN3 model fuses the DDM image and auxiliary parameters excluding SWH, wave direction and rainfall information. The GloWS-Net model includes SWH of swell, wave direction and rainfall information as input parameters.
Table 5 shows the retrieval accuracy statistics of different retrieval models for wind speed less than 10 m/s, 10–15 m/s, 15 m/s and 0–30 m/s. In the table, RMSE, mean absolute error (MAE), correlation coefficient (CC) and MAPE represent root mean square error, mean absolute error, correlation coefficient and mean absolute percentage error, respectively. The following conclusions can be drawn from the table:
(1) When the wind speed is within the range of 0-30 m/s, the RMSE of the FCN4 and the proposed GloWS-Net model architectures are the lowest (1.92 m/s). However, it can be observed from the table that the proposed GloWS-Net model architecture is superior to the FCN4 model in terms of MAE and MAPE, especially for MAPE. Generally, MAE and MAPE are less affected by extreme values. However, RMSE uses the square of error and it is more sensitive to outliers. The FCN3 and CNN3 model architectures with the same number of auxiliary parameter inputs also show a similar situation, which indicates that after adding DDM images, the GloWS-Net model architecture can obtain the same RMSE as the FCN model, but MAPE has been significantly improved by 16.56%. In addition, in terms of the four indicators (MRSE, MAE, CC and MAPE), the performance of the proposed GloWS-Net model architecture is significantly better than the model combing the results from NBRCS and LES based on the MVE method, with an increase of 23.98%, 27.95%, 11.02% and 32.52%, respectively. Compared with the NBRCS method, it increased by 20.67%, 22.78%, 8.95% and 30.92%, respectively. Compared with the LES method, it has increased by 29.16%, 27.85%, 29.19% and 36.21%, respectively.
(2) In the case of strong wind (>15 m/s), the RMSE of all architectures and MVE methods increases. However, adding auxiliary parameters, as done in FCN2, FCN3, FCN4, CNN3, and GloWS-Net model architectures, will reduce RMSE. Compared with other architectures, CNN1 and CNN2 that only handle BRCS DDM or power DDM have poor performance. This shows that even when the wind speed is lower than 9 m/s, it is necessary to add auxiliary parameters to obtain satisfactory results. It is encouraging to note that the GloWS-Net model architecture has the best performance in retrieving wind speed under strong wind conditions and has obtained the minimum RMSE.
(3) In the case of moderate wind speed (10–15 m/s), the retrieval performance of FCN4 and GloWS-Net model architecture is better than that of the GMF method. However, the performance of GloWS-Net is slightly inferior to that of FCN4. Under low wind speed (<10 m/s), the performance of GloWS-Net is slightly better than that of FCN4. In this case, although the RMSE of the MVE method is small, it is still worse than FCN (such as FCN2, FCN3 and FCN4), CNN (CNN1, CNN2 and CNN3) and GloWS-Net. In addition, it is worth noting that the accuracy of the FCN1 model with only NBRCS and LES observables as auxiliary parameters is much lower than that of the traditional MVE method, which indicates that more auxiliary parameters need to be added to the FCN1 model for wind speed retrieval to obtain satisfactory results (such as FCN2, FCN3 and FCN4). Another important aspect we can notice is that the CNN1 model with BRCS DDM as the input only achieves the same performance as the FCN4 and GloWS-Net model architectures under low wind speed. This shows that the CNN1 wind model for low wind speed can benefit from the accretion layer.
In sum, in terms of overall RMSE, except for FCN1, CNN1 and CNN2, other model architectures have better wind speed retrieval performance than the NBECS, LES and MVE methods. For the accuracy obtained from the wind speed range of 0–30 m/s, although the RMSEs of GloWS-Net and FCN4 are equivalent, the performance of the proposed GloWS-Net is better than that of FCN4 in terms of MAE and MAPE, especially for MAPE. In addition, FCN4 performs worse than GloWS-Net under low and high wind speeds. It should be emphasized that only a verification using in situ measurements provided by weather stations or a campaign with boats on the investigated area will fully validate the proposed approach.
In order to evaluate the performance of different models,
Figure 5 shows the wind speed scatter density plots of different models and ERA5. The color bar in the figure represents the data density, the red dotted line represents the 1:1 reference line, the magenta solid line represents the linear fitting result between the retrieved and ERA5 wind speeds, and the CC represents Pearson correlations between retrieved wind speed of the model and ERA5 wind speed are also given. The following observations can be seen from the figure:
(1) Although the FCN4 model analyzed above and the GloWS-Net model produce the same RMSE, the proposed GloWS-Net model and CNN3 model show better performance in terms of correlation between retrieved wind speed and ERA5 data. Obviously, there are more data points distributed symmetrically along the y = x line, and fewer data points scattered around the line.
(2) The MVE method obviously overestimated the wind speed of 5 to 12 m/s, which may be caused by insufficient power calibration of CYGNSS data and high noise [
48,
54]. Within this range, a large number of samples and some measured power values may be underestimated. This problem has been improved by FCN, CNN and GloWS-Net. In particular, the results retrieved by CNN3 and GloWS-Net both are more concentrated along the 1:1 reference line when wind speed is in the range of 5~12 m/s. It is also obvious from the figure that the MVE method, FCN1, FCN2, FCN3, FCN4, CNN1 and CNN2 model architectures have poor response capability to wind speed in the range of 0–5 m/s. However, CNN3 and GloWS-Net model architectures do not have this problem. This shows that after adding DDM images to the input layer of the CNN3 and GloWS-Net wind models, the architecture incorporating the convolution layer has better performance than those with only the full connection layer because they use the patterns in DDM. In addition, except for CNN3 and GloWS-Net, other models show overestimation at very low wind speeds (<5 m/s) and underestimation at high wind speeds (15–30 m/s). The correlation between the wind speed retrieved by CNN1 and CNN2 models and the ERA5 data is the worst, which clearly indicates that more auxiliary parameters need to be included in CNN1 and CNN2 to obtain better results.
(3) It is worth mentioning that, compared with the two most advanced deep learning model architectures (i.e., MCNN and CyGNSSnet) currently used for spaceborne GNSS-R wind speed retrieval [
40,
41], the GloWS-Net model architecture proposed in this paper performs very well in the case of high wind speed, that is, the GloWS-Net model significantly mitigates the underestimation phenomenon at high wind speed. Generally, for marine disasters caused by marine events with large wind speeds such as hurricanes and typhoons, high-precision wind speed estimation results at high wind speeds are very helpful for monitoring those disasters. Although limited to the current level of spaceborne GNSS-R technology, wind speed prediction under strong winds is still facing great challenges [
64,
65]. However, the excellent performance of the GloWS-Net model at high wind speeds is promising for future marine disaster monitoring.
Figure 6 shows the RMSE and MAE of different models for different wind speed ranges. It can be seen that CNN3 and GloWS-Net outperform other architectures especially for challenging high wind speeds (>20 m/s). In addition, when the wind speed is greater than 8 m/s, it can be seen that including SWH, wave direction and rainfall information enhances the model performance. However, when the wind speed is greater than 20 m/s, the CNN3 model shows equivalent performance as the GloWS-Net model architecture. The reason may be that strong wind speed is accompanied by heavy rainfall or strong wind speed causes dramatic changes in sea conditions. Even if rainfall, swell and wave direction information is introduced into the GloWS-Net model framework, it is difficult to correct these influencing factors completely. Therefore, future research needs to further optimize the GloWS-Net model architecture.
In order to compare the global performance of wind speed retrieved by GloWS-Net and the traditional model, we selected the period from July to August 2021 in the test data set for analysis.
Figure 7 shows the ERA5 wind speed and the results retrieved using the MVE method, FCN4 and GloWS-Net.
Figure 8 shows the deviation distribution histogram between ERA5 wind speed and the results obtained by the MVE method, FCN4 as well as GloWS-Net. In the figure, the average deviation (μ), standard deviation (σ), mean absolute error (MAE) and 80% quantile (Qua) of the deviation are also given. The blue bar chart depicts the error distribution, the red dotted line represents the fitting curve of the probability density function of the error and the green dotted line marks the wind speed deviation of 0 m/s. It can be observed from
Figure 7 that the performance of FCN4 and GloWS-Net is better than the MVE method in retrieving global wind speed, and the results from the MVE method are significantly different from the ERA5 data in multiple sea areas around the world (as shown by the magenta rectangle mark in the figure). Comparing the wind speed retrieval results of FCN4 and GloWS-Net with the ERA5 data, it is found that the performance of the GloWS-Net model is better. For example, in the sea area within the longitude of 60°E–120°E and the latitude of 10°S–40°S, the FCN4 model shows an underestimated wind speed, while the retrieved wind speed of the GloWS-Net model is highly consistent with the ERA5 data. From
Figure 8, it can also be seen that the performance of the GloWS-Net model is better than that of the FCN4 model. The deviation between the GloWS-Net model wind speed results and the ERA5 data is very concentrated (80% of the wind speed deviation is less than 2.28 m/s) and near the deviation line of 0 m/s, while the wind speed deviation of FCN4 is on the left side of the green line in the histogram. The negative deviation is more obvious, and 80% of the wind speed deviation is less than 2.34 m/s. From the wind speed deviation histograms of the three models, the global wind speed retrieved by the MVE method is the worst, and 80% of the wind speed deviation is 3.20 m/s. The above analysis further confirms that the GloWS-Net model architecture has strong advantages in retrieving global sea surface wind speed.
4.2. Comparison with CCMP Data
It is necessary to evaluate the wind speed retrieval performance of the proposed deep model through independent wind speed measurement. The CCMP wind speed is closer to conventional in situ measurements from ships than the ERA5 product. In [
66], CCMP winds and Tropical Atmosphere Ocean (TAO) mooring observations were compared; a good agreement with a root mean square error (RMSE) of 1 m/s and a correlation coefficient of 0.95 were obtained. The CCMP wind product is a newly released global ocean wind data set and suitable for scientific study at various temporal and spatial resolutions, which is widely used to verify the retrieval of wind speed by spaceborne GNSS-R [
55,
67,
68]. Therefore, the wind speed of CCMP was also collected and compared with the retrieved wind speed.
Figure 9 shows the correlation between the ERA5 wind speed data and CCMP wind speed data used in the test model in this study, as well as the probability density function (PDF) distribution curve of ERA5 and CCMP wind speeds. These data cover a wind speed range of 0 to 30 m/s, but only a few samples have wind speeds of 20–30 m/s, which may lead to low accuracy at high wind speeds. We can see that the PDF of the two data sets is almost the same, except that ERA5 wind speed is slightly higher than CCMP at 3–6 m/s and vice versa at 14–19 m/s. This may be due to the differences in the platforms, hardware and algorithms used to generate the two data sets. In addition, the RMSE and CC between the two data sets are 1.46 m and 0.95, respectively, which are highly correlated.
Table 6 shows the accuracy statistics by comparing the retrieved wind speed data of different models with the CCMP data. The following conclusions can be drawn from the table:
(1) When the wind speed is within the range of 0–30 m/s, the RMSE of the FCN4 model architecture is the lowest (2.14 m/s), followed by the proposed GloWS-Net model architecture (RMSE = 2.16 m/s) and FCN3 model architecture. The results of comparison between retrieved wind speed and CCMP are similar to those of ERA5, the difference of RMSE between the GloWS-Net model architecture and FCN model is very small, but MAPE has been significantly improved by 17.75%. Moreover, in terms of the four indicators (MRSE, MAE, CC and MAPE), the performance of the proposed GloWS-Net model architecture is much better than the model combining the results from NBRCS and LES based on the MVE method, with an increase of 20.27%, 22.21%, 11.20% and 29.02%, respectively. Compared with the NBRCS method, it is increased by 17.20%, 20.11%, 8.89% and 3.86%, respectively. Compared with the LES method, it is increased by 24.71%, 23.45%, 29.70% and 29.90%, respectively.
(2) In the case of strong wind (>15 m/s), except for the GloWS-Net model architecture, the RMSE of wind speed retrieved from other models is large. The RMSE of the GloWS-Net model is 2.8 m/s, and the retrieval accuracy is 37.42% and 11.58% higher than that of the MVE and FCN4 models, respectively.
(3) Under medium (10–15 m/s) and low (<10 m/s) wind speed, the results of the comparison between retrieved wind speed and CCMP are similar to those of ERA5. Among them, the GloWS-Net and FCN4 model retrieved wind speed has good correlation with the CCMP data.
Figure 10 shows the wind speed scatter density plots of different models and CCMP. It can be seen that the comparison results between the wind speed retrieved by different models and CCMP are consistent with those with the ERA5 data. This further shows that the GloWS-Net model proposed in this paper is reliable and has high generalization ability, which means it is good for practical application.
It is also necessary to evaluate the retrieval performance of different retrieval models in different wind speed ranges when the CCMP wind speed data are used as reference.
Figure 11 shows the RMSE and MAE for different wind speed ranges. It can be seen that CNN3 and GloWS-Net outperform other architectures especially for challenging high wind speeds (> 20 m/s). Furthermore, when the wind speed is greater than 8 m/s, it can be seen that including SWH, wave direction and rainfall information also improves the GloWS-Net model performance. When the wind speed is greater than 20 m/s, the GloWS-Net model shows the best retrieval performance.
Figure 12 shows the global deviation distribution histogram between CCMP wind speed and the results obtained by the MVE method and FCN4 as well as GloWS-Net. From
Figure 12, it can also be seen that the performance of the GloWS-Net model is better than that of the FCN4 model. The deviation between the GloWS-Net model wind speed results and the ERA5 data is very concentrated (80% of the wind speed deviation is less than 2.60 m/s) and near the deviation line of 0 m/s, while 80% of the wind speed deviation of FCN4 model is less than 2.62 m/s, the global wind speed retrieved by the MVE method is the worst, and 80% of the wind speed deviation is 3.39 m/s. The above analysis further proves that the GloWS-Net model architecture has the best generalization ability in retrieving global sea surface wind speed.
4.3. Discussion
Previous research has successfully utilized a fully connected network (FCN) model for wind speed retrieval [
37,
38,
39,
40,
41,
42,
43]. Inspired by image processing technology, Asgarimehr and Guo et al. studied spaceborne GNSS-R wind speed retrieval based on deep learning methods (CyGNSSnet and MCNN) [
40,
41]. Although both models employ the features extracted from BRCS DDM, MCNN also uses effective scattering area as the input image. However, their research shows that the GNSS-R wind speed retrieval model can benefit from the convolution layer. Because the image information in the DDM is integrated, the architecture that combines the convolution layer and the full connection layer has better performance than the architecture that only has the full connection layer. However, the retrieval results of both their models and the GloWS-Net model architecture proposed in this paper show that, mainly in the case of high wind speed, the model architecture integrating layer that takes DDM as input with the fully connection layer that takes auxiliary parameters as input can obtain more obvious accuracy improvement than the FCN model, but their RMSE improvement in the whole wind speed range of 0–30 m/s is not significant. This may require further optimization of the convolutional layer architecture or training of special models for different wind speed ranges [
50]. However, the GloWS-Net model architecture proposed in this paper is superior to the FCN model architecture in MAE and MAPE, especially in MAPE. From this point of view, GloWS-Net achieves better performance improvement than CyGNSSnet and MCNN for the wind speed range of 0–30 m/s.
Although CNN extracts features directly from the input, research shows that the prediction accuracy of CNN can be improved by using traditional features. For example, the combination of CNN and handmade features enhances image classification [
69,
70]. Similarly, theoretical remote sensing knowledge is increasingly combined with deep learning to further improve its performance [
71]. The analysis in this paper also shows that traditional CNN models (such as CNN1 and CNN2) without full connection layer to combine auxiliary parameters and DDM result in an RMSE of 2.46 m/s. The performance of fully connected architectures (such as FCN1) with NBRCS and LES as input characteristics is inferior to CNN1 and CNN2, resulting in an RMSE of 2.66 m/s. However, under high wind speed (>15 m/s), the performance of FCN1 is better than that of CNN1 and CNN2. Therefore, the use of convolution layers to directly extract features from DDM, combined with more auxiliary feature parameters including NBRCS and LES, can provide the best performance (such as CNN3 and GloWS-Net models), and RMSE is better than 1.92 m/s. By comparing the performance of FCN4 and GloWS-Net, it can be concluded that adding more auxiliary parameters does not necessarily reduce the RMSE of the test data. Although previous research and the results in this paper show that these auxiliary parameters can improve the retrieval performance of the model, and based on this fact, they have been selected as the input of the full connection layer. Therefore, it is necessary to further optimize the previously proposed model and GloWS-Net model architecture.
MVE is a classical method for retrieving sea surface wind speed. According to the statistical results in
Table 5 and
Table 6, the GloWS-Net model framework is superior to the MVE method in all wind speed ranges. Especially at high wind speeds (>15 m/s), RMSE is decreased by 37.45%, which shows significant improvement compared with previous studies.
Figure 8 and
Figure 12 also confirm that the global performance of the GloWS-Net model framework is better than MVE and FCN4, and the deviation has improved globally. Compared with the recent model based on the full connection layer for processing CYGNSS data [
36,
38], the GloWS-Net model architecture also shows encouraging performance. However, the differences in data length, version, quality control, filtering and validation methods should be considered to ensure the estimation of new data sets will have the same accuracy.
Both MVE and GloWS-Net underestimated high wind speed, which may be caused by the sensitivity saturation of DDM observed under strong wind conditions [
32]. This is a common problem of radar scatterometer whose performance will be reduced under high wind speed [
72]. However, compared with previous CyGNSSnet and MCNN models, the GloWS-Net model has significantly improved the wind speed retrieval performance in the case of large wind speed, showing good consistence with the ERA5 data. However, it is still challenging to use the GloWS-Net model to retrieve high wind speed. Due to the relatively small number of high wind speed samples involved in the training model, the performance of the deep learning algorithm is limited when the wind speed is greater than 15 m/s.
Figure 7 shows the area with overestimated wind speeds, especially in the Asia Pacific region with longitude between 50°W–0°. This is consistent with the situation reported by Asgarimehr et al. [
40]. This area is strongly affected by the L-band signal of the Quasi Zenith Satellite System (QZSS), which may be a potential source of Radio Frequency Interference (RFI). RFI caused by other L-band signals, especially satellite enhancement system (SBAS) signals, has been considered as the root cause of the reduction of signal to noise ratio and the wind speed overestimation of GNSS-R [
73]. Asgarimehr et al. also reported similar overestimation in the equatorial region in TDS-1 satellite measurements [
74]. Please note that the similarity of wind speed retrieved by MVE, CyGNSSnet and GloWS-Net confirms that overestimation is related to the data quality but not due to the retrieval methods.