Summary: Because of the difficulty of obtaining an analytic expression for Bayes error, a wide variety of separability measures has been proposed for feature selection. In this paper, we show that there is a general framework based on the criterion of mutual information (MI) that can provide a realistic solution to the problem of feature selection for high-dimensional data. We give a theoretical argument showing that the MI of multi-dimensional data can be broken down into several one-dimensional components, which makes numerical evaluation much easier and more accurate. It also reveals that selection based on the simple criterion of only retaining features with high associated MI values may be problematic when the features are highly correlated. Although there is a direct way of selecting features by jointly maximising MI, this suffers from combinatorial explosion. Hence, we propose a fast feature-selection scheme based on a ‘greedy’ optimisation strategy. To confirm the effectiveness of this scheme, simulations are carried out on 16 land-cover classes using the 92AV3C data set collected from the 220-dimensional AVIRIS hyperspectral sensor. We replicate our earlier positive results (which used an essentially heuristic method for MI-based band-selection) but with much reduced computational cost and a much sounder theoretical basis.