The paper considers the general problem of nonlinear filtering for the unobserved components of a Markov process in terms of the observable ones. It studies the sensitivity of the solution to errors arising through incorrect initialization of the filter. This is done using relative entropy as a measure of the discrepancy between the optimal and incorrect filters. A particular decomposition of the relative entropy allows to show that it is a positive supermartingale and as a consequence filters are stable (in the sense of average relative entropy) against erroneous initializations. Finally, for signals observed in additive white noise, it is shown that the incorrectly initialized estimate of the observation function is asymptotically optimal in the infinite time limit.