Summary: To monitor the dynamics and non-stationarity inherent in industrial processes, we propose a novel incipient fault detection and isolation scheme grounded in a probabilistic perspective, using the Cauchy-Schwarz (CS) divergence. Our innovation lies in the utilization of marginal CS divergence for incipient fault detection and the conditional CS divergence for fault isolation. This approach neither require prior parametric assumptions about the underlying data distribution nor depend on historical fault data, while simultaneously providing explanatory diagnostics. Beyond this, we develop a change point detection-base diagnosis technique for practical engineering applications. This online process monitoring technique guarantees timely intervention to uphold process stability and safety. We demonstrate the compelling performance, higher detection rate and lower alarm rate, of the CS divergence over prevalent divergence-based approaches, such as Kullback-Leibler divergence, Wasserstein distance and Mahalanobis distance. We also illustrate the explanatory insights offered by conditional CS divergence in fault isolation on synthetic data, benchmarks of continuous stirred-tank reactor process and continuous stirred-tank heater process and even a real-world continuous catalytic reforming process. Code of this CS divergence based fault detection and isolation is available at https://github.com/Feiya-Lv/Incipient-Fault-Detection-and-Isolation-with-Cauchy-Schwarz-Divergence.