For graph-based multi-view clustering, a critical issue is to capture consensus cluster structures via a two-stage learning scheme. Specifically, first learn similarity graph matrices of multiple views and then fuse them into a unified superior graph matrix. Most current methods learn pairwise similarities between data points for each view independently, which is widely used in single view. However, the consensus information contained in multiple views are ignored, and the involved biases lead to an undesirable unified graph matrix. To this end, we propose a bipartite graph based multi-view clustering (BIGMC) approach. The consensus information can be represented by a small number of representative uniform anchor points for different views. A bipartite graph is constructed between data points and the anchor points. BIGMC constructs the bipartite graph matrices of all views and fuses them to produce a unified bipartite graph matrix. The unified bipartite graph matrix in turn improves the bipartite graph similarity matrix of each view and updates the anchor points. The final unified graph matrix forms the final clusters directly. In BIGMC, an adaptive weight is added for each view to avoid outlier views. A low-rank constraint is imposed on the Laplacian matrix of the unified matrix to construct a multi-component unified bipartite graph, where the component number corresponds to the required cluster number. The objective function is optimized in an alternating optimization fashion. Experimental results on synthetic and real-world data sets demonstrate its effectiveness and superiority compared with the state-of-the-art baselines.