Summary: Learning causal models hidden in the background of observational data has been a difficult issue. Dealing with latent common causes and selection bias for constructing causal models in real data is often necessary because observing all relevant variables is difficult. Ancestral graph models are effective and useful for representing causal models with some information of such latent variables. The causal faithfulness condition, which is usually assumed for determining the models, is known to often be weakly violated in statistical view points for finite data. One of the authors developed a constraint-based causal learning algorithm that is robust against the weak violations while assuming no latent variables. In this study, we applied and extended the thoughts of the algorithm to the inference of ancestral graph models. The practical validity and effectiveness of the algorithm are also confirmed by using some standard datasets in comparison with FCI and RFCI algorithms.