Summary: We provide estimates of the kappa coefficient using multivariate probit models from a Bayesian perspective and investigate the performance of our method by assuming different prior distributions for the correlations of the underlying normal variables from multivariate probit models. We compare our method with J. L. Fleiss’ method [Psychol. Bull. 76, 378–382 (1971)] and the weighted moment method through two simulation studies. We further show that our method can be easily extended to detect treatment effects and estimate the kappa coefficients by treatment groups for clustered binary data using an example from an immunotherapy study in rheumatoid arthritis. C-code for our program is available on request.