Efficiently learning the preferences of people. (English) Zbl 1260.68320
Summary: This paper presents a framework for optimizing the preference learning process. In many real-world applications in which preference learning is involved the available training data is scarce and obtaining labeled training data is expensive. Fortunately in many of the preference learning situations data is available from multiple subjects. We use the multi-task formalism to enhance the individual training data by making use of the preference information learned from other subjects. Furthermore, since obtaining labels is expensive, we optimally choose which data to ask a subject for labelling to obtain the most of information about her/his preferences. This paradigm – called active learning – has hardly been studied in a multi-task formalism. We propose an alternative for the standard criteria in active learning which actively chooses queries by making use of the available preference data from other subjects. The advantage of this alternative is the reduced computation costs and reduced time subjects are involved. We validate empirically our approach on three real-world data sets involving the preferences of people.
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 68U35 | Computing methodologies for information systems (hypertext navigation, interfaces, decision support, etc.) |
Keywords:
learning preferences; active learning; experimental design; multi-task learning; hierarchical modelingReferences:
| [1] | Adomavicius, A., Sankaranarayanan, R., Sen, S., & Tuzhilin, A. (2005). Incorporating contextual information in recommender systems using a multidimensional approach. ACM Transactions on Information Systems, 23(1), 103–145. · doi:10.1145/1055709.1055714 |
| [2] | Anand, P. (1993). The philosophy of intransitive preferences. The Economic Journal, 103(417), 337–346. · doi:10.2307/2234772 |
| [3] | Arehart, K. H., Kates, J. M., Anderson, C. A., & Harvey, L. O. Jr. (2007). Effects of noise and distortion on speech quality judgments in normal-hearing and hearing-impaired listeners. Journal of the Acoustical Society of America, 122(2), 1150–1164. · doi:10.1121/1.2754061 |
| [4] | Arens, R. (2008). Learning SVM ranking function from user feedback using document metadata and active learning in the biomedical domain. In Proceedings of the ECML/PKDD workshop on preference learning (pp. 363–383). |
| [5] | Argyriou, A., Micchelli, C., & Pontil, M. (2008). When is there a representer theorem? Vector versus matrix regularizers. Journal of Machine Learning Research. · Zbl 1235.68128 |
| [6] | Bakker, B., & Heskes, T. (2003). Task clustering and gating for Bayesian multitask learning. Journal of Machine Learning Research, 4, 83–99. · Zbl 1085.68612 |
| [7] | Berger, M. P. F. (1994). D-optimal sequential sampling designs for item response theory models. Journal of Educational and Behavioral Statistics, 19, 43–56. · doi:10.3102/10769986019001043 |
| [8] | Birlutiu, A., & Heskes, T. (2007). Expectation propagation for rating players in sports competitions. In Proceedings of the 11th European conference on principles and practice of knowledge discovery in databases (pp. 374–381). |
| [9] | Birlutiu, A., Groot, P., & Heskes, T. (2009). Multi-task preference learning with an application to hearing aid personalization. Neurocomputing, 73(7–9), 1177–1185. · doi:10.1016/j.neucom.2009.11.025 |
| [10] | Bishop, C. M. (2006). Pattern recognition and machine learning. Berlin: Springer. · Zbl 1107.68072 |
| [11] | Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022. · Zbl 1112.68379 |
| [12] | Blythe, J. (2002). Visual exploration and incremental utility elicitation. In Proceedings of the 18th national conference on artificial intelligence (pp. 526–532). |
| [13] | Bordley, R. F. (1982). A multiplicative formula for aggregating probability assessments. Management Science, 28(10), 1137–1148. · Zbl 0492.90042 · doi:10.1287/mnsc.28.10.1137 |
| [14] | Boutilier, C. (2002). A POMDP formulation of preference elicitation problems. In Proceedings of the 18th national conference on artificial intelligence (pp. 239–246). |
| [15] | Boutilier, C., Zemel, R. S., & Marlin, B. (2003). Active collaborative filtering. In Proceedings of the 19th annual conference on uncertainty in artificial intelligence (pp. 98–106). |
| [16] | Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge: Cambridge University Press. · Zbl 1058.90049 |
| [17] | Bradley, R. A., & Terry, M. E. (1952). Rank analysis of incomplete block designs, I: the method of paired comparisons. Biometrika, 39, 324–345. · Zbl 0047.12903 |
| [18] | Brinker, K. (2004). Active learning of label ranking functions. In Proceedings of the 27th international conference on machine learning. |
| [19] | Brochu, E., de Freitas, N., & Ghosh, A. (2008). Active preference learning with discrete choice data. In J. C. Platt, Y. Koller, D. Singer & S. Roweis (Eds.), Advances in neural information processing systems (Vol. 20, pp. 409–416). Cambridge: MIT Press. |
| [20] | Caruana, R. (1997). Multitask learning. Machine Learning, 28(1), 41–75. · doi:10.1023/A:1007379606734 |
| [21] | Chajewska, U., Koller, D., & Parr, R. (2000). Making rational decisions using adaptive utility elicitation. In Proceedings of the 17th national conference on artificial intelligence (pp. 363–369). |
| [22] | Chaloner, K., & Verdinelli, I. (1995). Bayesian experimental design: a review. Statistical Science, 10, 273–304. · Zbl 0955.62617 · doi:10.1214/ss/1177009939 |
| [23] | Christensen, R. (1997). Log-linear models and logistic regression. Berlin: Springer. · Zbl 0880.62073 |
| [24] | Chu, W., & Ghahramani, Z. (2005a). Extensions of Gaussian processes for ranking: semi-supervised and active learning. In NIPS workshop on learning to rank. |
| [25] | Chu, W., & Ghahramani, Z. (2005b). Preference learning with Gaussian processes. In Proceedings of the 22nd international conference on machine learning. · Zbl 1222.68170 |
| [26] | Clyde, M., Muller, P., & Parmigiani, G. (1993). Optimal design for heart defibrillators. In Case studies in Bayesian statistics (Vol. II, pp. 278–292). · Zbl 0825.92074 |
| [27] | Cohn, D. A., Ghahramani, A., & Jordan, M. I. (1996). Active learning with statistical models. Journal of Artificial Intelligence Research, 4(1), 129–145. · Zbl 0900.68366 |
| [28] | Dagan, I., & Engelson, S. P. (1995). Committee-based sampling for training probabilistic classifiers. In Proceedings of the 12th international conference on machine learning (pp. 150–157). |
| [29] | Dasgupta, S., & Hsu, D. (2008). Hierarchical sampling for active learning. In Proceedings of the 25th international conference on machine learning (pp. 208–215). |
| [30] | Doyle, J. (2004). Prospects for preferences. Computational Intelligence, 20(2), 111–136. · doi:10.1111/j.0824-7935.2004.00233.x |
| [31] | Dror, H. A., & Steinberg, D. M. (2008). Sequential experimental designs for generalized linear models. Journal of the American Statistical Association, 103(481), 288–298. · Zbl 1471.62426 · doi:10.1198/016214507000001346 |
| [32] | Evgeniou, T., Micchelli, A., & Pontil, M. (2005). Learning multiple tasks with kernel methods. Journal of Machine Learning Research, 6, 615–637. · Zbl 1222.68197 |
| [33] | Fedorov, V. V. (1972). Theory of optimal experiments. New York: Academic Press. · Zbl 0258.62044 |
| [34] | Ford, I., & Silvey, S. D. (1980). A sequentially constructed design for estimating a nonlinear parametric function. Biometrika, 67, 381–388. · Zbl 0433.62054 · doi:10.1093/biomet/67.2.381 |
| [35] | Freund, Y., Shamir, E., & Tishby, N. (1997). Selective sampling using the query by committee algorithm. Machine Learning, 28(2–3), 133–168. · Zbl 0881.68093 · doi:10.1023/A:1007330508534 |
| [36] | Fürnkranz, J., & Hüllermeier, E. (2010). Preference learning. Berlin: Springer. · Zbl 1214.68285 |
| [37] | Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003). Bayesian data analysis (2nd ed.). London: Chapman & Hall/CRC. |
| [38] | Geman, S., Bienenstock, E., & Doursat, R. (1992). Neural networks and the bias/variance dilemma. Neural Computation, 4(1), 1–58. · doi:10.1162/neco.1992.4.1.1 |
| [39] | Gervasio, M. T., Moffitt, M. D., & Pollack, M. E. (2005). Active preference learning for personalized calender scheduling assistance. In Proceedings of the 10th international conference on intelligent user interfaces (pp. 90–97). |
| [40] | Glickman, M., & Jensen, S. (2005). Adaptive paired comparison design. Journal of Statistical Planning and Inference, 127, 279–293. · Zbl 1083.62001 · doi:10.1016/j.jspi.2003.09.022 |
| [41] | Groot, P. C., Birlutiu, A., & Heskes, T. (2010). Bayesian Monte Carlo for the global optimization of expensive functions. In Proceedings of the 19th European conference on artificial intelligence (pp. 249–254). · Zbl 1211.65078 |
| [42] | Guo, S., & Sanner, S. (2010). Real-time multiattribute Bayesian preference elicitation with pairwise comparison queries. In Proceedings of the 13th international conference on artificial intelligence and statistics (pp. 289–296). |
| [43] | Harpale, S., & Yang, Y. (2008). Personalized active learning for collaborative filtering. In Proceedings of the 31st annual international ACM SIGIR conference on research and development in information retrieval (pp. 91–98). |
| [44] | Heskes, T., & de Vries, B. (2005). Incremental utility elicitation for adaptive personalization. In Proceedings of the 17th Belgium-Netherlands conference on artificial intelligence (pp. 127–134). |
| [45] | Hollander, M., & Wolfe, D. A. (1973). Nonparametric statistical methods. New York: Wiley. · Zbl 0277.62030 |
| [46] | Jin, R., & Si, L. (2004). A Bayesian approach toward active learning for collaborative filtering. In Proceedings of the 20th conference on uncertainty in artificial intelligence (pp. 278–285). |
| [47] | Kanninen, B. (2002). Optimal design for multinomial choice experiments. Journal of Marketing Research, 39, 307–317. · doi:10.1509/jmkr.39.2.214.19080 |
| [48] | Lewi, J., Butera, R., & Paninski, L. (2009). Sequential optimal design of neurophysiology experiments. Neural Computation, 21(3), 619–687. · Zbl 1180.68219 · doi:10.1162/neco.2008.08-07-594 |
| [49] | Lewis, D., & Gale, W. (1994). A sequential algorithm for training text classifiers. In Proceedings of the 17th annual international ACM SIGIR conference on research and development in information retrieval (pp. 3–12). |
| [50] | MacKay, D. J. C. (1992). Information-based objective functions for active data selection. Neural Computation, 4, 590–604. · doi:10.1162/neco.1992.4.4.590 |
| [51] | MacKay, D. J. C. (2002). Information theory, inference & learning algorithms. Cambridge: Cambridge University Press. |
| [52] | McCallum, A., & Nigam, K. (1998). Employing EM and pool-based active learning for text classification. In Proceedings of the 15th international conference on machine learning (pp. 350–358). |
| [53] | Melville, P., & Mooney, R. (2004). Diverse ensembles for active learning. In Proceedings of the 21st international conference on machine learning (pp. 584–591). |
| [54] | Minka, T. (2001). A family of approximation methods for approximate Bayesian inference. PhD thesis, MIT. |
| [55] | Pahikkala, T., Waegeman, W., Tsivtsivadze, E., De Baets, B., & Salakoski, T. (2009). From ranking to intransitive preference learning: rock-paper-scissors and beyond. In Proceedings of the ECML/PKDD workshop on preference learning (pp. 84–100). |
| [56] | Qin, T., Liu, T. Y., Xu, J., & Li, H. (2010). LETOR: a benchmark collection for research on learning to rank for information retrieval. Information Retrieval, 13(4), 346–374. · doi:10.1007/s10791-009-9123-y |
| [57] | Sanborn, A. N., & Griffiths, T. L. (2008). Markov chain Monte Carlo with people. In: Advances in neural information processing systems. |
| [58] | Schein, A., & Ungar, L. (2007). Active learning for logistic regression: an evaluation. Machine Learning, 68(3), 235–265. · Zbl 1470.68170 · doi:10.1007/s10994-007-5019-5 |
| [59] | Seeger, M. W. (2008). Bayesian inference and optimal design for the sparse linear model. Journal of Machine Learning Research, 9, 759–813. · Zbl 1225.68213 |
| [60] | Settles, B., & Craven, M. (2008). An analysis of active learning strategies for sequence labeling tasks. In Proceedings of the conference on empirical methods in natural language processing (pp. 1070–1079). |
| [61] | Seung, H. S., Opper, M., & Sompolinsky, H. (1992). Query by Committee. In Proceedings of the 5th annual workshop on computational learning theory (pp. 287–294). |
| [62] | Thrun, S. (1995). Is learning the nth thing any easier than learning the first? In Advances in neural information processing systems (pp. 640–646). |
| [63] | Tversky, A. (1998). Preference, belief, and similarity. Cambridge: MIT Press. |
| [64] | Xu, Z., Kersting, K., & Joachims, T. (2010). Fast active exploration for link-based preference learning using Gaussian processes. In Proceedings of the 2010 European conference on machine learning and knowledge discovery in databases: part III (pp. 499–514). |
| [65] | Yu, K., Schwaighofer, A., Tresp, V., Ma, W. Y., & Zhang, H. J. (2003). Collaborative ensemble learning: combining collaborative and content-based information filtering. In Proceedings of the 19th conference on uncertainty in artificial intelligence (pp. 616–623). |
| [66] | Yu, K., Tresp, V., & Schwaighofer, A. (2005). Learning Gaussian processes from multiple tasks. In Proceedings of the 22nd international conference on machine learning. |
| [67] | Yu, K., Bi, J., & Tresp, V. (2006). Active learning via transductive experimental design. In Proceedings of the 23rd international conference on machine learning (pp. 1081–1088). |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
