×

Avoiding the dangers of averaging across subjects when using multidimensional scaling. (English) Zbl 1040.91082

Summary: F. G. Ashby, W. T. Maddox and W. W. Lee [Psychol. Sci. 5, 144–151 (1994)] argue that it can be inappropriate to fit MultiDimensional Scaling (MDS) models to similarity or dissimilarity data that have been averaged across subjects. They demonstrate that the averaging process tends to make dissimilarity data more amenable to metric representations, and conduct a simulation study showing that noisy data generated using one distance metric, when averaged, may be better fit using a different distance metric. This paper argues that a Bayesian measure of MDS models has the potential to address these difficulties, because it takes into account data-fit, the number of dimensions used by an MDS representation, and the precision of the data. A method of analysis based on the Bayesian measure is demonstrated through two simulation studies with accompanying theoretical analysis. In the first study, it is shown that the Bayesian analysis rejects those MDS models showing better fit to averaged data using the incorrect distance metric, while accepting those that use the correct metric. In the second study, different groups of simulated ‘subjects’ are assumed to use different underlying configurations. In this case, the Bayesian analysis rejects MDS representations where a significant proportion of subjects use different configurations, or when their dissimilarity judgments contain significant amounts of noise. It is concluded that the Bayesian analysis provides a simple and principled means for systematically accepting and rejecting MDS models derived from averaged data.

MSC:

91C15 One- and multidimensional scaling in the social and behavioral sciences
Full Text: DOI

References:

[1] Ashby, F. G.; Maddox, W. T.; Lee, W. W., On the dangers of averaging across subjects when using multidimensional scaling or the similarity-choice model, Psychological Science, 5, 3, 144-151 (1994)
[2] Cox, T. F.; Cox, M. A.A., Multidimensional scaling (1994), Chapman & Hall: Chapman & Hall London · Zbl 0853.62047
[3] Ekman, G., Dimensions of color vision, The Journal of Psychology, 38, 467-474 (1954)
[4] Furnas, G. W., Metric family portraits, Journal of Classification, 6, 7-52 (1989) · Zbl 0691.62004
[5] Garner, W. R., The processing of information and structure (1974), Erlbaum: Erlbaum Potomac, MD
[6] Gati, I.; Tversky, A., Representations of qualitative and quantitative dimensions, Journal of Experimental Psychology: Human Perception and Performance, 8, 2, 325-340 (1982)
[7] Glushko, R. J., Pattern goodness and redundancy revisitedMultidimensional scaling and hierarchical cluster analysis, Perception & Psychophysics, 17, 2, 158-162 (1975)
[8] Gregson, R. A.M., A comparative evaluation of seven similarity models, British Journal of Mathematical and Statistical Psychology, 29, 139-156 (1976) · Zbl 0345.92013
[9] Grünwald, P., Model selection based on minimum description length, Journal of Mathematical Psychology, 44, 1, 133-152 (2000) · Zbl 0968.62008
[10] Heaps, C.; Handel, S., Similarity and features of natural textures, Journal of Experimental Psychology: Human Perception and Performance, 25, 2, 299-320 (1999)
[11] Johnson, E. J.; Tversky, A., Representations of perceptions of risks, Journal of Experimental Psychology: General, 113, 1, 55-70 (1984)
[12] Jones, F. N.; Roberts, K.; Holman, E. W., Similarity judgments and recognition memory for common spices, Perception & Psychophysics, 24, 1, 2-6 (1978)
[13] Kass, R. E.; Raftery, A. E., Bayes factors, Journal of the American Statistical Association, 90, 430, 773-795 (1995) · Zbl 0846.62028
[14] Kruschke, J. K., ALCOVEAn exemplar-based connectionist model of category learning, Psychological Review, 99, 1, 22-44 (1992)
[15] Kruschke, J. K., Human category learningImplications for backpropagation models, Connection Science, 5, 3-36 (1993)
[16] Kruskal, J. B., Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis, Psychometrika, 29, 1, 1-27 (1964) · Zbl 0123.36803
[17] Lee, M. D., Determining the dimensionality of multidimensional scaling representations for cognitive modeling, Journal of Mathematical Psychology, 45, 1, 149-166 (2001) · Zbl 1003.91033
[18] More, J. J. (1977). The Levenberg-Marquardt algorithm: Implementation and theory. In G. A. Watson (Ed.), Lecture notes in mathematics; More, J. J. (1977). The Levenberg-Marquardt algorithm: Implementation and theory. In G. A. Watson (Ed.), Lecture notes in mathematics
[19] Myung, I. J.; Balasubramanian, V.; Pitt, M. A., Counting probability distributionsDifferential geometry and model selection, Proceedings of the National Academy of Sciences, 97, 11170-11175 (2000) · Zbl 0997.62099
[20] Myung, I. J.; Forster, M.; Browne, M. W., A special issue on model selection, Journal of Mathematical Psychology, 44, 1-2 (2000) · Zbl 0951.00029
[21] Myung, I. J.; Pitt, M. A., Applying Occam’s razor in modeling cognitionA Bayesian approach, Psychonomic Bulletin & Review, 4, 1, 79-95 (1997)
[22] Nosofsky, R. M., Choice, similarity, and the context theory of classification, Journal of Experimental Psychology: Learning, Memory, and Cognition, 10, 1, 104-114 (1984)
[23] Nosofsky, R. M., Attention, similarity, and the identification-categorization relationship, Journal of Experimental Psychology: General, 115, 1, 39-57 (1986)
[24] Pitt, M. A.; Myung, I. J.; Zhang, S., Toward a method of selecting among computational models of cognition, Psychological Review, 109, 3, 472-491 (2002)
[25] Raftery, A. E., Bayes factors and BICComment on Weakliem, Sociological Methods and Research, 27, 411-427 (1999)
[26] Schwarz, G., Estimating the dimension of a model, Annals of Statistics, 6, 2, 461-464 (1978) · Zbl 0379.62005
[27] Shepard, R. N., Stimulus and response generalizationA stochastic model relating generalization to distance in psychological space, Psychometrika, 22, 4, 325-345 (1957) · Zbl 0088.12906
[28] Shepard, R. N., The analysis of proximitiesMultidimensional scaling with an unknown distance function, I. Psychometrika, 27, 2, 125-140 (1962) · Zbl 0129.12103
[29] Shepard, R. N., Toward a universal law of generalization for psychological science, Science, 237, 1317-1323 (1987) · Zbl 1226.91051
[30] Shepard, R. N. (1991). Integrality versus separability of stimulus dimensions: From an early convergence of evidence to a proposed theoretical basis. In J. R. Pomerantz, & G. L. Lockhead (Eds.), The perception of structureEssays in honor of Wendell R Garner; Shepard, R. N. (1991). Integrality versus separability of stimulus dimensions: From an early convergence of evidence to a proposed theoretical basis. In J. R. Pomerantz, & G. L. Lockhead (Eds.), The perception of structureEssays in honor of Wendell R Garner
[31] Shepard, R. N., Perceptual-cognitive universals as reflections of the world, Psychonomic Bulletin & Review, 1, 1, 2-28 (1994)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.