Abstract
The popularity of GPS-equipped gadgets and mapping mashup applications has motivated the growth of geotagged Web resources as well as georeferenced multimedia applications. More and more research attention have been put on mining collaborative knowledge from mass user-contributed geotagged contents. However, little attention has been paid to generating high-quality geographical clusters, which is an important preliminary data-cleaning process for most geographical mining works. Previous works mainly use geotags to derive geographical clusters. Simply using one channel information is not sufficient for generating distinguishable clusters, especially when the location ambiguity problem occurs. In this paper, we propose a two-level clustering framework to utilize both the spatial and the semantic features of photographs for clustering. For the first-level geoclustering phase, we cluster geotagged photographs according to their spatial ties to roughly partition the dataset in an efficient way. Then we leverage the textual semantics in photographs' annotation to further refine the grouping results in the second-level semantic clustering phase. To effectively measure the semantic correlation between photographs, a semantic enhancement method as well as a new term weighting function have been proposed. We also propose a method for automatic parameter determination for the second-level spectral clustering process. Evaluation of our implementation on real georeferenced photograph dataset shows that our algorithm performs well, producing distinguishable geographical cluster with high accuracy and mutual information.
Similar content being viewed by others
Notes
The definition of geographical distance in Wikipedia, http://en.wikipedia.org/wiki/Geographical_distance.
References
Kennedy, L.S., Naaman, M.: Generating diverse and representative image search results for landmarks. In: WWW, pp. 297–306 (2008)
Quack, T., Leibe, B., Van Gool, L.J.: World-scale mining of objects and events from community photo collections. In: CIVR, pp. 47–56 (2008)
Crandall, D.J., Backstrom, L., Huttenlocher, D.P., Kleinberg, M.: Mapping the world’s photos. In: WWW, pp. 761–770 (2009)
Serdyukov, P., Murdock, V., van Zwol, R.: Placing flickr photos on a map. In: SIGIR, pp. 484–491 (2009)
Kleban, J., Moxley, E., Xu, J., Manjunath, B.S.: Global annotation on georeferenced photographs. In: CIVR (2009)
Joshi, D., Gallagher, A.C., Yu, J., Luo, J.: Inferring photographic location using geotagged web images. Multimed. Tools Appl. 56, 131–153 (2012)
Joshi, D., Luo, J., Yu, J., Lei, P., Gallagher, A.: Using geotags to derive rich tag-clouds for image annotation. In: Hoi, S.C.H, Luo, J., Boll, S., Xu, D., Jin, R., King, I. (eds.) Social Media Modeling and Computing, pp. 239–256. Springer, London (2011)
Cao, L., Yu, J., Luo, J., Huang, T.S.: Enhancing semantic and geographic annotation of web images via logistic canonical correlation regression. In: Proceedings of the 17th ACM international conference on Multimedia, New York, pp. 125–134 (2009)
Zhang, L., Han, Y., Yang, Y., Song, M., Yan, S., Tian, Q.: Discovering discriminative graphlets for aerial image categories recognition. IEEE Trans. Image Process. 22, 5071–5084 (2013)
Zhang, L., Gao, Y., Zimmermann, R., Tian, Q., Li, X.: Fusion of multichannel local and global structural cues for photo aesthetics evaluation. IEEE Trans. Image Process. 23, 1419–1429 (2014)
Zhang, L., Song, M., Zhao, Q., Liu, X., Chen, C.: Probabilistic graphlet transfer for photo cropping. IEEE Trans. Image Process. 22, 802–815 (2013)
Zhang, L., Gao, Y., Ji, R., Xia, Y., Dai, Q., Li, X.: Actively learning human gaze shifting paths for semantics-aware photo cropping. IEEE Trans. Image Process. 23, 2235–2245 (2014)
Zhang, L., Song, M., Liu, Z., Liu, X., Bu, J., Chen, C.: Probabilistic graphlet cut: exploiting spatial structure cue for weakly supervised image segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1908–1915 (2013)
Zhang, L., Song, M., Yang, Y., Zhao, Q., Zhao, C., Sebe, N.: Weakly supervised photo cropping. IEEE Trans. Multimed. 16, 94–107 (2014)
Avrithis, Y.S., Kalantidis, Y., Tolias, G., Spyrou, E.: Retrieving landmark and non-landmark images from community photo collections. In: Proceedings of the international conference on Multimedia, New York, pp. 153–162 (2010)
The Panoramio Website. http://www.panoramio.com, Google Panoramio (2007)
Ankerst, M., Breunig, M.M., Kriegel, H.-P., Sander, J.: OPTICS: ordering points to identify the clustering structure. In: SIGMOD, pp. 49–60 (1999)
Toyama, K., Logan, R., Roseway, A.: Geographic location tags on digital images. In: Proceedings of the eleventh ACM international conference on Multimedia, New York, pp. 156–166 (2003)
Jaffe, A., Naaman, M., Tassa, T., Davis, M.: Generating summaries and visualization for large collections of geo-referenced photographs. In: Proceedings of the 8th ACM international workshop on Multimedia information retrieval, New York, pp. 89–98 (2006)
Ahern, S., Naaman, M., Nair, R., Yang, J.H.-I.: World explorer: visualizing aggregate data from unstructured text in geo-referenced collections. In: JCDL, pp. 1–10 (2007)
Popescu, A., Mollic, P.A., Kanellos, L.: ThemExplorer: finding and browsing geo-referenced images. In: CBMI, pp. 576–583 (2008)
Zhu, Z., Shou, L., Mao, K., Chen, G.: Location disambiguation for geo-tagged images. In: SIGIR, pp. 1165–1166 (2011)
Naaman, M., Paepcke, A., Garcia-Molina, H.: From where to what: metadata sharing for digital photographs with geographic coordinates. In: CoopIS/DOA/ODBASE, pp. 196–217 (2003)
Davis, M., King, S., Good, N., Sarvas, R.: From context to content: leveraging context to infer media metadata. In: Proceedings of the 12th annual ACM international conference on Multimedia, New York, pp. 188–195 (2004)
Rattenbury, T., Good, N., Naaman, M.: Towards automatic extraction of event and place semantics from flickr tags. In: SIGIR, pp. 103–110 (2007)
Hoashi, K., Uemukai, T., Matsumoto, K., Takishima, Y.: Constructing a landmark identification system for Geo-tagged photographs based on Web data analysis. In: ICME, pp. 606–609 (2009)
Cao, X., Cong, G., Jensen, C.S.: Mining significant semantic locations from GPS data. PVLDB 3, 1009–1020 (2010)
Acknowledgments
This work is supported in part by the China 863 Project No. 2013AA040601.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhu, Z., Xu, C. Organizing photographs with geospatial and image semantics. Multimedia Systems 23, 53–61 (2017). https://doi.org/10.1007/s00530-014-0426-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00530-014-0426-5