2012 EvaluatingtheEffectivenessofSea

(Liao et al., 2012) ⇒ Zhen Liao, Yang Song, Li-wei He, and Yalou Huang. (2012). “Evaluating the Effectiveness of Search Task Trails.” In: Proceedings of the 21st International Conference on World Wide Web. ISBN:978-1-4503-1229-5 doi:10.1145/2187836.2187903

Subject Headings: Search Task Trail.

Notes

Cited By

Quotes

Abstract

In this paper, we introduce " task trail” as a new concept to understand user search behaviors. We define task to be an atomic user information need. Web search logs have been studied mainly at session or query level where users may submit several queries within one task and handle several tasks within one session. Although previous studies have addressed the problem of task identification, little is known about the advantage of using task over session and query for search applications. In this paper, we conduct extensive analyses and comparisons to evaluate the effectiveness of task trails in three search applications: determining user satisfaction, predicting user search interests, and query suggestion. Experiments are conducted on large scale datasets from a commercial search engine. Experimental results show that: (1) Sessions and queries are not as precise as tasks in determining user satisfaction. (2) Task trails provide higher web page utilities to users than other sources. (3) Tasks represent atomic user information needs, and therefore can preserve topic similarity between query pairs. (4) Task-based query suggestion can provide complementary results to other models. The findings in this paper verify the need to extract task trails from web search logs and suggest potential applications in search and recommendation systems.

References

1. ODP:http://www.dmoz.org/.
2. Paul N. Bennett, Krysta Svore, Susan T. Dumais, Classification-enhanced Ranking, Proceedings of the 19th International Conference on World Wide Web, April 26-30, 2010, Raleigh, North Carolina, USA doi:10.1145/1772690.1772703
3. Paolo Boldi, Francesco Bonchi, Carlos Castillo, Debora Donato, Aristides Gionis, Sebastiano Vigna, The Query-flow Graph: Model and Applications, Proceedings of the 17th ACM Conference on Information and Knowledge Management, October 26-30, 2008, Napa Valley, California, USA doi:10.1145/1458082.1458163
4. Huanhuan Cao, Daxin Jiang, Jian Pei, Qi He, Zhen Liao, Enhong Chen, Hang Li, Context-aware Query Suggestion by Mining Click-through and Session Data, Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 24-27, 2008, Las Vegas, Nevada, USA doi:10.1145/1401890.1401995
5. Lara D. Catledge, James E. Pitkow, Characterizing Browsing Strategies in the World-Wide Web, Computer Networks and ISDN Systems, v.27 n.6, p.1065-1073, April 1995 doi:10.1016/0169-7552(95)00043-7
6. Nick Craswell, Martin Szummer, Random Walks on the Click Graph, Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 23-27, 2007, Amsterdam, The Netherlands doi:10.1145/1277741.1277784
7. Hongbo Deng, Irwin King, Michael R. Lyu, Entropy-biased Models for Query Representation on the Click Graph, Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, July 19-23, 2009, Boston, MA, USA doi:10.1145/1571941.1572001
8. Debora Donato, Francesco Bonchi, Tom Chi, Yoelle Maarek, Do You Want to Take Notes?: Identifying Research Missions in Yahoo! Search Pad, Proceedings of the 19th International Conference on World Wide Web, April 26-30, 2010, Raleigh, North Carolina, USA doi:10.1145/1772690.1772724
9. Ted Dunning, Accurate Methods for the Statistics of Surprise and Coincidence, Computational Linguistics, v.19 n.1, March 1993
10. Steve Fox, Kuldeep Karnawat, Mark Mydland, Susan Dumais, Thomas White, Evaluating Implicit Measures to Improve Web Search, ACM Transactions on Information Systems (TOIS), v.23 n.2, p.147-168, April 2005 doi:10.1145/1059981.1059982
11. Jianfeng Gao, Wei Yuan, Xiao Li, Kefeng Deng, Jian-Yun Nie, Smoothing Clickthrough Data for Web Search Ranking, Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, July 19-23, 2009, Boston, MA, USA doi:10.1145/1571941.1572003
12. Ahmed Hassan, Rosie Jones, Kristina Lisa Klinkner, Beyond DCG: User Behavior As a Predictor of a Successful Search, Proceedings of the Third ACM International Conference on Web Search and Data Mining, February 04-06, 2010, New York, New York, USA doi:10.1145/1718487.1718515
13. Ahmed Hassan, Yang Song, Li-wei He, A Task Level Metric for Measuring Web Search Satisfaction and Its Application on Improving Relevance Estimation, Proceedings of the 20th ACM International Conference on Information and Knowledge Management, October 24-28, 2011, Glasgow, Scotland, UK doi:10.1145/2063576.2063599
14. Haveliwala T., Kamvar, S., and Jeh G. An Analytical Comparison of Approaches to Personalizing Page Rank. 2003.
15. Daqing He, Ayse Göker, David J. Harper, Combining Evidence for Automatic Web Session Identification, Information Processing and Management: An International Journal, v.38 n.5, p.727-742, September 2002 doi:10.1016/S0306-4573(01)00060-7
16. Chien-Kang Huang, Lee-Feng Chien, Yen-Jen Oyang, Relevant Term Suggestion in Interactive Web Search based on Contextual Information in Query Session Logs, Journal of the American Society for Information Science and Technology, v.54 n.7, p.638-649, May 2003 doi:10.1002/asi.10256
17. Bernard J. Jansen, Amanda Spink, Vinish Kathuria, How to Define Searching Sessions on Web Search Engines, Proceedings of the 8th Knowledge Discovery on the Web International Conference on Advances in Web Mining and Web Usage Analysis, p.92-109, August 20, 2006, Philadelphia, PA, USA
18. Thorsten Joachims, Optimizing Search Engines Using Clickthrough Data, Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, July 23-26, 2002, Edmonton, Alberta, Canada doi:10.1145/775047.775067
19. Rosie Jones, Kristina Lisa Klinkner, Beyond the Session Timeout: Automatic Hierarchical Segmentation of Search Topics in Query Logs, Proceedings of the 17th ACM Conference on Information and Knowledge Management, October 26-30, 2008, Napa Valley, California, USA doi:10.1145/1458082.1458176
20. Rosie Jones, Benjamin Rey, Omid Madani, Wiley Greiner, Generating Query Substitutions, Proceedings of the 15th International Conference on World Wide Web, May 23-26, 2006, Edinburgh, Scotland doi:10.1145/1135777.1135835
21. Alexander Kotov, Paul N. Bennett, Ryen W. White, Susan T. Dumais, Jaime Teevan, Modeling and Analysis of Cross-session Search Tasks, Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, July 24-28, 2011, Beijing, China doi:10.1145/2009916.2009922
22. Zhen Liao, Daxin Jiang, Enhong Chen, Jian Pei, Huanhuan Cao, Hang Li, Mining Concept Sequences from Large-Scale Search Logs for Context-Aware Query Suggestion, ACM Transactions on Intelligent Systems and Technology (TIST), v.3 n.1, p.1-40, October 2011 doi:10.1145/2036264.2036281
23. Yuting Liu, Bin Gao, Tie-Yan Liu, Ying Zhang, Zhiming Ma, Shuyuan He, Hang Li, BrowseRank: Letting Web Users Vote for Page Importance, Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 20-24, 2008, Singapore, Singapore doi:10.1145/1390334.1390412
24. Claudio Lucchese, Salvatore Orlando, Raffaele Perego, Fabrizio Silvestri, Gabriele Tolomei, Identifying Task-based Sessions in Search Engine Query Logs, Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, February 09-12, 2011, Hong Kong, China doi:10.1145/1935826.1935875
25. MacQueen, J. Some Methods for Classification and Analysis of Multivariate Observations. Pages 281--297, 1967.
26. Martin E., Kriegel, H., Sander, J. and Xu, X. A Density-based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. KDD '96, Pages 226--231. AAAI Press, 1996.
27. Qiaozhu Mei, Dengyong Zhou, Kenneth Church, Query Suggestion Using Hitting Time, Proceedings of the 17th ACM Conference on Information and Knowledge Management, October 26-30, 2008, Napa Valley, California, USA doi:10.1145/1458082.1458145
28. Christopher Olston, Ed H. Chi, ScentTrails: Integrating Browsing and Searching on the Web, ACM Transactions on Computer-Human Interaction (TOCHI), v.10 n.3, p.177-197, September 2003 doi:10.1145/937549.937550
29. Dou Shen, Rong Pan, Jian-Tao Sun, Jeffrey Junfeng Pan, Kangheng Wu, Jie Yin, Qiang Yang, Q²C@UST: Our Winning Solution to Query Classification in KDDCUP 2005, ACM SIGKDD Explorations Newsletter, v.7 n.2, p.100-110, December 2005 doi:10.1145/1117454.1117467
30. Xuehua Shen, Bin Tan, ChengXiang Zhai, Context-sensitive Information Retrieval Using Implicit Feedback, Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, August 15-19, 2005, Salvador, Brazil doi:10.1145/1076034.1076045
31. Craig Silverstein, Hannes Marais, Monika Henzinger, Michael Moricz, Analysis of a Very Large Web Search Engine Query Log, ACM SIGIR Forum, v.33 n.1, p.6-12, Fall 1999 doi:10.1145/331403.331405
32. Michael J. Swain, Dana H. Ballard, Color Indexing, International Journal of Computer Vision, v.7 n.1, p.11-32, Nov. 1991 doi:10.1007/BF00130487
33. Ryen W. White, Jeff Huang, Assessing the Scenic Route: Measuring the Value of Search Trails in Web Logs, Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, July 19-23, 2010, Geneva, Switzerland doi:10.1145/1835449.1835548
34. Ryen W. White, Paul N. Bennett, Susan T. Dumais, Predicting Short-term Interests Using Activity-based Search Context, Proceedings of the 19th ACM International Conference on Information and Knowledge Management, October 26-30, 2010, Toronto, ON, Canada doi:10.1145/1871437.1871565
35. Ryen W. White, Mikhail Bilenko, Silviu Cucerzan, Studying the Use of Popular Destinations to Enhance Web Search Interaction, Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 23-27, 2007, Amsterdam, The Netherlands doi:10.1145/1277741.1277771
36. Biao Xiang, Daxin Jiang, Jian Pei, Xiaohui Sun, Enhong Chen, Hang Li, Context-aware Ranking in Web Search, Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, July 19-23, 2010, Geneva, Switzerland doi:10.1145/1835449.1835525

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2012 EvaluatingtheEffectivenessofSea	Zhen Liao Yang Song Li-wei He Yalou Huang			Evaluating the Effectiveness of Search Task Trails				10.1145/2187836.2187903		2012

2012 EvaluatingtheEffectivenessofSea

Notes

Cited By

Quotes

Abstract

References

Navigation menu

Search