科技报告详细信息
A Novel Traffic Analysis for Identifying Search Fields in the
Forman, George ; Kirshenbaum, Evan ; Rajaram, Shyamsundar
HP Development Company
关键词: web data mining;    clickstream analysis;    machine learning classification;    active learning;   
RP-ID  :  HPL-2010-27
学科分类:计算机科学(综合)
美国|英语
来源: HP Labs
PDF
【 摘 要 】

Using a clickstream sample of 2 billion URLs from many thousand volunteer Web users, we wish to analyze typical usage of keyword searches across the Web. In order to do this, we need to be able to determine whether a given URL represents a keyword search and, if so, which field contains the query. Although it is easy to recognize 'q' as the query field in 'http://www.google.com/search?hl=en&q=music', we must do this automatically for the long tail of diverse websites. This problem is the focus of this paper. Since the names, types and number of fields differ across sites, this does not conform to traditional text classification or to multi-class problem formulations. The problem also exhibits highly non- uniform importance across websites, since traffic follows a Zipf distribution. We developed a solution based on manually identifying the query fields on the most popular sites, followed by an adaptation of machine learning for the rest. It involves an interesting case-instances structure: labeling each website case usually involves selecting at most one of the field instances as positive, based on seeing sample field values. This problem structure and soft constraint - which we believe has broader applicability - can be used to greatly reduce the manual labeling effort. We employed active learning and judicious GUI presentation to efficiently train a classifier with accuracy estimated at 96%, beating several baseline alternatives.

【 预 览 】
附件列表
Files Size Format View
RO201804100002780LZ 393KB PDF download
  文献评价指标  
  下载次数:25次 浏览次数:33次