효과적인 중복 포스트 검출 기법

초록

Recently, duplications of blog posts increase with invigoration of blogsphere. Identifying and filtering duplicates are major concerns of web search engines because users do not want duplicates to appear in their search results and indexing of duplicates not only incurs waste of storage space but also increases search time. In addition, it gives rise to copyright infringement. In this paper, we propose a method for identifying duplicate posts in large blog-database environment. Our method effectively makes it possible to exclude the duplicate posts from the index for blog post searches. We show the effectiveness of the proposed method via extensive experiments.

키워드