PreJoin: An Efficient Trie-based String Similarity Join Algorithm
8th International Conference on Informatics and Systems (INFOS) • 2012
Publication Information
Authors
Karam Gouda and Metwally Rashad
Keywords
Not Available
Journal
8th International Conference on Informatics and Systems (INFOS)
Publisher
ieeexplore
Volume
Not Available
Issue
Not Available
Pages
37-43
publication.type
International
Paper Link
Open Link
Supplementary Materials
Not Available
Abstract
A string similarity join finds all similar pairs between two collections of strings. It is an essential operation in many applications, such as data integration and cleaning, and has attracted significant attention recently. In this paper, we study string similarity joins with edit-distance constraints. Recently, a Trie-based similarity Join framework is proposed [3]. Existing Trie-based Join algorithms have shown that Trie Indexing is more suitable for Similarity Join on short strings. The main problem with current approaches is that they generate and maintain lots of candidate prefixes called active nodes which need to be further removed. With large edit distance, the number of active nodes becomes quite large. In this paper, we propose a new Trie-based Join algorithm called PreJoin, which improves over current Trie- based Join methods. It efficiently finds all similar string pairs using a new active-node set generation method, and a dynamic preorder traversal of the Trie index. Experiments show that PreJoin is highly efficient for processing short as well as long strings, and out performs the state-of-the-art Trie-based Join approaches by a factor five.
Staff Members - Benha University