Jaro–Winkler distance

Short description: String distance measure

In computer science and statistics, the Jaro–Winkler similarity is a string metric measuring an edit distance between two sequences. It is a variant of the Jaro distance metric metric (1989, Matthew A. Jaro) proposed in 1990 by William E. Winkler.

The Jaro–Winkler distance uses a prefix scale [math]\displaystyle{ p }[/math] which gives more favourable ratings to strings that match from the beginning for a set prefix length [math]\displaystyle{ \ell }[/math].

The higher the Jaro–Winkler distance for two strings is, the less similar the strings are. The score is normalized such that 0 means an exact match and 1 means there is no similarity. The original paper actually defined the metric in terms of similarity, so the distance is defined as the inversion of that value (distance = 1 − similarity).

Although often referred to as a distance metric, the Jaro–Winkler distance is not a metric in the mathematical sense of that term because it does not obey the triangle inequality.

Definition

Jaro similarity

The Jaro similarity [math]\displaystyle{ sim_j }[/math] of two given strings [math]\displaystyle{ s_1 }[/math] and [math]\displaystyle{ s_2 }[/math] is

[math]\displaystyle{ sim_j = \left\{ \begin{array}{l l} 0 & \text{if }m = 0\\ \frac{1}{3}\left(\frac{m}{|s_1|} + \frac{m}{|s_2|} + \frac{m-t}{m}\right) & \text{otherwise} \end{array} \right. }[/math]

Where:

• [math]\displaystyle{ |s_i| }[/math] is the length of the string [math]\displaystyle{ s_i }[/math];
• [math]\displaystyle{ m }[/math] is the number of matching characters (see below);
• [math]\displaystyle{ t }[/math] is the number of transpositions (see below).

Jaro similarity score is 0 if the strings do not match at all, and 1 if they are an exact match. In the first step, each character of [math]\displaystyle{ s_1 }[/math] is compared with all its matching characters in [math]\displaystyle{ s_2 }[/math]. Two characters from [math]\displaystyle{ s_1 }[/math] and [math]\displaystyle{ s_2 }[/math] respectively, are considered matching only if they are the same and not farther than [math]\displaystyle{ \left\lfloor\frac{\max(|s_1|,|s_2|)}{2}\right\rfloor-1 }[/math] characters apart. For example, the following two nine character long strings, FAREMVIEL and FARMVILLE, have 8 matching characters. 'F', 'A' and 'R' are in the same position in both strings. Also 'M', 'V', 'I', 'E' and 'L' are within three (result of [math]\displaystyle{ \lfloor\tfrac{\max(9, 9)}{2}\rfloor - 1 }[/math]) characters away. If no matching characters are found then the strings are not similar and the algorithm terminates by returning Jaro similarity score 0.

If non-zero matching characters are found, the next step is to find the number of transpositions. Transposition is the number of matching characters that are not in the right order divided by two. In the above example between FAREMVIEL and FARMVILLE, 'E' and 'L' are the matching characters that are not in the right order. So the number of transposition is one.

Finally, plugging in the number of matching characters [math]\displaystyle{ m }[/math] and number of transpositions [math]\displaystyle{ t }[/math] the Jaro similarity of FAREMVIEL and FARMVILLE can be calculated, [math]\displaystyle{ \frac{1}{3}\left(\frac{8}{9} + \frac{8}{9} + \frac{8-1}{8} \right) = 0.88 }[/math]

Jaro–Winkler similarity

Jaro–Winkler similarity uses a prefix scale [math]\displaystyle{ p }[/math] which gives more favorable ratings to strings that match from the beginning for a set prefix length [math]\displaystyle{ \ell }[/math]. Given two strings [math]\displaystyle{ s_1 }[/math] and [math]\displaystyle{ s_2 }[/math], their Jaro–Winkler similarity [math]\displaystyle{ sim_w }[/math] is:

[math]\displaystyle{ sim_w = sim_j + \ell p (1 - sim_j), }[/math]

where:

• [math]\displaystyle{ sim_j }[/math] is the Jaro similarity for strings [math]\displaystyle{ s_1 }[/math] and [math]\displaystyle{ s_2 }[/math]
• [math]\displaystyle{ \ell }[/math] is the length of common prefix at the start of the string up to a maximum of 4 characters
• [math]\displaystyle{ p }[/math] is a constant scaling factor for how much the score is adjusted upwards for having common prefixes. [math]\displaystyle{ p }[/math] should not exceed 0.25 (i.e. 1/4, with 4 being the maximum length of the prefix being considered), otherwise the similarity could become larger than 1. The standard value for this constant in Winkler's work is [math]\displaystyle{ p = 0.1 }[/math]

The Jaro–Winkler distance [math]\displaystyle{ d_w }[/math] is defined as [math]\displaystyle{ d_w = 1 - sim_w }[/math].

Although often referred to as a distance metric, the Jaro–Winkler distance is not a metric in the mathematical sense of that term because it does not obey the triangle inequality. The Jaro–Winkler distance also does not satisfy the identity axiom [math]\displaystyle{ d(x,y)=0 \leftrightarrow x = y }[/math].

Relationship with other edit distance metrics

There are other popular measures of edit distance, which are calculated using a different set of allowable edit operations. For instance,

• the Levenshtein distance allows deletion, insertion and substitution;
• the Damerau–Levenshtein distance allows insertion, deletion, substitution, and the transposition of two adjacent characters;
• the longest common subsequence (LCS) distance allows only insertion and deletion, not substitution;
• the Hamming distance allows only substitution, hence, it only applies to strings of the same length.

Edit distance is usually defined as a parameterizable metric calculated with a specific set of allowed edit operations, and each operation is assigned a cost (possibly infinite). This is further generalized by DNA sequence alignment algorithms such as the Smith–Waterman algorithm, which make an operation's cost depend on where it is applied.

See also

• Record linkage
• Census

References

• Cohen, W. W.; Ravikumar, P.; Fienberg, S. E. (2003). "A comparison of string distance metrics for name-matching tasks". KDD Workshop on Data Cleaning and Object Consolidation 3: 73–8. https://www.cs.cmu.edu/afs/cs/Web/People/wcohen/postscript/kdd-2003-match-ws.pdf. 
• Jaro, M. A. (1989). "Advances in record linkage methodology as applied to the 1985 census of Tampa Florida". Journal of the American Statistical Association 84 (406): 414–20. doi:10.1080/01621459.1989.10478785. 
• Jaro, M. A. (1995). "Probabilistic linkage of large public health data file". Statistics in Medicine 14 (5–7): 491–8. doi:10.1002/sim.4780140510. PMID 7792443. 
• Winkler, W. E. (1990). "String Comparator Metrics and Enhanced Decision Rules in the Fellegi-Sunter Model of Record Linkage". Proceedings of the Section on Survey Research Methods (American Statistical Association): 354–359. https://files.eric.ed.gov/fulltext/ED325505.pdf. 

• Winkler, W. E. (2006). "Overview of Record Linkage and Current Research Directions". Research Report Series, RRS. https://www.census.gov/content/dam/Census/library/working-papers/2006/adrm/rrs2006-02.pdf. 

External links

• strcmp.c - Original C implementation by the author of the algorithm
• nltk.metrics.distance module- Python implementation in the Natural Language Toolkit