tdebatty / java-string-similarity Goto Github PK

Similar to #59 and #19, I decided to make a pure Kotlin port of this library.

Although on the server side, Kotlin has compat with Java libraries, I wanted to make it a Multiplatform lib because why not.

Currently there is no README, but when I finish it, I'm going to mention this repo in the README.

The port is here, under kt-string-similarity: https://github.com/solo-studios/kt-fuzzy

These classes are not the latest release:

info.debatty.java.stringsimilarity.KShingling;
info.debatty.java.stringsimilarity.StringProfile;

Hello!
Can the java-string-similarity be applied to many languages such as Germany,French,Chinese and so on.Can it be is relative to language type?

The review of API changes for the Java String Similarity library since 0.1 version: https://abi-laboratory.pro/java/tracker/timeline/java-string-similarity/

The report is updated 3 times a week. Hope it will be helpful for users and maintainers of the library.

The report is generated by the https://github.com/lvc/japi-tracker tool for jars at http://central.maven.org/maven2/info/debatty/java-string-similarity/ according to the article https://wiki.eclipse.org/Evolving_Java-based_APIs_2.

Thank you.

@override annotation in NGram.java shows error in Java 6 and above. It should be removed as the method is not being overridden. It is just being implemented.

I am using this library in an Apache Spark application (using scala).

I have been seeing variable results using the NGram algorithm where exact matches result is either "0.0" or "1.0". Below are some examples.

`QGram dig = new QGram(2);

dig.distance("S","S")
//result = 1.0

dig.distance("Kirk","Kirk")
//result = 0.0

dig.distance("07426796542","07426796542")
//result = 0.0`

Should all these examples not result in a score of 1.0 as they are exactly the same?

Hi here,
I want to use this algorithm for a problem i'm trying to solve between and inventory excel and images related to the products inside the excel spreedsheet so i'm working into my own code in Netbeans 12.5 with Java but i'm not able to use them i tried to clone the code and the librarie into the project creat and object
Cosine cosine = new Cosine(2);
for example but there is no reference for importing the class. i tried also to make an import of the package
import src.test.java.info.debatty.java.*;
but still not working. I am not a full Java developer , i just trying to solve a problem but with Software, if you can give a better guide of how to implement it, much appreciate it.

🙏🙏🙏

Hi Team,

Can anyone imagine or already get any problem to use a that lib (RatcliffObershelp specificly) in a Singleton class?

My use :

import info.debatty.java.stringsimilarity.RatcliffObershelp;

public class StringSimilarityCalculator {

    private static StringSimilarityCalculator stringSimilarityCalculator;
    private static RatcliffObershelp calculo = new RatcliffObershelp();
    private static final double minSimilaridadeEndereco = 0.90;

    private StringSimilarityCalculator(){}

    public static StringSimilarityCalculator getInstance() {

        if(stringSimilarityCalculator == null){

            stringSimilarityCalculator = new StringSimilarityCalculator();
        }

        return stringSimilarityCalculator;
    }

    public boolean isStringEnderecoSimilar(String enderecoPagador, String enderecoPagadorBanco){

        return calculo.similarity(enderecoPagador.toLowerCase(),
                enderecoPagadorBanco.toLowerCase()) >= minSimilaridadeEndereco;
    }
}

Thanks !!

I have not found such perfect package on npm, so, could I translate this package from Java to TypeScript?

This is the semi-finished repo. Please look it over: https://github.com/hellojayjay/string-metric

Will you allow me to do so?

I am trying to use jaro wrinkler similarity to check colors strings coming from user inputted form against a palette of fixed colors.

Using jaro wrinkler similarity, I get these kind of results for very short strings:

• s1 = "ed" - s2 = "red" -> similarity = 0
• s1 = "nude" - s2 = "red" -> similarity = 0.5833333134651184

Is it correct to get similarity = 0 in the first case?

Hi. I noticed something whilst browsing this git...

The readme states that the triangle inequality does not hold for the Demerau-Levenshtein algorithm. This is only true if you use a restricted edit distance implementation of the algorithm. If you allow for the 'unrestricted' case (of adjacent transpositions) the inequality will hold. This is, however, at the expense of added computational complexity. See here

Implementing the adjacent transposition calc will allow you to use Demerau-Levenshtein in a metric tree, for example.

Cheers

Chris

Have I tried to build the library after downloading on my machine using maven, it failed and there an error in NGram.java distance() about overriding. I am not sure how to resolve this. Let me know how to execute the example you mentioned in read.me. Thank you.

Where is the SparseIntegerVector (and related) code?

There is a library which has an accurate implementation but its based on scala.

https://github.com/tdebatty/java-string-similarity/blob/master/src/main/java/info/debatty/java/stringsimilarity/KShingling.java

In this way precomputing profiles could happen in parallel:
https://github.com/tdebatty/java-string-similarity/blob/master/src/main/java/info/debatty/java/stringsimilarity/examples/PrecomputedCosine.java

Would it be possible for you to add a column called "Typical Uses" in the table in Overview? Perhaps to briefly mention where the algorithm is typically used (for example "diff utility, Git reconciliation" for LCS).

Not sure if you will have or can find this info. It will be helpful to pick out the algorithm best suited for the user's needs.

Can you attach a license to this project? I'm interested in using and possibly contributing, but without a license, I can't be sure of your intentions.

I've created a port of your library to the .NET Framework using C#. The repository for the port is here: https://github.com/feature23/StringSimilarity.NET

Aside from changes to C# syntax, I made a few changes to make the code a bit more idiomatic for the language, but it's as true to the original as possible. Also I have, of course, linked back here to your original library. In any case, I just wanted to make you aware of the port, and please let me know if you have any interested in collaborating!

Hi,

When using jaro-winkler method I see that when comparing empty strings '' and '' the score return is 1, indicating a complete mismatch.

here is an example below implemented in scala

val jaro = new JaroWinkler()
println(jaro.distance("",""))
println(jaro.distance("match","match"))

Can this be altered so that when comparing empty strings it returns an exact match of 0?

Thanks

SSK is a kernel method for measuring the similarity between strings.
Any implementations?
Hope to see.

Hi,

Could you please document the if NormalizedLevenshtein or other implementation class is thread safe or not.

Thanks

Hello,

I just tried cosine similarity and the result was above 1!
My code is attached.

ConsineSimilarity.zip

Can you check the sections about the Jaccard index and the Sorensen-Dice coefficient in the README file? They both report the distance as computed as 1 - cosine similarity.

I was trying to look at the Coveralls data here and it can't show the source code because it says the repo owner must re-authorize with GitHub:

SOURCE NOT AVAILABLE
The owner of this repo needs to re-authorize with github; their OAuth credentials are no longer valid so the file cannot be pulled from the github API.

I am using jaro wrinkler similarity to check similarities between names. In one of the use case, i found this issue.
s1 = "SOME NAME" - s2 = "" -> similarity = 1
Why is the output "1". shouldn't "1" be for exact matches ?
please help

version details : java-string-similarity -> 2.0.0

We are working on Record linkage project. We are observing a strange behavior from all of the standard technique like Jaro Winkler, Levenshtein, N-Gram, Damerau-Levenshtein, Jaccard index, Sorensen-Dice
which we used from your repository.

Say,
String 1= MINI GRINDER KIT
String 2= Weiler 13001 Mini Grinder Accessory Kit, For Use With Small Right Angle Grinders
String 3= Milwaukee Video Borescope, Rotating Inspection Scope, Series: M-SPECTOR 360, 2.7 in 640 x 480 pixels High-Resolution LCD, Plastic, Black/Red

In the above case string 1 and string 2 are related the score of all the methods as shown below.
Jaro Winkler -> 0.391666651
Levenshtein -> 75
N-Gram, -> 0.9375
Damerau -> 75
Jaccard index -> 0
Sorensen-Dice -> 0
Cosine -> 0

But string 1 and string 3 are not at all related, but distance method are giving very high score.
Jaro Winkler -> 0.435714275
Levenshtein -> 133
N-Gram, -> 0.953571439
Damerau -> 133
Jaccard index -> 1
Sorensen-Dice -> 0
Cosine -> 0

Any thoughts .?

1. Would you explain why you decided not to make StringDistance.distance() static? Why is it necessary to create an instance of StringDistance (or of a derived class) to compute the distance? The method does not use this anyway...

2. Also, why StringDistance derives from java.io.Serializable?

All of the metrics set a distance of 0 to imply exact string match and 1 to imply distance strings. Ngram.distance is backwards and returns 1 for an exact match and 0 for distant strings.

java-string-similarity/src/main/java/info/debatty/java/stringsimilarity/Jaccard.java:

        Map<String, Integer> profile1 = getProfile(s1);
        Map<String, Integer> profile2 = getProfile(s2);

        Set<String> union = new HashSet<String>();
        union.addAll(profile1.keySet());
        union.addAll(profile2.keySet());

        int inter = 0;

        for (String key : union) {
            if (profile1.containsKey(key) && profile2.containsKey(key)) {
                inter++;
            }
        }

        return 1.0 * inter / union.size();

can be modified to:

        Map<String, Integer> profile1 = getProfile(s1);
        Map<String, Integer> profile2 = getProfile(s2);

        Set<String> union = new HashSet<String>();
        union.addAll(profile1.keySet());
        union.addAll(profile2.keySet());

        double x = profile1.size();
        double y = profile2.size();
        double z = union.size();
        return (x + y - z) / z;

This issue was reported to my team's .NET port of your library but I confirmed that it is an issue here as well.

The example on the README shows that the N-Gram code expects values of 0.416666 and 0.97222. However, different results are given when the code is actually ran. I am not sure if this is a bug in the code, or that the README comment is outdated/incorrect.

I created a unit test for the README example, and sure enough it fails:

    @Test
    public void exampleFromReadme() {
        // produces 0.416666
        NGram twogram = new NGram(2);
        assertEquals(0.416666, twogram.distance("ABCD", "ABTUIO"), 0.001);

        // produces 0.97222
        String s1 = "Adobe CreativeSuite 5 Master Collection from cheap 4zp";
        String s2 = "Adobe CreativeSuite 5 Master Collection from cheap d1x";
        NGram ngram = new NGram(4);
        assertEquals(0.97222, ngram.distance(s1, s2), 0.001);
    }

Results:

java.lang.AssertionError: 
Expected :0.416666
Actual   :0.5833333134651184

This result (0.583) is the same that we get on the .NET side of things. As I am not an expert in these algorithms, I am unsure if this is a code bug or a need to update the README.

Hello!
Can the java-string-similarity be applied to many languages such as Germany,French,Chinese and so on.Can it be is relative to language type?

Use-case: Searching for a best match

Given a string Q, and a list of strings L, I want to find the distance between Q and every element of L and then pick the element with the shortest distance.

Current solution

The current solution is to call a distance function with two String parameters. This function assumes nothing about the Strings and hence some information is recomputed every time (for example, profiles of a string).

Proposal

Two additional APIs can be provided to improve performance:

   Profile getProfile(String)

   double distance(Profile p1, Profile p2)

For convenience, a third API would also be useful:

  double distance(Profile p, String s) {
    return distance(p, getProfile(s));
  }

This can be used like this:

  String query = "alex";
  QGram qg = new QGram(2);
  Profile queryProfile = qg.getProfile(query);
  list forEach { element ->
    println(qg.distance(queryProfile, element));
  }

Further, if the list is going to be persistent, it could also be possible to serialize the profile of each element of the list into the persistent store. Then, both the query string and the list element's profile need not be recomputed every time!

If the overall idea sounds good to you, I will write more about how the Profile type could be made type safe across the different implementations of StringSimilarityInterface.

Currently the latest version on maven central is 1.1.0
The latest released version in 1.2.0 according to releases page.

Thanks

Starting from this build, the build command of mvn clean cobertura:cobertura coveralls:report in .travis.yml failed. Hence the coveralls stats was not updated since this commit.

Error message:
[ERROR] Failed to execute goal org.codehaus.mojo:cobertura-maven-plugin:2.7:instrument (default-cli) on project java-string-similarity: Execution default-cli of goal org.codehaus.mojo:cobertura-maven-plugin:2.7:instrument failed: Plugin org.codehaus.mojo:cobertura-maven-plugin:2.7 or one of its dependencies could not be resolved: Could not find artifact com.sun:tools:jar:0 at specified path /usr/local/lib/jvm/openjdk11/../lib/tools.jar -> [Help 1]

Turned out it was because the cobertura-maven-plugin is not supporting java 8 and beyond. False alarm? Consider JaCoCo instead?

I could swear I successfully used version 0.15 before, whereas right now the latest published version seems to be 0.13. Am I only hallucinating? :)

"An Improved Algorithm for Matching Biological Sequences", Gotoh (1982):
The algorithm of Waterman et al. (1976) for malchirlg biological seyuences was modified under some limitations to be accomplished in essentially MN steps, instead of the M²N steps necessary in the original algorithm.
http://www.genome.ist.i.kyoto-u.ac.jp/~aln_user/archive/JMB82.pdf

In README paragraph:
https://github.com/tdebatty/java-string-similarity#shingle-n-gram-based-algorithms
There are import of classes that does not exist already:
import info.debatty.java.stringsimilarity.KShingling;
import info.debatty.java.stringsimilarity.StringProfile;

I've made a java port of the sift4 common algorithm. https://siderite.blogspot.com/2014/11/super-fast-and-accurate-string-distance.html

Is there any interest in adding it here?

传说中的字符串编辑距离