Fixed to work correctly with Cyrillic symbols.

[injDakov]

No description provided.

[JakeBayer]

Hello,

Thanks for your contribution! I would like to know how exactly you are using FuzzySharp such that you are running into issues. I need to better understand how users are encountering issues so that I am better equipped to solve them.

If you are using the "Extract" methods from the Process.cs class, you can provide your own "string processor". If you omit this parameter, it will by default choose the "DefaultPreProcessor", which will remove all characters not matched by alphanumeric characters in the standard English Alphabet and then call ToLower() on them.

However, if you provide your own processor function, it will use that processor function instead and skip using the "Default" one provided by FuzzySharp. This way you can define a string processor for any character set you want.

For instance, you can use var results = Process.ExtractAll("similar", new [] {"very similar", "v simliar", "super similar"}, (s) => { return Regex.Replace(s, "[^ a-zA-Z0-9а-зА-З]", " ").ToLower().Trim(); });

Let me know if this solves your issue. I will update the README to include a section regarding this issue

[injDakov]

Hi Jake,
I use the amount of Cyrillic strings and then I run into issues.

My workaround is similar to your suggested.

method ()
{
...
var results = Process.ExtractAll(queryName, namesList, input => ProcessorFunction(input), cutoff: cutoffValue).ToList();
...
}
private static string ProcessorFunction(string input)
{
input = Regex.Replace(input, "[^ a-zA-Z0-9а-яА-Я]", " ");
input = input.ToLower();
return input.Trim();
}

But my opinion is that the native library without workaround should support Latin and Cyrillic alphabet.

[ahamidou]

This is interesting and good initiative.
I think a better way to do this is by updating the PreprocessMode enum to accept, the enum is confusing and Full vs None does not make much sense.
Also flags makes sense in case I'm working with more than one language.
I propose the following:

[Flags]
public enum PreprocessMode
    {
        NotSet = 0,
        English = 1,
        Russian = 2,
        Gibberish = 5 
    }

Then here, in this method use the correct pattern(s).
If PreprocessMode==1 then pattern = "[^ a-zA-Z0-9]"; // English
If PreprocessMode==2 then pattern = "[^а-зА-З0-9]"; // Russian
If PreprocessMode==3 then pattern = "[^a-zA-Z0-9а-зА-З]"; //Both English & Russian

Finally, even the name PreprocessMode isn't very descriptive, maybe LanguageProcessor or something like that would be a better name.