Using dot in full text search works unexpectedly

Support
#1

Using full text search I found that when using dots in the search field, nothing is returned. When enclosing them in quotation marks it works fine. I cannot explain why this would work this way.

Example:
The entity contains a reference like: A.8.1. This reference is something the user would want to search for. When entering A.8.1 in the search field nothing is returned. When entering “A.8.1”, the entity is returned as expected.

Obviously, I would like to have the user be able to find the entity using the plain reference (without the quotation marks).

Also, replacing the dots with underscores works ok.

What does the dot imply on searching? I could not find any remark here: Searching - CUBA Platform. Full Text Search

#2

This happens because lucene IndexWriter in FTS module uses EntityAttributeTokenizer to split that indexed text into tokens, and this tokenizer thinks that dot always separates tokens. We have plans to replace this EntityAttributeTokenizer with StandardTokenizer (the last one will not split words like “A.8.1” into several tokens).
I looked to the code, and unfortunately the part of the FTS module that initializes the IndexWriter is not easily extendable. So, at the moment, there is no easy way to override the standard behavior in your project.

We’ve created issues on both these problems:
https://youtrack.cuba-platform.com/issue/PL-9642
https://youtrack.cuba-platform.com/issue/PL-9636

#3

Hi. I see there is little progress on this issue but I am pressed to somehow solve this issue anyway.

I thought to modify the search results window to add quotes on al terms containing a dot. Somewhat like this:

public class SearchResults extends SearchResultsWindow {

@Override
public void init(Map<String, Object> params) {
    // Override the super init as the "." is not handled well
    //super.init(params);

    fileMetaClass = metadata.getSession().getClassNN(FileDescriptor.class);

    contentLayout = (AbstractOrderedLayout) WebComponentsHelper.unwrap(getComponent("contentBox"));

    String searchTerm = (String) params.get("searchTerm");
    if (StringUtils.isBlank(searchTerm)) {
        initNoSearchTerm();
    } else {
        // Split terms and add quotes for all terms containing a dot
        String[] sTerms = searchTerm.split(" ");
        StringBuilder newSearchTerm = new StringBuilder();
        for (int i=0;i < sTerms.length; i++) {
            if (sTerms[i].contains(".")) {
                sTerms[i] = "\"" + sTerms[i] + "\"";
            }
            newSearchTerm.append(" ").append(sTerms[i]);
        }
        initSearchResults(params, newSearchTerm.toString());
    }
}

But is turns out that this doesn’t help at all as the search is actually performed at the class SearchLauncher class.

Is there any way to override the behaviour of the search launcher to get the behaviour as stated?

#4

Hi,
just create a class that extends the SeachLauncher and register it in your web-screens.xml

<screen id="ftsSearch" class="com.company.ftstest.web.MySearchLauncher"/>

As for the issues, we’ll try to implement them in the next release (6.9)

#5

Hi Max, thanks for following up on this. I actually thought the web-screens.xml only provides the option for overriding screens, not ‘simple’ classes. But it works which is great!

#6

In fact, SearchLauncher is a Callable that returns a Window object. Such classes can be used as screens, as described here: A.screens.xml - CUBA Platform. Developer’s Manual