VuFind

Edgecase search strings breaking search (empty parenteses; parentheses with colon)

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 1.0RC1, 1.0RC2
  • Fix Version/s: 1.0
  • Component/s: Search
  • Labels:
    None
  • Environment:
    r1777

Description

The following edgecase combinations result in an error:

1) An empty set of parenteses anywhere outside of a phrase search. Example- hello world ()

Error:
Unable to process query
Solr Returned: org.apache.lucene.queryParser.ParseException: Cannot parse '((title_short:(hello world ())^750 OR title_full:(hello world ())^400 OR title:(hello world ())^300 OR title:(hello world ())^250 OR title_alt:(hello world ())^200 OR title_new:(hello world ())^100)^50 OR series:(hello world ())^50 OR series2:(hello world ())^30 OR author:(hello world ())^300 OR author:(hello world ())^250 OR author2:(hello world ())^50 OR author_additional:(hello world ())^50 OR contents:(hello world ())^10 OR topic:(hello world ())^500 OR geographic:(hello world ())^300 OR genre:(hello world ())^300 OR allfields:(hello world ()))': Encountered " ")" ") "" at line 1, column 28. Was expecting one of: <NOT> ... "+" ... "-" ... "(" ... "*" ... <QUOTED> ... <TERM> ... <PREFIXTERM> ... <WILDTERM> ... "[" ... "{" ... <NUMBER> ... <TERM> ... "*" ...



2) Combination of a colon (anywhere) with a set of parentheses (empty or not empty). Example- hello : (world)

Error:
Unable to process query
Solr Returned: undefined field hello

RequestURI=/solr/biblio/select/

Issue Links

Activity

Hide
Greg Pendlebury added a comment -
We've got several other examples in our log files as well. Things like opening quotations but not closing them : "test

For us at least this is unfortunately common as students get sloppy in doing a copy/paste from course material and miss one of the quotes.

In particular the advanced (ie. non DisMax) searches are more trouble. Perhaps it's something that needs to update during search parsing/tokenisation?

Worth upgrading to Major too.
Show
Greg Pendlebury added a comment - We've got several other examples in our log files as well. Things like opening quotations but not closing them : "test For us at least this is unfortunately common as students get sloppy in doing a copy/paste from course material and miss one of the quotes. In particular the advanced (ie. non DisMax) searches are more trouble. Perhaps it's something that needs to update during search parsing/tokenisation? Worth upgrading to Major too.
Hide
Demian Katz added a comment -
The improved parse error detection in r1959 offers a cleaner response to most of these bad queries, though the colon issue is still unresolved. I'll work on that momentarily.
Show
Demian Katz added a comment - The improved parse error detection in r1959 offers a cleaner response to most of these bad queries, though the colon issue is still unresolved. I'll work on that momentarily.
Hide
Demian Katz added a comment -
As of r1960, I believe that all issues listed on this page are covered by the parse error detection code. If you find any new search queries that cause fatal errors or if you feel that any specific types of bad queries need different special handling, please open new tickets to address the issues.
Show
Demian Katz added a comment - As of r1960, I believe that all issues listed on this page are covered by the parse error detection code. If you find any new search queries that cause fatal errors or if you feel that any specific types of bad queries need different special handling, please open new tickets to address the issues.
Hide
Bill McClendon added a comment -
All,

If it is fixed, I don't see how. Either that, or the approach per site is not documented. There appears to be some custom "munge" capability. But with no documentation, I can;t tell if that would cover this issue or not.

All search strings that contain a colon return no results. MANY titles contain a colon and the autocomplete feature offers them up and you may simply select them. You then get no results.

Note: There is no error or failure. It just fails to properly search

We are far past r1960 and have this issue. We are currently on 1.1
Show
Bill McClendon added a comment - All, If it is fixed, I don't see how. Either that, or the approach per site is not documented. There appears to be some custom "munge" capability. But with no documentation, I can;t tell if that would cover this issue or not. All search strings that contain a colon return no results. MANY titles contain a colon and the autocomplete feature offers them up and you may simply select them. You then get no results. Note: There is no error or failure. It just fails to properly search We are far past r1960 and have this issue. We are currently on 1.1
Hide
Demian Katz added a comment -
The problem solved on this ticket relates specifically to colons in combination with parentheses. Colons in and of themselves are problematic, since VuFind tries to treat them as Solr field specifiers, which isn't always the desired behavior.

Could you open a new ticket and provide some specific examples of searches that fail? I'm surprised that autocomplete suggestions are causing problems, because colons are only treated as field specifiers if they are not followed by a space -- i.e. field:keyword, not field: keyword -- and MARC records normally include spaces after colons.

If your titles have spaces around colons and searching is still failing, that's a bug that we need to fix.

If your titles do not have spaces around colons, maybe we need to add a configurable option to disable field specifiers in searches -- in general, it seems to cause more trouble than it's worth, but I don't want to simply take away the option since it has been part of VuFind for a long time and some people may actually use it.
Show
Demian Katz added a comment - The problem solved on this ticket relates specifically to colons in combination with parentheses. Colons in and of themselves are problematic, since VuFind tries to treat them as Solr field specifiers, which isn't always the desired behavior. Could you open a new ticket and provide some specific examples of searches that fail? I'm surprised that autocomplete suggestions are causing problems, because colons are only treated as field specifiers if they are not followed by a space -- i.e. field:keyword, not field: keyword -- and MARC records normally include spaces after colons. If your titles have spaces around colons and searching is still failing, that's a bug that we need to fix. If your titles do not have spaces around colons, maybe we need to add a configurable option to disable field specifiers in searches -- in general, it seems to cause more trouble than it's worth, but I don't want to simply take away the option since it has been part of VuFind for a long time and some people may actually use it.
Hide
Demian Katz added a comment -
One other thought -- implementing the Extended Dismax handler (VUFIND-466) might also help with this problem.
Show
Demian Katz added a comment - One other thought -- implementing the Extended Dismax handler (VUFIND-466) might also help with this problem.

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: