Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.2
    • Component/s: Import Tools
    • Labels:
      None

      Description

      The addition of normalised call numbers to the index would allow a more accurate sorting of items by call number.

      Bob Haschart has suggested that

      "In SolrMarc in the Utils class there is a method that Naomi Dushay created that takes a LC call number and transforms it into an expanded version that should be directly sortable. So for a random sampling of records with LC call numbers "near" PQ239"

      PQ239.Z56
      PQ239.H63 2008
      PQ239.S62 1982
      PQ239.B68 1983
      PQ2390.S35 A5
      PQ2390.S35 B8 1898
      PQ2389 .R65 F3 1854 t.1
      PQ239.A7 1969
      PQ2.N6 1959
      PQ22.A4 D47 1949
      PQ238.L57 1985

      the expanded sortable version returned by Naomi's routine would be:-

      PQ 0239.000000 Z0.560000
      PQ 0239.000000 H0.630000 002008
      PQ 0239.000000 S0.620000 001982
      PQ 0239.000000 B0.680000 001983
      PQ 2390.000000 S0.350000 A0.500000
      PQ 2390.000000 S0.350000 B0.800000 001898
      PQ 2389.000000 R0.650000 F0.300000 001854 T.000001
      PQ 0002.000000 N0.600000 001959
      PQ 0022.000000 A0.400000 D0.470000 001949
      PQ 0238.000000 L0.570000 001985

      Which is easily sorted to produce the desired ordering.

      I don't know whether there is an existing "standard" indexing function that invokes the CallNumUtils.getLCShelfkey() method that produces the above expanded strings, but it could be easily added as a custom method, or as a scripted method.

      If user entered call numbers were also normalised, it would be possible to perform call number ranged searches. It has been suggested that a custom solr plugin might achieve the latter.
      1. callnumber_normalize.bsh
        1.0 kB
        Alan Rykhus
      2. callnumber_normalize.bsh
        1.0 kB
        Alan Rykhus
      3. callnumber.bsh
        3 kB
        Nathan Tallman
      4. normalizedCallnumber.patch
        3 kB
        Luke O'Sullivan
      1. Solr admin page_20130403-140631.png
        125 kB

        Activity

        Hide
        Demian Katz added a comment -
        What is the exact error you are seeing? It doesn't look like it's intentionally failing when the field is missing, but I think it may need to be rewritten slightly to be more error-tolerant. Perhaps adding:

                if (fieldSpec == null) {
                    return null;
                }

        right after:

                fieldSpec= indexer.getFirstFieldVal(record, fieldSpec);

        would help.
        Show
        Demian Katz added a comment - What is the exact error you are seeing? It doesn't look like it's intentionally failing when the field is missing, but I think it may need to be rewritten slightly to be more error-tolerant. Perhaps adding:         if (fieldSpec == null) {             return null;         } right after:         fieldSpec= indexer.getFirstFieldVal(record, fieldSpec); would help.
        Hide
        Nathan Tallman added a comment -
        Below is the error message. Adding what you suggest will probably do the trick.

        Nov 13, 2013 12:50:49 AM org.solrmarc.marc.MarcImporter importRecords
        SEVERE: Unable to index record vtls000027057 (record count 27582) -- Error while trying to evaluate script: callnumber.bsh
        java.lang.IllegalArgumentException: Error while trying to evaluate script: callnumber.bsh
        at org.solrmarc.index.SolrIndexer.handleScript(SolrIndexer.java:1170)
        at org.solrmarc.index.SolrIndexer.addFieldValueToMap(SolrIndexer.java:914)
        at org.solrmarc.index.SolrIndexer.map(SolrIndexer.java:821)
        at org.solrmarc.marc.MarcImporter.addToIndex(MarcImporter.java:399)
        at org.solrmarc.marc.MarcImporter.importRecords(MarcImporter.java:313)
        at org.solrmarc.marc.MarcImporter.handleAll(MarcImporter.java:607)
        at org.solrmarc.marc.MarcImporter.main(MarcImporter.java:867)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:622)
        at com.simontuffs.onejar.Boot.run(Boot.java:334)
        at com.simontuffs.onejar.Boot.main(Boot.java:170)
        Caused by: Typed variable declaration : Method Invocation CallNumUtils.getLCShelfkey : at Line: 29 : in file: inline evaluation of: ``/** * Custom call number script. * * This can be used to override built-in So . . . '' : CallNumUtils .getLCShelfkey ( fieldSpec , recordID )

        Target exception: java.lang.NullPointerException

        at bsh.BSHMethodInvocation.eval(Unknown Source)
        at bsh.BSHPrimaryExpression.eval(Unknown Source)
        at bsh.BSHPrimaryExpression.eval(Unknown Source)
        at bsh.BSHVariableDeclarator.eval(Unknown Source)
        at bsh.BSHTypedVariableDeclaration.eval(Unknown Source)
        at bsh.BSHBlock.evalBlock(Unknown Source)
        at bsh.BSHBlock.eval(Unknown Source)
        at bsh.BSHBlock.eval(Unknown Source)
        at bsh.BSHIfStatement.eval(Unknown Source)
        at bsh.BSHBlock.evalBlock(Unknown Source)
        at bsh.BSHBlock.eval(Unknown Source)
        at bsh.BshMethod.invokeImpl(Unknown Source)
        at bsh.BshMethod.invoke(Unknown Source)
        at bsh.BshMethod.invoke(Unknown Source)
        at org.solrmarc.index.SolrIndexer.handleScript(SolrIndexer.java:1149)
        ... 12 more
        Nov 13, 2013 12:50:49 AM org.solrmarc.marc.MarcImporter importRecords
        SEVERE: ******** Halting indexing! ********
        Show
        Nathan Tallman added a comment - Below is the error message. Adding what you suggest will probably do the trick. Nov 13, 2013 12:50:49 AM org.solrmarc.marc.MarcImporter importRecords SEVERE: Unable to index record vtls000027057 (record count 27582) -- Error while trying to evaluate script: callnumber.bsh java.lang.IllegalArgumentException: Error while trying to evaluate script: callnumber.bsh at org.solrmarc.index.SolrIndexer.handleScript(SolrIndexer.java:1170) at org.solrmarc.index.SolrIndexer.addFieldValueToMap(SolrIndexer.java:914) at org.solrmarc.index.SolrIndexer.map(SolrIndexer.java:821) at org.solrmarc.marc.MarcImporter.addToIndex(MarcImporter.java:399) at org.solrmarc.marc.MarcImporter.importRecords(MarcImporter.java:313) at org.solrmarc.marc.MarcImporter.handleAll(MarcImporter.java:607) at org.solrmarc.marc.MarcImporter.main(MarcImporter.java:867) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:622) at com.simontuffs.onejar.Boot.run(Boot.java:334) at com.simontuffs.onejar.Boot.main(Boot.java:170) Caused by: Typed variable declaration : Method Invocation CallNumUtils.getLCShelfkey : at Line: 29 : in file: inline evaluation of: ``/** * Custom call number script. * * This can be used to override built-in So . . . '' : CallNumUtils .getLCShelfkey ( fieldSpec , recordID ) Target exception: java.lang.NullPointerException at bsh.BSHMethodInvocation.eval(Unknown Source) at bsh.BSHPrimaryExpression.eval(Unknown Source) at bsh.BSHPrimaryExpression.eval(Unknown Source) at bsh.BSHVariableDeclarator.eval(Unknown Source) at bsh.BSHTypedVariableDeclaration.eval(Unknown Source) at bsh.BSHBlock.evalBlock(Unknown Source) at bsh.BSHBlock.eval(Unknown Source) at bsh.BSHBlock.eval(Unknown Source) at bsh.BSHIfStatement.eval(Unknown Source) at bsh.BSHBlock.evalBlock(Unknown Source) at bsh.BSHBlock.eval(Unknown Source) at bsh.BshMethod.invokeImpl(Unknown Source) at bsh.BshMethod.invoke(Unknown Source) at bsh.BshMethod.invoke(Unknown Source) at org.solrmarc.index.SolrIndexer.handleScript(SolrIndexer.java:1149) ... 12 more Nov 13, 2013 12:50:49 AM org.solrmarc.marc.MarcImporter importRecords SEVERE: ******** Halting indexing! ********
        Hide
        Demian Katz added a comment -
        Yeah, I think that should help -- if it works, please upload an updated .bsh for future reference! Thanks for your help!
        Show
        Demian Katz added a comment - Yeah, I think that should help -- if it works, please upload an updated .bsh for future reference! Thanks for your help!
        Hide
        Nathan Tallman added a comment -
        Updated callnumber.bsh from patch, updated to handle records without a 099:090:050 field.
        Show
        Nathan Tallman added a comment - Updated callnumber.bsh from patch, updated to handle records without a 099:090:050 field.
        Hide
        Demian Katz added a comment -
        I have added the getFullCallNumberNormalized routine to callnumber.bsh and have also committed it to the SolrMarc trunk so that it will be available in the next SolrMarc release (2.7). This isn't currently being used for anything in the default configuration -- it's just available as an option. We should consider the options offered by VUFIND-657 before making schema/default indexing changes.
        Show
        Demian Katz added a comment - I have added the getFullCallNumberNormalized routine to callnumber.bsh and have also committed it to the SolrMarc trunk so that it will be available in the next SolrMarc release (2.7). This isn't currently being used for anything in the default configuration -- it's just available as an option. We should consider the options offered by VUFIND-657 before making schema/default indexing changes.

          People

          • Assignee:
            Demian Katz
            Reporter:
            Luke O'Sullivan
          • Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: