[VUFIND-382] Add a timeline graph/visualisation for Publication dates Created: 16/Feb/11 Updated: 19/Sep/11 Resolved: 19/Sep/11 |
|
| Status: | Resolved |
| Project: | VuFind® |
| Components: | Search, User Interface |
| Affects versions: | None |
| Fix versions: | 1.3 |
| Type: | New Feature | Priority: | Minor |
| Reporter: | Eoghan Ó Carragáin | Assignee: | Till Kinstler |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Environment: | All | ||
| Attachments: |
flot-0.1.patch
flot-0.2.patch
flot-0.3.patch
flot-0.4.patch
flot-0.5.patch
flot-0.6.patch
|
| Description |
|
Add a timeline visualisation for publication dates. This could possibly by done as recommendation module which makes a second call to Solr. Examples in the wild: https://blacklight.mse.jhu.edu/demo http://finden.nationallizenzen.de/Search/Results?lookfor=Joyce&submit=Finden http://openlibrary.org/subjects/steam-engines More info/discussion: http://bibwild.wordpress.com/2010/07/29/cool-range-limitprofile-function-in-blacklight/ http://code.google.com/p/flot/ http://www.simile-widgets.org/timeline/ Related JIRA tickets: |
| Comments |
| Comment by Till Kinstler [ 17/Feb/11 ] |
|
Here is what we did in Suchkiste: In the SearchObject class we added a method getDateRange that sends a search request to Solr asking for an unlimited number of facet values for the publishDate field sorted lexicographically (which is: by year). That method is called in services/Search/Results.php, so we do an additional search there. We need to do that additional search, because we want the date distribution for the user query without filters applied and an unlimited number of facet values sorted by year, not by facet count. The result of that search is passed to the Smarty frontend, where it is displayed using am ugly combination of jQuery flot (http://code.google.com/p/flot/) and jQuery Slider (http://docs.jquery.com/UI/Slider). We pass the values to flot by embedding them in a Javascript variable right in the HTML source code. Not nice, blows up the page and nothing is displayed until the second search is finished... Might be worth doing this in an AJAX call. We played with SVG to plot the diagram as well, but browser support for SVG is still too much a mess (or was about half a year ago)... We had some issues with flot, for example we first had the graph in a scalable user interface (no fixed page width) and got lots of display errors when resizing the browser window. Now we have a fixed width layout (for other reasons as well), and flot works reasonably. I'd be happy to contribute that into mainline VuFind, but I first should have a look at current code in that area to see how it might fit in. Or, if you have ideas, just speak up... |
| Comment by Till Kinstler [ 21/Feb/11 ] |
| Started working on that |
| Comment by Till Kinstler [ 21/Feb/11 ] |
|
The attached patch (flot-0.1.patch) adds a recommendations module to the top of search result lists that plots the publication years in the search result. The patch uses jQuery flot (http://code.google.com/p/flot/) for the blueprint theme and yui-flot (https://github.com/bluesmoon/yui-flot/wiki) for the default and classic themes to display the plot. The data to be plotted is fetched in a new method getDateRange() which is in the Solr SearchObject (sys/SearchObjects/Solr.php). I am not sure, if that is the right place for it. Please comment. Obviously, this only works for Solr backends at the moment. I have no idea, if it is possible to get the needed data out of Summon or WorldCat. Because I haven't changed the SearchObject classes for Summon and WorldCat, VuFind will crash if you use them, because they don't have a method getDateRange()... I didn't bother about that, as long as I don't know, if the SearchObject is the right place for that method. I didn't put much effort into the optical style of the plot, designers feel free to make it nice. The javascript that calls flot is placed right in the Smarty template files at the moment, including all the data necessary to plot the graph. Comments on how to change that are very welcome. Might call an external JS file that is generated on the fly with the data? In theory, it should be possible to have plots for several different Solr index fields. But I haven't tested that, yet. |
| Comment by Demian Katz [ 22/Feb/11 ] |
|
A few comments on the patch: 1.) Would it be better to load the library prerequisites in TopPubDateVis.tpl rather than layout.tpl so that they are not always present if the plug-in is disabled? 2.) You should be able to use {js} tags inside the <!--[if IE]> blocks -- you don't have to use hard-coded <script> tags there. 3.) It probably wouldn't hurt to load the data via AJAX instead of embedding it in the templates since the functionality is all dependent on Javascript anyway. 4.) Rather than having a dateVis setting in facets.ini, would it be better to just pass the array through as a delimited list on the recommendation entry in searches.ini? 5.) I think the SearchObject is the right place for the getPublicationDates method -- it should be possible to implement this for Summon as well as Solr, though I have my doubts about WorldCat. Perhaps the recommendation module should be smart enough to fail gracefully if no getPublicationDates method is defined on the SearchObject that it receives. I'll actually try running the code in the near future and may have more comments after that... but these are my initial thoughts after simply looking at the code. Thanks for taking the time to contribute this! |
| Comment by Demian Katz [ 22/Feb/11 ] |
| One other minor issue -- I notice that the caption on the graph is the name of the facet field (i.e. publishDate) rather than the corresponding description (i.e. "Year of Publication"). |
| Comment by Till Kinstler [ 23/Feb/11 ] |
|
Thanks for the comments. The main thing is to make this work through AJAX, yes. And, if there are multiple visualisations for different index fields, to fetch the data in only one additional search. Another nice feature would be different visualisation types for different data, eg. pie charts for language, authors, formats etc... flot only does bar and line graphs, so another data visualisation library would be necessary... I looked around a little bit, there are some nice ones, eg. jQuery Visualize (http://filamentgroup.com/lab/update_to_jquery_visualize_accessible_charts_with_html5_from_designing_with/) ... Someone having experience with any of these? OK, but first stick to flot and get the data transfer AJAXed... |
| Comment by Till Kinstler [ 24/Feb/11 ] |
|
New patch for current trunk (flot-0.2.patch), first remove previous one... Facet values are now AJAXed. JS files now loaded from recommender templates added check for Solr SearchObject configuration still in searches,yaml and facets.ini. No real idea, how to refactor that, yet. translation of field names into headings for recomender modules not yet fixed, depends on where configuration will go... Problem: When defining more than one visualisation, the selection in the YUI-flot based implementations for default and classic theme fails. Values are always picked from the last plot object on the page?!? Have to investigate the event handling in YUI-flot, I guess... |
| Comment by Till Kinstler [ 25/Feb/11 ] |
|
Bug: $this->index is not set in getFullFieldFacets in SearchObject/Solr.php. Fix will be included in next patch. |
| Comment by Till Kinstler [ 17/Mar/11 ] |
|
I am looking for some ideas, where to put configuration for this. I agree with Demian, that it is not necessary to split configuration between two files (searches.yaml and facets.ini). On the other hand, translation of facet field names to display strings is handled in facets.ini, so we need to read that anyway. But I agree, the setting of fields to be visualised in facets.ini breaks a bit the concept there and is not necessary. So how to do it? Translation of the facet field name is a second issue. It would be convenient to use the existing SearchObject->getFacetLabel method, but it is a protected method of SearchObject, and I would access it from the recommender module Object, not from within a SearchObject... |
| Comment by Demian Katz [ 18/Mar/11 ] |
| I think the simplest solution is that, rather than specifying a facets.ini file to load as the parameter to the recommendations module, instead provide a colon-separated list of fields to visualize as the parameter to the recommendations module. Using getFacetLabel for display purposes seems like a good idea -- I don't have a problem with changing that from protected to public in order to support your needs. |
| Comment by Till Kinstler [ 02/Sep/11 ] |
|
added file flot-0.3.patch: sligtly improved patch against VuFind 1.2: - Configuration now only in saerches.ini by adding a default_top_recommend[] module of type "PubDateVisAjax" there. List index fields to "visualize" one by one seperated by colons, eg default_top_recommend[] = PubDateVisAjax:publishDate - translations for field names are pulled through getFacetLabel($field) in web/sys/SearchObject/Base.php. This function needs to be public to be accessible from the recommendation module. I hope the change protected->public doesn't have side effects - the earlier mentioned bug in getFullFieldFacets() is fixed. - limitations in "classic" theme (not possible to add multiple visualisations due to hassles with the YUI flot implementation) still apply Please test and comment. |
| Comment by Demian Katz [ 12/Sep/11 ] |
| I'm attaching a patch which simplifies the new "get search parameters" search object method, improves protected vs. public changes (fixes a problem caused in Summon and avoids the need to make one method public) and adds some comments to searches.ini. |
| Comment by Demian Katz [ 13/Sep/11 ] |
| Attaching flot-0.5.patch, featuring an improved getFullFieldFacets method courtesy of Lutz Biedinger. |
| Comment by Demian Katz [ 14/Sep/11 ] |
| Attaching flot-0.6.patch, containing a couple of small fixes courtesy of Lutz Biedinger. |
| Comment by Demian Katz [ 19/Sep/11 ] |
| Committed as r4261. |