The fourteenth VuFind® instructional video discusses VuFind®'s internationalization (i18n) capabilities, explaining how to configure languages, how to create new translations, and how to incorporate i18n into your code.

Video is available as an mp4 download or through YouTube.

1. Localization Documentation

This is a raw machine-generated transcript; it will be cleaned up as time permits.

for this month's viewfinder video we are taking a deep dive into viewfinder's internationalization system we're going to start at the surface talking about what it does and how you can configure it uh when we'll go a little bit deeper to talk about how you can add additional languages to the software and then even deeper uh to talk about how internationalization can be used uh when you're developing on the software adding new features or building local customizations i've tried to organize this content in order by complexity so you can leave the video whenever you think you've had enough but you can keep going to get deeper and deeper into the system so viewfind is a piece of software that's used throughout the world and of course in different parts of the world people speak different languages so viewfinder needs a way to present its interface in multiple languages and that's where internationalization comes into play if you have a default viewfinder installation you'll have this language control at the top right as you can see which shows many different languages all rendered using their native representations so people who speak the language can instantly pick out their native preference and if you choose one of these the whole interface draws itself uh in the new language all of this like everything else in viewfind uh is highly configurable so we're going to go into some of that today it also is useful even if you're presenting your viewfinder in only one language because the internationalization system provides a way to override a lot of the text that viewfind uses so if you need to customize specific language in the interface you can often do it uh just by changing one line of text making it a really easy way to customize your viewfind experience we'll talk about that too so let's start by taking a look at some configuration files i'm just bringing up visual studio which is pointed at my viewfinder home directory so i can quickly navigate through the files so if we look at the config.ini main configuration there are a few settings that are of interest first of all fairly near the top there are a few language related settings there's browser detect language which by default is turned on at true and this means that viewfind will try to read http headers sent by your web browser to determine your native language and if that language is supported by viewfind it will turn it on by default so if you have a viewfinder instance that's used by many people speaking many different languages having this setting on just ensures that they won't be inconvenienced by having to switch the language manually right below the detect language setting is just language where you can set the default language of your viewfind installation so this is the language that will be turned on uh if you either turn off the detect language setting or if a user's browser specifies a language that viewfind isn't currently configured to support so you aren't necessarily forcing this language but this is the one that will be used if nothing more specific is identified i should also note that right under language is the locale setting and this indicates where your viewfind instance is located and this is completely independent of the language setting this is mainly used for things like determining what type of currency is uh displayed when users look at their fines and so forth so you want to set this to where you are or where your library is and it's independent of the language setting because the language is controlled by the end user in any case there's one more part of config.ini we should look at which is further down and this is just a section called languages and this is a list of language codes mapped to those language names in english and we use english language names in the configuration here just for consistent readability they get translated into their native forms elsewhere and i'll show you that later but the point is this lists all of the languages that viewfind currently supports and they are all turned on by default the order of this list also controls the order of the language drop down in the viewfind interface so if your community is more likely to speak a certain set of languages you might want to move them to the top of this list and if there are languages here that are very narrowly used and don't apply to your audience you might want to turn them off just to simplify the interface you'll also notice that there are some languages that are turned off by default and these are regional variations so for example uh you could switch english to use british rather than american spellings or you can turn on both versions if you wanted to uh similarly uh there's a separate file for flemish dutch but that's turned off by by default we do offer both portuguese and brazilian portuguese by default so this is something you might even want to consider turning off if that's not relevant to your users while we're in the configuration files uh there's just one other thing that i would like to show you which is uh in facets.ini the facet configuration uh in the advanced section the advanced settings more specifically uh there's a section called translated facets and this tells viewfind when it's displaying facet lists uh which facet fields should be run through the translation system and potentially translated as you can see by default we're only translating the format names uh and the high level library of congress call numbers uh in the future we may add more translated fields but of course it's a lot of work to translate all possible values in a facet so we only add those as time permits that work to be done and this is certainly something members of the community could contribute back if it's important to them for example there's a long-standing project to translate language names into all supported languages which i'd love to see completed but it just hasn't been done yet because it's a lot of work but it is certainly technically possible within this system uh one little detail i will highlight is that when translating a facet you can specify either that it comes from the main language file by specifying the name of the field by itself or you can follow the field name with a colon and a name of a text domain containing relevant translations text domains are a way of organizing our translations and i will show these to you in more detail in just a moment so now that i've showed you how to configure internationalization both in terms of what languages are provided and also in terms of what facet fields can get translated let me show you how the translations are actually organized within viewfind uh inside your viewfinder home directory you'll find a languages subdirectory that contains many many uh any files and as you can see most of these use the two letter or two letter and regional subdivision codes the same ones that are found in the config.ini file this is of course how we organize each language's translations into a separate file there's also this special native.ini file and as i mentioned this is how we translate the english language names from the configuration file into their native forms for display in the drop down it's the one special exception within this directory where otherwise every file name corresponds to a code so let's look at one of the actual language files en.ini where all of the english codes are found the any files containing viewfinds language translations use a subset of the broader any file standard they support semicolons for comments um and beyond that everything is in the format of a translation key an equal sign and then the translation in double quotes viewfinds language files are standardized to always use this format every string is always in double quotes and the keys are always sorted alphabetically just for consistency and to make it easier to find things if you scroll through you'll notice that some of the translation strings include placeholder values of the format percent percent some name percent percent these placeholder tokens are used uh in cases where a translation needs to have a value inserted into it at a particular position and using these tokens is really important for translation because if you're assembling a sentence different languages have different grammars and so the word order may be different so by using a token this makes it possible for translators to always put the changeable parts of a phrase in the correct position while it's being translated you'll also notice that some of the translation keys end in underscore html uh this indicates that the translation string itself may include some html and when it is being displayed in the viewfind interface it should not be escaped because we want to actually render the html for the end user rather than display the literal less than em greater than in this particular example uh most of the other keys that do not have the html suffix on them are going to be assumed to be plain text with no html and when they get rendered they will have any special characters escaped to ensure that they display to the end user exactly as they are written in the translation file with no characters being interpreted as html you'll also notice that there's kind of a mix in here of actual uh english phrases being used as keys and more abstract tokens being used as keys this is sort of a side effect of viewfinder code evolving over time when viewfind was first written everything was just an english key but it pretty quickly became apparent that it was sometimes more useful to use a more abstract name for a translation this is particularly critical when the same english word might have different words for different meanings in another language so we started using keys that offer more context about how the translation is used so that even if the text is identical for two different things in english it's possible for a translation to use different phrases for each context in which the words are used if that happens to be necessary you'll also notice that there are some subdirectories under the languages directory uh and these contain more any files these are the text domains that i mentioned earlier um if you have a specific set of related translations rather than putting them in the top level language file which is already very long and contains a lot of different things you could instead decide that they belong in a text domain which is just in any file in a named subdirectory so for example when i showed you the facet translations earlier i showed that the call number first field had its own text domain so if we look uh in these files you'll see that all of the top level library of congress classifications are translated in these files in some cases complete translations haven't been provided yet but the point is uh this way we can sort out our translations into a more logical arrangement it becomes easier to find things and easier to read things again there are still many things in the top level translation files that would probably better belong in a separate text domain but because text domains were introduced to viewfind later in its development not everything has been refactored yet so over time you may start to see more text domains and shorter top level language files but for now we are just refactoring as time permits and circumstances allow the language files are really useful for translating short phrases and words to build larger interfaces but there are also some other translation facilities in viewfind for situations where you have a huge block of text that perhaps would not be appropriate to include as a line in a language file one of the most obvious examples of this is view finds help pages if i go over here there are a few different search screens in viewfind like the search tips page which pops up here that have a whole lot of text in them but which we want to be able to provide to users in multiple languages so you know for example here if i uh switch this to spanish and pop open the search tips i get that whole page in spanish but if i had to break that down into a separate translation for every sentence in the file that would be uh unreasonably tedious so instead for help screens we have a template based system if you go into the themes you will find in the root theme where our widely shared content lives under the templates directory there is a help translations folder and in here as you can see there are folders that correspond to language codes and not every help screen has been translated into every language yet but where they have been you'll find that there are templates with parallel names so for example you know here in english is search phtml which has the html template for search tips and under es for espanol we have the spanish version of the same file so for help screens uh viewfind just has some code that looks at the user's current language checks to see if a help template exists in that language uh and displays it if it's available if a translation is not available the help system will just display the english template with a warning message apologizing that a translation is not available this is another area of course where members of the community could contribute by translating more of the help screens into more languages one final thing which i will briefly mention and we'll show in more detail in a future video when we talk about managing static content in viewfind there are a few different ways that viewfind can display static pages of content if you want to use it as a lightweight content management system for example if you want to build local pages with frequently asked questions or information about your library all of these also have template-based mechanisms for providing translations just as a quick example let me switch into the bootstrap three theme go into templates and in here there is a folder called content and as you can see uh there are a few different uh example content templates here all of these are really designed to be overridden locally so you know for example the frequently asked questions page is very short um the ask a librarian page just says this is the default page this isn't meant to be used in production but what i want to show here uh relating to internationalization is that all of these content pages have a fixed name but if you put an underscore and language code after that you can create an english specific version of the page that will be used when that language is selected so for example you know you can see ask a librarian has a default page and an english page so if i go back to english and click on ask a librarian to view this it says this is the english ask a librarian page so now that i've showed you where all of the language files are located uh and how they work let me do a quick demo of how you can easily override some language strings to customize something uh in your your local instance of viewfinder so suppose i don't like the message that is displayed when i perform a search and don't get any results so right now it says your search no results did not match any resources but say i want to change that language the first thing i need to do is figure out what language string is actually causing this message to appear the easiest way to do that is to go into the main language file and search for uh a phrase that matches this the string i want to change and of course if i don't find it in the main file i can search in all of the text domains as well but most things you see in the viewfind interface are going to come from the main translation file so let me go back to visual studio here and look in the english language file i'm just going to search for any resources because that's a fairly distinctive part of the string i want to override and sure enough i find that there is a string here called no hit look for html so this is the string i can customize if i want to change this particular message in the user interface like many things in viewfind uh you can create files in your local settings directory to override uh settings from the core languages are no exception so if i go to my local directory and i create a folder called languages and inside that folder i create a file called en.ini i can customize english language strings here and override the defaults a nice thing about the language file overwriting is that you don't have to copy the whole language file from the core you can override only the strings that you want to change here and they'll get merged automatically with the defaults from the core so suppose i want this to instead say sorry we could not find any matches for your search i just edit the string here in my local language file but there's one more important step that we need to remember and this is uh because the the language translations are distributed across a number of files a viewfinder maintains a cache of language strings so that it doesn't have to spend a lot of time gathering things together every time it renders a page so if you change a language file you also need to remember to empty out the language cache so that your changes will immediately take effect i can do this by going to my view find home directory and then simply running sudo rm minus rf local slash cache languages and that will clean out my cache and now uh if i refresh my page over here sure enough my custom translation has kicked in and it says sorry we could not find any matches for no results so customizing any piece of text in viewfind is that easy and of course if you need to customize the text in multiple languages you just need to create multiple language files uh inside your local languages directory so now that we've talked about how to use the language system and how to customize uh what is displayed let's go a little bit deeper and talk about how you can add additional languages uh to viewfind you can probably guess that it's really just a matter of creating some new files um you just need to figure out what the language code is for your new language uh then you can take one of the existing language files uh rename it to match the code of the new language and translate all the strings within it you of course have to repeat that process for all of the text domains if you want your translation to be comprehensive uh you also need to add the new language to config.ini in the languages section so that it's included in the list of available languages and you want to uh make sure that there's an entry in the native.ini so that we can represent the language name in english in config.ini for consistency with the language within that file but we can also represent the language uh in its native form uh in the user interface so it's really just those few steps the hard part of course is doing the actual translation work and of course if anybody does want to add a new language to viewfinder has any questions about the context of particular strings or needs any help please feel free to reach out to me or to the community at large and you'll get some assistance because we're always pleased to expand the reach of ufind by adding more languages but outside of that general process for creating languages i also wanted to highlight one useful tool that viewfind includes that's helpful while managing languages which i use every time a new viewfinder release is being planned and we need to update all of the translations viewfind actually includes several tools within it that are geared toward developers and as such these tools are only available when viewfinder is switched into development mode i showed this in a past video but just as a reminder if you go to your apache configuration in the local directory httpd viewfinder.conf there is a line in there which sets development mode and right now in the demo instance i'm teaching with development mode is already turned on but if this were not already turned on you would have to make sure that any comment marker in front of it was removed and after making that change you would need to restart apache uh to make the change take effect uh if this were setting were not in place you would not be able to see the developers tools but since in my case it is already turned on i can just go ahead and show you that if you go to viewfind slash devtools you will see a page summarizing the available development tools and one of these is language details if you look at the language details page you will see a list of all of the languages currently supported by vuefind and all of these are getting compared against the english language file to show you where there are gaps because while we try to keep all of our translations up to date not all of our volunteer translators are always available so some languages have more complete translations than others and this tool allows you to find out what strings are missing so for example uh if i wanted to see uh which lines still need to be translated into welsh i could click show here and i get a pop-up summarizing all of the english lines that still need translation if you click on this it automatically highlights the whole thing for you so it's easy to copy and paste and this is the process i follow when i request new translations for every release i go through here copy and paste all the lists of missing lines and email them to the volunteer translators who then send them back to me for incorporation into the project this extra lines column is also useful because if you accidentally create a translation in another language file that doesn't exist in english it will be highlighted here and sometimes this can catch minor typographical errors or formatting problems because they'll be interpreted as inappropriate lines in the file this will help you track them down and fix them you can also see this extra help files column gives you a count of how many files exist uh in that help template directory i showed you for each language so as you can see there are some opportunities to fill gaps here if anybody cares to do so also note that there's been some uh recent work to improve the look and feel of this screen so by the time you're watching this video uh it may actually look a little bit different but the functionality will remain the same uh now that i've showed you where to create files to add a new language and how to manage languages through this development tool i'd also like to highlight a few useful command line tools that you can use for managing the language files and these are mainly used during development of the core project they're less useful for local customizations so you would find yourself using these uh primarily if you're adding new features to the viewfinder core and those features introduce new language strings so i'm going to pop to the command line here and just a reminder you can always get a summary of all of viewfind's command line capabilities by just running the index.php file in the public directory through php so this gives me a summary of many commands and you'll notice that there are a few language specific ones whose names all start with language so there is language add using template this uh provides a mechanism for creating new language strings by combining existing ones and i will actually do a demo of this a little bit later so it will become more clear then there is a copy string command which simply copies one string in all of the language files to another string this could be useful uh if you want to differentiate something so suppose you have a text string that's being used in two places but you decide that maybe you want to refine the language in one of those two places to be more specific you could use copy string to create two different identical strings in all of the language files and then you could customize them where appropriate this way you can add a new string without creating gaps in the translation but you still provide the opportunity to customize more specifically as needed there's also language delete which is fairly self-explanatory it removes a particular translation key from all of the language files uh this can be useful uh if something becomes obsolete again i'll demonstrate this for you in a moment and finally there is language normalize the language file normalizer for this one you give it the name of a directory containing any files and it will format all of them to meet a viewfind's language file standards as i mentioned earlier it alphabetizes everything by key and it puts all of the text to the right of the equal sign into double quotes it's just a nice way to be sure that everything is neatly formatted and if you're submitting new language strings to viewfind our continuous integration process which looks at submissions and checks their validity will double check that all of your language files are normalized correctly so running this tool can just help prevent some bumps during the contribution process so with all of that introduced let's uh dive into a hands-on example of using these command line tools and i'm actually going to use this to create a real change to the viewfind code that i think will be useful which i'll submit as a pull request following this call so if i go to the home page of view find by default i see all of these browse by headings for the the different facets that we display on the home page and i happen to notice that this isn't formatted in the best possible way let me show you what i mean if i go into the english language file and i search for browse by i have this uh home browse key which just translates to browse by so what this means and actually i can show you this in a moment is that we are just concatenating a facet name onto a translation to create those headings and as i mentioned that may not be appropriate in every language because perhaps in some other language the grammar specifies that the facet name should be before the string or even in the middle of the string this is a situation where we really should be using a placeholder token instead of combining strings together and i'm sure the reason this is the way it is is that this browse by string was added to viewfind in its early days before we improved our practices so this seems like a perfect opportunity to improve that little detail and demonstrate uh all of the processes around it along the way so if we're going to change a string in the language files the first thing we need to do is figure out where it is being used uh i like to use the grep command to do this it's a standard unix command line tool for finding things in files so let me just do that i'll go to the command line i'm going to go into the themes directory because i think in this instance it's safe to assume that this text is only being used in templates it's probably not found in any of our controller or service logic and then i'm just going to say grep minus r home browse star so grep minus r means recursively search through all the subdirectories of the current directory look for the phrase home browse and star just means look in every file so grep is telling us here that home browse is only found in one file but within that file it's actually used in three places so let me open up this file templates content block facet list html content block facetlist.phtml so there are a few different conditions in this file that cause the headings to be generated in different contexts but in all three uh situations it's doing exactly the same thing and as i said before it's first translating the home browse text then it's adding a space to it and then uh it is displaying the label but this is not as flexible as we would like so let's change this so that uh it instead uses a token in the appropriate position and this is a perfect use case for the add using template command line tool for managing language files so i'm moving back to my viewfind home directory and then i'm just going to show you you can run php public slash index.php add using template minus minus help to get help on uh how to use this particular command in this instance the help screen shows us that the first parameter is the name of the new key that we want to create and the second is a template showing it how to build that string and this uses a double piped placeholders to embed existing translation strings so this will become more clear as i show the example so right now we have an existing translation string called home and we need to create a new one because this tool creates a new key it doesn't overwrite an existing key but let's take this opportunity to use a more specific key name anyway so let's call this home browse by facet and then we're going to create the template to match the current behavior so i'm going to use double quotes to surround my template because it will have spaces in it and i want it to be treated as a single piece of text when it gets passed in as a command line parameter to my tool so i'm going to double pipe home browse so this is going to take the existing translation of the home browse key and put it into my new home browse by facet key then i'm going to add a space and i'm just going to create a token called percent percent facet percent percent and then i'm going to end my double quoted phrase for the template so what this is going to do is it's going to go through every single language file it's going to create a new home browse by facet translation which will be the existing home browse translation followed by a space and followed by this percent percent facet percent percent token of course this may not be optimal in every language but this is going to match the current behavior uh and it's at least going to give us the possibility of improving this in the future whereas as currently implemented there's nothing you can do to change the order of this display so when i run it it runs through all of my language files you'll notice that it skips a couple of files because they don't contain a home browse key and thus uh cannot translate it and these files that got skipped are the regional translations so the the british english and the flemish dutch and these don't contain this particular key because they only contain text that provides a regional variation and in both of these instances there's no difference between regions for this particular piece of text in any case i've now edited a whole bunch of files so if i do a git diff it will show me uh what has been added and as you can see it's just taken the existing home browse translation and added uh facet on the end of it which is exactly what we wanted uh but now um our intent is to replace the home browse with home browse by facet so this is an opportunity to also use the language delete command now that we've moved home browse we can get rid of it and so that works just the same we say you know php public index language delete the name of the key to delete it runs through all the files and deletes it it reports files that don't contain the key so if i do a git diff now we can see that it has removed home browse but it's still adding home browse by facet so now that we've created the home browse by facet we need to actually use it this is a great opportunity to demonstrate uh viewfinds translation view helpers which are used throughout the uh language fi uh throughout the themes in the templates there are two of them uh one is called this translate and one is called this transesque and they both work exactly the same way except trans-esque html escapes the translation when it's done running whereas translate leaves the text raw so the the escaping helper is used in most places but the translate helper is used if we need to render html which is why we have that html suffix to indicate uh templates that contain html or it's used in context where we're not generating html for example if we're building the body of a text email so let's refactor this code a little bit to make it easier to modify so since we have identical logic in three different places i think it will be cleaner if we just uh take this whole bit of logic and we create a variable called label heading so then we can do the calculation in one place in the code and display it three times over so i'm just going to put some php logic right here that says uh sorry label heading equals and that's the current logic which we will change in a moment i'm just going to uh finish refactoring this everywhere i know there are three of these where's the third one two three okay so now we have one piece of code that's generating the label heading and we're using that value in three different places but we now have a token so instead of concatenating this stuff together uh and just to refresh memory on that what we used to be doing was translating the home browse key then adding a space then translating the name of the facet field being displayed let's be smarter about this and instead let's translate home browse by facet and the both of the translate view helpers accept a second parameter which is optional and contains an array of tokens to values so in this case we created a token called a percent percent facet and we want that to instead [Music] be the name of that facet so i'm going to take away the concatenation and instead plug this value right into this array but there's one more important detail in this instance which is that here we're translating and escaping the label but we're also translating and escaping this whole thing we don't want to escape the value twice because if we do that we might end up causing some html entities to be rendered to the end user and that would be confusing so when generating the list of tokens here we'll just do a flat out translate and then i will escape it at the end after we've combined all the parts together so that completes my refactoring but you will recall that i mentioned that the language cache needs to be cleared in order for uh changes to the language files to take effect and let me just demonstrate that uh to prove that what i've done has actually had an effect if i refresh the page now all of my headings just changed to home browse by facet because i created a new string but that string isn't in the cache yet so now if i go down to the terminal and i remove local cache languages again to clear out that part of the cache and then i refresh my page again now we're back to having appropriate headings on all of our columns but now we're using a more uh appropriate tokenized version of the language string that will be easier to customize in the future so uh we've just about run out of time here but the last thing i wanted to very quickly highlight is i've showed you uh how translation works in the template files you can also occasionally need to do translations deeper in the code when you're writing controllers or building services and viewfind provides a couple of useful mechanisms to help you with that if you look inside the core viewfinder module code under viewfind i18n translator there's something called the translator aware interface and something called the translator aware trait and to make a a long story short if you implement the translator aware interface on a service or controller many of viewfind service managers will automatically call its set translator method and pass in the laminas class that actually does the work of translating things uh and even if it doesn't get auto-injected this makes it easy to inject through a factory so depending on context the point is anything that has a translator aware interface can receive a translator and use it to translate strings the translator aware trait is a trait that can be mixed into any of your classes uh it gives you the set translator method to match the translator aware interface and it provides some useful utility methods that you might find helpful it has this get translator locale method which will find out what user language is currently active so if you need to find out what user language code has been selected this will tell you and it has the translate method which we showed in the view helper earlier that takes a string to translate an array of tokens and also a third parameter a default value that can be used if no translation is found at all so this is used throughout the viewfind code it can be useful in your custom code i don't have time today to go much deeper into it but just wanted to make you aware that this is available uh should you need it and that's all we have time for so i hope this has been a useful introduction to internationalization uh and that it will help you customize your local interface and perhaps inspire you to contribute some more languages to viewfind thank you as always for your time and see you next time