performanceoptimizationmultilingual

Which multilingual web design solution is fastest for the user, if this is indeed an issue?


Context:

I'm in the design phase of what I'm hoping will be a big website (lots of traffic, lots of users reading and writing to database). I want to offer this website in the three languages I speak myself (English, French, and by the time I finish the website, I will hopefully have learned enough Spanish to offer that too)

Dilemma:

I'm wondering how I should go about offering these various languages (and perhaps more in the future).

Criteria:

Many methods exist for designing multi-language websites. I'm looking for the technique that will result in a faster browsing experience for the user.

Choices:

Currently, I can think of (and have read about) the following choices. They are sorted in order of preference up to now.

  1. Store all language-specific strings in a database and fetch the good one depending on prefered-language (members can choose which language they prefer), browser-default-language and which language is selected during the current session, in that order.

    Pros:

    • Most of the time, a single test at the beggining of the session confirms which language to use for the remainder of the session (stored in a SESSION variable). Otherwise, a user logging in also fetches the right language and keeps it until he/she logs out (no further tests). So the testing part should be pretty fast.

    Cons:

    • I'm afraid that accessing the database all the time would be quite time-consuming (longer page load for the user), especially considering that lots of users could also be accessing the database at the same time for the same reason (getting the website text in the correct language), but also for posting comments and the such.
    • Strings which include variables (e.g. "Hello " + user.name + ", how are you?") are harder to store because the variable (e.g. user name) changes for each user.
    • A direct link to a portal for a specific language would be ugly (e.g. www.site.com?lang=es)
  2. Store all language-specific strings in a text file and fetch the good one depending on prefered-language (members can choose which language they prefer), browser-default-language and which language is selected during the current session, in that order.

    Pros:

    • Most of the time, a single test at the beggining of the session confirms which language to use for the remainder of the session (stored in a SESSION variable). Otherwise, a user logging in also fetches the right language and keeps it until he/she logs out (no further tests). So the testing part should be pretty fast.

    Cons:

    • I'm afraid that accessing the text file all the time would be quite time-consuming (longer page load for the user), especially considering that lots of users could also be accessing the file at the same time for the same reason (getting the website text in the correct language).
    • Strings which include variables (e.g. "Hello " + user.name + ", how are you?") are harder to store because the variable (e.g. user name) changes for each user.
    • I don't think multiple users could access the text file concurrently, though I may be wrong. If that's the case though, every user loading a page would have to wait for his/her turn to access the text file.
    • Fetching the very last string of the text file could be pretty long...
    • A direct link to a portal for a specific language would be ugly (e.g. www.site.com?lang=es)
  3. Creating multiple versions of the website in seperate folders, where each version is in a different language.

    Pros:

    • No extra-treatment is needed for handling languages, so no extra waiting time.

    Cons:

    • Maintaining the website will be like going to school: painfull, long, makes you stupid after doing the same thing over and over again.
    • ugly url (e.g. www.site.com/es/ instead of www.site.com)

Additionnaly, the coices above could be combined with one or more of the following techniques:

  1. Caching certain frequently requested pages (in a singleton or static PHP function?). Certain sentences could also be cached for every language.

    Pros

    • Quicker access for frequently-requested pages.
    • Which pages need caching can be determined dynamically, with time.

    Cons

    • I'm not sure about this one, but would this end up bloating the server's RAM?
  2. Rewritting the url could be used for many things.

    • A user looking for direct access to one language could do so using www.site.com/fr/somefile and would be redirected to www.site.com/somefile, but with the language selected beign stored in a session variable.

    Pros

    • Search engines like this because they have two different pages to show for two different languages

    Cons

    • Bookmarking a page doesn't mean you'll en up with the right language when you come back, unless I put the language information in the url (www.site.com/somefile?lang=fr)

A little more info

I usually user the following technologies to make a website:

This being said, if a solution requires that I learn a new language or something, I'm very open to doing so. I have no deadline for this project and I do intend to learn a lot from doing it!

Conclusion:

What I'm looking for is a method that allows me to offer multiple languages while not increasing page load time and not going crazy when trying to maintain the website. If you guys/gals have other ideas I should consider, I will try adding them to my list. Another possibility is that I'm overdoing this. Maybe I won't gain enough time with these methods for this all to be worth it, I just don't know how to verify if I need to worry about this or not.. so if you have any ideas for that, it would also help me.


Solution

  • Whether you use a database or a filesystem to store the translations, you should be loading the text all at once and then serving it from memory. Most applications will typically not have so much text that this becomes a problem. In Java or .Net this could be accomplished by storing the text in a singleton or static object. Then all the strings are in RAM and do not need to be loaded or parsed. If your platform does not have a convenient way to store data in ram, you could run a separate caching application such as memcached.

    The rest of your concerns can be mitigated by hiding the details. Build or find a framework that lets you load your translations and then look them up by some key. If you decide to switch to files or a database later, the rest of your code is unaffected. In the short term do whichever is easier for you. I've found that it's best to have a mix: it's easier to manage application text along with the source code in a version control system. But some text changes often, or needs to change without requiring a build+deployment cycle, and that text should be in the DB.

    Finally, don't build strings with substitutions in them. Use some kind of format string, because otherwise your translators will go crazy trying to translate sentence fragments.

    (Warning: Java code sample)

    //WRONG
    String msg = "Hello, " + username + ", welcome back.";
    
    //RIGHT
    String fmt = "Hello, %s, welcome back."; // in real code: load this string from a file or the db
    String msg = fmt.format(username);
    

    Another person mentioned encoding the language in the URL. This is the preferred way to do it if you care what a search engine thinks of your site. Google recommends using different hostnames or a different subdirectory. This means that the language headers sent by the user can't be used for anything, except perhaps initially sending them to one landing page or another. You will need to determine the language for each request based on the incoming URL (this actually simplifies your code a lot later on). In Java I'd store the language code in the Request and just grab it whenever I need it.

    The easiest way to handle language codes in the URL is to use re-writing. A client sends a request for www.yoursite.com/de/somepage and internally you re-write the request to www.yoursite.com/somepage and store the language identifier somewhere. In Java each request has an HttpServletRequest object where you can store attributes for the lifecycle of the request. If your framework doesn't have anything like that you can just add a parameter to the url: www.yoursite.com/de/somepage => www.yoursite.com/somepage?lang=de. If you are using hostname-based languages you can use hostnames such as de.yoursite.com or www.yoursite.de. There are pros and cons to using this approach. For one thing, using country-code TLDs means registering new TLDs and trying to figure out whether a country code is appropriate to represent a language (it's often not). Using differnet hostnames/domains means you have to consider under what domains cookies are stored. If you want a cookie-free subdomain you need to plan this carefully. But from the coding side a language-based hostname doesn't need any additional re-writing; you can read the hostname (it's the Host header in the HTTP request) and parse that to determine the language.