I18N Performance Analysis

A recent in-depth performance analysis of WordPress core showed that loading translations had a significant hit on a site’s server response time. Given that more than half of all WordPress sites use a language other than English, the performance team identified this as an area worth looking into more closely. The team spent the last couple of months exploring this in more detail and the results are now shared in this blog post.

Context

Initial benchmarks showed that the median loading time for a localized site can be up to 50% slower than for non-localized sites, depending on which themes and plugins are being used. This was measured using both the wpp-research CLI tool and also a dedicated benchmark environment (as elaborated in the Comparison section towards the end).

The WordPress i18n system is based on gettext, which uses source .po (Portable Object) files and binary .mo (Machine Object) files for storing and loading translations. It is not using the C gettext API itself but a custom userland implementation that works without any external dependencies.

In addition to core itself, each plugin and theme has its own translation file, which has to be loaded and parsed on every request. Loading and parsing all these translation files is an expensive task.

In the past, various solutions have been discussed and explored to improve the i18n performance of WordPress. A non-exhaustive list:

• Use a more lightweight MO parser
• Improve translation lookups by using the hash map in MO files (e.g. with DynaMo)
• Caching translations in the object cache
• Caching translations in APCu (an in-memory key-value store for PHP)
• Other more elaborated forms of caching (e.g. per request)
• Using the native PHP gettext extension
• Use a custom PHP extension to handle the MO file parsing)
• Using lazily evaluated translation calls (see #41305 for details)
• Using a different file format than .mo files, e.g. plain .php files

A more recent discussion touching on all of these solutions can be found over at the wordpress/performance repository. It’s a great way to get some context on this topic.

For this analysis, many of these solutions were looked at, focusing on their advantages and disadvantages. At the end of this post there is a comparison table with some much needed numbers as well, based on custom-built benchmarks.

Solutions

Solution A: Use different file format

Use a different file format for translations instead of .mo files to avoid the overhead of loading and parsing binary files.

Design considerations

With this solution, translations will be stored in plain .php files returning an associative array of translation strings. Whenever a .php file is available, it will be preferred over the .mo file, which is still used as a fallback. The rest of the architecture remains the same.

When a localized WordPress site downloads language packs from the translate.wordpress.org translation platform, it downloads .po and .mo files containing all the translations. This will be modified to include .php files. GlotPress, which the platform is built on, will be updated to support this new output format. Additionally, WordPress core itself could be modified to generate PHP files whenever they are missing.

In theory, nothing is faster in PHP than loading and executing another PHP file. .json, .ini, or .xml would all be much slower.

Proof of concepts using the PHP files can be found at swissspidy/wp-php-translation-files and swissspidy/ginger-mo.

Benefits

• Initial benchmarks show consistent significant performance improvements
• Relatively trivial to implement
• Maintains backward compatibility thanks to graceful fallback
• Makes it easier for users to inspect and change translations (no need to compile .po to .mo)
• Avoids loading and parsing binary .mo files, which is the main bottleneck
• Lets PHP store translations in OPcache for an additional performance benefit
• Battle-tested approach in the PHP ecosystem (for example in Laravel)

Caveats and risks

• Requires not only changes to WordPress core, but also tools like GlotPress and WP-CLI
• Adds maintenance overhead by introducing a new file format on top of the existing one
  • As shown by the proof of concept, the overhead is minimal
  • In the long term, .mo support could be deprecated
• Security considerations due to essentially executing remotely fetched PHP files
  • Not really different from downloading plugins/themes from WordPress.org
  • WordPress considers translations to be trusted
  • Hosting providers could be blocking PHP execution in wp-content/languages
  • Could potentially use checksum verifications or static analysis at install time to detect anomalies

Effort and timeline

The proof of concept using PHP files is in a very solid state already. There are also examples for changes to WP-CLI (PR) and GlotPress (PR). This makes it suitable for a feature project to expand testing with very little effort required. Even a core merge would be very straightforward in a relatively short time, potentially already in Q4 2023. The security aspect when using PHP files could be a potential blocker, so it’s important to loop in the WordPress security team and hosting providers early on.

More time is required to test other file formats and compare results.

Solution B: Native gettext extension

Use the native gettext PHP extension written in C when available, instead of the custom built-in parser in WordPress.

Design considerations

WordPress has always used a custom MO file parser, because the native gettext extension is not necessarily available on the server. With this solution, the existing system is adapted to use the extension whenever available and falling back to the custom implementation if not.

This has been previously explored in #17268 and implemented in WP Performance Pack and Native Gettext. These implementations can serve as inspiration for the initial design. They all work similarly in that they symlink or copy the translation files to a new directory structure that is compatible with the gettext extension.

As of July 2023, around 66% of all localized WordPress sites have the gettext extension installed, according to information from the WordPress update requests.

Benefits

• Significant performance improvements for eligible sites
  • Initial benchmarks show that loading time and memory usage basically do not differ from non-localized sites

Caveats and risks

• The gettext extension is not commonly available
  • Smaller incentive to implement and lower impact overall
• Requires locales to be installed on the server
  • Servers rarely have many installed locales
    • Locales often need to be compiled first and take up a lot of space
    • WordPress on the other hand supports over 200 locales
  • Potential clashes with the custom locales WordPress supports
    • For example, locales like pt_PT_ao90, de_DE_formal or roh might not even be supported
  • Outreach to hosting providers would be necessary
• Adds maintenance overhead by essentially adding a second gettext implementation
• Poor API
  • Requires setting environment variables (such as LC_MESSAGES and LANGUAGE), which might not be possible or cause conflicts on certain servers/sites
• Requires symlinks or hard file copies
  • Symlinks might not be possible on the server; copying all translation files means doubling disk usage
• Translation files are cached by PHP, thus any translation change requires restarting the web server
  • There are workarounds such as cache busting using random file names or fstat, however they might not work on all environments
• Has not been tested on a wider scale, despite being discussed for years

Check out the code of WP Performance Pack and Native Gettext to get a better idea of the extension’s poor API.

Effort and timeline

While there are existing implementations that could be leveraged for this solution, further field testing is required to assess whether the extension actually works under all circumstances. Given the limitations around the poor API and requirements for installing locales, it does not seem like a viable solution at all.

Solution C: Cache translations

Cache translations somehow to avoid expensive .mo parsing.

Design considerations

Cache translations either on disk, in the database, or the object cache to avoid expensive .mo file parsing on subsequent requests. This can be done in a generalized manner or also on a per-request basis to only load translations required for the current URL.

Many different caching strategies have been explored in various forms in the past, each with their own pros and cons. Some could even be combined. Defining the exact implementation requires further exploration and testing, which warrants its own exploration post.

Benefits

• Caching translations after one time .mo parsing potentially improves performance for future requests

Caveats and risks

• Caching using persistent object cache (e.g. Memcached, Redis) or APCu:
  • Not available on most sites, making this not an ideal solution
    • Availability according to data from WordPress update requests:
      • Memcached: ~25%
      • Redis: ~25%
      • APCu: ~6%
  • Can potentially significantly increase cache size or exceed cache key limits
• Database caching:
  • Moves the problem from disk reads to database reads
  • Can potentially significantly increase database size
  • Alternatively, use sqlite as a cache backend
    • Untested approach
    • Available on around 90% of sites
• Disk caching:
  • Not always possible, depending on server environment
  • Still causes file reads, only with fewer or other files
• Multiple cache groups (e.g. per-request or frontend/admin split)
  • Smarter cache logic to only load translations that are needed for the majority of requests
  • Can potentially significantly increase cache size
  • Unlikely that different requests use very different translations
• Cache retrieval adds overhead
  • Exact performance gains depend on implementation method and need to be measured first
  • No performance gains with cold cache
  • Cache invalidation logic TBD

Effort and timeline

Given the existing solutions in the ecosystem, the engineering effort itself would not be too big, but the right caching implementation (e.g. disk cache or object cache) needs to be evaluated first.

However, the right caching strategy probably does not exist because of all the different hosting environments. Since it’s unrealistic for core to support multiple types of caching, this solution seems better suited for plugins rather than core.

Solution D: Lazily evaluated translation calls

Use lazily evaluated translation calls to reduce the number of function calls in certain cases, leading to improved performance.

Design considerations

The idea of lazily evaluated translation calls has been first discussed in #41305. It enables avoiding string-specific expensive translation lookups until the translations are actually needed, by passing around proxy objects.

In other words: beyond just-in-time loading of translation files (which WordPress already does), this would add just-in-time lookup of individual strings in the translations. Check out this proof of concept to get a better picture.

It can be integrated essentially in two ways, both of which are explained on the core ticket:

1. Change all translation calls to be lazily evaluated by default
2. Make this opt-in, either with new function arguments or new functions altogether

Benefits

• Reduces the number of translation lookups, in some scenarios drastically
  • On a regular home page request there are ~60% less translation calls, saving around ~10ms (as measured by XHProf)
• As a side effect, solves UX issues such as #38643

Caveats and risks

• Depending on implementation this either breaks backward compatibility or risks not gaining enough adoption
  • Documentation, tooling, and developer education can help mitigate this to a certain extent
  • Adoption could be done gradually, e.g. starting with an opt-in approach and eventually making it the default
• Likely will not have a significant impact on typical frontend page loads, as it’s mostly useful for areas like the REST API schema output, where a lot of translation calls are made without actually using the translations
  • Needs analysis in more scenarios to measure impact
  • The REST API schema already has a workaround by using a cache in a static variable
• Does not improve situation for actually loading translation files
• Initial testing shows that this actually hurts performance due to the additional thousands of proxy objects being created

Effort and timeline

Gradual adoption would mean a multi-year effort to establish lazily evaluated translation calls, while enabling this by default is a significant backward compatibility break that could affect thousands of plugins and themes in the ecosystem. And since it does actually slow down performance in some cases, this solution is not a great candidate for implementation.

Solution E: Optimize/Rewrite existing MO parser

Refactor the existing MO parser in WordPress to be more performant.

Design considerations

Completely overhaul the existing MO translation file parser in WordPress with performance in mind. For example by using Ginger MO, WP Performance Pack, or other existing solutions as a base.

While for instance Altis DXP (Human Made) have actually replaced the existing MO parser with a custom-made PHP extension written in Rust, such an approach is obviously not feasible for core. The new solution needs to be written in userland PHP.

Initial testings with an updated fork of Ginger MO show some noticeable speedups and lower memory usage. It also supports multiple translation files per text domain and multiple locales loaded at once, which could prove beneficial for improving the locale switching functionality in WordPress core.

Besides that, plugins like WP Performance Pack and DynaMo have implemented partial lookups using the MO hash table or binary search, avoiding reading the whole file and storing it in memory. That slightly reduces memory usage and performance.

Benefits

• Can be used without necessarily introducing another file format
• Opens up potential performance enhancements in other areas, i.e. locale switching
• Mostly maintains backward compatibility

Caveats and risks

• Still a risk of breaking backward compatibility

Effort and timeline

There already is a working proof of concept for this solution, but more testing is required to further refine it and improve its backward compatibility layer. With such an effort being an ideal candidate for a feature plugin, this could be achieved relatively quickly in a few months.

Solution F: Splitting up translation files

Split translation files from plugins and themes into smaller chunks to make loading them more efficient.

Design considerations

Depending on the project’s size, translation files can be quite big. That’s why WordPress itself uses separate translation files for the admin and everything else, so that not too many strings are unnecessarily loaded.

This strategy could be applied to plugins and themes as well. Either by allowing them to use multiple text domains (which would require developer education and changes to tooling), or by somehow doing this automatically (exact method TBD)

Benefits

• Faster loading times due to loading smaller files

Caveats and risks

• Risk of breaking backward compatibility
• Opt-in approach requires tooling and distribution changes and risks slow adoption

Effort and timeline

Further research is required to evaluate this properly.

Comparison

At first glance, solution A (PHP translation files) is a relatively straightforward enhancement that maintains backward compatibility and shows promising improvements. However, it does not only require changes to core itself, but also to the translation platform. The security aspect remains a risk, although discussing it early on with stakeholders and gathering more testers would help mitigate it.

Leveraging the native gettext extension as in solution B shows stunning results, but the lack of availability and the non-ideal API are a concern. Still, it’s a progressive enhancement that cannot be ignored. Especially since it could pretty much eliminate the need for additional caching as in solution C.

Caching already loaded translations as in solution C does not eliminate the root cause of the i18n slowness, but can speed up subsequent requests. Unfortunately, persistent object caches or APCu are rather uncommon (though we do not have exact data on the former yet, see #58808), and implementing more complex types of caching (e.g. per-request caching) would require significant exploration effort before becoming a viable option.

Lazily evaluated translation calls (solution D) can shave time off translation calls in some situations, but overall actually decrease performance. While it could help solve some actual UX issues in core, the backward compatibility and adoption concerns make it even less of a suitable solution.

Existing plugins like Ginger MO and WP Performance Pack show that the existing MO parser in WordPress can be further improved (solution E).

Benchmarks

Now to the most interesting part: the hard numbers!

These benchmarks are powered by a custom-built performance testing environment using @wordpress/env and Playwright. The environment has been configured with some additional plugins and the PHP extensions required for some of the solutions. Tests have been performed against the 6.3 RC by visiting the home page and the dashboard 30 times each and then using the median values.

You can find the exact setup in this wp-i18n-benchmarks GitHub repository.

Block Theme

Locale	Scenario	Object Cache	Memory Usage	Total Load Time
en_US	Default		15.60 MB	133.58 ms
de_DE	Default		29.14 MB	181.95 ms
de_DE	Ginger MO (MO)		19.24 MB	159.18 ms
de_DE	Ginger MO (PHP)		16.98 MB	138.14 ms
de_DE	Ginger MO (JSON)		19.24 MB	153.39 ms
de_DE	Native Gettext		15.99 MB	142.12 ms
de_DE	DynaMo		19.62 MB	157.93 ms
de_DE	Cache in APCu		50.37 MB	181.51 ms
en_US	Default		29.01 MB	167.67 ms
de_DE	Ginger MO (MO)		16.85 MB	127.97 ms
de_DE	Ginger MO (JSON)		15.86 MB	129.19 ms
de_DE	DynaMo		50.30 MB	170.19 ms
de_DE	Cache in object cache		29.07 MB	173.19 ms

Classic Theme

Locale	Scenario	Object Cache	Memory Usage	Total Load Time
en_US	Default		15.35 MB	120.79 ms
de_DE	Default		28.79 MB	172.10 ms
de_DE	Ginger MO (MO)		18.85 MB	145.68 ms
de_DE	Ginger MO (PHP)		16.56 MB	124.73 ms
de_DE	Ginger MO (JSON)		18.84 MB	140.78 ms
de_DE	Native Gettext		15.58 MB	128.26 ms
de_DE	DynaMo		19.24 MB	146.09 ms
de_DE	Cache in APCu		50.13 MB	167.28 ms
en_US	Default		28.59 MB	154.30 ms
de_DE	Ginger MO (MO)		16.37 MB	112.94 ms
de_DE	Ginger MO (JSON)		15.38 MB	115.11 ms
de_DE	DynaMo		49.99 MB	151.82 ms
de_DE	Cache in object cache		28.65 MB	156.36 ms

Admin

Locale	Scenario	Object Cache	Memory Usage	Total Load Time
en_US	Default		15.42 MB	139.83 ms
de_DE	Default		31.92 MB	187.76 ms
de_DE	Ginger MO (MO)		20.07 MB	164.94 ms
de_DE	Ginger MO (PHP)		17.09 MB	139.66 ms
de_DE	Ginger MO (JSON)		20.06 MB	160.87 ms
de_DE	Native Gettext		15.95 MB	143.43 ms
de_DE	DynaMo		20.58 MB	166.79 ms
de_DE	Cache in APCu		58.13 MB	190.38 ms
en_US	Default		31.84 MB	164.26 ms
de_DE	Ginger MO (MO)		17.01 MB	118.52 ms
de_DE	Ginger MO (JSON)		15.87 MB	120.01 ms
de_DE	DynaMo		58.07 MB	162.41 ms
de_DE	Cache in object cache		31.86 MB	164.28 ms

Conclusion

Finding the right path forward means weighing all the pros and cons of each solution and looking at both horizontal and vertical impact, i.e. how much faster can i18n be made for how many sites.

When looking at all these factors, it appears that a revamped translations parser (solution E) could bring the most significant improvements to all localized WordPress sites. Especially when combined with a new PHP translation file format (solution A), which Ginger MO supports, the i18n overhead becomes negligible. Of course the same risks associated with introducing a new format apply.

On top of that, a revamped i18n library like Ginger MO could also be combined with other solutions such as caching or dynamic MO loading to potentially gain further improvements. However, those routes have yet to be explored.

Next steps

The WordPress performance team wants to further dive into this topic and test some of the above solutions (and combinations thereof) on a wider scale through efforts like the Performance Lab feature project. We are looking forward to hearing your feedback on this analysis and welcome any additional comments, insights, and tinkering.

---

Thank you to @flixos90, @westonruter, @joemcgill, @spacedmonkey, and @adamsilverstein for reviewing and helping with this post. Thank you to @nbachiyski, @ocean90, @akirk, @rmccue, @dd32 for providing valuable insights and context.

#core, #i18n, #performance