An update on what’s next for ex_cldr and friends

A very occasional update on what’s coming next for ex_cldr and related libraries.

In October 2022 (Proleptic Gregorian)

• CLDR 42 is in beta and due for release in October. In line with usual practise, updated ex_cldr and related libraries will be available on the day of release. As usual, some formatted results may chance between releases but the API and semantics do not change.

• Update ex_cldr_routes to support Phoenix 1.7 and to introduce a new sigil to localise the new verified_routes capability. Probably going to be sigil_q (q being after p) - alternative suggestions welcome!

By Year End 2022

• Introduce ex_cldr_person_names. With CLDR 42 comes data to support person name formatting. ex_cldr_person_names will leverage that data to provide person name formatting. Input on user requirements are especially welcome.

By March 2023

• Implement the Unicode Message Format version 2.0 as a new library. ex_cldr_messages implements the ICU Message Format and thanks to great work by @maennchen it’s fully integrated with Elixir gettext making for a seamless developer experience for what I believe is a better message format. The new Message Format 2.0 syntax is designed to be more widely adopted and address known issues with the 1.0 format. All the heavy-weight players from industry have been part of formulating this specification.

In Calendar Year 2023

• Collaborate on writing book about localizing applications. There is very little literature on this topic let alone practical guides.

• Implement some Liveview components on Phoenix 1.7+. Input, thoughts, ideas, suggestions very welcome. My initial list:

  • Locale selector. A panel to select an available locale and customize it (calendar, currency, etc etc)
  • Calendar picker. There are some great javascript calendars and they should be the preferred UI mechanism. But in the liveview world there is room for a calendar too. And, well, I just like calendars. This implementation will work on any calendar that can be defined with the Cldr.Calendar protocol and be completely localized so it will have different capabilities to the normal JS ones.

• Implement an interactive notepad like numpad. ex_cldr and friends have a lot of inbuilt knowledge of numbers, dates, times, units of measure and it would be fun to wrap this in a notebook style interface. I’m hoping this can be done on top of Livebook - I don’t think there is a way to override the evaluator loop but maybe there is a way to put this all together. Definitely interested in ideas, feedback and suggestions.

This week, downloads of ex_cldr hits 500,000.

A little over 4 years ago I decided to follow @josevalim to Elixir and having spent a bit of time hanging around on the edges of the Ruby i18n Gem I thought working on an I18n project for Elixir would be a good way to spend a weekend getting to know the language.

That weekend turned into a 4 year journey. Fast forward and now there have been

• ex_cldr and 17 other libraries in the I18n and CLDR domain published on hex.
• 2,628 commits across all libraries
• 139 releases of the main library
• 17 contributors across all libraries
• Two additional libraries: ex_cldr_languages by @LostKobrakai and ex_cldr_territories by @Schultzer

Many thanks to @lostkobrakai, @danschultzer, @szTheory, @KronidDeth, @michalmuskala, @zaack, @tcitworld, @ jueberschlag, @ Schultzer, @Zurga, @yuchunc, @kwando, @bglusman, @kianmeng, @Qqwy, @NickNeck and @dbernheisel for their support. It’s been a wild ride and there’s still more to come!

To celebrate what is, for me, a pretty big milestone I’ve started a blog on i18n with Elixir with the first two articles at Adventures in Internationalization and Localization.

Yeah, creative it isn’t and my writing is really rusty. Like software, I assume it will get better with practise!

CLDR 48 was released last week which has triggered new versions of several of the core ex_cldr libraries. Here’s a summary of newsworthy changes:

Ex_cldr

Bug Fixes

• Parsing language tags with the und (unknown) language now correctly substituted for a known language name if one can be found in Cldr.Locale.likely_subtags/0. For example:

iex> Cldr.validate_locale("und-TW")
{:ok, TestBackend.Cldr.Locale.new!("zh-Hant-TW")}

• The :gettext_locale_name field of a t:Cldr.LanguageTag.t/0 is now set exactly as returned from Gettext.known_locale_names/1. Previously it was being tranformed to a BCP 47 format locale (replacing “_” with “-”). That is no longer the case.

• Territory containment was previously not considering nested containers and is now doing so. For example, territory “019” (Americas) includes “419” (Latin America and the Carribbean) but “419” was not appearing in the list for “019” since CLDR categorises “grouping” territories separately. This should not affect any consumers, only library writers.

Enhancements

• Update to CLDR 48 data.

• Add Cldr.Locale.Match.best_match/2 and Cldr.Locale.Match.match_distance/3 to implement the CLDR Language Matching algorithm. This is a more formal and testable approach that the previous mechanism. This new function will be use to implement better matching between a known CLDR locale name and supported Gettext locale names. Therefore it is possible that the gettext_locale_name field of a t:Cldr.LanguageTag.t/0 may change with this release.

Cldr Numbers

• Adds Cldr.Number.to_ratio_string/3 to format numbers as a rational string. Numbers can also be formatted as an integer and a ratio. Since different rendering systems (browsers, terminals etc) reproduce ratios with differing fidelity, there are options to format the ratio as a precomposed fraction, super- and subscript numerals separated by a fraction slash or normal numerals separated by a fraction slash.

• For number systems other than Latn, the :prefer: :precomposed and prefer: :super_sub options have no effect.

Examples

Cldr Dates Times

Breaking Changes

• Changes in the implemementation of time zone formatting may result in different (but semantically the same) strings for formats that include time zone information. See Bug Fixes below.

Bug Fixes

• Fixes the generation of time zone formats to be in accordance with CLDR Time Zone Names. This is particularly true of the following time zone formats:

  • Generic non-location format (eg “Pacific Time”, “PT”)
  • Generic partial location format (eg “Pacific Time (Canada)”)
  • Generic location format (eg “France Time”, “Adelaide Time”)
  • Specific non-location format (eg “Pacific Standard Time”, “PST”)
  • Localized GMT format (eg “GMT+03:30”, “Гринуич+03:30”, “GMT+?”)

• Fixes Cldr.DateTime.Relative.to_string/3 when the relative value is an integer of 2 or -2 and the locale provides a specific word for that value. For example, in English there are words for 1 day in advance or behind (“today” and “yesterday”). When specifing 2 days ago, English has no specific word so the result is “2 days ago”. In German the result is “vorgestern”. Similary for the English “in 2 days”, the German result will be “übermorgen”.

Enhancements

• Updates to CLDR 48 data.

• Adds Cldr.DateTime.Format.gmt_unknown_format/1 to return a string representing an unknown GMT offset.

• Adds style: :at to Cldr.DateTime.Relative.to_string/2. This allows formatting of string like “tomorrow at 3:30 PM”. The default is style: :standard.

• Significant improvement in support of date time skeletons. Skeletons are a flexible way to express desired formatting in a locale-indepdendent way.

• Support is added for the “j”, “J” and “C” input skeleton symbols which will be substituted with the locales preferred hour formats. These changes also improve implicit format generation derived from whatever date and time data is passed to the Cldr.Date.to_string/3, Cldr.Time.to_string/3, Cldr.DateTime.to_string/3 functions.

Cldr Calendars

Primarily bug fixes to the Julian calendar and enhancements to make the Julian calendar configurable (in history, the Julian calendar new year has celebrated at different times of year).

Bug Fixes

• Fix year/1, month/2 and days_in_month/2 for Julian calendars.

• CLDR 48 fixes the dates for some Japanese eras.

Enhancements

• Use CLDR 48 calendar data. The primary change for calendars is to the era dates for the Japanese calendar.

• Adds use Cldr.Calendar.Julian, new_year_starting_month_and_day: {month_of_year, day_of_month} to allow modelling Julian calendars that don’t start on January 1st.

• Adds Cldr.Calendar.convert/2 which converts either a date or a date range to another calendar.

Cldr Calendars Japanese

Bug Fixes

• Several era dates were fixed in CLDR 48.

• The calendar_year/3 callback was fixed to return the regnal year (year of the reign of the then-current emperor) as expected by date time formatting.

Other library updates

These are updates primarily to be compatible with CLDR 48

• ex_cldr_units
• ex_cldr_routes including REAME update to recognise the original work by @BartOtten
• ex_cldr_locale_display
• ex_cldr_person_names
• ex_cldr_territories

Evolution of ex_cldr

Work continues on localize as the “version 3” of ex_cldr. The objective is to get the initial release covering the core, numbers, calendars and date/times released in the first quarter of 2026.

It’s my intention to continue to maintain ex_cldr and friends for as long as there is usage and bug reports. However I hope this will be the last update for ex_cldr to use a new version of CLDR data so that I can focus more time on localize. That scenario can be revised if required around March 2026 with CLDR 49.

Today ex_cldr hit one million downloads on hex.pm.

I wrote the first message on this topic in 2018 so it’s been a while. And maybe it’s three enthusiasts with a non-caching CI/CD pipeline only But it’s the most popular (by downloads) OSS library I’ve ever written. And because it’s Elixir it’s still fun!

Thanks to everyone who has used it, tried it, reported an issue or found a bug. The support and encouragement is much appreciated.

For early 2022 I’ll be launching:

• Chinese, Japanese and Korean Lunisolar calendars
• Updated ICU Message formatter with Gettext integration (due primarily to the work of @maennchen)

And then in the background I’m still working on transformations and collations.

Here’s to an exciting 2022!

CLDR version 43 was released this month, triggering a release cycle for the ex_cldr family of libraries. All-in-all, 15 ex_cldr libraries have been updated, polished and published as part of this release cycle. Please consult the relevant changelogs for more information.

Hightlights

• ex_cldr data is now versioned. This means that when installing locales, ex_cldr does a better job of detecting if there is any existing locale data and if that data is stale, the updated data will be downloaded and installed. This addresses a long standing issue.

• ex_money_sql now has a set of helper macros for querying money amounts. This was contributed by @am-kantox - really appreciated.

• All the additional calendar implementations (Ethiopian, Coptic, Persian, Lunisolar, Composite) are now all at version 1.0.0.

• Lots of work on the lunisolar calendars to make the API more approachable for common requirements like determining the data of the lunar new year for a given Gregorian year. Many thanks to @nineclue for pushing me to get this library into good shape.

• Adds a new calendar implementation, the Japanese calendar. This calendar is the Gregorian calendar but using the Japanese eras which count the number of years of an Emporer’s reign.

Upcoming

Despite this round of updates there are still some open issues which I know some very patient people are waiting for. With this round of updates complete I can now focus on the following over the next week:

1. Updating ex_cldr_routes to support localised verified routes. Highest priority.
2. Publishing ex_cldr_person_names which provides localised name formatting. The final two actions required are test conformance and documentation.
3. Improving ex_cldr_units to generalise the algebra for unit multiplication and division and to add a :wrapper option to Cldr.Unit.to_string/2 similar to the implementations for Cldr.Number.to_string/2 and Cldr.DateTime.to_string./2.