josevalim
Proposal: strftime-based calendar/datetime formatting
NOTE: this is a focused thread, so we appreciate if everybody stayed on topic. Feel free to comment anything in regards to calendar formatting but avoid off-topic or loosely related topics. For example, if you would like to discuss or propose other Calendar/DateTime features, please use a separate thread.
Hi everyone,
This is take two for calendar/datetime formatting in Elixir. This time, we are exploring strftime-based syntax which is much simpler in scope than the Unicode’s Locale Date Markup Language discussed previously.
Here is how the API will look like:
Calendar.format(date_or_time_or_datetime, "%Y-%m-%d %H:%M:%S.%f")
#=> {:ok, "2018-11-29 13:19:41.032412"}
The Calendar.format/2 entry point accepts any calendar type, using structural typing. This means we will be able to format any map that has the fields being formatted. In case a map field is missing, an appropriate error message will be raised.
The formatting function will also support multiple options to customize different aspects of formatting. Let’s take a look at them:
Options
The options can be broken into 2 distinct categories.
The first one is about localization:
:preferred_date- configures the default date:preferred_time- configures the default time:preferred_datetime- configures the default datetime:hours_in_am_pm- a function that receives hour, minute, second and returns thehours_in_am_pmtuple (as seen inc:Calendar.hours_in_am_pm/3)
Then we have options that control translations:
:am_pm_names- a function that receives:am,:pmand returns the relevant “am”/“pm” string:month_names- a function that receives the month as an integer and returns the month name as a string. For example,fn index -> {"January", "February", ...} |> elem(index - 1) end:abbreviated_month_names- a function that receives the month as an integer and returns the abbreviated month name as a string. For example,fn index -> {"Jan", "Feb", ...} |> elem(index - 1) end:day_of_week_names- a function that receives the day of the week as an integer and returns the day of the week as a string. For example,fn index -> {"Monday", "Tuesday", ...} |> elem(index - 1) end:abbreviated_day_of_week_names- a function that receives the day of the week as an integer and returns the abbreviated day of the week as a string. For example,fn index -> {"Mon", "Tue", ...} |> elem(index - 1) end
The default values of all options will be returned by the calendar, which should implement a formatter_config callback.
With the options out of the way, let’s talk about the formatting syntax.
strftime syntax
strftime has a simpler notation while still covering a wide range of use cases. This leaves it open for the community to support more complex formats such as ICU/Unicode/CLDR if desired.
The proposed syntax is an extension of strftime that also allows the padding width to be given as argument:
%<flag>?<width>?<format>
The flag is limited to certain characters, the width is a positive integer without leading zeros and the format is always a letter. Examples are %d. %-d, %4d and %_4d.
| Format | Description | Example (in ISO) | Source |
|---|---|---|---|
| %a | Abbreviated name of day | Mon | Calendar.day_of_week + :abbreviated_day_of_week_names |
| %A | Full name of day | Monday | Calendar.day_of_week + :day_of_week_names |
| %b | Abbreviated month name | Jan | struct.month + :abbreviated_month_names |
| %B | Full month name | January | struct.month + :month_names |
| %c | Preferred date+time representation | 2018-10-17 12:34:56 | :preferred_datetime |
| %d | Day of the month | 01, 12 | struct.month |
| %f | Microseconds | 000000, 999999, 0123 | struct.microsecond |
| %H | Hour using a 24-hour clock | 00, 23 | struct.hour |
| %I | Hour using a 12-hour clock | 01, 12 | struct.hour |
| %j | Day of the year | 001, 366 | Calendar.day_of_year |
| %m | Month | 01, 12 | struct.month |
| %M | Minute | 00, 59 | struct.minute |
| %p | “AM” or “PM” (noon is “PM”, midnight as “AM”) | AM, PM | Calendar.hours_in_am_pm + :am_pm_names |
| %P | “am” or “pm” (noon is “pm”, midnight as “am”) | am, pm | Calendar.hours_in_am_pm + :am_pm_names |
| %q | Quarter | 1, 2, 3, 4 | Calendar.quarter_of_year |
| %S | Second | 00, 59, 60 | struct.second |
| %u | Day of the week | 01 (monday), 07 (sunday) | Calendar.day_of_week |
| %x | Preferred date (without time) representation | 2018-10-17 | :preferred_date |
| %X | Preferred time (without date) representation | 12:34:56 | :preferred_time |
| %y | Year as 2-digits | 01, 01, 86, 18 | struct.year |
| %Y | Year | -0001, 0001, 1986 | struct.year |
| %z | +hhmm/-hhmm time zone offset from UTC (empty string if naive) | +0300, -0530 | struct.utc_offset + struct.std_offset |
| %Z | Time zone abbreviation (empty string if naive) | CET, BRST | struct.zone_abbr |
| %% | Literal “%” character | % |
The source column is used as a reference for the implementation and it won’t be present in the final documentation.
Flags
By default the modifiers above are all padded with zeros according to the ISO standard. The user can disable padding or use spaces with the flags below:
_(underscore) - pad a result with spaces, such as%_d-(dash) - do not pad a result, such as%-d0(zero) - pad with zeros, such as%0d
Rationale
Last but not least, it is worth discussing the rationale for date/time formatting. If you have an application that works with calendar types, it is likely that you have to format them at some point. If your application mostly interfaces with other systems, then there is a chance the built-in ISO format is enough, but not always. For example, some HTTP headers use a different format than the recommended ISO one. Therefore adding formatting to the standard library feels like a natural next step to the existing functionality. Furthermore, by choosing to support strftime, we guarantee that the implementation will have tiny footprint compared to larger standards.
Another discussion, which may or may not impact this one, is about parsing. The parsing specification is often the same as the formatting specification but we have explicitly decided to not support parsing in Elixir. First of all, it is really hard to support a general but efficient runtime date/time parsing strategy. If you expect certain formats, it is almost always better to define functions that parse specifically those formats. Things get trickier if we consider the fact we need to support internalization, which is trivial for formatting, but quite expensive for parsing. In other words, while we can provide a general and efficient implementation for formatting, we can’t do so for parsing. Since different trade-offs can be made here, ranging from performance to flexibility, we are not comfortable in picking one or another.
Roadmap
We don’t plan to add this functionality directly to Elixir. Instead we will develop it as a library and collect feedback. The complexity of the implementation will also dictate if this will become part of core or not, but we believe the implementation will be relatively simple.
Log
Log of changes done to the proposal.
- 2018/12/14 - proposal submitted
- 2018/12/15 - removed the Formatter callback from the proposal in favor of an option/config based API
- 2018/12/17 - removed
week_of_yearto align with current Elixir master - 2018/12/18 - added width and %q
- 2018/12/19 - remove calendar extensions section
Feedback
Your turn.
Most Liked
josevalim
We have finally implemented a library based on this proposal: https://github.com/plataformatec/nimble_strftime
Everyone, please do give it a try in your application! And @kip, let us know if it provides the necessary hooks for i18n/l10n.
axelson
I like the reduced scope! I think it will cover the majority of use-cases very well.
Is there a chance of providing a means for developers to flexibly provide their own custom formatting? It could be used to provide the ordinal formatting that @Nicd is requesting.
Here’s one example syntax:
iex> Calendar.format(now, `"%B %{d_ord}"`, MyApp.OrdinalFormatter)
December 14th
Another syntax might be more like %B %Cd where the C indicates that the next character represents a custom formatting directive.
Of course a setup like this would mean that you’d always have to pass in your formatter when formatting your date strings, but you could relatively easily create a MyApp.calendar_format/2 that would bake it in.
kip
@josevalim very practical proposal and probably easier to consume in most use cases than the CLDR encoding I’d agree.
No surprise that I look at this and consider how this could be used in a locale-specific way. Injecting a formatter is great. But I think it would be even better if there is an option to inject an ma tuple instead of just the module. That way a locale could also be injected (or any other parameter). Otherwise for a locale aware application one would need to either:
- Set a locale prior to calling
strftimewhich seems very brittle and not very clear. - Or, as your original example illustrates, there would need to be one module per locale and a lookup table to translate a locale into a module name which is a lot of scaffolding just to inject the right formatter for a locale.
An example of the ma approach would be:
Calendar.format(date_or_time_or_datetime, "%Y-%m-%d %H:%M:%S.%f", {MyApp, ["pt-BR"]})
This would make the intent clearer and be easier to adapt for Gettext and Cldr and any other locale-specific library. Implementation is just another function head that can easily be pattern matched. The actual date being formatted would be prepended to the other tuple arguments.
kip
With Calendar.strftime/2 being merged into Elixir master (from the NimbleStrftime lib) I’d like to open a proposal on some additional formatting options and gain community feedback.
There are three sub-proposals and any and all feedback is welcome.
Weeks (yes, again with the weeks conversation
)
Linux strftime has formatting flags for weeks. These being:
%U The week number of the current year as a decimal number, range
00 to 53, starting with the first Sunday as the first day of
week 01. See also %V and %W. (Calculated from tm_yday and
tm_wday.)
%V The ISO 8601 week number (see NOTES) of the current year as a
decimal number, range 01 to 53, where week 1 is the first week
that has at least 4 days in the new year. See also %U and %W.
(Calculated from tm_year, tm_yday, and tm_wday.) (SU)
%W The week number of the current year as a decimal number, range
00 to 53, starting with the first Monday as the first day of
week 01. (Calculated from tm_yday and tm_wday.)
Adding this formatting directives would require an update to the Calendar behaviour to provide support.
Options:
- No need, don’t add
- Just add
%Vfor the ISO week number and addCalendar.iso_week_of_year/1to theCalendarbehaviour - Maximise compatibility, go with all three and add the callbacks
Add support for Era
Calendar.day_of_era/1 returns {day, era} but that’s an integer, not a display format. Calendar behaviour doesn’t do any display format translation today. The new Calendar.strftime/2 does display format translation for AM and PM (and variants). So one approach is to simply enhance Calendar.strftime/2 to also map Calendar.ISO eras 0 -> display_name and 1 -> display_name. And there would need to be agreement on display name (AD versus CE and BC versus BCE).
Era isn’t used in day-to-day formatting for Gregorian calendars except for dates before year 1. It is used as a standard part of formatting Japanese calendars and for some format of Chinese calendars (and derivatives on the 60 year cycle).
strftime defines the following directives:
| Specifier | Meaning |
|---|---|
| %Ec | Date/time for current era. |
| %EC | Era name. |
| %Ex | Date for current era. |
| %EX | Time for current era. |
| %Ey | Era year. This is the offset from the base year. |
| %EY | Year for current era. |
These can be implemented without a change to the calendar behaviour if Calendar.strftime/2 treats era like it treats am/pm and provides an internal translation. It would perhaps be better that both of these (ie am/pm and era translations became behaviours since that would also help in international projects using Gettext or Cldr.
Options
- Era? Who cares, forget about it
- Compatibility is a good idea and implementation seems simple. Do it.
- I care about calendars beyond
Calendar.ISOso I need this (I’m not holding my breath on this one
)
Localised number systems (ie not Latin alphabet)
strftime supports localising the date format to use non-latin alphabets. These is also supported in ex_cldr and friends. The default for these directives is to use the Latin alphabet so implementing these directives is quite trivial. If no configuration is provided in options, use the fallback to the Latin characters.
The formatting directives are:
| Specifier | Meaning |
|---|---|
| %Od | Represents the day of the month, using the locale’s alternative numeric symbols, filled as needed with leading 0’s if an alternative symbol for 0 exists. If an alternative symbol for 0 does not exist, the %Od modified conversion specifier uses leading space characters. |
| %Oe | Represents the day of the month, using the locale’s alternative numeric symbols, filled as needed with leading 0’s if an alternative symbol for 0 exists. If an alternative symbol for 0 does not exist, the %Oe modified conversion specifier uses leading space characters. |
| %OH | Represents the hour in 24-hour clock time, using the locale’s alternative numeric symbols. |
| %OI | Represents the hour in 12-hour clock time, using the locale’s alternative numeric symbols. |
| %Om | Represents the month, using the locale’s alternative numeric symbols. |
| %OM | Represents the minutes, using the locale’s alternative numeric symbols. |
| %OS | Represents the seconds, using the locale’s alternative numeric symbols. |
| %Ou | Represents the weekday as a number using the locale’s alternative numeric symbols. |
| %OU | Represents the week number of the year, using the locale’s alternative numeric symbols. Sunday is considered the first day of the week. Use the rules corresponding to the %U conversion specifier. |
| %OV | Represents the week number of the year (Monday as the first day of the week, rules corresponding to %V) using the locale’s alternative numeric symbols. |
| %Ow | Represents the number of the weekday (with Sunday equal to 0), using the locale’s alternative numeric symbols. |
| %OW | Represents the week number of the year using the locale’s alternative numeric symbols. Monday is considered the first day of the week. Use the rules corresponding to the %W conversion specifier. |
| %Oy | Represents the year (offset from %C) using the locale’s alternative numeric symbols. |
Options
- Who cares about countries and people that don’t use the Latin alphabet. Forget it.
- Good idea - easy to implement, no impact if I’m not doing non-latin alphabet code. Glad to see Elixir embracing global cultures (ok, I’m pitching I know
)
kip
I’ve been thinking about this a lot on a long plane flight so here goes:
Calendar module responsibilities
This part is not about formatting but separation and encapsulation of concerns. But if there is agreement with this, and my proposal about augmenting the format flags, then it greatly simplifies the contract between strftime and a calendar.
I think a calendar should encapsulate everything relevant to calculating days, weeks, months, years etc. That is, any configuration should be baked into the module that implements the Calendar behaviour.
This seems obvious when we talk about the iSO calendar versus the Balinese, or Ancient Egyptian or the Islamic calendars. But its less obvious if we consider the myriad of calendars derived from the Gregorian.
For example, in the US, a company may have a fiscal calendar that starts on an arbitrary day of the year. All months, quarters, weeks and days are calculated from this date. Cisco Systems has a financial year calendar that ends on the last Saturday of July. There are a whole class of calendars called 445 calendars and their cousins the 454 and 544 which are common in retail and are designed to make it better for comparing this year to prior year periods.
Although I started thinking this would be best done as configuration passed around in each structurally compatible Date, Time or DateTime struct or map, the variations are too many. I now feel it is better that this detail be backed into a module that implements the Calendar behaviour. This doesn’t make implementing a calendar any more complex. It does potentially have the side effect of having to generate calendar modules at runtime for some use cases.
Calendar Behaviour
If you accept that a Calendar module encapsulates all the required configuration and implementation required for the behaviour then implementing strftime comes back to mostly calling the behaviour functions. And week_of_year should definitely stay - in all of those business calendars is used a lot and the notion of a week is, I think we would all agree, a very common case.
If you got this far then the missing pieces of the Calendar behaviour that would be required to support strftime would be day_name, month_name (short and long) and the am/pm functionality described by @josevalim above. Since these will need to be implemented anyway it feels as if they should be part of the Calendar behaviour.
Formatting
Formatting for strftime should not be the calendars concern (in my opinion). Thats a representational issue and other than names for days and months, formatting should be the strftime formats responsibility. In examining the Ruby and Python implementations I see that the Python version is quite sparse but the Ruby version includes an approach that I think is pragmatic.
A Proposal
@spec strftime(date_or_time_or_datetime, format :: String.t(), options :: Map.t()) ::
String.t | {:error, String.t}
format is a format string as described in the original proposal however it includes an optional width that is used to determine padding. This is similar to how printf and friends in C define padding.
Format encoding: %<flags><width><conversion>.
Flags (based upon the Ruby implementation)
| Flag | Description |
|---|---|
| - | don’t pad a numerical output |
| _ | use spaces for padding |
| 0 | use zeros for padding |
| ^ | upcase the result string |
| # | change case |
| : | use colons for %z |
Using this approach allows a calendar implementor to use the %x directive to return a format string that uncludes the required padding. That is, %x is really just an interpolation of a format defined in the calendar. And that format can include the appropriate padding. Same would apply for %X and %c. The format for these “preferred formats” can also then vary based upon locale without too much difficulty in any locale-aware calendar.
Examples
| Format | Output |
|---|---|
| %m | 1, 12 |
| %2m | 01, 12 |
| %A | Monday |
| %^A | MONDAY |
Summary
This proposal intents to:
-
Recommend reinforcing the
Calendarbehaviour and add to it those missing pieces that support formatting which are primarily theday_nameandmonth_name(short and long) functions. -
Simplify the interaction between
strftimeand aCalendarmodule by reverting to the original plan of simply calling the relevant functions in the calendar module. -
Making formatting decisions part of the
formatstring (ie flags and width modifiers)
Options
I have included an options argument to strftime above because for localisation a locale name needs to be specified. In Python a locale is part of the stdlib and runtime environment but not so Elixir. Therefore it should be passed in to the calendar. It could be argued that since there is no other part of Elixir that embodies the idea of localisation then this is not the place to start. I admit my bias given my work on Cldr, but I think the idea of a locale should be able to be specified.
A Calendar should feel free to return {:error, "unknown locale"} if a locale is requested that it doesn’t support. Of if a more loose contract is preferred then treat the locale as entirely optional.







