josevalim

josevalim

Creator of Elixir

Proposal: strftime-based calendar/datetime formatting

NOTE: this is a focused thread, so we appreciate if everybody stayed on topic. Feel free to comment anything in regards to calendar formatting but avoid off-topic or loosely related topics. For example, if you would like to discuss or propose other Calendar/DateTime features, please use a separate thread.

Hi everyone,

This is take two for calendar/datetime formatting in Elixir. This time, we are exploring strftime-based syntax which is much simpler in scope than the Unicode’s Locale Date Markup Language discussed previously.

Here is how the API will look like:

Calendar.format(date_or_time_or_datetime, "%Y-%m-%d %H:%M:%S.%f")
#=> {:ok, "2018-11-29 13:19:41.032412"}

The Calendar.format/2 entry point accepts any calendar type, using structural typing. This means we will be able to format any map that has the fields being formatted. In case a map field is missing, an appropriate error message will be raised.

The formatting function will also support multiple options to customize different aspects of formatting. Let’s take a look at them:

Options

The options can be broken into 2 distinct categories.

The first one is about localization:

  • :preferred_date - configures the default date
  • :preferred_time - configures the default time
  • :preferred_datetime - configures the default datetime
  • :hours_in_am_pm - a function that receives hour, minute, second and returns the hours_in_am_pm tuple (as seen in c:Calendar.hours_in_am_pm/3)

Then we have options that control translations:

  • :am_pm_names - a function that receives :am, :pm and returns the relevant “am”/“pm” string
  • :month_names - a function that receives the month as an integer and returns the month name as a string. For example, fn index -> {"January", "February", ...} |> elem(index - 1) end
  • :abbreviated_month_names - a function that receives the month as an integer and returns the abbreviated month name as a string. For example, fn index -> {"Jan", "Feb", ...} |> elem(index - 1) end
  • :day_of_week_names - a function that receives the day of the week as an integer and returns the day of the week as a string. For example, fn index -> {"Monday", "Tuesday", ...} |> elem(index - 1) end
  • :abbreviated_day_of_week_names - a function that receives the day of the week as an integer and returns the abbreviated day of the week as a string. For example, fn index -> {"Mon", "Tue", ...} |> elem(index - 1) end

The default values of all options will be returned by the calendar, which should implement a formatter_config callback.

With the options out of the way, let’s talk about the formatting syntax.

strftime syntax

strftime has a simpler notation while still covering a wide range of use cases. This leaves it open for the community to support more complex formats such as ICU/Unicode/CLDR if desired.

The proposed syntax is an extension of strftime that also allows the padding width to be given as argument:

%<flag>?<width>?<format>

The flag is limited to certain characters, the width is a positive integer without leading zeros and the format is always a letter. Examples are %d. %-d, %4d and %_4d.

Format Description Example (in ISO) Source
%a Abbreviated name of day Mon Calendar.day_of_week + :abbreviated_day_of_week_names
%A Full name of day Monday Calendar.day_of_week + :day_of_week_names
%b Abbreviated month name Jan struct.month + :abbreviated_month_names
%B Full month name January struct.month + :month_names
%c Preferred date+time representation 2018-10-17 12:34:56 :preferred_datetime
%d Day of the month 01, 12 struct.month
%f Microseconds 000000, 999999, 0123 struct.microsecond
%H Hour using a 24-hour clock 00, 23 struct.hour
%I Hour using a 12-hour clock 01, 12 struct.hour
%j Day of the year 001, 366 Calendar.day_of_year
%m Month 01, 12 struct.month
%M Minute 00, 59 struct.minute
%p “AM” or “PM” (noon is “PM”, midnight as “AM”) AM, PM Calendar.hours_in_am_pm + :am_pm_names
%P “am” or “pm” (noon is “pm”, midnight as “am”) am, pm Calendar.hours_in_am_pm + :am_pm_names
%q Quarter 1, 2, 3, 4 Calendar.quarter_of_year
%S Second 00, 59, 60 struct.second
%u Day of the week 01 (monday), 07 (sunday) Calendar.day_of_week
%x Preferred date (without time) representation 2018-10-17 :preferred_date
%X Preferred time (without date) representation 12:34:56 :preferred_time
%y Year as 2-digits 01, 01, 86, 18 struct.year
%Y Year -0001, 0001, 1986 struct.year
%z +hhmm/-hhmm time zone offset from UTC (empty string if naive) +0300, -0530 struct.utc_offset + struct.std_offset
%Z Time zone abbreviation (empty string if naive) CET, BRST struct.zone_abbr
%% Literal “%” character %

The source column is used as a reference for the implementation and it won’t be present in the final documentation.

Flags

By default the modifiers above are all padded with zeros according to the ISO standard. The user can disable padding or use spaces with the flags below:

  • _ (underscore) - pad a result with spaces, such as %_d
  • - (dash) - do not pad a result, such as %-d
  • 0 (zero) - pad with zeros, such as %0d

Rationale

Last but not least, it is worth discussing the rationale for date/time formatting. If you have an application that works with calendar types, it is likely that you have to format them at some point. If your application mostly interfaces with other systems, then there is a chance the built-in ISO format is enough, but not always. For example, some HTTP headers use a different format than the recommended ISO one. Therefore adding formatting to the standard library feels like a natural next step to the existing functionality. Furthermore, by choosing to support strftime, we guarantee that the implementation will have tiny footprint compared to larger standards.

Another discussion, which may or may not impact this one, is about parsing. The parsing specification is often the same as the formatting specification but we have explicitly decided to not support parsing in Elixir. First of all, it is really hard to support a general but efficient runtime date/time parsing strategy. If you expect certain formats, it is almost always better to define functions that parse specifically those formats. Things get trickier if we consider the fact we need to support internalization, which is trivial for formatting, but quite expensive for parsing. In other words, while we can provide a general and efficient implementation for formatting, we can’t do so for parsing. Since different trade-offs can be made here, ranging from performance to flexibility, we are not comfortable in picking one or another.

Roadmap

We don’t plan to add this functionality directly to Elixir. Instead we will develop it as a library and collect feedback. The complexity of the implementation will also dictate if this will become part of core or not, but we believe the implementation will be relatively simple.

Log

Log of changes done to the proposal.

  • 2018/12/14 - proposal submitted
  • 2018/12/15 - removed the Formatter callback from the proposal in favor of an option/config based API
  • 2018/12/17 - removed week_of_year to align with current Elixir master
  • 2018/12/18 - added width and %q
  • 2018/12/19 - remove calendar extensions section

Feedback

Your turn.

Most Liked

josevalim

josevalim

Creator of Elixir

We have finally implemented a library based on this proposal: https://github.com/plataformatec/nimble_strftime

Everyone, please do give it a try in your application! And @kip, let us know if it provides the necessary hooks for i18n/l10n.

axelson

axelson

Scenic Core Team

I like the reduced scope! I think it will cover the majority of use-cases very well.

Is there a chance of providing a means for developers to flexibly provide their own custom formatting? It could be used to provide the ordinal formatting that @Nicd is requesting.

Here’s one example syntax:

iex> Calendar.format(now, `"%B %{d_ord}"`, MyApp.OrdinalFormatter)
December 14th

Another syntax might be more like %B %Cd where the C indicates that the next character represents a custom formatting directive.

Of course a setup like this would mean that you’d always have to pass in your formatter when formatting your date strings, but you could relatively easily create a MyApp.calendar_format/2 that would bake it in.

kip

kip

ex_cldr Core Team

@josevalim very practical proposal and probably easier to consume in most use cases than the CLDR encoding I’d agree.

No surprise that I look at this and consider how this could be used in a locale-specific way. Injecting a formatter is great. But I think it would be even better if there is an option to inject an ma tuple instead of just the module. That way a locale could also be injected (or any other parameter). Otherwise for a locale aware application one would need to either:

  1. Set a locale prior to calling strftime which seems very brittle and not very clear.
  2. Or, as your original example illustrates, there would need to be one module per locale and a lookup table to translate a locale into a module name which is a lot of scaffolding just to inject the right formatter for a locale.

An example of the ma approach would be:

 Calendar.format(date_or_time_or_datetime, "%Y-%m-%d %H:%M:%S.%f", {MyApp, ["pt-BR"]})

This would make the intent clearer and be easier to adapt for Gettext and Cldr and any other locale-specific library. Implementation is just another function head that can easily be pattern matched. The actual date being formatted would be prepended to the other tuple arguments.

kip

kip

ex_cldr Core Team

With Calendar.strftime/2 being merged into Elixir master (from the NimbleStrftime lib) I’d like to open a proposal on some additional formatting options and gain community feedback.

There are three sub-proposals and any and all feedback is welcome.

Weeks (yes, again with the weeks conversation :slight_smile: )

Linux strftime has formatting flags for weeks. These being:

   %U     The week number of the current year as a decimal number, range
          00 to 53, starting with the first Sunday as the first day of
          week 01.  See also %V and %W.  (Calculated from tm_yday and
          tm_wday.)

   %V     The ISO 8601 week number (see NOTES) of the current year as a
          decimal number, range 01 to 53, where week 1 is the first week
          that has at least 4 days in the new year.  See also %U and %W.
          (Calculated from tm_year, tm_yday, and tm_wday.)  (SU)

   %W     The week number of the current year as a decimal number, range
          00 to 53, starting with the first Monday as the first day of
          week 01.  (Calculated from tm_yday and tm_wday.)

Adding this formatting directives would require an update to the Calendar behaviour to provide support.

Options:

  1. No need, don’t add
  2. Just add %V for the ISO week number and add Calendar.iso_week_of_year/1 to the Calendar behaviour
  3. Maximise compatibility, go with all three and add the callbacks

Add support for Era

Calendar.day_of_era/1 returns {day, era} but that’s an integer, not a display format. Calendar behaviour doesn’t do any display format translation today. The new Calendar.strftime/2 does display format translation for AM and PM (and variants). So one approach is to simply enhance Calendar.strftime/2 to also map Calendar.ISO eras 0 -> display_name and 1 -> display_name. And there would need to be agreement on display name (AD versus CE and BC versus BCE).

Era isn’t used in day-to-day formatting for Gregorian calendars except for dates before year 1. It is used as a standard part of formatting Japanese calendars and for some format of Chinese calendars (and derivatives on the 60 year cycle).

strftime defines the following directives:

Specifier Meaning
%Ec Date/time for current era.
%EC Era name.
%Ex Date for current era.
%EX Time for current era.
%Ey Era year. This is the offset from the base year.
%EY Year for current era.

These can be implemented without a change to the calendar behaviour if Calendar.strftime/2 treats era like it treats am/pm and provides an internal translation. It would perhaps be better that both of these (ie am/pm and era translations became behaviours since that would also help in international projects using Gettext or Cldr.

Options

  1. Era? Who cares, forget about it
  2. Compatibility is a good idea and implementation seems simple. Do it.
  3. I care about calendars beyond Calendar.ISO so I need this (I’m not holding my breath on this one :slight_smile: )

Localised number systems (ie not Latin alphabet)

strftime supports localising the date format to use non-latin alphabets. These is also supported in ex_cldr and friends. The default for these directives is to use the Latin alphabet so implementing these directives is quite trivial. If no configuration is provided in options, use the fallback to the Latin characters.

The formatting directives are:

Specifier Meaning
%Od Represents the day of the month, using the locale’s alternative numeric symbols, filled as needed with leading 0’s if an alternative symbol for 0 exists. If an alternative symbol for 0 does not exist, the %Od modified conversion specifier uses leading space characters.
%Oe Represents the day of the month, using the locale’s alternative numeric symbols, filled as needed with leading 0’s if an alternative symbol for 0 exists. If an alternative symbol for 0 does not exist, the %Oe modified conversion specifier uses leading space characters.
%OH Represents the hour in 24-hour clock time, using the locale’s alternative numeric symbols.
%OI Represents the hour in 12-hour clock time, using the locale’s alternative numeric symbols.
%Om Represents the month, using the locale’s alternative numeric symbols.
%OM Represents the minutes, using the locale’s alternative numeric symbols.
%OS Represents the seconds, using the locale’s alternative numeric symbols.
%Ou Represents the weekday as a number using the locale’s alternative numeric symbols.
%OU Represents the week number of the year, using the locale’s alternative numeric symbols. Sunday is considered the first day of the week. Use the rules corresponding to the %U conversion specifier.
%OV Represents the week number of the year (Monday as the first day of the week, rules corresponding to %V) using the locale’s alternative numeric symbols.
%Ow Represents the number of the weekday (with Sunday equal to 0), using the locale’s alternative numeric symbols.
%OW Represents the week number of the year using the locale’s alternative numeric symbols. Monday is considered the first day of the week. Use the rules corresponding to the %W conversion specifier.
%Oy Represents the year (offset from %C) using the locale’s alternative numeric symbols.

Options

  1. Who cares about countries and people that don’t use the Latin alphabet. Forget it.
  2. Good idea - easy to implement, no impact if I’m not doing non-latin alphabet code. Glad to see Elixir embracing global cultures (ok, I’m pitching I know :slight_smile: )
kip

kip

ex_cldr Core Team

I’ve been thinking about this a lot on a long plane flight so here goes:

Calendar module responsibilities

This part is not about formatting but separation and encapsulation of concerns. But if there is agreement with this, and my proposal about augmenting the format flags, then it greatly simplifies the contract between strftime and a calendar.

I think a calendar should encapsulate everything relevant to calculating days, weeks, months, years etc. That is, any configuration should be baked into the module that implements the Calendar behaviour.

This seems obvious when we talk about the iSO calendar versus the Balinese, or Ancient Egyptian or the Islamic calendars. But its less obvious if we consider the myriad of calendars derived from the Gregorian.

For example, in the US, a company may have a fiscal calendar that starts on an arbitrary day of the year. All months, quarters, weeks and days are calculated from this date. Cisco Systems has a financial year calendar that ends on the last Saturday of July. There are a whole class of calendars called 445 calendars and their cousins the 454 and 544 which are common in retail and are designed to make it better for comparing this year to prior year periods.

Although I started thinking this would be best done as configuration passed around in each structurally compatible Date, Time or DateTime struct or map, the variations are too many. I now feel it is better that this detail be backed into a module that implements the Calendar behaviour. This doesn’t make implementing a calendar any more complex. It does potentially have the side effect of having to generate calendar modules at runtime for some use cases.

Calendar Behaviour

If you accept that a Calendar module encapsulates all the required configuration and implementation required for the behaviour then implementing strftime comes back to mostly calling the behaviour functions. And week_of_year should definitely stay - in all of those business calendars is used a lot and the notion of a week is, I think we would all agree, a very common case.

If you got this far then the missing pieces of the Calendar behaviour that would be required to support strftime would be day_name, month_name (short and long) and the am/pm functionality described by @josevalim above. Since these will need to be implemented anyway it feels as if they should be part of the Calendar behaviour.

Formatting

Formatting for strftime should not be the calendars concern (in my opinion). Thats a representational issue and other than names for days and months, formatting should be the strftime formats responsibility. In examining the Ruby and Python implementations I see that the Python version is quite sparse but the Ruby version includes an approach that I think is pragmatic.

A Proposal

@spec strftime(date_or_time_or_datetime, format :: String.t(), options :: Map.t()) :: 
    String.t | {:error, String.t}

format is a format string as described in the original proposal however it includes an optional width that is used to determine padding. This is similar to how printf and friends in C define padding.

Format encoding: %<flags><width><conversion>.

Flags (based upon the Ruby implementation)

Flag Description
- don’t pad a numerical output
_ use spaces for padding
0 use zeros for padding
^ upcase the result string
# change case
: use colons for %z

Using this approach allows a calendar implementor to use the %x directive to return a format string that uncludes the required padding. That is, %x is really just an interpolation of a format defined in the calendar. And that format can include the appropriate padding. Same would apply for %X and %c. The format for these “preferred formats” can also then vary based upon locale without too much difficulty in any locale-aware calendar.

Examples

Format Output
%m 1, 12
%2m 01, 12
%A Monday
%^A MONDAY

Summary

This proposal intents to:

  1. Recommend reinforcing the Calendar behaviour and add to it those missing pieces that support formatting which are primarily the day_name and month_name (short and long) functions.

  2. Simplify the interaction between strftime and a Calendar module by reverting to the original plan of simply calling the relevant functions in the calendar module.

  3. Making formatting decisions part of the format string (ie flags and width modifiers)

Options

I have included an options argument to strftime above because for localisation a locale name needs to be specified. In Python a locale is part of the stdlib and runtime environment but not so Elixir. Therefore it should be passed in to the calendar. It could be argued that since there is no other part of Elixir that embodies the idea of localisation then this is not the place to start. I admit my bias given my work on Cldr, but I think the idea of a locale should be able to be specified.

A Calendar should feel free to return {:error, "unknown locale"} if a locale is requested that it doesn’t support. Of if a more loose contract is preferred then treat the locale as entirely optional.

Where Next?

Popular in Proposals Top

josevalim
NOTE: this is a focused thread, so we appreciate if everybody stayed on topic. Feel free to comment anything in regards to calendar forma...
New
josevalim
Hi everyone, This is a proposal for introducing local accumulators to Elixir. This is another attempt of solving the comprehension probl...
1043 9939 246
New
josevalim
Hi everyone, as you may be aware, we are researching a type system for Elixir. As preparation for my upcoming ElixirConf US talk, I woul...
New
josevalim
I am resubmitting the proposal from earlier today with more context and more focus on the important parts. Some concerns and praises stay...
452 17667 124
New
josevalim
UPDATE: This proposal has been retracted. Read the new proposal here: Local accumulators for cleaner comprehensions Hi everyone, This i...
New
josevalim
One of the major differences between running your application as a release and as a Mix project is the differences in configuration. Mix ...
382 17497 108
New
michalmuskala
TL;DR: The Elixir Core team is announcing a call for proposals to extend support for time zones in Elixir’s standard library. The reason...
New
josevalim
Hi everyone, Erlang/OTP 21 comes with two new guards contributed by @michalmuskala: map_get/2 and is_map_key/2. Now we need to discuss h...
New
josevalim
Hi everyone, We are considering deprecating 'charlists' in Elixir in favor of ~c"charlist". In many languages, 'foobar' is equivalent to...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New

Other popular topics Top

chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30048 115
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
beno
I will often find my self writing things similar to: case some_value do nil -&gt; something() "" -&gt; something() _ -&gt; someth...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
Nvim
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New

We're in Beta

About us Mission Statement