tc39 / proposal-intl-enumeration

Return supported values of options, such as timeZone, calendars, numberingSystems, currencies, units

Intl Enumeration API Specification

List supported values of options in pre-existing ECMA 402 API.

Stage 3

• Advanced to Stage 1 in TC39 June 2020 meeting.
  • 2020-06-03 Meeting Notes
• Advanced to Stage 2 in TC39 September 2020 meeting.
  • 2020-09-22 Meeting Notes
• Update in TC39 Nov 2020 meeting.
  • 2020-11-18 Meeting Notes
• Update in TC39 Apr 2021 meeting.
  • 2021-04-21 Meeting Notes
• Update in TC39 May 2021 meeting.
  • 2021-05-25 Meeting Notes
• Advanced Stage 3 advancement in TC39 July 2021 meeting
  • 2021-07-13 Meeting Notes
• Update in TC39 Dec 2021 meeting.

TG2 (ECMA402) Working Meeting Notes:

• 2020-05-20 TG2 Meeting Notes
• 2020-08-13 TG2 Meeting Notes
• 2020-11-05 TG2 Meeting Notes
• 2021-03-11 TG2 Meeting Notes
• 2021-04-08 TG2 Meeting Notes
• 2021-05-06 TG2 Meeting Notes
• 2021-06-03 TG2 Meeting Notes
• 2021-07-01 TG2 Meeting Notes

Need Help!

This proposal is now reaching stage 3, we need help to add tests into Test262 now. See Issue 3131 in test262 for details. We would also love to see polyfill available to web developers.

Motivation

ECMA402 allows the set of supported local time zones, collations, calendars, numbering systems and currency as implementation dependents. This proposal provides an API to identify the supported values of these options in the implementation.

The proposed API empower the caller to feature detect the support in the implementation easily and could be used to download polyfill for missing support easily in the beginning of the loading time. For example, a web page could exam the return value and decide it need to import and install a DateTimeFormat polyfill because the return time zones does not contains the server stored user time zone preference and will not be able to use Intl.DateTimeFormat to format the time according to the server side stored time zone preferences. Users may set up their account and store their time zone preferences in the server side while previously using a different user agent which already supports such a time zone and need to use the user agent in another machine in an internet cafe to access the web application. The proposed API allows the web application to do an up front check and import polyfill from server side while the prefered time zone is not available in the user agent. Similarly, the web application may check for the supported set of currency and unit to implement the bootstrap logic of dynamic import polyfill for some javascript library. For example, engineers in Google are currently changing the closure library implementation to use ECMA402 support if the functionality is available to avoid unnecessary data download, but still download the necessary data and import polyfill while the desired support is not available. The proposed API allows the library to determine the logic with a very minimum amount of data.

Web applications can also combine the use of the proposed API with Intl.DisplayNames and Intl.NumberFormat to build a powerful UI without downloading a lot of data. For example, web applications can use the proposed API to get the list of supported currency, then use Intl.DisplayNames to get back the display name of the currency to build a menu for user to select the calendar, and format the currency amount by using Intl.NumberFormat API. The return value could also support the code to decide which set of the currency conversion information should be downloaded from the server and only download the set of currency the implementation knows how to format correctly, or download all the currency tables and then import polifil to support the one which is not supported. Without the proposed API the web application may either download an extra currency exchange table for currency which won’t be formatted correctly by Intl.NumberFormat and neither be able to efficiently determine which set of currency format data need to be imported with the polyfill. The proposed API empowers the web developer to design an efficient system to import minimum information from the server to implement such a fallback mechanism.

ECMA 402 is also used on the server side. A server side application code running on top of Node.js, which is built based on v8 and can access ECMA402 code can call the proposed API to generate a list of time zone names known by the server into html and return to the client in a calendar application.

In summary, the proposed API is an important facility to allow web developers to detect missing support and import fallback polyfills efficiently with a minimum amount of data in the application initialization time. It also allows web application, in conjunction with using other ECMA402 API to program powerful internationalization UI with very minimum information from server side. It is also a vital facility for server side usage of ECMA402.

Overview

One method of Intl, return an array.

Intl.supportedValuesOf(key)

Supported Keys

• calendar
• collation
• currency
• numberingSystem
• timeZone
• unit

Background

tc39/ecma402#435

Use case

Use Case 1 - Detect Missing Features to Trigger the Import of Polifill

The user stores their application user preference in the server side to express they want to use the ROC calendar. While the user logs into the application from his new mobile phone, the application code calls this API and found the "roc" calendar is not in the support set, the application imports polyfill from the server side which implements the ROC calendar and uses the polyfill. The user then logs into the application from his desktop, and the application calls this API and found the “roc” calendar is supported, the application therefore just calls the Intl.DateTimeFormat directly.

Use Case 2 - Server Side Programming

The web programmer is programming javascript running on top of Node.js by using ECMA402 code. The application (server side javascript running on top of Node.js) call this API to find out all the support set of calendar, collation, currency, numbering system, and time zone, and unit, and use that to generate UI selector in html and send to the client to let the user to select in their application preference. After the user submits their choice, the server side application stores the user’s application preference into server side storage. The application code (server side javascript running on Node.js) later uses these preference values to output the html to the client.

Use Case 3 - Building Time Zone picker inside WebGL

Developer is working on a video game by using JavaScript to program WebGL and therefore all what users see on the screen is not HTML rendering and everything is 3D in WebGL. The programmer is writing the code to let the character the player control go enter a virtual store and select the currency they like to use to trade the virtual goodie, the programmer therefore need to build a 3D GUI in WebGL which list all the currency Intl.NumberFormat can support to format. A non-existing HTML solution won’t help them even if all the user agents support such a feature in HTML one day.

Prior Arts

Get the List of TimeZone

• Use Cases:
  • How to get list of all timezones in javascript on Stack Overflow
• Prior Arts:
  • momentjs moment.tz.names
  • tzdb
  • countries-and-timezones
  • country-timezone

Get the List of Currency Codes

• Use Cases:
  • User request on Stack Overflow
• Prior Arts:
  • currency-codes

Usage example

// Find out the supported calendars
Intl.supportedValuesOf("calendar").forEach(function(calendar) {
   // 'buddhist', 'chinese',  ... 'islamicc'
});

// Find out the supported currencies
Intl.supportedValuesOf("currency").forEach(function(currency) {
   // 'AED', 'AFN', 'ALL', ... 'ZWL'
});

// Find out the supported numbering systems
Intl.supportedValuesOf("numberingSystem").forEach(function(nu) {
  // 'adlm', 'ahom', 'arab', ...  'wara', 'wcho'
});

// Find out the supported time zones
Intl.supportedValuesOf("timeZone").forEach(function(timeZone) {
  // 'Africa/Abidjan', 'Africa/Accra', ... 'Pacific/Wallis'
});

// Find out the supported units
Intl.supportedValuesOf("unit").forEach(function(unit) {
  // 'acre', 'bit', 'byte', ... 'year'
});

Authors

• Frank Tang (@FrankYFTang)

Designated reviewers

• Shane Carr @sffc
• Jordan Harband @ljharb

ECMAScript editors

• Richard Gibson @gibson042

Proposal

Spec

• https://tc39.es/proposal-intl-enumeration/

References

Analysis of Privacy / Fingerprinting Concerns

• “Intl.Enumeration Privacy Implications Mozilla’s Recommendation”., Tom Ritter, Anne van Kesteren, Steven Englehardt, Zibi Braniecki, Mozilla, Feb 4, 2021.
• Discussion during 2021-03 ECMA402 Meeting
• Duscussion during 2021-04 ECMA402 Meeting (link to be added soon)

Implementation Status

• v8 tracking bug
• Mozilla
• JSC
• Test262

============================ Ignore Text Below ============================

The following are from the template, will be deleted later

This repo is setup by following instruction on TC39 template-for-proposals

  Your explainer can point readers to the `index.html` generated from `spec.emu`
  via markdown like

  ```markdown
  You can browse the [ecmarkup output](https://ACCOUNT.github.io/PROJECT/)
  or browse the [source](https://github.com/ACCOUNT/PROJECT/blob/master/spec.emu).
  ```

  where *ACCOUNT* and *PROJECT* are the first two path elements in your project's Github URL.
  For example, for github.com/**tc39**/**template-for-proposals**, *ACCOUNT* is "tc39"
  and *PROJECT* is "template-for-proposals".

Maintain your proposal repo

1. Make your changes to spec.emu (ecmarkup uses HTML syntax, but is not HTML, so I strongly suggest not naming it ".html")
2. Any commit that makes meaningful changes to the spec, should run npm run build and commit the resulting output.
3. Whenever you update ecmarkup, run npm run build and commit any changes that come from that dependency.

proposal-intl-enumeration's Issues

Intl.supportedValuesOf('timeZone') and UTC

As stated in #36, probably the most common use case for enumeration is showing a list in a UI like a dropdown or other list-based picker.

How come this new API does not list valid values such as the common UTC or CET in this list?

I recon if the common use case is to show a dropdown list of allowed values, there should be a way to get those included in the list.

I preemptively apologize if I am out of my way asking about this in this repo.

Transfer from tc39-transfer repo to tc39 repo

I just transfer the proposal into tc39-transfer after Stage 1 advancement consensus in 2020-06-03 TC39 meeting. @ljharb @bterlson @gesa @codehag please transfer this repository to the TC39 organization in the time fit. Thanks

AvailableUnits should contain exactly all simple units sanctioned and not more entries

https://tc39.es/proposal-intl-enumeration/#sec-availableunits:

The AvailableUnits abstract operation returns a List, ordered as if an Array of the same values had been sorted using %Array.prototype.sort% using undefined as comparefn, that contains the unique values of simple unit identifiers listed in every row of Table 1, except the header row.

The verb "contains" can be interpreted that more units can be returned. But because IsWellFormedUnitIdentifier restricts the list of supported units and because no additional units are allowed per https://tc39.es/ecma402/#annex-implementation-dependent-behaviour, it's not useful to return more units.

Maybe change the description to use "consists" instead of "contains":

The AvailableUnits abstract operation returns a List, ordered as if an Array of the same values had been sorted using %Array.prototype.sort% using undefined as comparefn, that consists of the unique values of simple unit identifiers listed in every row of Table 1, except the header row.

stage 3 review

unrelated to the review: i noticed https://tc39.es/ecma402/#sec-canonicalizelocalelist and https://tc39.es/ecma402/#sec-getoption don't auto-link from the proposal spec, altho https://tc39.es/ecma402/#table-numbering-system-digits and https://tc39.es/proposal-intl-enumeration/#table-sanctioned-simple-unit-identifiers do (cc @bakkot, maybe ecmarkup doesn't handle linking 402 AOs properly?)

• Confirming that Intl.getSupportedValuesOf is supposed to throw on a null options argument? (seems fine, just making sure)
• editorial: "The list must includes" should be "The list must include", throughout
• editorial: "list of String" should be "list of Strings"
• https://tc39.es/proposal-intl-enumeration/#sec-availablecalendars doesn't specify that the list must be unique, nor does it specify its ordering. I would hope that it's always unique, and always sorted deterministically, even if the contents of strings are defined by the host or implementation. It would also be helpful to use 262's terms for these, to indicate it's up to the host/implementation. Same feedback on all the "AvailableX" AOs.
• normative: Why do we need a "SupportedValues.prototype" object? the only purpose of its internal slot and single prototype method (Symbol.iterator) seems to be so it's an iterable object - is there a reason not to just return an iterator directly, without creating a new kind of thing?

(Feel free to check the boxes as you address the comments)

Remove region option for timeZone

Split from #22

6. AvailableTimeZones ( region )

• Define behaviour when region is an unsupported region.
• For example what is the return value of AvailableTimeZones("ZX"), where "ZX" isn't a registered region subtag, or what is returned for AvailableTimeZones("ZZ"), where "ZZ" is the unknown region placeholder, also what should be returned for larger regions like AvailableTimeZones("019"), where "019" is the region subtag for the Americas?

9. Intl.supportedValuesOf ( key [ , options ])

• region must be validated and canonicalised.
• Does it accept only region subtags or can it extract the region subtag from a full-fledged language tag?

10. Intl.supportedValuesOf ( key [ , options ])

• Do we still need the region support for AvailableTimeZones() at all, given that get Intl.Locale.prototype.timeZones from the "Intl Locale Info API" proposal will basically provide the same information?

Return iterators instead of array of elements

Some of the enumerations can be quite large, e.g. time zones. Instead of returning full set, return iterator only.

MDN documentation

Privacy evaluation of the API

Based on tc39/ecma402#435 and tc39/ecma402#443 , I'm filing an issue to request evaluation of this API from the privacy experts.

Should the simple units tabel contain all values from IEEE 1541-2002?

If your table contain bit and byte, could you add a kibibyte, mebibyte and other values from this standart?

API doesn't take into account `locale`s?

Since CLDR is sliced by locale I can selectively load locale data for 1 but not another. Does this API handle this scenario?
What does it mean to support values in this scenario?

Add a note about currently unused "options" parameter

The options parameter isn't used anymore after #29, so it maybe confusing why Intl.supportedValuesOf() is still accepting it and passing it to GetOptionsObject(). There should probably be an <emu-note> explaining why this parameter is unused.

Returns array instead of iterator

Split from #22
13. CreateSupportedValuesIterator ( list )

• There needs to be a separate %SupportedValuesIterator% object, otherwise there's no next method to iterate over the elements.
• I guess ? Yield(E) should be changed to ! Yield(E) given that there's no way to resume this generator with an abrupt completion. (I have no idea why ECMA-262 always uses fallible ? Yield() for all its iterators, even when the operation is infallible.)

14. Intl.supportedValuesOf ( key [ , options ])
    I have some doubts about returning iterators instead of for example arrays, because:

• The motivation for iterators was time zones, which has ~465 elements when called with no region. All other lists, with the exception of currencies, are significantly smaller.
• But time zones can't be lazily retrieved from ICU, because we need to canonicalise and sort all entries.
• So unless we account for the case where implementations cache the sorted and canonicalised list of time zones, using iterators over arrays doesn't buy us much. Especially because right now we don't yet have any good built-in Iterator support API and I don't see any real progress on https://github.com/tc39/proposal-iterator-helpers in the last 10 months.
  • Last relevant meeting notes are https://github.com/tc39/notes/blob/master/meetings/2020-07/july-21.md#iterator-helpers-update and https://github.com/tc39/notes/blob/master/meetings/2020-09/sept-21.md#need-another-stage-3-reviewer-for-iterator-helpers where Stage 3 reviewers were requested, but no reviews have been performed as of yet.
  • So, if we stick with iterators, please at least make sure that the "Iterator Helpers" proposal is still on track.
• And if we end up in a place where users always have to perform var array = [...Intl.supportedValuesOf(key)] to get something useful to operate upon, we may have made a design mistake.
• The "Usage example" only shows how to use the new API, but not really when to use it. If I had a better feeling when to use this API, I could more easily say which return type is more appropriate.
  • I do see use cases for time zones and currencies, but other types may not be useful at the moment, because we don't provide means to localise the returned strings. For example currencies can be translated via Intl.DisplayNames and time zones can be translated via Intl.DateTimeFormat (and the "Extend TimeZoneName Option" proposal will even provide better localisations!), but neither calendars, nor collations, nor numbering systems, nor units can be localised right now. Let's say an application wants to provide a date control with a customisable calendar, just providing the BCP47 names to an end user won't be super useful. (Later: I totally forgot about the "Intl.DisplayNames v2" proposal, so "unit" is a better example for missing localisation atm.)
• Please note: I can't say with 100% certainty which is the correct way to go forward, I'm just always a bit hesitant when new patterns are added, because that may also shape future APIs and because I think we should aim to have a consistent overall API approach.
• Also: The "Intl Locale Info Proposal" proposal uses arrays for its return values, so it might be prudent to align the design decisions across these two proposals.

getSupportedUnits does not include compound units

ECMA-402 supports single units and compound units that are a composition of two single units. For example: "meter", "second", and "meter-per-second" are all supported. Intl.getSupportedUnits() is misleading because it doesn't (and shouldn't) return any compound units.

Action Items from TC39 May 2021 meeting

1. Work with @ljharb to make sure the specification of iterator will not introduce unnecessary object and fulfill Mark Miller's expectation.
2. Improve Motivation text in the Explanatory part.

List Locales

Does this proposal offer a way to list locales as well?
https://stackoverflow.com/questions/52549577/javascript-get-the-complete-list-of-locales-supported-by-the-browser

Need Stage 3 Reviewers sign up

Entrance Criteria for Stage 3 (Candidate) require
Designated reviewers have signed off on the current spec text

So we need at least 2 stage 3 reviewer for this proposal.
I somehow thought the Stage 3 reviewers are needed AFTER advanced to Stage 3 to review spec DURING Stage 3 so I didn't ask people to sign up us Stage 3 Reviewers while I proposed to move from Stage 1 to Stage 2.

@sffc @ryzokuken

Review notes

I did make a quick review of the proposal. Below are my notes:

---

1. "the functionality of the constructed objects" term is unclear:

• The term was copied from https://tc39.es/ecma402/#sec-internal-slots, where "constructed objects" refers to the objects created by the constructors listed in that section.
• But when the term is used in the newly added sections, there aren't any constructors which it can refer to, which makes "constructed objects" unclear.

Possible solution:

• Maybe explicitly list the constructors which apply to the relevant sections?
  • For example currency codes only apply to Intl.NumberFormat and Intl.DisplayNames objects. But this approach doesn't seem too elegant.

2. "abstract operation return a List"

• Every occurence of "abstract operation return a List" should instead read "abstract operation returns a List", i.e. "returns" instead of "return".

3. AvailableCurrencies ( )

• "unique well-formed" doesn't make clear in which case the returned currencies are.
• For example both "usd" and "USD" are well-formed according to IsWellFormedCurrencyCode().

4. AvailableTimeZones ( region )

• Add Oxford comma in "the unique, canonical and case-regularized form" -> "the unique, canonical, and case-regularized form".

5. AvailableTimeZones ( region )

• "If the region is undefined, then it return all the names." could be misread that it doesn't require the result to contain only unique, canonical, and case-regularised names.

We could provide a complete algorithm to make this case more clear:

1. Let names be a List of all Zone and Link names of the IANA Time Zone Database.
2. Let result be a new empty List.
3. For each element name of names, do
4. Assert: ! IsValidTimeZoneName(name) is true.
5. Let canonical be ! CanonicalizeTimeZoneName(name).
6. Append canonical to the end of result.
7. Sort result in simple lexicographic order.
8. Return result with all duplicate elements removed.

6. AvailableTimeZones ( region )
   (split the content of item 6 to #24)

7. AvailableUnits ( ) and AvailableNumberingSystems ( )

• Spelling mistakes around singular/plural terms throughout these sections, possibly in more places.

8. Numbering System Identifiers

• "Unicode Technical Standard #35, Part 3, Section 1" should be link to the relevant UTS section.
• Also all other references to UTS should be linkified.

(split the content of item 9 and 10 to #24)

9. Intl.supportedValuesOf ( key [ , options ]) ... see #24

10. Intl.supportedValuesOf ( key [ , options ]) ... see #24

11. Intl.supportedValuesOf ( key [ , options ])

• Missing full stop in the final step.

12. CreateSupportedValuesIterator ( list )

• The abstract operation should get into its own section and shouldn't be part of "Function Properties of the Intl Object"
• Either make it a subsection of Intl.supportedValuesOf(...) or add a new sub-section named "Abstract Operations for Intl" under "The Intl Object".

( Move the content of the item 13 and 14 to #23 )
13. CreateSupportedValuesIterator ( list ) ... see #23
14. Intl.supportedValuesOf ( key [ , options ]) ... see #23

Detail spec out AvailableTimeZones to be sorted and canonicalized

Split item 5 of @anba review of #22 here so we can track the issue independently.

5. AvailableTimeZones ( region )
"If the region is undefined, then it return all the names." could be misread that it doesn't require the result to contain only unique, canonical, and case-regularised names.
We could provide a complete algorithm to make this case more clear:

Let names be a List of all Zone and Link names of the IANA Time Zone Database.
Let result be a new empty List.
For each element name of names, do
Assert: ! IsValidTimeZoneName(name) is true.
Let canonical be ! CanonicalizeTimeZoneName(name).
Append canonical to the end of result.
Sort result in simple lexicographic order.
Return result with all duplicate elements removed.

Special treatment for "gregory" calendar

The current definition of AvailableCalendars says

The list must include "gregory".

I agree it's pretty unlikely that any implementation would omit gregory. But it seems to me that if any calendar should be required, it should be iso8601, as that's the only calendar that we are going to require that implementations without 402 support for Temporal.

AvailableCalendars, AvailableCollations, and AvailableNumberingSystems should return canonical identifiers

Similar to tc39/proposal-intl-locale-info#32, AvailableCalendars, AvailableCollations, and AvailableNumberingSystems should all return canonical identifiers. For example AvailableCalendars should canonicalise "islamicc" to "islamic-civil".

More things that we might enumerate over

These are some things that I could imagine using in a picker

• Languages
• Regions

In general, if something is supported by Intl.DurationFormat, I wonder if you might want to be able to enumerate over supported values.

For example, this set includes scripts, but maybe enumerating over scripts is only useful in conjunction with data about which scripts may be meaningful for a given locale, which is a whole other project...

No use cases listed

The README.md does not contain any use cases.

In particular, it would be valuable to see use cases that require actual enumeration, rather than determining whether a particular value is present.

The polyfill `Intl.getSupportedTimeZones` uses incorrect data

Intl.getSupportedTimeZones does not contain items such as UTC and CET, which it proably should.

The polyfill seems to use the wrong source data. It links to https://raw.githubusercontent.com/unicode-org/cldr/master/common/supplemental/metaZones.xml, but that's not the normative source for IANA time zone names. Instead it should generate its data from https://github.com/eggert/tz.

Originally posted by @anba in #47 (comment)

Fetching a list of IDs along with localized text for each ID

Probably the most common use case for enumeration is showing a list in a UI like a dropdown or other list-based picker. A list UI requires both the list of IDs and the human-readable, localized text corresponding to each ID. Is the code below the suggested way to use this proposal to get the data for this use case?

names = new Intl.DisplayNames('fr', { type: 'timeZone' });
listData = Intl.supportedValuesOf('timeZone').map(id => ({ id, name: names.of(id) })); 

Also, out of curiosity, was it considered to also provide a single-call solution like below?

listData = Intl.DisplayNames('fr', { type: 'timeZone' }).getAll(); 

A single-call solution could be added in a later proposal, and I see the value of supporting enumeration in cases where localized text is not needed. But I was curious about why the most ergonomic solution for the most common use case wasn't included in V1.

A Link for Currency Use Cases

Here's a link that could be added here (under the "Use Cases" bullet):
https://stackoverflow.com/questions/58029344/get-list-of-supported-currencies