Why are SSL CAs prohibiting double dash in third and fourth characters?

Question

Digicert has disallowed "double dashes" in the third and fourth characters in new certs:

Effective October 1, 2021, for publicly trusted TLS/SSL certificates, we no longer allow the use of double dashes (--) in the third and fourth characters in domain names, unless the double dashes proceed the letters xn (xn--example.com).

Similarly, AWS has made such certs ineligible for ACM renewal.

Digicert references ballot 202, which I found on cabforum.

CAs MUST NOT include Domain Labels which have hyphens as the third and fourth characters unless the first character is “x” or “X”, the second character is “n” or “N”, and the fifth and later characters are a valid Punycode string.

This is my first time encountering punycode, and it seems rather interesting itself. But why are CAs prohibited from using hyphens when the domain is NOT punycode? Is there some security concern at play here? Digicert mentions that sites like es--xyz.loudsquid.com are not allowed. Why is es-- undesirable?

Answer 1

Adding an answer because too long in a comment, but on the specific point of why reserving everything if xn-- is enough.

In one of first iteration of IDNA standard ("Internationalizing Domain Names in Applications"), in a draft in November 2001 (draft-ietf-idn-idna-04) there was this:

5. ACE prefix The ACE prefix, used in the conversion operations (section 4), will be specified in a future revision of this document. It will be two alphanumeric ASCII characters followed by two hyphen-minuses. It MUST be recognized in a case-insensitive manner.

The scheme allowed interoperability tests when there was multiple encodings proposed. So in fact it seems there was at least bl--, bq--, dq--, lq--, mq--, ra--, wq-- and zq-- (and when things solidified, xn was chosen at random so that no one had a head start and no collisions with actual existing names -- see https://web.archive.org/web/20100427154004/http://www.atm.tut.fi/list-archive/ietf-announce/msg13572.html to see exactly how xn was chosen by the IETF). If you are old enough, you would remember that Network Solutions/Verisign then was selling bq-- domain names, as IDN testbed.

In February 2003:

An eligible subset of that list of 42 entries will be determined by eliminating the following codes due to their use, in one or more top-level domain zone files that have been reviewed, as the first two characters of second-level domain labels that have hyphens in their third and fourth character positions: AA, QM to QZ, XA, XZ, and ZZ.

Going back to December 2000 at IETF San Diego has these notes:

ACE identifier candidates

• prefixes: AA--, AB--, ..., 99--
• suffixes: --AA, --AB, ..., --99

Relevant domain names: aa--a.com, aa-b.org, ..99--zzzz.net, aa--x.co.jp, etc. a-aa.com, b--aa.org, ..., zzzzz--99.or.kr, etc.

Proposal

step 1: tentative suspension of registering relevant domain names for ACE identifier candidates

step 2: conduct a survey of relevant domain names already registered

step 3: select about 10 to 20 identifiers one of which is for test and others for real use, based on the survey

step 4: permanent blocking of registrations of domain names relevant to the selected identifiers (except for registrations compliant to MDN semantics).

In November 2000 in draft-ietf-idn-aceid-00 we have:

All strings starting with a combination of two alpha-numericals, followed by two hyphens, are defined to be ACE prefix identifier candidates. All strings starting with one hyphen followed by three alpha-numericals, and strings starting with two hyphens followed by two alpha-numericals are defined as ACE suffix identifier candidates. ACE prefix identifier candidates and ACE suffix identifier candidates are collectively called ACE identifier candidates.

which got simplified in following June to just:

All strings starting with a combination of two alpha-numericals, followed by two hyphens, are defined to be ACE prefix identifier candidates. All strings starting with two hyphens followed by two alpha-numericals are defined as ACE suffix identifier candidates.

And the mailing list archives before 2001-01 seems to be lost forever so no way to find more about that, I fear.

New prefixes?

In the comments, this question was raised:

Are there new/current proposals to introduce standard prefixes other than xn--?

There aren't as of now, July 2023, new/current proposals but in fact there are old ones…

From RFC 4690 "Review and Recommendations for Internationalized Domain Names (IDNs)" (September 2006 - so right between IDNA2003 and IDNA2008 when people where clearly saying the problems in IDNA2003 but not yet being sure how to best address them), §4.1.2 has the following:

It is worth noting that sufficiently extreme changes to IDNA would require a new Punycode prefix, probably with long-term support for both the old prefix and the new one in both registration arrangements and applications. An alternative, which is almost certainly impractical, would be some sort of "flag day", i.e., a date on which the old rules are simultaneously abandoned by everyone and the new ones adopted. However, preliminary analysis indicates that few, if any, of the changes recommended for consideration elsewhere in this document would require this type of version change. For example, suppose additional restrictions, such as those implied above, are imposed on what can be registered. Those restrictions might require policy decisions about how labels are to be disposed of if they conformed to the earlier rules but not to the new ones. But they would not inherently require changes in the protocol or prefix.

So IDNA2008 happened few years after that with major changes (not being tied to a specific Unicode version anymore, not using a first step of Nameprep, etc.) but without a change of prefix, xn-- remaining the IDNA prefix.

This is discussed again in fact in one of the RFC about IDNA2008 itself, RFC5894 "Background, Explanation, and Rationale" has a whole section (§7.4): "The Question of Prefix Changes"

The conditions that would have required a change in the IDNA ACE prefix ("xn--", used in IDNA2003) were of great concern to the community. A prefix change would have clearly been necessary if the algorithms were modified in a manner that would have created serious ambiguities during subsequent transition in registrations. This section summarizes the working group's conclusions about the conditions under which a change in the prefix would have been necessary and the implications of such a change.

and

7.4.3. Implications of Prefix Changes

While it might have been possible to make a prefix change, the costs of such a change are considerable.

June 2023 update

I stumbled upon a document enforcing the prohibition. At https://www.icann.org/en/system/files/files/idn-guidelines-22sep22-en.pdf you can find "Guidelines for the Implementation of Internationalized Domain Names - Version 4.1 - 22 September 2022" which says this in §2.1:

4. No label containing hyphens in both the third and the fourth positions may be registered unless it is a valid A-label, with reservation for transitional action. Labels with hyphens in both the third and the fourth positions are explicitly reserved to indicate encoding schemes, of which IDNA is only one instantiation. These guidelines are not intended to assist with any other instantiations.

I don't think it is like an actual policy from ICANN, but surely something at least all gTLD registries have to follow. There is only this mention on ICANN's website:

Registries seeking to deploy IDNs under their agreements with ICANN have been authorized to do so on the basis of the Guidelines.

(from https://www.icann.org/resources/pages/implementation-guidelines-2012-02-25-en)

July 2023 update with some digging on prefixes

By random luck, I stumbled upon various documents describing various prefixes used back in the day (in no specific order, and not exhaustive by any stretch):

• https://datatracker.ietf.org/doc/html/draft-duerst-dns-i18n (July 1998) suggests using a TLD instead of prefixes, being .i or .i18n.int (but it is called a Zero Level Domain, as it would be hidden from users display); it also defines a transformation later referenced as "UTF-5"
• https://datatracker.ietf.org/doc/html/draft-ietf-idn-utf6-00 (November 2000) defines "UTF-6" (as extension out of the previous "UTF-5") and uses wq--
• https://datatracker.ietf.org/doc/html/draft-ietf-idn-lace-01 (January 2001) uses prefix lq--
• https://datatracker.ietf.org/doc/html/draft-ietf-idn-brace-00 (September 2000) uses a signature aka suffix of -8q9
• https://datatracker.ietf.org/doc/html/draft-ietf-idn-race-01 (August 2000) uses prefix bq--
• https://datatracker.ietf.org/doc/html/draft-ietf-idn-sace-00 (August 2000) describes just a mapping algorithm, without indications of prefixes. It also says "It is needed during a transition period when non-ASCII names are introduced in DNS to avoid breaking programs expecting ASCII only.". I guess this will be an eternal transition :-)
• https://datatracker.ietf.org/doc/html/draft-ietf-idn-idne-02 (March 2001) directly uses UTF-8 for names in the DNS (with the use of EDNS mechanism to signal it to DNS systems) and as such does not need any new mapping algorithm
• https://datatracker.ietf.org/doc/html/draft-skwan-utf8-dns-06 (May 2001) also uses UTF-8 directly
• https://datatracker.ietf.org/doc/html/draft-ietf-idn-udns-03 (August 2001) also using names directly without a new algorithm, but defining a new DNS label type
• https://datatracker.ietf.org/doc/html/draft-ietf-idn-compare-01 (July 2000) compares in fact various of the previous proposals for various metrics; it mentions one of the proposal was to add a trailing hyphen for the new names (which is specifically prohibited by RFC 1034).

Answer 2

The double hyphen is reserved as a generalized extensibility mechanism of which Punycode is one example.

RFC 5891: 4.2.3.1. Hyphen Restrictions

The Unicode string MUST NOT contain "--" (two consecutive hyphens) in the third and fourth character positions and MUST NOT start or end with a "-" (hyphen).

Answer 3

This is due to the double dash's usage in internationalized domain names. xn-- has a special meaning in domain names, and it is technically a violation of the IDNA2008 standard if the -- series of characters is in the 3rd and 4th spot, unless the first 2 characters are xn.

The specific RFCs that were defined for IDNA2008 are RFC 5890 to 5894.

Answer 4

The purpose of leaving certain patterns of alphanumeric-and-dash strings reserved is to allow for the possibility that they might be used to represent something that can't presently be represented. There's no way a certificate authority can know what a string like aa--bcde might mean in future, and who might be entitled to use such a thing as a domain name. If a CA were to issue certificates for that domain name to Acme Enterprises and then the committee in charge of domain name formats decided that names starting with aa-- should be issued by the Accreditation Agency, which issued Binary Coded Decimal Enterprises the name aa-bcde, the fact that Acme Enterprise had a certificate for that name would be a problem. To be sure, it might be mitigated by issuing a revocation for the issued certs, but it's better to simply avoid such problems in the first place.