X.509 algorithm identifier parameters vs. extension values

Question

From the X.509 specs:

AlgorithmIdentifier  ::=  SEQUENCE  {
     algorithm               OBJECT IDENTIFIER,
     parameters              ANY DEFINED BY algorithm OPTIONAL  }
                                -- contains a value of the type
                                -- registered for use with the
                                -- algorithm object identifier value

Why isn't Extension defined similarily? Here's how it's defined right now:

Extension  ::=  SEQUENCE  {
     extnID      OBJECT IDENTIFIER,
     critical    BOOLEAN DEFAULT FALSE,
     extnValue   OCTET STRING
                 -- contains the DER encoding of an ASN.1 value
                 -- corresponding to the extension type identified
                 -- by extnID
     }

Why is it not this?:

Extension  ::=  SEQUENCE  {
     extnID      OBJECT IDENTIFIER,
     critical    BOOLEAN DEFAULT FALSE,
     extnValue   ANY DEFINED BY extnID
     }

Answer 1

Nominally, the contents of the extension (the extnValue) could be any sequence of bytes. The "value bytes" are to be interpreted relatively to the extension ID; each extension is free to define its own syntax.

The extension which are defined by X.509 itself all use ASN.1: for each "standard" extension, the syntax for the corresponding value uses ASN.1, so the extension value, to be stored in the OCTET STRING, shall be the DER encoding of some ASN.1-based object. Hence the comment. However, extensions can be, by definition, arbitrary, and X.509 uses an OCTET STRING so that other standards or implementations may define their own extensions which are not necessarily based on ASN.1 (I have seen that in the wild).

There is nothing really profound here. X.509 could have defined the Extension type as you suggest, with an ANY; in which case people who want to use non-ASN.1 extension values would have had to encapsulate them in an OCTET STRING. The discrepancy between Extension and AlgorithmIdentifier just shows that these structures were defined by distinct people, and the whole ASN.1 ecosystem is known never to have exhibited tremendous consistency. Also, for backward compatibility, decisions such as using an ANY instead of an OCTET STRING (or vice versa) tend to stick for a long time.