XSD pattern for new IdentifierType supporting Short Ids

[cdanger]

Currently, the draft spec says:

5.4 Simple type IdentifierType

A value of the IdentifierType simple type refers to a specific attribute category, attribute, data type, function, notice, status code, combining algorithm or XPath version.

<xs:simpleType name="IdentifierType">
   <xs:restriction base="xs:string">
      <xs:pattern value="[^{}]*({[A-Za-z][0-9A-Za-z]*(-[0-9A-Za-z]+)*}[^{}]*)*"/>
   </xs:restriction>
</xs:simpleType>

A value of this simple type is either:

• an absolute URI [RFC2396],
• the name of a short identifier or
• a character string with one or more short identifier names enclosed by curly brackets (i.e., { and }; U+007B and U+007D) optionally preceded, followed and/or separated by other characters allowed in a URI.

A few issues:

• XSD-aware XML editor (e.g. IntelliJ IDEA) reports an error in the pattern on the second left curly brace: Unexpected start of quantifier '{' . A simple fix is to escape the curly braces:
  <xs:pattern value="[^{}]*(\{[A-Za-z][0-9A-Za-z]*(-[0-9A-Za-z]+)*\}[^{}]*)*"/>
• RFC 2396 is obsoleted by RFC 3986 so we may upgrade the reference for the absolute URI (Note: the reference for xs:anyURI has been upgraded to RFC 3986 in XSD 1.1)
• Some values match the regex although they are not supposed to be valid, e.g. empty string, whitespace, :, :x, 0:, ...

We should build the regex by increment, defining one regex for each case at a time:

Case 1: absolute URI (with fragment)

The fragment part is required to support standard XACML datatypes from XML Schema in particular, e.g. https://www.w3.org/2001/XMLSchema#string.

RFC 2396 is the reference for URI in XACML 3.0. However, it has been obsoleted by RFC 3986 which gives the following ABNF definition for an absolute URI with optional fragment (Appendix A):
URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ]

A corresponding regex can be found here, and the regex for each component in the URI as well (scheme, hier-part, query, fragment). We can reuse that but with a few tweaks:

• The non-capturing group operator (?:...) is not supported in XSD pattern therefore all these operators must be removed from the regex;
• In XML content (the XSD pattern in this case), '&' is a special character which must be replaced with &.
• The regex for the dec-octet ABNF rule (decimal in IPv4 address) looks wrong because it matches 00, 01, 001, etc. therefore needs to be fixed in the XSD pattern.

The regex excludes curly braces '{}' in particular, which prevents any mix-up with the other two cases of Identifier (ShortId or ShortId-based URI expression).

To be clear, the regex checks the syntax according to the RFC 3986 only, i.e. the generic scheme-agnostic syntax for URI, and nothing more. Any scheme-specific syntax rule is out of scope (defined in other RFCs). It would be impractical to try and capture in a single XSD pattern every scheme-specific rule of every possible URI scheme (every RFC) out there. Ultimately, XACML applications need to implement their own scheme-specific checks if there are schemes they're interested in particularly.

N.B.: the XSD standard has a standard datatype anyURI for URI validation but is not enough in this case because:

• anyURI represents any URI, and even any IRI in XSD 1.1, including relative URIs which we don't allow here, therefore the xsd regex to match only absolute URIs.
• The anyURI type allows characters which are illegal in a URI, provided that they can be escaped to produce a legal URI by using the escaping procedure specified in the XSD standard (e.g. XSD 1.0 refers to the escaping procedure defined in Section 5.4 Locator Attribute of [XML Linking Language]). In other words, the anyURI type may accept these characters, including curly braces {}, either escaped or unescaped. Therefore the need for the XSD pattern to exclude {} as mentioned before. See also Michael Kay's post on stackoverflow (a anyURI is a wannabee URI).

The regex is quite large, so it seems more readable to define a dedicated XSD type for this case (AbsoluteUriType), and define a union type for IdentifierType that combines with the others, rather than mixing all in one regex for the three cases together. Also this allows to derive the dedicated type from standard anyURI in this case, whereas it is not possible for the second case (ShortId name). Besides, AbsoluteUriType could be used for identifiers that don't support ShortId, such as PolicyId identifiers.

The proposal for a new XSD type for absolute URIs, using the proper regex (as a reminder, an XSD pattern is matched against the entire string, so the ^ and $ anchors are omitted):

<xs:simpleType name="AbsoluteUriType">
   <xs:restriction base="xs:anyURI">
      <!--
      Absolute URI with optional fragment (fragment is needed to support the standard XACML datatypes from XML Schema, e.g. 'http://www.w3.org/2001/XMLSchema#string').

      We need to define a pattern for further restriction because the anyURI type is for any URI including relative ones, and actually this type accepts also characters which are illegal in URI, including the curly braces '{}' we want to exclude, provided that they can be escaped to produce a
      legal URI according to the escaping procedure specified in the XSD standard (e.g. XSD 1.0 refers to the section 5.4 Locator Attribute of [XML Linking Language]). In other words, the anyURI type may accept these characters (e.g. `{}`) either escaped or unescaped.

      Pattern for an absolute URI (with optional fragment) based on ABNF definition of 'URI' in RFC 3986 (de facto, this pattern checks the syntax according the RFC only, i.e. the generic scheme-agnostic syntax, while scheme-specific syntax rules are out of scope, defined in other RFCs):
            ^<scheme> : <hier-part> (\? <query> )? (# <fragment> )?$
         with the following sub-patterns:
         - <scheme>:
               [A-Za-z][A-Za-z0-9+\-.]*
         - <hier-part>:
               (// <authority> <path-abempty> | <path-absolute> | <path-rootless> | <path-empty> )
         - <authority>:
               ( <userinfo> @)? <host> (: <port> )?
         - <userinfo>:
               ( <unreserved> | <pct-encoded> | <sub-delims> |:)*
            expanded to (after merging <unreserved>, <sub-delims> and ':'):
               ([A-Za-z0-9\-._~!$&'()*+,;=:]|%[0-9A-Fa-f]{2})*
         - <unreserved>:
               [A-Za-z0-9\-._~]
         - <pct-encoded>:
               %[0-9A-Fa-f]{2}
         - <sub-delims>:
               [!$&'()*+,;=]
         - <host>:
               ( <IP-literal> | <IPv4address> | <reg-name> )
            <IPv4address> pattern is included in <reg-name> (any IPv4 address matches <reg-name>), therefore the pattern can be simplified to:
               ( <IP-literal> | <reg-name> )
         - <IP-literal>:
               \[( <IPv6address> | <IPvFuture> )\]
         - <IPv6address> (<ls32> is factored out from the first 7 lines of the ABNF rule):
            (
               (
                  (<h16>:){6}
                  | ::(<h16>:){5}
                  | <h16>?::(<h16>:){4}
                  | ( (<h16>:){0,1} <h16> )?::(<h16>:){3}
                  | ( (<h16>:){0,2} <h16> )?::(<h16>:){2}
                  | ( (<h16>:){0,3} <h16> )?::<h16>:
                  | ( (<h16>:){0,4} <h16> )?::
               ) <ls32>
               | ( (<h16>:){0,5} <h16> )?:: <h16>
               | ( (<h16>:){0,6} <h16> )?::
            )
         - <h16>:
               [0-9A-Fa-f]{1,4}
         - <ls32>:
               ( <h16> : <h16> | <IPv4address> )

         - <IPvFuture> (leading "v" is case-insensitive, cf. section 3.2.2 (Host)):
               [Vv][0-9A-Fa-f]+\.( <unreserved> | <sub-delims> |:)+
            expanded to (after merging <unreserved>, <sub-delims> and ':'):
               [Vv][0-9A-Fa-f]+\.[A-Za-z0-9\-._~!$&'()*+,;=:]+
         - <IPv4address>:
               ( <dec-octet> \.){3} <dec-octet>
         - <dec-octet>:
               (25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9][0-9]|[0-9])
         - <reg-name>:
               ( <unreserved> | <pct-encoded> | <sub-delims> )*
            expanded to (after merging <unreserved> and <sub-delims>):
               ([A-Za-z0-9\-._~!$&'()*+,;=]|%[0-9A-Fa-f]{2})*
         - <port>: [0-9]*
         - <path-abempty>:
               (/<segment>)*
         - <segment>:
               <pchar>*
         - <pchar>:
               ( <unreserved> | <pct-encoded> | <sub-delims> |:|@)
            expanded to (after merging <unreserved>, <sub-delims>, ':' and '@'):
               ([A-Za-z0-9\-._~!$&'()*+,;=:@]|%[0-9A-Fa-f]{2})
         - <path-absolute>:
               /( <segment-nz> (/<segment>)*)?
         - <segment-nz>:
               <pchar>+
         - <path-rootless>:
               <segment-nz> (/<segment>)*
         - <path-empty>: empty string (zero character)
         - <query>:
               ( <pchar> | / | ? )*
            expanded to (after merging <pchar>, '/' and '?'):
               ([A-Za-z0-9\-._~!$&'()*+,;=:@/?]|%[0-9A-Fa-f]{2})*
         - <fragment>: same as <query>

      The pattern requires a colon ':' and forbids curly braces {} therefore this type of string cannot be confused with ShortIdNameType or ShortIdBasedUriExpression.
      An XSD pattern is matched against the entire string, so the ^ and $ anchors are omitted.
      Since '&' is a special character in XML content, it must be replaced with '&amp;' in the XSD pattern.
      -->
      <xs:pattern
               value="[A-Za-z][A-Za-z0-9+\-.]*:(//(([A-Za-z0-9\-._~!$&amp;'()*+,;=:]|%[0-9A-Fa-f]{2})*@)?(\[(((([0-9A-Fa-f]{1,4}:){6}|::([0-9A-Fa-f]{1,4}:){5}|([0-9A-Fa-f]{1,4})?::([0-9A-Fa-f]{1,4}:){4}|(([0-9A-Fa-f]{1,4}:){0,1}[0-9A-Fa-f]{1,4})?::([0-9A-Fa-f]{1,4}:){3}|(([0-9A-Fa-f]{1,4}:){0,2}[0-9A-Fa-f]{1,4})?::([0-9A-Fa-f]{1,4}:){2}|(([0-9A-Fa-f]{1,4}:){0,3}[0-9A-Fa-f]{1,4})?::[0-9A-Fa-f]{1,4}:|(([0-9A-Fa-f]{1,4}:){0,4}[0-9A-Fa-f]{1,4})?::)([0-9A-Fa-f]{1,4}:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?))|(([0-9A-Fa-f]{1,4}:){0,5}[0-9A-Fa-f]{1,4})?::[0-9A-Fa-f]{1,4}|(([0-9A-Fa-f]{1,4}:){0,6}[0-9A-Fa-f]{1,4})?::)|[Vv][0-9A-Fa-f]+\.[A-Za-z0-9\-._~!$&amp;'()*+,;=:]+)\]|([A-Za-z0-9\-._~!$&amp;'()*+,;=]|%[0-9A-Fa-f]{2})*)(:[0-9]*)?(/([A-Za-z0-9\-._~!$&amp;'()*+,;=:@]|%[0-9A-Fa-f]{2})*)*|/(([A-Za-z0-9\-._~!$&amp;'()*+,;=:@]|%[0-9A-Fa-f]{2})+(/([A-Za-z0-9\-._~!$&amp;'()*+,;=:@]|%[0-9A-Fa-f]{2})*)*)?|([A-Za-z0-9\-._~!$&amp;'()*+,;=:@]|%[0-9A-Fa-f]{2})+(/([A-Za-z0-9\-._~!$&amp;'()*+,;=:@]|%[0-9A-Fa-f]{2})*)*|)(\?([A-Za-z0-9\-._~!$&amp;'()*+,;=:@/?]|%[0-9A-Fa-f]{2})*)?(\#([A-Za-z0-9\-._~!$&amp;'()*+,;=:@/?]|%[0-9A-Fa-f]{2})*)?"/>
   </xs:restriction>
</xs:simpleType>

Possible improvement:
The host part of an RFC 3986 URI may be an IP address (IP-literal and IPv4address rules) or registered host name (reg-name rule), and such hostname doesn't have to follow the syntax for DNS domain names because it is meant to be generic / DNS-agnostic in the RFC, therefore the syntax is far more permissive. I think IP addresses should not be allowed in a absolute URI used as a unique identifier for attributes, categories, datatypes, etc. and the hostname should follow DNS-naming-syntax. My suggestion would be to apply these two rules to the pattern above. Removing the IP address patterns in particular would simplify the regex significantly.

Case 2: regex for ShortId

Reusing the definition from the current xacml 4.0 draft, section 5.3 Element ShortId:

<xs:simpleType name="ShortIdNameType">
   <xs:restriction base="xs:string">
      <xs:pattern value="[A-Za-z][0-9A-Za-z]*(-[0-9A-Za-z]+)*"/>
   </xs:restriction>
</xs:simpleType>

Case 3: regex for a string containing one or more {shortId}s, optionally preceded, followed and/or separated by characters allowed in a URI

In this case, the string is expected to be an expression of an absolute URI, based on short identifiers (the short identifiers are the variables waiting to be expanded in this expression), therefore the suggested name for the XSD type: ShortIdBasedUriExpressionType.

Based on the ABNF definition of 'URI' in Appendix A of RFC 3986, a regex for matching a character allowed in a URI:
[A-Za-z0-9\-._~!$&'()*+,;=:/?#\[\]@%]

The proposed XSD type for ShortId-based URI expressions:

<xs:simpleType name="ShortIdBasedUriExpressionType">
   <xs:restriction base="xs:string">
      <!--
      Pattern for a sequence of one or more {ShortId}, each one possibly preceded, separated and/or followed by valid URI character(s) (also curly braces are special characters in regex, therefore must be
      escaped):
            ^( <URI_char>* \{ <ShortId> \} )+ <URI_char>*$
      with the following sub-patterns:
      - <ShortId> (pattern for a ShortId from ShortIdNameType definition):
            [A-Za-z][0-9A-Za-z]*(-[0-9A-Za-z]+)*
      - <URI_char> (pattern for a valid URI character based on the ABNF definition of 'URI' in Appendix A of RFC 3986):
            [A-Za-z0-9\-._~!$&amp;'()*+,;=:/?#\[\]@%]

      The pattern requires curly braces {} therefore this type of string cannot be confused with AbsoluteUriType or ShortIdNameType where {} are forbidden.
      An XSD pattern is matched against the entire string, so the ^ and $ anchors are omitted.
      In XML content, special character '&' must be replaced with '&amp;'.
      -->
      <xs:pattern value="([A-Za-z0-9\-._~!$&amp;'()*+,;=:/?#\[\]@%]*\{[A-Za-z][0-9A-Za-z]*(-[0-9A-Za-z]+)*\})+[A-Za-z0-9\-._~!$&amp;'()*+,;=:/?#\[\]@%]*"/>
   </xs:restriction>
</xs:simpleType>

New IdentifierType definition

The resulting new IdentifierType definition, based on previous types, would be:

<xs:simpleType name="IdentifierType">
   <xs:union memberTypes="AbsoluteUriType ShortIdNameType ShortIdBasedUriExpressionType" />
</xs:simpleType>

[humantypo]

Wow. The patchwork nature of the Internet has certainly created some interesting situations.

FYI, these URI variants are matches with the "scheme" portion of the regex (in addition to the intentional leading "text:"):

/[A-Za-z][A-Za-z0-9+\-.]*:/

https://sub.domain.co.uk: 8080/page?param1=value1&param2=value2
https://user: password@secure-site.net/data
http://localhost: 3000/api/v1/users
http://example.com: 8080/test
urn:isbn: 0451450523
file:///C: /Users/John/Documents/report.pdf
(spaces added so markdown would work)

I always wondered why they don't explicitly use "^" in the regex expressions in cases like this.

[cdanger]

Beware that an XSD pattern is always matched against the whole string, i.e. the '^' and '$' anchors are implicit! So if you want to test it with some other regex tester out there (not XSD-aware), make sure to add these anchors first (this changes everything):

^[A-Za-z][A-Za-z0-9+\-.]*:.*$

So the string must be a valid xs:anyURI (as defined in XML schema) AND match the pattern above.

I forgot the .* after the colon initially, now it is fixed. It should work with your examples now.

more info

[humantypo]

Ergo my distaste for incomplete regex. Never made any sense to me that one would verbally specify this criteria when the regex can enforce it natively.

As to the ".*", we might want to think about that some more. Here are invalid URIs that match with that addition:

^[A-Za-z][A-Za-z0-9+\-.]*:.*$

https://example.com\path
http://.example.com
http://user:pass@/no-host
http://example.com:invalid/path
http://example.com:80:80
https://[fe80::1%eth0]/
http://ex:port/
http://[::1]:
https://example.com/pa%th

[cdanger]

Hm... this pattern is just here to make sure the URI is absolute (has a scheme part), but don't forget the first part of the XSD type definition: <xs:restriction base="xs:anyURI">.

This means the XSD validator first checks the value is valid with respect to anyURI type, i.e. valid URI (as defined in XML schema which refers to the URI RFC).

Then, if the first check is OK, the XSD validator checks that the value matches the pattern in <xs:pattern>, which means the URI here has a scheme part, therefore is absolute (and not a relative URI).

Both checks matter.

The full definition again:

<xs:simpleType name="AbsoluteUriType">
   <xs:restriction base="xs:anyURI">
      <!-- Add regex match for the "scheme:" part that an absolute URI must have -->
      <xs:pattern value="[A-Za-z][A-Za-z0-9+\-.]*:.*"/>
   </xs:restriction>
</xs:simpleType>

See the XSD definition of AbsoluteUriType in the top comment, section Case 1: absoluteURI.

[steven-legg]

I've been quiet this week because I've had to work around a planned power outage.

I'm not sure it is worth the effort to try to tie down an absolute URI with a pattern or an appeal to xs:anyURI. It wouldn't be complete anyway. Take a value like this:

{scheme}{domain}{path}myAttribute

It is supposed to be an absolute URI once all the short identifier names have been expanded out but there is no way a pattern on the string can test that. It can only test that the short identifier names have been referenced correctly. If the validity can only be checked later by an implementation-defined method then that will also cover a value that didn't contain short identifier names in the first place.

Pragmatically, we don't even need the final string to actually be an absolute URI because we just compare these identifiers code point by code point.

Does JSON Schema have an equivalent for union types and xs:anyURI?

[cdanger]

I'm not sure it is worth the effort to try to tie down an absolute URI with a pattern or an appeal to xs:anyURI. It wouldn't be complete anyway. Take a value like this:

{scheme}{domain}{path}myAttribute

Indeed, for this example, it doesn't fit. Just to clarify, the XSD type / regex for absolute URI (AbsoluteUriType) that was discussed in the four comments earlier was only for the first case / first bullet (*absolute URI [RFC 2396] *) of the description of IdentifierType that I quoted from the spec (section 5.4) at the top of this page. I had not intention to apply it to the third case (third bullet) which your example {scheme}{domain}{path}myAttribute falls into. So I've proposed a different pattern and XSD type actually for this case - ShortIdBasedUriExpressionType (name is just a proposal) . And the union type in the end to merge the XSD types for each case.

Does JSON Schema have an equivalent for union types and xs:anyURI?

Yes, for union types, you may use the oneOf keyword;

"Identifier": { 
        "oneOf": [    
            { "$ref": "#/$defs/AbsoluteURI" },
            { "$ref": "#/$defs/ShortIdName" },
            { "$ref": "#/$defs/ShortIdBasedUriExpression" }
        ] 
    }

For URI, you can use the format keyword with value uri:

{ "type": "string", "format": "uri" }

More info in section 7.3.5 of JSON Schema Validation spec

-- EDIT 2025-08-21 --
The "anyOf" keyword is a better alternative in this case because it's usually faster than "oneOf". "anyOf" checks that at least one of the alternatives is valid, so it's done as soon as one is found valid; whereas "oneOf" always checks all alternatives to make sure there is one and only one that is valid, which is not necessary here.

"Identifier": { 
        "anyOf": [    
            { "$ref": "#/$defs/ShortIdName" },
            { "$ref": "#/$defs/ShortIdBasedUriExpression" },
            { "$ref": "#/$defs/AbsoluteURI" }
        ] 
    }

Another possibility is also the if-then-else keywords.

[steven-legg]

I thought about using anyURI and a union type but decided it was needlessly complicated and suboptimal.

There are three cases:

• an absolute URI,
• a short identifier name or
• a string with multiple short identifier references.

We expect that the second case will overwhelmingly be used. The example I added to the spec only uses the second case. A specification that always forces an attempt to test if the IdentifierType value is an absolute URI is wasting time when that attempt will nearly always fail. An implementation is obliged to verify that the second and third cases result in an absolute URI and whatever way the implementation chooses to do that can equally well be applied to the first case, and post-checking outside the syntax leaves scope for an implementation to optimize the processing.

For example, when the implementation loads a short identifier set into memory it can tag each short identifier to indicate whether it evaluates to an absolute URI. Then when it is parsing an IdentifierType in a request it can do a quick check that it is a short identifier name, look up the name and know from the tag whether it is good or bad - without testing for an absolute URI.

[steven-legg]

This

{ "type": "string", "format": "uri" }

appears to exclude relative URIs (good), but xs:anyURI allows relative URIs (bad). XSD 1.1 also allows IRIs.

[cdanger]

This

{ "type": "string", "format": "uri" }

appears to exclude relative URIs (good),

This accepts any RFC 3986 URI. The JSON Schema Validation spec §7.3.5 (cf. link at the end of my previous comment) only says:

uri: A string instance is valid against this attribute if it is a valid URI, according to [RFC3986].

To restrict to absolute URIs, we add a regex (similar to the one of the AbsoluteUriType definition proposed in my first comment at the top (updated), section Case1: absolute URI), with a few tweaks: must be adapted to the ECMA-262 regex flavor used in JSON schema, and the anchors ^$ are not implicit (unlike XML schema!), therefore they must be added to make an equivalent regex:

{ "type": "string", "format": "uri", "pattern": "^...(same as AbsoluteUriType pattern)...$" }

EDIT 2025-08-21: based on @steven-legg 's comment and the official test suite for the "uri" format on JSON Schema's github, it's confirmed the "uri" format excludes relative URIs (now called relative (URI) references in RFC 3986) already.

but xs:anyURI allows relative URIs (bad). XSD 1.1 also allows IRIs.

Yes, I mentioned all that in my first comment, this is why we need a regex to restrict to absolute URIs. See the XSD definition of AbsoluteUriType in the first comment, section Case 1: absolute URI.

[humantypo]

Perhaps it would be helpful to come up with some examples of desirable and undesirable URI strings that we can test against/discuss viability? The regex in the "first comment" is clearly too loose, but I can't offer a suggestion on modification without something tangible to work with.

On a side note, while XML requires string "trimming" outside of the regex match, is there a downside to using "^$" framing consistently in our regex matching from a practical perspective? I believe this should remove language specific implementations in the vast majority of cases. The only place where I can think of it not working would be if a string had "intentional whitespace" (e.g. " string", "string ", etc.). Otherwise XML's process of match(trim(string), /regex/) should be equal to the more universal match(string, /^regex$/).

[cdanger]

Perhaps it would be helpful to come up with some examples of desirable and undesirable URI strings that we can test against/discuss viability? The regex in the "first comment" is clearly too loose, but I can't offer a suggestion on modification without something tangible to work with.

Maybe not too loose anymore? Since our initial discussion, I've updated the XSD definition for absolute URI (<xs:simpleType name="AbsoluteUriType">...) in the first comment, especially the XSD regex, to match the RFC 3986 syntax for absolute URI (cf. the ABNF definition of URI in Appendix A). So this ABNF definition defines the syntax of valid absolute URI strings (with optional fragment). For translating that to an XSD regex, I reused this work actually, but I had to fix a few things to make it a valid XSD regex (the XSD regex flavor is more limited than most flavors) :-(.

So the regex checks the syntax for an absolute URI according to the RFC 3986 only, which is the generic / scheme-agnostic syntax. Scheme-specific rules are out of scope (as it has always been the case with the anyURI type used for URIs in XACML so far by the way). I think going beyond that by trying to address scheme-specific rules defined in other RFCs for every possible URI scheme out there (e.g. http/https, ftp/ftps, file, etc.), is almost impractical and sub-optimal to do in a XSD regex. They are so many, and whenever a new URI scheme comes up, we would need to update the XSD. Instead, we can let the PDP implementers do the extra checks for certain specific URI schemes if they want (in fact, they can just customize the XSD pattern for their need, or do the checks in the code which is usually more efficient than regex evaluation).

If you wish to test for yourself, please use an actual XML Schema implementation, because XML schema has its own regex dialect (for example, start/end anchors ^$ are implicit here) and some characters in the regex have been escaped/replaced because they are special in XML, e.g. & replaced with & so another regex flavor will misinterpret that). If you don't have one at hand (e.g. via XML tool support in your favorite IDE or via a library / CLI), you can use the online XSD validator from Liquid Technologies for example. To facilitate, I've attached a small XML schema which defines an element of type AbsoluteUriType ( <absUri>) for testing (Github rejects files with .xsd extension, therefore I added an extra .txt extension to the file but you remove / rename it to XSD_test.xsd) . With that, you can test the validity of the URI by putting it in a absUri element in a XML doc like this:

<?xml version="1.0" encoding="UTF-8"?>
<test xmlns="urn:example:schema:1">
    <absUri>https://www.w3.org/2001/XMLSchema#string</absUri>
</test>

If you find a string that is invalid (resp. valid) according to the RFC 3986 absolute URI definition but accepted (resp. rejected) by the AbsoluteUriType (<absUri> value), let me know. Happy to fix.

Regarding desirable URI strings, I checked, among others, the standard URIs in XACML from the list of of short identifiers created by @steven-legg. So that's a first list of desirable URI strings.

Regarding undesirable strings, I also checked the list of URIs you mentioned in a previous comment. Some of them are actually valid according to the RFC as I found out:

https://example.com\path
http://example.com:invalid/path
http://example.com:80:80
http://ex:port/
https://[fe80::1%eth0]/
https://example.com/pa%th

All invalid according to RFC 3986 and confirmed invalid with respect to AbsoluteUriType too

http://.example.com

Valid according to RFC 3986 (and confirmed valid with respect to AbsoluteUriType too). The host part matches the reg-name rule. Doesn't have to be a valid DNS domain name according to the RFC section 3.2.2: This specification does not mandate a particular registered name lookup technology and therefore does not restrict the syntax of reg-name beyond what is necessary for interoperability.
Note: the intent here is to check only compliance to RFC 3986 (for absolute URIs). As explained before (second paragraph), http-scheme-specific syntax rules are not considered in the regex as it is not in the RFC 3986.

http://user:pass@/no-host

Valid according to RFC 3986 (and confirmed valid with respect to AbsoluteUriType too). The empty host part matches the reg-name rule. Section 3.2.2 of the RFC says If the URI scheme defines a default for host, then that default applies when the host subcomponent is undefined or when the registered name is empty (zero length).
Note: the intent here is to check only compliance to RFC 3986 (for absolute URIs). As explained before (second paragraph), http-scheme-specific syntax rules are not considered in the regex as it is not in the RFC 3986.

http://[::1]:

Valid according to RFC 3986 and confirmed valid with respect to AbsoluteUriType too. The host part [::1] is a valid IPv6 address and the port is empty, which is also allowed by the ABNF syntax in section 3.2.3. See also section 6.2.3 of the RFC for another valid example with an empty port (http://example.com:/).

I've tested each case with two different XSD implementations.

On a side note, while XML requires string "trimming" outside of the regex match, is there a downside to using "^$" framing consistently in our regex matching from a practical perspective? I believe this should remove language specific implementations in the vast majority of cases. The only place where I can think of it not working would be if a string had "intentional whitespace" (e.g. " string", "string ", etc.). Otherwise XML's process of match(trim(string), /regex/) should be equal to the more universal match(string, /^regex$/).

Not sure this is in line with what you mean, but what I had in mind is that we can define the regex in ACAL with the ^$ anchors, so the pattern looks like ^regex$, where the regex must remain compatible with all regex dialects used in XML, JSON, YAML schemas. We just need to make sure it is XSD-compatible, therefore limited to the XSD regex dialect (more info on limitations) because it is the most reduced/limited one as far as I can see, especially in comparison to the ECMAScript regex dialect used in JSON schema. Then, when we map ^regex$ to the pattern in the XML schema (of XACML), we just remove the ^$ anchors which are implicit (and escape/replace the special characters in XML as usual, e.g. the ampersand). But when we map it to the pattern in JSON schema (of JACAL), we keep them because unlike XML schema, they are not implicit in JSON schema. More info in section 6.3.3 of JSON Schema validation spec.

[steven-legg]

{ "type": "string", "format": "uri" }

This accepts any RFC 3986 URI. The JSON Schema Validation spec §7.3.5 (cf. link at the end of my previous comment) only says:

uri: A string instance is valid against this attribute if it is a valid URI, according to [RFC3986].

So want then is the difference between "uri" and "uri-reference"?

uri-reference: A string instance is valid against this attribute if it is a valid URI Reference (either a URI or a relative-reference), according to [[RFC3986].

"uri" seems to be alluding to the URI ABNF rule in RFC3986 (an absolute URI) and "uri-reference" seems to be alluding to the URI-reference ABNF rule which allows absolute and relative URIs.

But I still think this is needlessly complicated. It will be irrelevant for most IdentifierType values.

[steven-legg]

Instead, we can let the PDP implementers do the extra checks for certain specific URI schemes if they want (in fact, they can just customize the XSD pattern for their need, or do the checks in the code which is usually more efficient than regex evaluation).

Well I decided even earlier to let the PDP implementers do the whole absolute URI check and not go down this URI rat hole. We're the ACAL police; we don't need to be the URI police.

[cdanger]

I thought about using anyURI and a union type but decided it was needlessly complicated and suboptimal.

There are three cases:

* an absolute URI,

* a short identifier name or

* a string with multiple short identifier references.

We expect that the second case will overwhelmingly be used. The example I added to the spec only uses the second case. A specification that always forces an attempt to test if the IdentifierType value is an absolute URI is wasting time when that attempt will nearly always fail. An implementation is obliged to verify that the second and third cases result in an absolute URI and whatever way the implementation chooses to do that can equally well be applied to the first case, and post-checking outside the syntax leaves scope for an implementation to optimize the processing.

That's OK, I can change the order in the union as you wish, so that the second case (ShortIdNameType) is always hit first:

<xs:simpleType name="IdentifierType">
   <xs:union memberTypes="ShortIdNameType ShortIdBasedUriExpressionType AbsoluteUriType" />
</xs:simpleType>

The XSD 1.1 standard, Part 2 section 2.4.1.3 says:

The order in which the ·member types· are specified in the definition [...] is significant. During validation, an element or attribute's value is validated against the ·member types· in the order in which they appear in the definition until a match is found.