Support big integers in JSON documents

[dlo]

Hi all! Thanks so much for open sourcing this tool.

I noticed that the linter fails when checking a file with a large integer (9223372036854776000), even if it's valid JSON. I ran into this when trying to bundle several files, one of which included the one that failed the linter. Here's the file in its entirety (you'll want to remove the // FAILS HERE text, obviously):

{
  "$id": "main.v1.Pricing.schema.json",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "additionalProperties": false,
  "properties": {
    "amount": {
      "$ref": "google.type.Money.schema.json"
    },
    "count": {
      "anyOf": [
        {
          "exclusiveMaximum": 9223372036854776000, // FAILS HERE
          "minimum": -9223372036854775808,
          "type": "integer"
        },
        {
          "pattern": "^-?[0-9]+$",
          "type": "string"
        }
      ]
    },
  },
  "title": "Pricing",
  "type": "object"
}

Invocation:

$ jsonschema lint main.v1.Pricing.schema.json
error: Failed to parse the JSON document at line 12 and column 31
  REDACTEDPATH/main.v1.Pricing.schema.json

[jviotti]

Hey @dlo ,

That's an interesting one!

The ECMA-404 standard for JSON defines numbers as having arbitrary infinite precision. i.e. according to it, you could have a 512-bit integer there! While the IETF RFC 8259 version of it clarifies that anything beyond 64-bit is not guaranteed to be interoperable:

numbers that are integers and are in the range [-(2**53)+1, (2**53)-1] are interoperable in the sense that implementations will agree exactly on their numeric values.

We maintain our own JSON parser (here: https://github.com/sourcemeta/core/tree/main/src/core/json) and we constraint integers as signed 64-bit integers indeed. The problematic integer you have there, 9223372036854776000, is slightly over the 2^64 -1 limit (9223372036854775807), so it indeed won't fit.

I'm not sure what language you are using in general, but what makes this whole thing more confusing is that languages like JavaScript by default represent integers as IEEE 764 floating point numbers (https://stackoverflow.com/questions/9643626/does-javascript-support-64-bit-integers). That means that Node.js will "take" numbers like 9223372036854776000, but with loss of precision and very strange behaviour. For example:

$ node
Welcome to Node.js v23.7.0.
Type ".help" for more information.
> 9223372036854776000
9223372036854776000
> 9223372036854776000 + 1
9223372036854776000
> 9223372036854776000 - 1
9223372036854776000

As you can see, basic arithmetic is completely broken.

In general, do you have a strong reason to do integers beyond the 64-bit range? If possible, it would recommend avoiding it, as most JSON parsers and JSON Schema validators out there will exhibit weird behaviour and pool interoperability, which is why the RFC constraints are in there!

That said:

• I think I should really improve the error message. Current one sucks
• I am interested in supporting arbitrary precision integers on my JSON implementation at some point for some more niche numerical uses of JSON (see Implement BigInt support (for JSON Schema) core#838). That said, I don't think I'll get into it any time soon, plus it might be complex to pull off properly (can always re-consider prioritising it on a contract or for a reasonable donation or license in case anybody else is reading this 😅)

Let me get into the error message one, hopefully today

[jviotti]

Though actually... maybe it is not that hard to do 128-bit integers (given __int128_t) in some compilers after all...

[jviotti]

See v6.0.2 (https://github.com/sourcemeta/jsonschema/releases/tag/v6.0.2). That version at least improves the error message. I'll keep this issue open as I might play around with __int128_t at some point soon.