Proposal - Define JSON Logic on the JSON Data Model

[gregsdennis]

JSON Schema, JSON Path, and JSON-e are all defined on the JSON data model. That is, they are defined on a data model that consists of:

• objects (unordered string-key/value pairs)
• arrays (ordered [not sorted] values)
• strings
• numbers
• true literal
• false literal
• null literal

true and false are typically combined as a "boolean" type. Also, an object property being present with a null value is distinct from the property missing.

Most importantly, operating on this data model:

• removes any lexical issues from the scope of the spec (whitespace, number formatting, etc.)
• allows processing of other compatible formats, e.g. YAML

---

Important

@TotalTechGeek is appending the following suggestion to this proposal https://github.com/orgs/json-logic/discussions/17#discussioncomment-11674812

[JTeeuwissen]

Not sure if within the scope of this proposal, but I think most programming languages can describe this model relatively straightforward, but I think numbers require some extra considerations. Should they be represented as floating binary points or floating decimal points, and how precise? Json itself does not put any constraints on the precision of numbers, but that might result in an inefficient or unworkable solution in most languages.

[TotalTechGeek]

But users shouldn't have to know more than "adheres to spec" when using Json logic across multiple implementations.

This is essentially what I'm promoting though.

JSON Logic as an org would mandate its recommended baseline / main profile / whatever we want to call it, but the IETF proposal or whatever route we go with is generic enough to be architecture / runtime portable.

That way you get AST -- Core -- Community 1.1.

And the community profile can be promoted easily to include other common extensions that are supported later on, without revising core. (If we add ISO date extensions or something)

[gregsdennis]

For the most explicitly clarity on this issue, I present the multipleOf excerpt from the spec and a test case from the JSON Schema Test Suite.

6.2.1. multipleOf

The value of "multipleOf" MUST be a number, strictly greater than 0.
A numeric instance is valid only if division by this keyword's value results in an integer.

{
  "description": "by small number",
  "schema": {
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "multipleOf": 0.0001
  },
  "tests": [
      {
          "description": "0.0075 is multiple of 0.0001",
          "data": 0.0075,
          "valid": true
      },
      {
          "description": "0.00751 is not multiple of 0.0001",
          "data": 0.00751,
          "valid": false
      }
  ]
}

Just about every implementation of JSON Schema across a multitude of languages passes this test case, despite many of them using IEEE754 double-precision floats.

In C#, to illustrate the difference, I created this test:

double a = .0075;  // IEEE754
double b = .0001;
decimal c = .0075m;
decimal d = .0001m;

Console.WriteLine("double");
Console.WriteLine(a % b);
Console.WriteLine();
Console.WriteLine("decimal");
Console.WriteLine(c % d);

which outputs

double
9.999999999999937E-05

decimal
0.0000

You can see that simply using the built-in modulus operator % presents a problem with IEEE754 math (which is what @rmannibucau is arguing). Despite this, the implementations pass these tests.

JSON Schema is not made unusable by requiring a mathematical interpretation of division for multipleOf. I also argue that most people find this easier to understand.

Implementations are not burdened with any kind of difficulty in supporting this, and it's easy to understand because it's very simple math.

I grant that a "multipleOf" calculation isn't as extensive as the math that JSON Logic offers, but my point is the same: defining the spec based on math makes sense to most people. Enforcing it with the test suite will ensure that implementations align. It may put a small burden on the implementations, but that's where the burden should lie, not with the users.

[JTeeuwissen]

Just as a note, the c# implementation of json schema from ricosuter uses decimal for multipleOf.

[rmannibucau]

Just means the test suite coverage is bad and doesnt consider edge cases which lead to all issues (just check on github) so this is abad practise for a spec.

[rmannibucau]

@JTeeuwissen moving to 16bytes just moves the issue, if spec says it is infinite as proposed your impl still has issues. Even in string based impl you have limits on the precision.

[TotalTechGeek]

Warning

Revote necessary because of new appended suggestion; this is an outdated vote.

Approved; +1

[codetiger]

Warning

Revote necessary because of new appended suggestion; this is an outdated vote.

Agreed 100%.
One of the reasons why I named my repo DataLogic-rs is to support the model is different formats.

[TotalTechGeek]

Okay: We have 3 TC Approvals (I'm assuming @gregsdennis implicitly approves this), but some community dissent.

I think I'd like to switch to Apache Voting,

Warning

Revote necessary because of new appended suggestion; these are outdated votes.

TC Approval Votes:

• @gregsdennis (Assumed +1)
• @TotalTechGeek (+1)
• @codetiger (+1)

Community Dissent:

• @rmannibucau (Assumed -1)

Assuming @JTeeuwissen is a -0.2, and @dslmeinte is a -0.1 (maybe +0.1)

The dissent is on whether the JSON Logic Core specification should incorporate any additional specifications around "numbers" beyond what JSON specifies.

Dissent Summary

Community Concern: Because JSON does not specify a range for numbers, many JSON parsers use IEEE754, and cannot represent large integers.

@gregsdennis believes that the JSON / YAML Parser is responsible for deciding the range, not the JSON Logic Core spec, which follows prior art with what JSON / JSON-E / JSON Schema decided.

[TotalTechGeek]

Because we haven't "officially" tweaked our voting system, I'm going to use the previously communicated rules for "If there's dissent, we'll try to reach consensus, and if it can't be reached, a 4th TC member's vote is required"

[rmannibucau]

FTR i'm not against, I'm just pointing out that "numbers" doesn't mean anything if not defined

[TotalTechGeek]

Ah; apologies if I assumed -1 a bit overzealously!

I'm not sure if I find myself disagreeing with the community dissent myself, but I think I'm comfortable with the convention exercised by other projects, and I'd be for incorporating the supported ranges into our compatibility tables.

[toddbaert]

Consider me on-board as well.

[TotalTechGeek]

Okay folks! I don't think we're going to reach perfect consensus here.

I think everyone had great input, and I think debate truly helps us achieve a better outcome.

To be clear, I did appreciate the points / thoughts raised by @JTeeuwissen, @dslmeinte and @rmannibucau, and I hope everyone felt heard in this discussion.

I think based on the approvals, we will be moving forward with the compromise:

• Core remains precision agnostic and a general specification.
• JSON Logic will maintain profiles and a recommended standard for implementations.
• If we introduce a spec operator, named profiles will come from JSON Logic organization, not from "Core".

The points about the logic & behavior portability are acknowledged; and we agree that there should be a mandate, but the implementation threshold will be maintained outside of the RFC.

This will allow the org to upgrade the base threshold more rapidly as we get mass support for community extensions.

---

TC Approvers:

• @TotalTechGeek (+1)
• @toddbaert (+1
• @gregsdennis (+1)

Assumed Abstaining:

• @codetiger

[TotalTechGeek]

imo; this was one of four major proposals we need to knock out before we begin the mass re-evaluation of operators, though another member might spot something I haven't.

We needed to decide:

• Backwards Compat Policy ✅
• Data model / precision policies for JSON Logic ✅
• Do we replace var with something else?
  (To define tests properly, getting out context operators locked in is relatively critical)
• Do we erase sugaring from JSON Logic? (This seems to be a regret of @jwadhams)
  (Inconsistent behavior of truthiness in "!!" operator jwadhams/json-logic#44 (comment). My prescribed approach is rather different from Jeremy's, but I do think a change should be made. In compliance with the policy, the change should also be backwards compatible)

Once we decide these guys, I think it opens the flood gates for re-evaluating each operator and designing a robust suite of tests for each of them.

[codetiger]

After the discussion I am changing my vote to +1 for the proposal.