Skip to content

GH-3404: Canonicalize decimals during inlining for TDB2. #3405

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

GH-3404: Canonicalize decimals during inlining for TDB2. #3405

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions jena-arq/src/main/java/org/apache/jena/sparql/util/XSDNumUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,18 @@ public static String canonicalDecimalStrNoIntegerDot(BigDecimal bd) {
return bd.stripTrailingZeros().toPlainString();
}

/** Return a canonical decimal with a trailing ".0". */
public static BigDecimal canonicalDecimalWithDot(BigDecimal decimal) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this differ from canonicalDecimalStrWithDot?

Copy link
Contributor Author

@Aklakan Aklakan Aug 30, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new method is used on input after parsing the lexical form into a decimal - just before inlining. The Str Version ist used on output, after converting the decimal to string. I think that the Str Version could be removed in favor of Decimal.toPlainString() since the binary representation is now canonicalized on input.

BigDecimal result = decimal;
if (result.scale() > 1) {
result = decimal.stripTrailingZeros();
}
if (result.scale() <= 0) {
result = result.setScale(1);
}
return result;
}

/**
* Integer-valued decimals have a trailing ".0".
* (In XML Schema Datatype 1.1 they did not have a ".0".)
Expand Down
1 change: 1 addition & 0 deletions jena-tdb2/src/main/java/org/apache/jena/tdb2/store/NodeIdInline.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,7 @@ public static boolean hasInlineDatatype(Node node) {
// But at this point we know it's a valid literal so the excessive
// chopping by .trim is safe.
BigDecimal decimal = new BigDecimal(lit.getLexicalForm().trim());
decimal = XSDNumUtils.canonicalDecimalWithDot(decimal);

// Does range checking.
DecimalNode56 dn = DecimalNode56.valueOf(decimal);
Expand Down
67 changes: 59 additions & 8 deletions jena-tdb2/src/test/java/org/apache/jena/tdb2/store/value/TestNodeIdInline.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,35 +91,68 @@ public class TestNodeIdInline
@Test public void nodeId_int_21()
{ testNoInline("'300'^^xsd:byte"); }

@Test public void nodeId_decimal_1()
@Test public void nodeId_decimal_01()
{ test("3.14", "3.14"); }

@Test public void nodeId_decimal_2()
@Test public void nodeId_decimal_02()
{ testNoInline("123456789.123456789"); }

// Just this once, directly create the Node.
@Test public void nodeId_decimal_3()
@Test public void nodeId_decimal_03()
{ test("12.89", NodeFactory.createLiteralDT("12.89", XSDDatatype.XSDdecimal)); }

@Test public void nodeId_decimal_4()
@Test public void nodeId_decimal_04()
{ test("-1.0", "-1.0"); }

// This number has > 47 bits of value : 2412.80478192688
@Test public void nodeId_decimal_5()
@Test public void nodeId_decimal_05()
{ testNoInline("2412.80478192688"); }

// This number has > 47 bits of value : -2412.80478192688
@Test public void nodeId_decimal_6()
@Test public void nodeId_decimal_06()
{ testNoInline("-2412.80478192688"); }

@Test public void nodeId_decimal_7()
@Test public void nodeId_decimal_07()
{ test("'0.00000001'^^xsd:decimal",
NodeFactory.createLiteralDT("0.00000001", XSDDatatype.XSDdecimal));
}

@Test public void nodeId_decimal_8()
@Test public void nodeId_decimal_08()
{ test("0.00000001", NodeFactory.createLiteralDT("0.00000001", XSDDatatype.XSDdecimal)); }

@Test public void nodeId_decimal_09()
{ testNodeIdRoundtripDecimal("0"); }

@Test public void nodeId_decimal_10()
{ testNodeIdRoundtripDecimal("-0"); }

@Test public void nodeId_decimal_11()
{ testNodeIdRoundtripDecimal("0.0"); }

@Test public void nodeId_decimal_12()
{ testNodeIdRoundtripDecimal("-0.0"); }

@Test public void nodeId_decimal_13()
{ testNodeIdRoundtripDecimal(".0"); }

@Test public void nodeId_decimal_14()
{ testNodeIdRoundtripDecimal("-.0"); }

@Test public void nodeId_decimal_15()
{ testNodeIdRoundtripDecimal("18"); }

@Test public void nodeId_decimal_16()
{ testNodeIdRoundtripDecimal("18."); }

@Test public void nodeId_decimal_17()
{ testNodeIdRoundtripDecimal("18.0"); }

@Test public void nodeId_decimal_18()
{ testNodeIdRoundtripDecimal("18.00"); }

@Test public void nodeId_decimal_19()
{ testNodeIdRoundtripDecimal("18.000"); }

@Test public void nodeId_dateTime_01()
{ test("'2008-04-28T15:36:15+01:00'^^xsd:dateTime"); }

Expand Down Expand Up @@ -306,4 +339,22 @@ private void test(String x, Node correct) {
// Term equality.
assertEquals(correct, n2, ()->"Not same term");
}

private static void testNodeIdRoundtripDecimal(String decimalStr) {
Node node = NodeFactory.createLiteralDT(decimalStr, XSDDatatype.XSDdecimal);
testNodeIdRoundtrip(node);
}

/** For a Node n assert: nodeId(n) == nodeId(extract(nodeId(n)) */
private static void testNodeIdRoundtrip(Node node) {
NodeId nodeId = NodeId.inline(node);
testNodeIdRoundtrip(nodeId);
}

/** For a NodeId n assert: n == nodeId(extract(n)) */
private static void testNodeIdRoundtrip(NodeId expected) {
Node extractedNode = NodeId.extract(expected);
NodeId actual = NodeId.inline(extractedNode);
assertEquals(expected, actual);
}
}