Removing untagged nodes, impact on node id range

[mmd-osm]

Even though a fair amount of nodes could be eliminated, we still seem to be using the same node ids space in the range of 1 to >10 billion as before, now much more sparsely populated.

While I would assume that the osm2pgsql flat node store will disappear, there may be other applications like mentioned in https://forum.openstreetmap.org/viewtopic.php?pid=871998#p871998, that are not that happy about an ever growing node id space. Other apps like Tilemaker are frequently run alongside an extra renumbering step before the actual process to cut down on memory requirements (systemed/tilemaker#386 et al.)

I'm wondering if we can take full advantage of a much smaller number of nodes, even without any sort of re-numbering. I'm aware that renumbering is opening up a can of worms, but still.

What I'm looking for is some kind of guiding design principles for the new data model like "we will (or will not) renumber nodes, only delete them...".

[AndrewHain]

Are you thinking about less bits in node numbers, less space for contiguous storage or something else?

[mmd-osm]

It’s about finding out in the first place, why they’re using a renumbering step. It’s likely the latter, but in other use cases it might be even both. I added a comment on the tilemaker repo and asked for advise. First hand knowledge…

[mmd-osm]

Pretty insightful response: systemed/tilemaker#386 (comment)

[flacombe]

As osm ids aren't stable identifiers, I find interesting to renumber nodes as 98% of them will be deleted, yes.

It's not an easy operation and should be done only a very few times.
Deleting 98% of existing objects is a valid reason to me.

[ZeLonewolf]

I currently store and track changes in untagged nodes (geometries of streets). I would like to better understand what "removing untagged nodes" actually means and how I would migrate from the current scheme to whatever the future scheme is. Node IDs allow me to track incremental changes in a street. So if you have a street with 10 nodes, and one of the nodes is moved a little bit, I can re-process data on just that node that moved and not the rest of the street.

In my application, I track whether or not a pedestrian has passed within a certain distance of a node. If a node moves (because someone has edited it), I clear the status of that node and re-process it against user data. If there are no node IDs, how would I do that?

Is there more in-depth documentation on the proposed before/after?

[1ec5]

I guess the assumption is that data consumers would key external data off portions of a way using linear referencing rather than the way’s nodes. In the routing domain, traffic data is often matched to ways using linear referencing. However, more general-purpose map matching facilities do match to ways’ nodes, if not by node ID, then by way ID plus an index into the way’s segments.

[mmd-osm]

Assuming a way is just an ordered list of lat/lons in the future, you can compare each lat/lon in way v_n-1 with way v_n, and determine all the locations where lat/lons differ.