"Meta" tags

[501Ghost]

This is related to chapter 3.14 of Jochen Topf's study on the OSM data model.

Our general understanding of tags is that each of them describes an aspect of a real life object. Over time, we have introduced many tags that do not refer to aspects of real life objects, but are instead meta tags that are only relevant to OSM users. Think about keys like note, fixme, source, created_by, type, survey:date, check_date and the list goes on.

I propose that we put these tags into a separate "meta tags" category, so people who look at "tags" will only see the information that they expect to see.

This should mostly be relevant and helpful for data users who use OSM tags directly to get information about objects but who do not require any metadata. This includes maps that display a list of tags when you click on OSM objects, like gk.historic.place and openstreetbrowser.org.

Disclaimer: I do not possess the technical know-how to come up with suggestions on the implementation of such a change. I am just presenting a draft proposal here.

[sdicke]

If a meta tag section should be added to the data model, the edit API should allow to exclude it from downloading to save memory footprint and bandwidth.

[501Ghost]

The meta tags are very helpful for editing, especially note, fixme, tiger:reviewed=no etc. If mappers disregard them in their downloads, they might overlook them and miss relevant information about the objects they plan to edit.

I was thinking more about making it possible for end users to exclude these tags from their downloads.

[tordanik]

Because the API intentionally doesn't know about the meaning of tags, this would require either all editors to have a list of all "meta keys" (which is inevitably going to be incomplete given "Any Tag You Like") or mappers to manually put tags into the correct category. Either way, it seems like this would create an extra source of mistakes when information is accidentally added as a tag instead of a meta tag, or vice versa.

At the same time, the practical benefits seem quite minor. These tags are a relatively small fraction of all OSM data, so I predict removing them would not make a meaningful performance difference. And if you're serious about filtering OSM data for your use case, you almost certainly want to remove all the "real" tags your application doesn't care about in addition to the meta tags.

I do appreciate that there is a certain sense of "cleanliness" in distinguishing metadata from information about the real world.

maps that display a list of tags when you click on OSM objects

That's not something I would recommend for any application which cares about usability for a non-mapper audience. For starters, it's not even an option unless your website is in English. And even then, tags aren't (and shouldn't be expected to be) designed as an appealing presentation of data for a general audience.

[SK53]

@tordanik This is not really a fair comparison. Let's say the API provides a way to populate a metatags hstore column in the API database, then broadly speaking it would extend the current API and how editors might interact with it, would be, as at present, up to them. They could ignore it entirely and continue to place fixme, source, note etc. in the current tags column, or individual editors could, as you say, create their own whitelists of values which would be placed into metatags. However, editors currently maintain huge lists of presets (including for some meta tags such as fixme and note in iD), so some of this exists already. As the API never needs to know what the difference is I think it fits cleanly into how things are done in OSM.

From the viewpoint of "any tags you like' nothing changes. An editor might say "you are using a source tag, would you like it moved to metatags?" perhaps, but this would only apply to advanced users. Editors like StreetComplete could hide the addition of things like check_date entirely from the user.

At present the OSM data model offers poor support for regular validation of existing features: I believe the API will have to evolve to support such mechanisms as in heavily mapped areas we move more from data creation to data maintenance (as others have said, see also #26). This is a small step in such a direction.

Unlike others I would not propose to touch ANY existing objects, but if editors want to implement it, then tags may be moved when an object is edited. (Broadly, I'm against changes to keys and tags for cosmetic, as apposed to semantic, reasons.

There are indeed aspects of practicality: is it something useful for maintainers of editors?; how would it look? are the overheads of an extra hstore column commensurate with putative benefits? But I don't think these are a basis for rejecting the idea at this stage.

There are more complex and sophisticated approaches which could be envisaged, but these would involve breaking API changes, so aren't relevant at the moment.

@501Ghost I've been entertaining this idea for about four years, so glad you have advanced it here.

[501Ghost]

I do appreciate that there is a certain sense of "cleanliness" in distinguishing metadata from information about the real world.

This is indeed my main point. I think it would be practical if the API has a concept of metadata that users can edit.

Either way, it seems like this would create an extra source of mistakes when information is accidentally added as a tag instead of a meta tag, or vice versa.

As you said, the API is currently indifferent to what tags are related to the real world and which are meta tags, so separating these tags will undoubtly require some manual labour. Most of this can be done by visiting Taginfo and making a list of what keys should go in the future "meta" section.

We probably won't find all meta tags before an update takes place, so there should probably be a way to (mass) edit tags from one category to the other.

These tags are a relatively small fraction of all OSM data, so I predict removing them would not make a meaningful performance difference.

It could make a slight difference for Overpass queries (out; versus out meta;).

maps that display a list of tags when you click on OSM objects

That's not something I would recommend for any application which cares about usability for a non-mapper audience.

I would also not recommend it, but it would be a bit more viable if the meta tags were listed separately. It's a minor benefit, but for these use cases I think it would be a welcome one.

[mmd-osm]

The topic of handling survey dates as extra data is already being discussed in #26.

While I don't think it makes sense from API point of view, that users can manipulate the list of meta tags (this is only asking for trouble and abuse), I see there's at least some need to keep track of "meta" information like survey dates.

I'm still a bit undecided if this is all massive over-engineering, given how little those tags are used in real life, and given the significant impact that would have on the API. So introducing the extra complexity that comes with those meta tags needs to be very carefully evaluated from an effort vs. benefit point of view.

[sdicke]

But such meta tags are used more and more frequently, for example by StreetComplete.

I'm still a bit undecided if this is all massive over-engineering, given how little those tags are used in real life, and given the significant impact that would have on the API. So introducing the extra complexity that comes with those meta tags needs to be very carefully evaluated from an effort vs. benefit point of view.

[501Ghost]

@mmd-osm As you say, the meta tags have little use in real life. That is precisely why I would put them in a separate category. Then we can say that the non-meta tags are (and should be) relevant in the real world outside OSM.