Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish minutely diffs for the dev instance (for Overpass) #1134

Open
NeatNit opened this issue Aug 6, 2024 · 5 comments
Open

Publish minutely diffs for the dev instance (for Overpass) #1134

NeatNit opened this issue Aug 6, 2024 · 5 comments

Comments

@NeatNit
Copy link

NeatNit commented Aug 6, 2024

The developer instance https://master.apis.dev.openstreetmap.org/ is useful for testing applications against, but applications that rely on Overpass (e.g. for automated or sparse editing) can't be properly tested because there is no Overpass for this instance.
Over at drolbr/Overpass-API#593, the answer was that it's currently impossible because there's no minutely diffs for the dev server. That was 4 years ago but I couldn't find any newer information, so I assume this is still the case. So would it be possible to start publishing what's needed to get Overpass going?

I will freely admit I have very little idea of how much I'm asking for. But it seems like no one even opened this issue in these 4 years so I figured it's worth asking.

@tomhughes
Copy link
Member

Running minutely diffs is a quite a complicated thing to do so I don't really think we want to go there.

@NeatNit
Copy link
Author

NeatNit commented Aug 6, 2024

I am curious about why it is so complicated, seeing as it's already implemented for production? If the frequency is the problem (which I kind of doubt), every 10 minutes would probably be good enough IMHO.

If you don't feel like explaining then don't, I'll be okay :)

@tomhughes
Copy link
Member

It uses https://github.com/openstreetmap/osmdbt which requires a custom postgres plugin to be installed in the database and then a bunch of fiddling about to enable replication and some scripts to do a bunch of steps every minute to extract the data as well as finding somewhere to publish the results.

@NeatNit
Copy link
Author

NeatNit commented Aug 6, 2024

I see. So it's a lot of manual work to set up basically? Plus the "find a place" thing.

Well, I hope you at least consider it, as I said it would be really useful for certain use cases.

@mmd-osm
Copy link

mmd-osm commented Aug 6, 2024

The other issue is that you would usually need a planet file as a starting point for Overpass API, then apply minutely diffs on top. So in addition to setting up osmdbt, we also need to set up frequent backups to be used by planet-dump-ng. I think at this point is fairly clear why this hasn't been set up before. You could of course do this all locally, but there's a bit of a learning curve here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants