Integrating BTCMap.org

[vindard]

BTCMap.org

This is a new project that aims to leverage the power of community-curated open datasets to create a source of data for physical places that accept bitcoin around the world.

The idea is that anyone will be able to tap into this dataset and create user-facing interfaces for it in their own custom ways. One of the drivers behind this is to remove the duplicate recording of map data for physical merhcants that different services currently curate in their own silos. It's also to help bootstrap new applications that would like to integrate some subset of this data into their own apps.

The data is hosted via the OpenStreetMaps platform.

The folks from the project also recently did a podcast episode with Stephan Livera talking about their intentions.

Galoy Maps

Galoy instances have built-in merchant mapping functionality via location data stored as a key on user objects in our internal databases. This is exposed through the API via a businessMapMarkers query. The Bitcoin Beach Wallet app has a built-in map interface for example that users can access to see who accepts bitcoin in their local communities. Map data is sourced from the API and used to populate a map visualisation in the app that users can then explore.

Storage

In the Galoy context, map data doesn't have its own collection as yet, but rather is tied to a user object in our system. The upside to this is, each set of co-ordinates is guaranteed to have a Galoy wallet attached to it for sending/receiving bitcoin.

This instance-level storage means that while different instances cannot share directly map data right now, they can call each other's API endpoints to pull in data from multiple sources. This is complicated and not done in practice though, so the current state of things is that the Bitcoin Beach and Bitcoin Jungle maps look very different for example.

Editing

Co-ordinates can be attached to a user via the admin panel or via the admin API by authorized users for a given instance. In the current implementation, users have no access to control their map data.

Future

The map feature is one that hasn't been given much attention since its earliest iterations. There are a few plans for this feature set, with no priority assigned as yet to these:

• separate the data from the User object
• expand permissions for who can edit this data
• extract the functionality entirely from our core code and bring it into our API via a GraphQL Federation setup
  • this opens the door to instance owners bringing their own interoperable map data to the API from a 3rd-party for example, and not necessarily needing to manage all this in-house

Potential integrations with BTCMap.org

There are a few pathways we can take to leverage the efforts being done with the BTCMap.org work:

• Exporting our current internal data to the dataset
• Updating the dataset whenever internal data is updated
• Reading from the dataset as a source of data

Exporting current data

This might actually be the lowest-hanging-fruit action to take to start participating in the project. Our mapping data is freely available via our API (see schema) which opens the door to anyone scraping this data and adding it to BTCMap.org.

GraphQL query:

{
  businessMapMarkers {
    mapInfo {
      title
      coordinates {
        latitude
        longitude
      }
    }
  	username
  }
}

curl to fetch data:

curl 'https://api.mainnet.galoy.io/graphql' \
    -H 'Accept-Encoding: gzip, deflate, br' \
    -H 'Content-Type: application/json' \
    -H 'Accept: application/json' \
    -H 'Connection: keep-alive' \
    -H 'DNT: 1' \
    -H 'Origin: https://api.mainnet.galoy.io' \
    --data-binary '{"query":"{\n  businessMapMarkers {\n    mapInfo {\n      title\n      coordinates {\n        latitude\n        longitude\n      }\n    }\n  \tusername\n  }\n}"}' \
    --compressed \
    > bbw-map-data-raw.json

# Format with jq if installed
jq . bbw-map-data-raw.json > bbw-map-data.json

Fetched data looks like:

{
  "data": {
    "businessMapMarkers": [
      {
        "mapInfo": {
          "title": " SEN servicios y productos",
          "coordinates": {
            "latitude": 10.288626,
            "longitude": -68.028272
          }
        },
        "username": "sanelo"
      },
      {
        "mapInfo": {
          "title": " Saranghe Look Kawaii",
          "coordinates": {
            "latitude": 13.350283,
            "longitude": -88.348252
          }
        },
        "username": "Helena"
      },
      .
      .
      .
    ]
  }
}

A potential point to consider here is adding some sort of custom tag to signal that this was sourced from a given Galoy instance (e.g. adding a galoy tag, or adding a key like pay.bbw.sv, which would also double as the domain for instance Lightning Addresses). This could help different instances, and at the same time signal a set of common expectations of the data e.g. being both onchain/lightning ready.

Updating the current data

There are two pathways to doing this.

Internal integration

One is an internal integration that sends off a write operation to OpenStreetMaps alongside our internal database updates from our code. Since we are currently in the mode of not expanding non-essential functionality in our core codebase, this may not be a feasible pathway right now.

External cronjob

This one is a more intensive way to keep things updated, but anyone can theoretically run an update script on a cronjob to periodically scrap Galoy data & OSM data, compare the two, and update the data wherever relevant. The upside to this is no changes need to be made to the Galoy codebase to do this, and instance owners don't even need to be involved in this effort.

Reading in data as a source

This is a more future-facing subheading, but in theory instances can eventually get to a place where they curate a subset of data on the OpenStreetMaps project and then read from that dataset to populate their local applications.

The upside to this is it encourages more open participation/curation of the data, and it makes it easier to share data across Galoy instances if some common standard is adopted.

The downside is that with an open dataset, instances could be open to spam or data tampering/deletion which could mess with the experience they are trying to go for. This isn't currently a problem since Galoy maps are administered internally, and it would be something that needs to be figured out.

[nicolasburtey]

I think I would prioritize the extraction of the map to its own codebase and glue it back together with Apollo federation for our current use case. then it will be easier to get the community to help on this.

BTCMap.org could then easily create the cron or other callback task they might need very easily

[dadofsambonzuki]

I like the idea of an initial data load into BTC Map and then a cron job on our side that looks for diffs. We can create GH Issues on the diffs that our Supertaggers can handle.

We can then explore other options for deeper integration if you wanted to leverage the BTC Map API for merchant data more generally.

[bubelov]

Here is a pre-processed list: https://data.btcmap.org/galoy.html

We can't fully automate the addition of those places because we need to check for duplicates but we have quite a few mappers who can help us with the initial import. Re-importing the diffs is probably the easiest approach to keep us in sync.

[mnalis]

There seems to be unapproved import of data to OpenStreetMap related to this discussion? Can you confirm if that is true?

If so, please see https://www.openstreetmap.org/changeset/128909577
and
https://community.openstreetmap.org/t/shadowy-supertaggers/5543

and engage with OSM community in discussion there.

Note that there are requirements that must be met before data may be imported to OSM, so it is crucial to familiarize oneself with following before engaging in such activity:

• https://wiki.openstreetmap.org/wiki/Import/Guidelines and
• https://wiki.openstreetmap.org/wiki/Automated_Edits_code_of_conduct

[dadofsambonzuki]

Hi,

Nathan here from BTCMap.org.

Understood and agreed. We're working with Galoy on a better process going forward.

I'll respond on the the main thread shortly so that everything is together.

Thanks,

Nathan
Co-founder
BTCMap.org

[hyperfree]

Just leaving my 2 sats as a BBW user:

I'd love to have a selection on the map where I can see ALL shops that accept BTC in any way. This would add a lot of value because I have more options as a user and it also adds to the network effect.

[stackingsaunter]

Bump. Blink should use open map standard!