Potential issue with custom points of interest

[gigioLR]

Hi, we are testing some customized points of interest for Porto Alegre, but a problem emerged.

During the "compile destinations" subprocess, it shows an error that "name 'engine' is not defined", even though we ran the software for the same city without the custom data.

Traceback (most recent call last):
  File "/home/ghsci/process/subprocesses/_05_compile_destinations.py", line 207, in <module>
    main()
  File "/home/ghsci/process/subprocesses/_05_compile_destinations.py", line 203, in main
    compile_destinations(codename)
  File "/home/ghsci/process/subprocesses/_05_compile_destinations.py", line 184, in compile_destinations
    custom_destination_setup(engine, r)
NameError: name 'engine' is not defined

In the YML for Porto Alegre, we used those parameters for the custom POI:

custom_destinations:
    ## nome do arquivo relativo ao diretório de dados do projeto
    file: other_custom_data/estabelecimentos_classificados_JCBV.csv
    ## nome simples da categoria
    dest_name: DESC_CNAE
    ## nome completo da categoria
    dest_name_full: DESC_CNAE
    ## coordenada y
    lat: LATITUDE
    ## coordenada x
    lon: LONGITUDE
    ## código EPSG
    epsg: 4326
    ## uma citação para esses dados
    citation: "Projeto Ambientes Alimentares"
    #########

It seems like the error happens before reading the CSV, is it our mistake or should we open an issue?

Also, can we use different categories than the ones in the "osm_destination_definitions.csv"? What the columns should have?

[carlhiggs]

Hi @gigioLR , thanks for letting me know about this. It seems the custom destinations code hasn't been fully updated to keep pace with the refactoring. We should introduce tests for use of custom destinations --- but in the mean time, I'll create an issue and push up a fix that I've tested on my end.

[carlhiggs]

Hi @gigioLR , I've just pushed up a fix to address this to the enhancements branch. I think this should address the issue for you, but there may still be some other things to work through on the enhancements branch before I merge this into main (a few things to neaten out with the spatial report template). I'll look at your GTFS issue (#356 ) before merging too in case we make changes for that --- but will be good to check out your config file, just in case.

[carlhiggs]

Regarding the question -- yes --- you can bring in completely new things here.

The mechanism for bringing in new destinations could certainly be more elegant; it was written in a highly task focused way, to allow for use of custom destinations for Maiduguri in our 25-cities analysis. This has changed little since then, but ideally we would allow for more flexibility (eg a list of data sources, and not just CSV in a specific format).

One thing I just changed, at least for custom destinations, instead of 'dest_name' and 'dest_name_full' which are not great field names, I have changed these for the configuration to name_field and description_field which should be more intuitive.

    ## nome do arquivo relativo ao diretório de dados do projeto
    file: 
    ## nome simples da categoria
    name_field: DESC_CNAE
    ## nome completo da categoria
    description_field: DESC_CNAE
    ## coordenada y
    lat: LATITUDE
    ## coordenada x
    lon: LONGITUDE
    ## código EPSG
    epsg: 4326
    ## uma citação para esses dados
    citation: "Projeto Ambientes Alimentares"

The terms dest_name and dest_name_full are still used behind the scenes in the dest_type and destinations tables, but I will leave changing those to be part of a bigger change, later, in case there are unforeseen complications.

Any contributions to the code for importing custom destinations are welcome -- but otherwise, I'll aim to get to this one day. Of course, we want it to work, so thank you for pointing out the refactoring oversights! I hope the changes I've pushed through to the enhancements branch address this issue.

[gigioLR]

Great, we will test it soon @carlhiggs, thank you!

P.S.: A suggestion for future versions is detailing a bit more of the customization process. Maybe include in the software an alternative example based on custom data (POI, population grid) and custom indicators.

[carlhiggs]

Great suggestions, thanks @gigioLR --- I've been aiming to provide some examples of this in code and video, hopefully will get to it in the coming weeks