Merge pull request #3517 from lonvia/improve-custom-formatter

Extend use of custom result formatters to CLI tool

Author: lonvia

docs/customize/Result-Formatting.md

@@ -66,6 +66,8 @@ For example, let us extend the result for the status call in text format
 and add the server URL. Such a formatter would look like this:
 
 ``` python
+from nominatim_api import StatusResult
+
 @dispatch.format_func(StatusResult, 'text')
 def _format_status_text(result, _):
     header = 'Status for server nominatim.openstreetmap.org'
@@ -86,19 +88,39 @@ as adding formatting functions for all result types using the custom
 format name:
 
 ``` python
+from nominatim_api import StatusResult
+
 @dispatch.format_func(StatusResult, 'chatty')
 def _format_status_text(result, _):
     if result.status:
         return f"The server is currently not running. {result.message}"
 
-    return f"Good news! The server is running just fine."
+    return "Good news! The server is running just fine."
 ```
 
 That's all. Nominatim will automatically pick up the new format name and
-will allow the user to use it. Make sure to really define formatters for
-**all** result types. If they are for endpoints that you do not intend to
-use, you can simply return some static string but the function needs to be
-there.
+will allow the user to use it. There is no need to implement formatter
+functions for all the result types, when you invent a new one. The
+available formats will be determined for each API endpoint separately.
+To find out which formats are available, you can use the `--list-formats`
+option of the CLI tool:
+
+```
+me@machine:planet-project$ nominatim status --list-formats
+2024-08-16 19:54:00: Using project directory: /home/nominatim/planet-project
+text
+json
+chatty
+debug
+me@machine:planet-project$
+```
+
+The `debug` format listed in the last line will always appear. It is a
+special format that enables debug output via the command line (the same
+as the `debug=1` parameter enables for the server API). To not clash
+with this built-in function, you shouldn't name your own format 'debug'.
+
+### Content type of new formats
 
 All responses will be returned with the content type application/json by
 default. If your format produces a different content type, you need
@@ -117,6 +139,67 @@ The `content_types` module used above provides constants for the most
 frequent content types. You set the content type to an arbitrary string,
 if the content type you need is not available.
 
+## Formatting error messages
+
+Any exception thrown during processing of a request is given to
+a special error formatting function. It takes the requested content type,
+the status code and the error message. It should return the error message
+in a form appropriate for the given content type.
+
+You can overwrite the default formatting function with the decorator
+`error_format_func`:
+
+``` python
+import nominatim_api.server.content_types as ct
+
+@dispatch.error_format_func
+def _format_error(content_type: str, msg: str, status: int) -> str:
+    if content_type == ct.CONTENT_XML:
+        return f"""<?xml version="1.0" encoding="UTF-8" ?>
+                     <message>{msg}</message>
+                """
+    if content_type == ct.CONTENT_JSON:
+        return f'"{msg}"'
+
+    return f"ERROR: {msg}"
+```
+
+
+## Debugging custom formatters
+
+The easiest way to try out your custom formatter is by using the Nominatim
+CLI commands. Custom formats can be chosen with the `--format` parameter:
+
+```
+me@machine:planet-project$ nominatim status --format chatty
+2024-08-16 19:54:00: Using project directory: /home/nominatim/planet-project
+Good news! The server is running just fine.
+me@machine:planet-project$
+```
+
+They will also emit full error messages when there is a problem with the
+code you need to debug.
+
+!!! danger
+    In some cases, when you make an error with your import statement, the
+    CLI will not give you an error but instead tell you, that the API
+    commands are no longer available:
+
+        me@machine: nominatim status
+        usage: nominatim [-h] [--version] {import,freeze,replication,special-phrases,add-data,index,refresh,admin} ...
+        nominatim: error: argument subcommand: invalid choice: 'status'
+
+    This happens because the CLI tool is meant to still work when the
+    nominatim-api package is not installed. Import errors involving
+    `nominatim_api` are interpreted as "package not installed".
+
+    Use the help command to find out which is the offending import that
+    could not be found:
+
+        me@machine: nominatim -h
+        ... [other help text] ...
+        Nominatim API package not found (was looking for module: nominatim_api.xxx).
+
 ## Reference
 
 ### FormatDispatcher

src/nominatim_api/__init__.py

@@ -39,6 +39,7 @@
                       SearchResult as SearchResult,
                       SearchResults as SearchResults)
 from .localization import (Locales as Locales)
-from .result_formatting import (FormatDispatcher as FormatDispatcher)
+from .result_formatting import (FormatDispatcher as FormatDispatcher,
+                                load_format_dispatcher as load_format_dispatcher)
 
 from .version import NOMINATIM_API_VERSION as __version__

src/nominatim_api/v1/__init__.py

@@ -11,9 +11,3 @@
 #pylint: disable=useless-import-alias
 
 from .server_glue import ROUTES as ROUTES
-
-from . import format as _format
-
-list_formats = _format.dispatch.list_formats
-supports_format = _format.dispatch.supports_format
-format_result = _format.dispatch.format_result

src/nominatim_db/cli.py

@@ -243,7 +243,8 @@ def get_set_parser() -> CommandlineParser:
             raise ex
 
         parser.parser.epilog = \
-            '\n\nNominatim API package not found. The following commands are not available:'\
+            f'\n\nNominatim API package not found (was looking for module: {ex.name}).'\
+            '\nThe following commands are not available:'\
             '\n    export, convert, serve, search, reverse, lookup, details, status'\
             "\n\nRun 'pip install nominatim-api' to install the package."