[feature] add robust mode

traveltime-dev · Apr 25, 2022 · 48709dc · 48709dc
1 parent ea4fa33
commit 48709dc
Show file tree

Hide file tree

Showing 4 changed files with 79 additions and 1 deletion.
diff --git a/docs/reference.md b/docs/reference.md
@@ -247,6 +247,10 @@ It can be used to get information such as a service area polygon based on travel
 - UNION will return the union of all polygons for all departure/arrivals searches.
 - INTERSECTION will return the intersection of all departure/arrival searches.
 
+**Robust mode:**
+
+By default, if any error happens, even on just one search, the algorithm fails and returns no result. In cases where partial results are still desired, you can enable robust mode. This will prevent batching and ignore failing requests, returning partial results. Note that this will consume your API quotas much faster.
+
 **Output layer:** Where to save the output layer. If you leave this empty, the result will be loaded as a temporary layer. It is still possible to save a temporary layer afterwards by right-clicking it in the legend and choosing "make permanent".
 
 ### ![](images/icons/timefilter_simple.svg#icon) Time filter (Simplified)
@@ -277,6 +281,10 @@ It can be used to get filter a point layer using a time search.
 
 **Load fares information:** Load informations about fares in the resulting attributes.
 
+**Robust mode:**
+
+By default, if any error happens, even on just one search, the algorithm fails and returns no result. In cases where partial results are still desired, you can enable robust mode. This will prevent batching and ignore failing requests, returning partial results. Note that this will consume your API quotas much faster.
+
 **Output layer:** Where to save the output layer. If you leave this empty, the result will be loaded as a temporary layer. It is still possible to save a temporary layer afterwards by right-clicking it in the legend and choosing "make permanent".
 
 ### ![](images/icons/route_simple.svg#icon) Route (Simplified)
@@ -305,6 +313,10 @@ It can be used to get the best routes between two sets of points.
 
 **Load fares information:** Load informations about fares in the resulting attributes.
 
+**Robust mode:**
+
+By default, if any error happens, even on just one search, the algorithm fails and returns no result. In cases where partial results are still desired, you can enable robust mode. This will prevent batching and ignore failing requests, returning partial results. Note that this will consume your API quotas much faster.
+
 **Output style:**
 
 - BY_ROUTE : return one linestring per route
@@ -343,6 +355,10 @@ Besides, while the API supports INTERSECTION and UNION of searches, the plugin p
 
 It happens that the API returns invalid geometries. While usually making no visible difference, invalid geometries can cause issues when further processing the features. The algorithm by default fixes all geometries in a post-processing step. In most cases, this option should be left enabled.
 
+**Robust mode**
+
+By default, if any error happens, even on just one search, the algorithm fails and returns no result. In cases where partial results are still desired, you can enable robust mode. This will prevent batching and ignore failing requests, returning partial results. Note that this will consume your API quotas much faster.
+
 ### ![](images/icons/timefilter_advanced.svg#icon) Time filter (Advanced)
 
 The parameters match closely the API. They are best documented on [the API documentation](http://docs.traveltimeplatform.com/reference/time-filter/).
@@ -351,8 +367,14 @@ All parameters are expressions, which allow you to freely use the QGIS expressio
 
 #### Differences with the API
 
+**Searches and locations**
+
 In the API, all coordinates are to be provided in the `locations` parameter, and then searches specify locations ids. As this doesn't suit well GIS workflows, the algorithm separates source for searches and for location.
 
+**Robust mode**
+
+By default, if any error happens, even on just one search, the algorithm fails and returns no result. In cases where partial results are still desired, you can enable robust mode. This will prevent batching and ignore failing requests, returning partial results. Note that this will consume your API quotas much faster.
+
 ### ![](images/icons/route_advanced.svg#icon) Route (Advanced)
 
 The parameters match closely the API. They are best documented on [the API documentation](http://docs.traveltimeplatform.com/reference/routes/).
@@ -361,8 +383,14 @@ All parameters are expressions, which allow you to freely use the QGIS expressio
 
 #### Differences with the API
 
+**Searches and locations**
+
 In the API, all coordinates are to be provided in the `locations` parameter, and then searches specify locations ids. As this doesn't suit well GIS workflows, the algorithm separates source for searches and for location.
 
+**Robust mode**
+
+By default, if any error happens, even on just one search, the algorithm fails and returns no result. In cases where partial results are still desired, you can enable robust mode. This will prevent batching and ignore failing requests, returning partial results. Note that this will consume your API quotas much faster.
+
 ### ![](images/icons/geocoding.svg#icon) Geocoding
 
 This algorithm provides a simpified access to the geocoding endpoint.

diff --git a/travel_time_platform_plugin/algorithms/advanced.py b/travel_time_platform_plugin/algorithms/advanced.py
@@ -21,6 +21,7 @@
     QgsLineSymbol,
     QgsPoint,
     QgsProcessing,
+    QgsProcessingException,
     QgsProcessingParameterBoolean,
     QgsProcessingParameterEnum,
     QgsProcessingParameterExpression,
@@ -258,6 +259,19 @@ def initAlgorithm(self, config):
                 ).format(prop),
             )
 
+        self.addParameter(
+            QgsProcessingParameterBoolean(
+                "INPUT_ROBUST_MODE",
+                "Robust mode",
+                optional=True,
+                defaultValue=False,
+            ),
+            advanced=True,
+            help_text=tr(
+                "Ignore error instead of failing. This will consume your API quota much faster as searches won't be batched, and may yield incomplete results."
+            ),
+        )
+
         # Define output parameters
         self.addParameter(
             QgsProcessingParameterFeatureSink(
@@ -292,7 +306,7 @@ def _processAlgorithmYieldSlices(self, parameters, context, feedback):
             else 0
         )
 
-        slicing_size = 10
+        slicing_size = 1 if self.params["INPUT_ROBUST_MODE"] else 10
         slicing_count = math.ceil(max(departure_count, arrival_count) / slicing_size)
 
         for i in range(slicing_count):
@@ -391,6 +405,31 @@ def processAlgorithmComputeSearchCountForThrottling(self, data):
             data.get("arrival_searches", [])
         )
 
+    def processAlgorithmMakeRequest(
+        self, parameters, context, feedback, data=None, params={}
+    ):
+        """Calls AlgorithmBase.processAlgorithmMakeRequest but optionally catching exception"""
+
+        try:
+            return super().processAlgorithmMakeRequest(
+                parameters, context, feedback, data, params
+            )
+        except QgsProcessingException as e:
+            if self.params["INPUT_ROBUST_MODE"]:
+                feedback.reportError(
+                    tr(
+                        "An API error was ignored due to robust mode. You will likely get incomplete result."
+                    )
+                )
+                return {"results": []}
+            else:
+                feedback.reportError(
+                    tr(
+                        "Use robust mode if you want the algorithm to return results despite this error."
+                    )
+                )
+                raise e from None
+
     def enabled_properties(self):
         """Returns the list of properties that are enabled"""
         return [

diff --git a/travel_time_platform_plugin/algorithms/simple.py b/travel_time_platform_plugin/algorithms/simple.py
@@ -83,6 +83,14 @@ def initAlgorithm(self, config):
                 defaultValue=utils.timezones.index(utils.default_timezone),
             )
         )
+        self.addParameter(
+            QgsProcessingParameterBoolean(
+                "SETTINGS_ROBUST_MODE",
+                tr("Robust mode"),
+                optional=True,
+                defaultValue=False,
+            )
+        )
 
         # OUTPUT
         self.addParameter(
@@ -110,6 +118,7 @@ def processAlgorithmPrepareSubParameters(self, parameters, context, feedback):
             "INPUT_THROTTLING_STRATEGY": THROTTLING_STRATEGIES.index(
                 THROTTLING_PER_SETTINGS
             ),
+            "INPUT_ROBUST_MODE": self.params["SETTINGS_ROBUST_MODE"],
             "OUTPUT": "memory:results",
         }
 

diff --git a/travel_time_platform_plugin/metadata.txt b/travel_time_platform_plugin/metadata.txt
@@ -88,6 +88,8 @@ changelog=- 2019-02-12 : version 0.1-alpha
         - fix : fix geoclick
     - 2022-01-18 : version 1.6.0
         - feature : include geometry cleanup step
+    - 2022-04-25 : version 1.7.0
+        - feature : added robust mode
 
 tags=travel,time,platform,api,distance,matrix,geocoder,geocode,geocoding,isochrone,isochrones,routing,polygon,polygons,network analysis,shortest path
 homepage=https://qgis.traveltimeplatform.com/