Merge remote-tracking branch 'origin/main' into mapserver-emptylayers…

…-precached
TerriaJS · Nov 30, 2023 · c83976c · c83976c
2 parents a0e9ed5 + 57a5e8c
commit c83976c
Show file tree

Hide file tree

Showing 131 changed files with 3,754 additions and 1,576 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -19,9 +19,9 @@ jobs:
       # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
       - uses: actions/checkout@v2
 
-      - uses: actions/setup-node@v2
+      - uses: actions/setup-node@v4
         with:
-          node-version: "14.x"
+          node-version-file: ".nvmrc"
       - run: npm install -g yarn@^1.19.0 && yarn install
 
       - name: Check formatting with prettier

diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml
@@ -7,9 +7,9 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v2
-      - uses: actions/setup-node@v2
+      - uses: actions/setup-node@v4
         with:
-          node-version: "14.x"
+          node-version-file: ".nvmrc"
       - uses: google-github-actions/[email protected]
         with:
           service_account_key: ${{ secrets.GCP_CREDENTIALS }}

diff --git a/.github/workflows/npm-publish.yml b/.github/workflows/npm-publish.yml
@@ -33,10 +33,10 @@ jobs:
 
       - name: Set up Node.js for NPM
         if: steps.detect.outputs.previous-version != steps.detect.outputs.current-version
-        uses: actions/setup-node@v2
+        uses: actions/setup-node@v4
         with:
           registry-url: "https://registry.npmjs.org"
-          node-version: "14.x"
+          node-version-file: ".nvmrc"
 
       - name: Install yarn
         if: steps.detect.outputs.previous-version != steps.detect.outputs.current-version

diff --git a/.nvmrc b/.nvmrc
@@ -0,0 +1 @@
+v16.20.2
diff --git a/CHANGES.md b/CHANGES.md
@@ -1,6 +1,20 @@
 # Change Log
 
-#### next release (8.3.8)
+#### next release (8.3.10)
+
+- Added `apiColumns` to `ApiTableCatalogItem` - this can now be used to specify `responseDataPath` per table column.
+- [The next improvement]
+
+#### 8.3.9 - 2023-11-24
+
+- **Breaking change:** new Search Provider model
+  - Added SearchProviderMixin to connect searchProviders with a model system
+  - Created a simple base Mixin (`SearchProviderMixin`) to attach SearchProviders to the Model system and enable easier creation of new search providers.
+  - Made SearchProviders configurable from `config.json`.
+  - See [0011-configurable-search-providers ADR](./architecture/0011-configurable-search-providers.md) and [Search providers customization](./doc/customizing/search-providers.md) for more details
+- Make all icons in `CatalogGroup` black by default and white when a catalog group is focused, selected or hovered over. Improve lock icon position in workbench.
+
+#### 8.3.8 - 2023-11-15
 
 - Fix maximum call stack size exceeded on Math.min/max when creating Charts
 - Fix boolean flag in `MyDataTab` displaying number
@@ -17,6 +31,9 @@
 - Fix WMS `GetMap`/`GetFeatureInfo` requests not having `styles` parameter (will use empty string instead of `undefined`)
 - `ArcGisMapServerCatalogItem` will now use "pre-cached tiles" if available if no (or all) `layers` are specified.
 - [The next improvement]
+- Add CesiumIon geocoder
+- `CatalogGroup` will now not show members until loaded
+- Add `GetTimeseries` support to `WebMapServiceCatalogItem`. This adds a new `supportsGetTimeseries` trait, which when true will replace `GetFeatureInfo` with `GetTimeseries` requests. It will also change `info_format` to `text/csv`, and show a chart in the feature info panel. Servers which advertise `GetTimeseries` capability will have this trait set to true by default. `GetTimeseries` requests will have `time = ""`.
 
 #### 8.3.7 - 2023-10-26
 

diff --git a/architecture/0011-configurable-search-providers.md b/architecture/0011-configurable-search-providers.md
@@ -0,0 +1,36 @@
+# 11. Configuration of search providers
+
+Date: 2021-01-19
+
+## Status
+
+Proposed
+
+## Context
+
+Ticket.
+https://github.com/TerriaJS/terriajs/issues/5141.
+
+### Intro
+
+The existing approach to the definition of SearchProviders requires the development team's involvement and rebuild of the application, which can be undesired behavior in highly dynamic environments.
+It's much better to enable the administrators to maintain the search providers.
+
+## Proposal
+
+- SearchProviders could greatly use the benefits of the new model system used for Catalog.
+- Create a simple base Mixin (`SearchProviderMixin`) to attach SearchProviders to the Model system and enable easier creation of new search providers.
+- Make SearchProviders configurable from `config.json`.
+- Provide sensible defaults for everything.
+- Typescript everything.
+- Make everything translateable (administrator can specify i18next keys for all names)
+
+## Benefits
+
+- Much easier to implement new search providers.
+- Much easier to update existing search providers, `urls` and `keys`.
+- Offer administrators an option to decide wheter they want to load group members using `CatalogSearchProvider`.
+
+## Consequences
+
+This is quite a large change and should be thoroughly tested to avoid the possible bugs in the search providers migration.
diff --git a/buildprocess/ci-deploy.sh b/buildprocess/ci-deploy.sh
@@ -18,8 +18,8 @@ gh api /repos/${GITHUB_REPOSITORY}/statuses/${GITHUB_SHA} -f state=pending -f co
 
 # Install some tools we need from npm
 npm install -g https://github.com/terriajs/sync-dependencies
-npm install request@^2.83.0
 npm install -g yarn@^1.19.0
+yarn add -W [email protected]
 
 # Clone and build TerriaMap, using this version of TerriaJS
 TERRIAJS_COMMIT_HASH=$(git rev-parse HEAD)

diff --git a/buildprocess/ci-values.yml b/buildprocess/ci-values.yml
@@ -46,7 +46,6 @@ terriamap:
     initializationUrls:
       - simple
     parameters:
-      bingMapsKey: "ApZeR4iLSH_Pl2w5OSXrIqjvc_KzgPn_UXjn6jTtQDiueg72InBThCeIO4OSs6Fq"
       cesiumIonAccessToken: "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJqdGkiOiIxZDY5MDE1YS0yNGFlLTQ1MzctYWNkNy0wNDQ1YWNiNTM5MDIiLCJpZCI6Mjk5Miwic2NvcGVzIjpbImFzciIsImdjIl0sImlhdCI6MTUzODAzMzgyNn0.a2uTotdiHUo8FiHsO4MVNZ1KT5pOF0rb7CFdwbRAsOo"
       useCesiumIonBingImagery: true
       googleAnalyticsKey:
@@ -67,6 +66,12 @@ terriamap:
       mobileDefaultViewerMode: "2d"
       experimentalFeatures: true
       feedbackUrl: "feedback"
+      searchProviders:
+        - id: search-provider/cesium-ion
+          type: cesium-ion-search-provider
+          name: "translate#viewModels.searchLocations"
+          flightDurationSeconds: 1.5
+          minCharacters: 3
       languageConfiguration:
         enabled: true
         languages:

diff --git a/doc/customizing/client-side-config.md b/doc/customizing/client-side-config.md
@@ -67,7 +67,7 @@ Specifies various options for configuring TerriaJS:
 | `defaultMaximumShownFeatureInfos` | no       | **number**                                                                                               | `100`                                                              | The maximum number of "feature info" boxes that can be displayed when clicking a point.                                                                                                                                                                                                                |
 | `regionMappingDefinitionsUrl`     | no       | **string**                                                                                               |                                                                    | **Deprecated** please use `regionMappingDefinitionsUrls` array instead. If this is defined, it will override `regionMappingDefinitionsUrls`                                                                                                                                                            |
 | `regionMappingDefinitionsUrls`    | no       | **string[]**                                                                                             | `["build/TerriaJS/data/regionMapping.json"]`                       | URLs of JSON files that define region mapping for Tabular data (eg CSV). This option only needs to be changed in unusual deployments. It has to be changed if deploying as static site, for instance. It multiple URLs are provided then the first matching region will be used (in order of URLs)     |
-| `catalogIndexUrl`                 | no       | **string**                                                                                               |                                                                    | URL of the JSON file that contains index of catalog. See [CatalogIndex](#catalogindex)                                                                                                                                                                                                                 |
+| `catalogIndexUrl`                 | no       | **string**                                                                                               |                                                                    | URL of the JSON file that contains index of catalog. See [CatalogIndex](search-providers.md#catalogindex)                                                                                                                                                                                              |
 | `proj4ServiceBaseUrl`             | no       | **string**                                                                                               | `"proj4def/"`                                                      | URL of Proj4 projection lookup service (part of TerriaJS-Server). This option only needs to be changed in unusual deployments. It has to be changed if deploying as static site, for instance.                                                                                                         |
 | `corsProxyBaseUrl`                | no       | **string**                                                                                               | `"proxy/"`                                                         | URL of CORS proxy service (part of TerriaJS-Server). This option only needs to be changed in unusual deployments. It has to be changed if deploying as static site, for instance.                                                                                                                      |
 | `proxyableDomainsUrl`             | no       | **string**                                                                                               | `"proxyabledomains/"`                                              | Deprecated, will be determined from serverconfig.                                                                                                                                                                                                                                                      |
@@ -115,6 +115,8 @@ Specifies various options for configuring TerriaJS:
 | `storyVideo.videoUrl`             | no       | **string**                                                                                               | https://www.youtube-nocookie.com/embed/fbiQawV8IYY                 | Video to show in Story Editor panel under Getting Started.                                                                                                                                                                                                                                             |
 | `relatedMaps`                     | no       | **[RelatedMap](#relatedmap)[]**                                                                          | See [`lib/Models/RelatedMaps.ts`](../../lib/Models/RelatedMaps.ts) | Maps to show in "Related Maps" menu panel                                                                                                                                                                                                                                                              |
 | `aboutButtonHrefUrl`              | no       | **string**                                                                                               | `"about.html"`                                                     | About button URL. If set to `null`, then the About button will not be shown                                                                                                                                                                                                                            |
+| `searchBar`                       | no       | **[SearchBar](#searchbar)**                                                                              | `new SearchBar()`                                                  | Search bar configuration                                                                                                                                                                                                                                                                               |
+| `searchProviders`                 | no       | \*\*[SearchProviders](search-providers.md)                                                               | `[]`                                                               | Search providers that will be used for search                                                                                                                                                                                                                                                          |
 
 ### MagdaReferenceHeaders
 
@@ -249,30 +251,15 @@ Credits/Attribution shown at the bottom of the map. Supports internationalizatio
 
 ---
 
-### CatalogIndex
+### SearchBar
 
-If your TerriaMap has many (>50) dynamic groups (groups which need to be loaded - for example CKAN, WMS-group...) it may be worth generating a static catalog index JSON file. This file will contain ID, name and description fields of all catalog items, which can be used to search through the catalog very quickly without needing to load dynamic references/groups (for example `MagdaReference` -> `WebMapServiceCatalogGroup` -> `WebMapServiceCatalogItem`).
+Configuration for the search bar. Some of the values will be used as default for
+search provider values.
 
-The https://github.com/nextapps-de/flexsearch library is used to index and search the catalog index file.
-
-To generate the catalog index:
-
--   `yarn build-tools`
--   `node .\build\generateCatalogIndex.js config-url base-url` where
-
-    -   `config-url` is URL to client-side-config file
-    -   `base-url` is URL to terriajs-server (this is used to load `server-config` and to proxy requests)
-    -   For example `node .\build\generateCatalogIndex.js http://localhost:3001/config.json http://localhost:3001`
-
--   This will output three files
-    -   `catalog-index.json`
-    -   `catalog-index-errors.json` with any error messages which occurred while loading catalog members
-    -   `catalog-index-errors-stack.json` with errors stack
--   Set `catalogIndexUrl` config parameter to URL to `catalog-index.json`
-
-This file will have to be re-generated manually every time the catalog structure changes - for example:
-
--   if items are renamed, or moved
--   dynamic groups are updated (for example, WMS server publishes new layers)
-
-For more details see [/buildprocess/generateCatalogIndex.ts](/buildprocess/generateCatalogIndex.ts)
+| Name                  | Required | Type          | Default                        | Description                                                                                       |
+| --------------------- | -------- | ------------- | ------------------------------ | ------------------------------------------------------------------------------------------------- |
+| placeholder           | no       | **string**    | `translate#search.placeholder` | Input text field placeholder shown when no input has been given yet. The string is translateable. |
+| recommendedListLength | no       | **number**    | `5`                            | Maximum amount of entries in the suggestion list.                                                 |
+| flightDurationSeconds | no       | **number**    | `1.5`                          | The duration of the camera flight to an entered location, in seconds.                             |
+| minCharacters         | no       | **number**    | 3                              | Minimum number of characters required for search to start                                         |
+| boundingBoxLimit      | no       | **Rectangle** | `Cesium.Rectangle.MAX_VALUE`   | Bounding box limits for the search results {west, south, east, north}                             |
diff --git a/doc/customizing/cloning-and-building.md b/doc/customizing/cloning-and-building.md
@@ -21,8 +21,8 @@ If you run into trouble or want more explanation, read on.
 TerriaJS can be built and run on almost any macOS, Linux, or Windows system. The following are required to build TerriaJS:
 
 -   The Bash command shell. On macOS or Linux you almost certainly already have this. On Windows, you can easily get it by installing [Git for Windows](https://gitforwindows.org/). In the instructions below, we assume you're using a Bash command prompt.
--   [Node.js](https://nodejs.org) v14.0 or later. You can check your node version by running `node --version` on the command-line.
--   [npm](https://www.npmjs.com/) v6.0 or later. npm is usually installed automatically alongside the above. You can check your npm version by running `npm --version`.
+-   [Node.js](https://nodejs.org) v16.0. You can check your node version by running `node --version` on the command-line.
+-   [npm](https://www.npmjs.com/) v8.0. npm is usually installed automatically alongside the above. You can check your npm version by running `npm --version`.
 -   [yarn](https://yarnpkg.com/) v1.19.0 or later. This can be installed using `npm install -g yarn@^1.19.0`
 
 ### Cloning TerriaMap