Merge branch 'master' of https://github.com/preset-io/superset into f…

…ix-permlink-save

CHANGELOG.md

@@ -19,6 +19,7 @@ under the License.
 
 ## Change Log
 
+- [2.1.1](#211-sun-apr-23-154421-2023-0100)
 - [2.1.0](#210-thu-mar-16-211305-2023--0700)
 - [2.0.1](#201-fri-nov-4-103402-2022--0400)
 - [2.0.0](#200-tue-jun-28-085302-2022--0400)
@@ -29,6 +30,51 @@ under the License.
 - [1.4.2](#142-sat-mar-19-000806-2022-0200)
 - [1.4.1](#141)
 
+
+### 2.1.1 (Sun Apr 23 15:44:21 2023 +0100)
+
+**Database Migrations**
+- [#23980](https://github.com/apache/superset/pull/23980) fix(migration): handle permalink edge cases correctly (@villebro)
+- [#23888](https://github.com/apache/superset/pull/23888) chore(key-value): use json serialization for main resources (@villebro)
+
+**Fixes**
+- [#23723](https://github.com/apache/superset/pull/23723) fix: add enforce URI query params with a specific for MySQL (@dpgaspar)
+- [#24970](https://github.com/apache/superset/pull/24970) fix: update permalink schema (@eschutho)
+- [#24995](https://github.com/apache/superset/pull/24995) fix: Revert "fix(chart): Time Series set showMaxLabel as null for time xAxis (#20627) (@eschutho)
+- [#24513](https://github.com/apache/superset/pull/24513) fix(sqllab): normalize changedOn timestamp (@villebro)
+- [#23512](https://github.com/apache/superset/pull/23512) fix: Dashboard not loading with default first value in filter (@geido)
+- [#24482](https://github.com/apache/superset/pull/24482) fix(permalink): Incorrect component schema reference (@Nisden)
+- [#24166](https://github.com/apache/superset/pull/24166) fix(permalink): migrate to marshmallow codec (@villebro)
+- [#24697](https://github.com/apache/superset/pull/24697) fix: import database engine validation (@dpgaspar)
+- [#24390](https://github.com/apache/superset/pull/24390) fix: FAB CSS on Superset (@dpgaspar)
+- [#24249](https://github.com/apache/superset/pull/24249) fix: dashboard ownership check (@betodealmeida)
+- [#23801](https://github.com/apache/superset/pull/23801) fix(plugin-chart-handlebars): Fix TypeError when using handlebars columns raw mode (@fmannhardt)
+- [#23566](https://github.com/apache/superset/pull/23566) fix: Filter values are not updating when dependencies are set (@michael-s-molina)
+- [#23400](https://github.com/apache/superset/pull/23400) fix: Select all issue with "Dynamically search all filter values" in FilterBar (@geido)
+- [#23865](https://github.com/apache/superset/pull/23865) fix: Native time range filter in legacy charts (@kgabryje)
+- [#24054](https://github.com/apache/superset/pull/24054) fix: handle temporal columns in presto partitions (@giftig)
+- [#23882](https://github.com/apache/superset/pull/23882) fix: handle comments in `has_table_query` (@betodealmeida)
+- [#24137](https://github.com/apache/superset/pull/24137) fix: disable SHOW_STACKTRACE by default (@dpgaspar)
+- [#24185](https://github.com/apache/superset/pull/24185) fix: db validate parameters permission (@dpgaspar)
+- [#23769](https://github.com/apache/superset/pull/23769) fix: allow db driver distinction on enforced URI params (@dpgaspar)
+- [#23600](https://github.com/apache/superset/pull/23600) fix: load examples as anon user (@betodealmeida)
+- [#23200](https://github.com/apache/superset/pull/23200) fix: permission checks on import (@betodealmeida)
+- [#23901](https://github.com/apache/superset/pull/23901) fix: check sqlalchemy_uri (@dpgaspar)
+- [#23751](https://github.com/apache/superset/pull/23751) fix(mssql): apply top after distinct (@villebro)
+- [#23586](https://github.com/apache/superset/pull/23586) fix(dashboard-rbac): use normal rbac when no roles chosen (@villebro)
+- [#23582](https://github.com/apache/superset/pull/23582) fix(dash import): Ensure old datasource ids are not referenced in imported charts (@jfrag1)
+- [#23506](https://github.com/apache/superset/pull/23506) fix(generic-x-axis): skip initial time filter for legacy charts (@villebro)
+- [#23507](https://github.com/apache/superset/pull/23507) fix(legacy-plugin-chart-heatmap): fix adhoc column tooltip (@villebro)
+- [#23441](https://github.com/apache/superset/pull/23441) fix(chart): non existent time grain no longer breaks the application (@rdubois)
+- [#23393](https://github.com/apache/superset/pull/23393) fix(Pivot Table v2): resolved full width issue (@AkashBoora)
+- [#22851](https://github.com/apache/superset/pull/22851) fix: Validate jinja rendered query (@geido)
+
+**Others**
+- [#24586](https://github.com/apache/superset/pull/24586) chore(metastore-cache): add codec support (@villebro)
+- [#23113](https://github.com/apache/superset/pull/23113) chore(sqla): Address performance tradeoff with eager loading (@john-bodley)
+- [#24294](https://github.com/apache/superset/pull/24294) chore: update UPDATING for 2.1.0 (@eschutho)
+- [#24056](https://github.com/apache/superset/pull/24056) chore: Remove unnecessary information from response (@geido)
+
 ### 2.1.0 (Thu Mar 16 21:13:05 2023 -0700)
 
 **Database Migrations**

CONTRIBUTING.md

@@ -578,6 +578,7 @@ npm ci
 ```
 
 Note that Superset uses [Scarf](https://docs.scarf.sh) to capture telemetry/analytics about versions being installed, including the `scarf-js` npm package. As noted elsewhere in this documentation, Scarf gathers aggregated stats for the sake of security/release strategy, and does not capture/retain PII. [You can read here](https://docs.scarf.sh/package-analytics/) about the package, and various means to opt out of it, but one easy way to opt out is to add this setting in `superset-frontent/package.json`:
+
 ```json
 // your-package/package.json
 {
@@ -598,10 +599,13 @@ There are three types of assets you can build:
 3. `npm run build-instrumented`: instrumented application code for collecting code coverage from Cypress tests
 
 If this type of error comes while building assets(i.e using above commands):
+
 ```bash
 Error: You must provide the URL of lib/mappings.wasm by calling SourceMapConsumer.initialize
 ```
+
 Then put this:
+
 ```bash
 export NODE_OPTIONS=--no-experimental-fetch
 ```
@@ -925,28 +929,22 @@ For debugging locally using VSCode, you can configure a launch configuration fil
 
 ```json
 {
-    "version": "0.2.0",
-    "configurations": [
-        {
-            "name": "Python: Flask",
-            "type": "python",
-            "request": "launch",
-            "module": "flask",
-            "env": {
-                "FLASK_APP": "superset",
-                "SUPERSET_ENV": "development"
-            },
-            "args": [
-                "run",
-                "-p 8088",
-                "--with-threads",
-                "--reload",
-                "--debugger"
-            ],
-            "jinja": true,
-            "justMyCode": true
-        }
-    ]
+  "version": "0.2.0",
+  "configurations": [
+    {
+      "name": "Python: Flask",
+      "type": "python",
+      "request": "launch",
+      "module": "flask",
+      "env": {
+        "FLASK_APP": "superset",
+        "SUPERSET_ENV": "development"
+      },
+      "args": ["run", "-p 8088", "--with-threads", "--reload", "--debugger"],
+      "jinja": true,
+      "justMyCode": true
+    }
+  ]
 }
 ```
 
@@ -1031,24 +1029,24 @@ You are now ready to attach a debugger to the process. Using VSCode you can conf
 
 ```json
 {
-    "version": "0.2.0",
-    "configurations": [
+  "version": "0.2.0",
+  "configurations": [
+    {
+      "name": "Attach to Superset App in Docker Container",
+      "type": "python",
+      "request": "attach",
+      "connect": {
+        "host": "127.0.0.1",
+        "port": 5678
+      },
+      "pathMappings": [
         {
-            "name": "Attach to Superset App in Docker Container",
-            "type": "python",
-            "request": "attach",
-            "connect": {
-                "host": "127.0.0.1",
-                "port": 5678
-            },
-            "pathMappings": [
-                {
-                    "localRoot": "${workspaceFolder}",
-                    "remoteRoot": "/app"
-                }
-            ]
-        },
-    ]
+          "localRoot": "${workspaceFolder}",
+          "remoteRoot": "/app"
+        }
+      ]
+    }
+  ]
 }
 ```
 
@@ -1349,7 +1347,7 @@ To do this, you'll need to:
   but perfect for testing (stores cache in `/tmp`)
 
   ```python
-  from cachelib.file import FileSystemCache
+  from flask_caching.backends.filesystemcache import FileSystemCache
   RESULTS_BACKEND = FileSystemCache('/tmp/sqllab')
   ```
 
@@ -1415,11 +1413,11 @@ Note not all fields are correctly categorized. The fields vary based on visualiz
 
 ### Time
 
-| Field              | Type     | Notes                                 |
-| ------------------ | -------- | ------------------------------------- |
-| `granularity_sqla` | _string_ | The SQLA **Time Column** widget       |
-| `time_grain_sqla`  | _string_ | The SQLA **Time Grain** widget        |
-| `time_range`       | _string_ | The **Time range** widget             |
+| Field              | Type     | Notes                           |
+| ------------------ | -------- | ------------------------------- |
+| `granularity_sqla` | _string_ | The SQLA **Time Column** widget |
+| `time_grain_sqla`  | _string_ | The SQLA **Time Grain** widget  |
+| `time_range`       | _string_ | The **Time range** widget       |
 
 ### GROUP BY
 

UPDATING.md

@@ -24,14 +24,14 @@ assists people when migrating to a new version.
 
 ## Next
 
+- [25053](https://github.com/apache/superset/pull/25053): Extends the `ab_user.email` column from 64 to 320 characters which has an associated unique key constraint. This will be problematic for MySQL metadata databases which use the InnoDB storage engine with the `innodb_large_prefix` parameter disabled as the key prefix limit is 767 bytes. Enabling said parameter and ensuring that the table uses either the `DYNAMIC` or `COMPRESSED` row format should remedy the problem. See [here](https://dev.mysql.com/doc/refman/5.7/en/innodb-limits.html) for more details.
 - [24939](https://github.com/apache/superset/pull/24939): Augments the foreign key constraints for the `embedded_dashboards` table to include an explicit CASCADE ON DELETE to ensure the relevant records are deleted when a dashboard is deleted. Scheduled downtime may be advised.
 - [24938](https://github.com/apache/superset/pull/24938): Augments the foreign key constraints for the `dashboard_slices` table to include an explicit CASCADE ON DELETE to ensure the relevant records are deleted when a dashboard or slice is deleted. Scheduled downtime may be advised.
 - [24911](https://github.com/apache/superset/pull/24911): Changes the column type from `TEXT` to `MediumText` in table `logs`, potentially requiring a table lock on MySQL dbs or taking some time to complete on large deployments.
 - [24657](https://github.com/apache/superset/pull/24657): Bumps the cryptography package to augment the OpenSSL security vulnerability.
 - [24628](https://github.com/apache/superset/pull/24628): Augments the foreign key constraints for the `dashboard_owner`, `report_schedule_owner`, and `slice_owner` tables to include an explicit CASCADE ON DELETE to ensure the relevant ownership records are deleted when a dataset is deleted. Scheduled downtime may be advised.
 - [24488](https://github.com/apache/superset/pull/24488): Augments the foreign key constraints for the `sql_metrics`, `sqlatable_user`, and `table_columns` tables to include an explicit CASCADE ON DELETE to ensure the relevant records are deleted when a dataset is deleted. Scheduled downtime may be advised.
 - [24335](https://github.com/apache/superset/pull/24335): Removed deprecated API `/superset/filter/<datasource_type>/<int:datasource_id>/<column>/`
-- [24185](https://github.com/apache/superset/pull/24185): `/api/v1/database/test_connection` and `api/v1/database/validate_parameters` permissions changed from `can_read` to `can_write`. Only Admin user's have access.
 - [24232](https://github.com/apache/superset/pull/24232): Enables ENABLE_TEMPLATE_REMOVE_FILTERS, DRILL_TO_DETAIL, DASHBOARD_CROSS_FILTERS by default, marks VERSIONED_EXPORT and ENABLE_TEMPLATE_REMOVE_FILTERS as deprecated.
 - [23652](https://github.com/apache/superset/pull/23652): Enables GENERIC_CHART_AXES feature flag by default.
 - [23226](https://github.com/apache/superset/pull/23226): Migrated endpoint `/estimate_query_cost/<int:database_id>` to `/api/v1/sqllab/estimate/`. Corresponding permissions are can estimate query cost on SQLLab. Make sure you add/replace the necessary permissions on any custom roles you may have.
@@ -42,6 +42,7 @@ assists people when migrating to a new version.
   make it more clear which envrionment your are in.
   `SUPERSET_ENV=production` and `SUPERSET_ENV=development` are the two
   supported switches based on the default config.
+- [19242](https://github.com/apache/superset/pull/19242): Adhoc subqueries are now disabled by default for security reasons. To enable them, set the feature flag `ALLOW_ADHOC_SUBQUERY` to `True`.
 
 ### Breaking Changes
 
@@ -82,6 +83,12 @@ assists people when migrating to a new version.
 
 - [24982](https://github.com/apache/superset/pull/24982): By default, physical datasets on Oracle-like dialects like Snowflake will now use denormalized column names. However, existing datasets won't be affected. To change this behavior, the "Advanced" section on the dataset modal has a "Normalize column names" flag which can be changed to change this behavior.
 
+## 2.1.1
+- [24185](https://github.com/apache/superset/pull/24185): `/api/v1/database/test_connection` and `api/v1/database/validate_parameters` permissions changed from `can_read` to `can_write`. Only Admin user's have access.
+
+### Other
+- [23888](https://github.com/apache/superset/pull/23888): Database Migration for json serialization instead of pickle should upgrade/downgrade correctly when bumping to/from this patch version
+
 ## 2.1.0
 
 - [22809](https://github.com/apache/superset/pull/22809): Migrated endpoint `/superset/sql_json` and `/superset/results/` to `/api/v1/sqllab/execute/` and `/api/v1/sqllab/results/` respectively. Corresponding permissions are `can sql_json on Superset` to `can execute on SQLLab`, `can results on Superset` to `can results on SQLLab`. Make sure you add/replace the necessary permissions on any custom roles you may have.

docker/pythonpath_dev/superset_config.py

@@ -23,8 +23,8 @@
 import logging
 import os
 
-from cachelib.file import FileSystemCache
 from celery.schedules import crontab
+from flask_caching.backends.filesystemcache import FileSystemCache
 
 logger = logging.getLogger()
 

docs/docs/databases/db-connection-ui.mdx

@@ -28,7 +28,7 @@ We added a new configuration option where the admin can define their preferred d
 # displayed prominently in the "Add Database" dialog. You should
 # use the "engine_name" attribute of the corresponding DB engine spec
 # in `superset/db_engine_specs/`.
-PREFERRED_DATABASES: List[str] = [
+PREFERRED_DATABASES: list[str] = [
     "PostgreSQL",
     "Presto",
     "MySQL",

docs/docs/frequently-asked-questions.mdx

@@ -6,35 +6,52 @@ sidebar_position: 7
 
 ## Frequently Asked Questions
 
+
+### How big of a dataset can Superset handle?
+
+Superset can work with even gigantic databases! Superset acts as a thin layer above your underlying
+databases or data engines, which do all the processing.  Superset simply visualizes the results of
+the query.
+
+The key to achieving acceptable performance in Superset is whether your database can execute queries
+and return results at a speed that is acceptable to your users. If you experience slow performance with
+Superset, benchmark and tune your data warehouse.
+
+### What are the computing specifications required to run Superset?
+
+The specs of your Superset installation depend on how many users you have and what their activity is, not
+on the size of your data.  Superset admins in the community have reported 8GB RAM, 2vCPUs as adequate to
+run a moderately-sized instance. To develop Superset, e.g., compile code or build images, you may
+need more power.
+
+Monitor your resource usage and increase or decrease as needed. Note that Superset usage has a tendency
+to occur in spikes, e.g., if everyone in a meeting loads the same dashboard at once.
+
+Superset's application metadata does not require a very large database to store it, though
+the log file grows over time.
+
+
 ### Can I join / query multiple tables at one time?
 
 Not in the Explore or Visualization UI. A Superset SQLAlchemy datasource can only be a single table
 or a view.
 
-When working with tables, the solution would be to materialize a table that contains all the fields
+When working with tables, the solution would be to create a table that contains all the fields
 needed for your analysis, most likely through some scheduled batch process.
 
-A view is a simple logical layer that abstract an arbitrary SQL queries as a virtual table. This can
-allow you to join and union multiple tables, and to apply some transformation using arbitrary SQL
-expressions. The limitation there is your database performance as Superset effectively will run a
+A view is a simple logical layer that abstracts an arbitrary SQL queries as a virtual table. This can
+allow you to join and union multiple tables and to apply some transformation using arbitrary SQL
+expressions. The limitation there is your database performance, as Superset effectively will run a
 query on top of your query (view). A good practice may be to limit yourself to joining your main
 large table to one or many small tables only, and avoid using _GROUP BY_ where possible as Superset
 will do its own _GROUP BY_ and doing the work twice might slow down performance.
 
-Whether you use a table or a view, the important factor is whether your database is fast enough to
-serve it in an interactive fashion to provide a good user experience in Superset.
+Whether you use a table or a view, performance depends on how fast your database can deliver
+the result to users interacting with Superset.
 
 However, if you are using SQL Lab, there is no such limitation. You can write SQL queries to join
 multiple tables as long as your database account has access to the tables.
 
-### How BIG can my datasource be?
-
-It can be gigantic! Superset acts as a thin layer above your underlying databases or data engines.
-
-As mentioned above, the main criteria is whether your database can execute queries and return
-results in a time frame that is acceptable to your users. Many distributed databases out there can
-execute queries that scan through terabytes in an interactive fashion.
-
 ### How do I create my own visualization?
 
 We recommend reading the instructions in
@@ -192,8 +209,9 @@ only a few database engines are supported for use as the OLTP backend / metadata
 
 Superset is tested using MySQL, PostgreSQL, and SQLite backends. It’s recommended you install
 Superset on one of these database servers for production.  Installation on other OLTP databases
-may work but isn’t tested. Column-store, non-OLTP databases are not designed for this type of workload.
-
+may work but isn’t tested.  It has been reported that [Microsoft SQL Server does *not*
+work as a Superset backend](https://github.com/apache/superset/issues/18961). Column-store,
+non-OLTP databases are not designed for this type of workload.
 
 ### How can I configure OAuth authentication and authorization?
 

docs/docs/installation/async-queries-celery.mdx

@@ -66,7 +66,7 @@ celery --app=superset.tasks.celery_app:app beat
 ```
 
 To setup a result backend, you need to pass an instance of a derivative of from
-cachelib.base.BaseCache to the RESULTS_BACKEND configuration key in your superset_config.py. You can
+from flask_caching.backends.base import BaseCache to the RESULTS_BACKEND configuration key in your superset_config.py. You can
 use Memcached, Redis, S3 (https://pypi.python.org/pypi/s3werkzeugcache), memory or the file system
 (in a single server-type setup or for testing), or to write your own caching interface. Your
 `superset_config.py` may look something like:
@@ -79,7 +79,7 @@ S3_CACHE_KEY_PREFIX = 'sql_lab_result'
 RESULTS_BACKEND = S3Cache(S3_CACHE_BUCKET, S3_CACHE_KEY_PREFIX)
 
 # On Redis
-from cachelib.redis import RedisCache
+from flask_caching.backends.rediscache import RedisCache
 RESULTS_BACKEND = RedisCache(
     host='localhost', port=6379, key_prefix='superset_results')
 ```