From 6f2eea9156484f7874b365e8176a8ae90bdd1f73 Mon Sep 17 00:00:00 2001 From: Oscar Kraemer Date: Thu, 24 Oct 2024 13:56:31 +0300 Subject: [PATCH 01/16] Update postgresql versions --- docs/cloud/dbaas/advanced.md | 6 +++--- docs/cloud/dbaas/cli.md | 6 +++--- docs/cloud/dbaas/web-interface.md | 4 +++- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/docs/cloud/dbaas/advanced.md b/docs/cloud/dbaas/advanced.md index a37e87a29f..38c0a0c6e0 100644 --- a/docs/cloud/dbaas/advanced.md +++ b/docs/cloud/dbaas/advanced.md @@ -14,11 +14,11 @@ engine. The configuration groups can be modified from the web-GUI as well as the ### Example how to create a configuration group with the CLI 1. Figure out which datastore, datastore-version and what values you want to create a configuration -group for. In this example we will use datastore `postgresql` and datastore-version `14.4`. +group for. In this example we will use datastore `postgresql` and datastore-version `17.0`. 2. Figure out which parameters can be modified: ```bash - openstack database configuration parameter list --datastore postgresql 14.4 + openstack database configuration parameter list --datastore postgresql 17.0 ``` Note that some parameters require restarting the database instance. @@ -28,7 +28,7 @@ group for. In this example we will use datastore `postgresql` and datastore-vers ``` openstack database configuration create group-name --datastore postgresql \ - --datastore-version 14.4 '{"max_connections": 234 }' + --datastore-version 17.0 '{"max_connections": 234 }' ``` 4. You can see the configuration group with: diff --git a/docs/cloud/dbaas/cli.md b/docs/cloud/dbaas/cli.md index 5b48eb90cd..29d2dfc9d7 100644 --- a/docs/cloud/dbaas/cli.md +++ b/docs/cloud/dbaas/cli.md @@ -49,8 +49,8 @@ Remember that you can use the help command as `openstack help database` and the ```sh openstack datastore list ``` - - * `datastore version`. This depends on the datastore you have chosen and you should usually choose the latest. If you use PostgreSQL you can probably use `14.5`. You can find out the available datastore versions with: + + * `datastore version`. This depends on the datastore you have chosen and you should usually choose the latest. If you use PostgreSQL you can probably use `17.0`. You can find out the available datastore versions with: ```sh openstack datastore version list postgresql @@ -68,7 +68,7 @@ Remember that you can use the help command as `openstack help database` and the --databases my_first_database \ --users databaseuser:myPassword568 \ --datastore postgresql \ - --datastore-version 14.5 \ + --datastore-version 17.0 \ --is-public \ --size 1 \ --allowed-cidr ${MY_IP}/32 diff --git a/docs/cloud/dbaas/web-interface.md b/docs/cloud/dbaas/web-interface.md index 25096d5011..d3c680d7a1 100644 --- a/docs/cloud/dbaas/web-interface.md +++ b/docs/cloud/dbaas/web-interface.md @@ -10,7 +10,8 @@ Now you can define what settings you want for your database: 1. `Instance name` - What ever you want to name the instance. 2. `Volume size` - How much disk space you will need for your database. If you just want to test the DBaaS then 1 GiB is probably enough. To increase the disk size later, downtime is required for the database. If you already know how much data you will use then it is easy to estimate how large volume you need. 3. `Volume type` - This can be left empty. -4. `Datastore` - What type of database you want. It is recommend to use the latest version of your preferred database, if you don't have a specific reason for using an older version. PostgreSQL is the only available database at the moment. +4. `Datastore` - What type of database you want. It is recommend to use the latest version of your preferred database, if you don't have a specific reason for using an older version. +Currently Pukki supports [PostgreSQL](postgresql.md) and [MariaDB](mariadb.md). 5. `Flavor` - How large database instances you want. For small use cases the `standard.small` is probably enough. If you later find out that it is not large enough you can always change it later. Changing flavor will require downtime. 6. `Locality` - Is not needed. In the future DBaaS will support clustered databases and at that point anti-affinity should be the preferred option in most cases. @@ -33,6 +34,7 @@ On the fourth page `Advanced`: 15. Now you can go to the database specific documentation to find out further instructions on how to use the database: * [PostgreSQL](postgresql.md) + * [MariaDB](mariadb.md) ## Modify user accounts in the database instance From 079553d4264c3e125456cad43ad9883ec9f2c2cd Mon Sep 17 00:00:00 2001 From: Oscar Kraemer Date: Thu, 24 Oct 2024 13:57:06 +0300 Subject: [PATCH 02/16] Change space to tab --- docs/cloud/dbaas/web-interface.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/cloud/dbaas/web-interface.md b/docs/cloud/dbaas/web-interface.md index d3c680d7a1..4f3c15dce1 100644 --- a/docs/cloud/dbaas/web-interface.md +++ b/docs/cloud/dbaas/web-interface.md @@ -34,7 +34,7 @@ On the fourth page `Advanced`: 15. Now you can go to the database specific documentation to find out further instructions on how to use the database: * [PostgreSQL](postgresql.md) - * [MariaDB](mariadb.md) + * [MariaDB](mariadb.md) ## Modify user accounts in the database instance From 47fe6494cb482e88210ef1785c5a17350fc36e6e Mon Sep 17 00:00:00 2001 From: Oscar Kraemer Date: Thu, 24 Oct 2024 14:11:51 +0300 Subject: [PATCH 03/16] Add some skeleton for MariaDB documentation --- docs/cloud/dbaas/databases.md | 1 + docs/cloud/dbaas/mariadb-accessing.md | 19 +++++++++++++++++++ docs/cloud/dbaas/mariadb-permissions.md | 11 +++++++++++ docs/cloud/dbaas/mariadb.md | 14 ++++++++++++++ 4 files changed, 45 insertions(+) create mode 100644 docs/cloud/dbaas/mariadb-accessing.md create mode 100644 docs/cloud/dbaas/mariadb-permissions.md create mode 100644 docs/cloud/dbaas/mariadb.md diff --git a/docs/cloud/dbaas/databases.md b/docs/cloud/dbaas/databases.md index d8c81a6651..181a7217d9 100644 --- a/docs/cloud/dbaas/databases.md +++ b/docs/cloud/dbaas/databases.md @@ -4,3 +4,4 @@ Currently Pukki only supports PostgreSQL. If you think that Pukki should support database type please [contact us](../../support/contact.md) with your suggestions. * [PostgreSQL](postgresql.md) +* [MariaDB](mariadb.md) diff --git a/docs/cloud/dbaas/mariadb-accessing.md b/docs/cloud/dbaas/mariadb-accessing.md new file mode 100644 index 0000000000..02d5f12a82 --- /dev/null +++ b/docs/cloud/dbaas/mariadb-accessing.md @@ -0,0 +1,19 @@ +# Accessing your MariaDB instance +## Graphical user interface +TODO add links to some graphical users interfaces +## Command-line +TODO Some example how to use command line tool, mention --ssl +### Common issues with CLI connections +### Accessing your Pukki MariaDB database from Puhti +### Basic Puhti batch job example using mysql +## Some useful SQL commands + List databases + List tables + Show table descriptions + Change database + Example query + Show all database settings + Show all users + Extended display + Import database dump + diff --git a/docs/cloud/dbaas/mariadb-permissions.md b/docs/cloud/dbaas/mariadb-permissions.md new file mode 100644 index 0000000000..ae98b425e9 --- /dev/null +++ b/docs/cloud/dbaas/mariadb-permissions.md @@ -0,0 +1,11 @@ +# MariaDB permissions and privilages +TOOD +## About privilages +TOOD +## Example of giving a user read-only access to a table +TOOD +## Example of giving a user read-write access to a table +TOOD +## When to use root user +TOOD + diff --git a/docs/cloud/dbaas/mariadb.md b/docs/cloud/dbaas/mariadb.md new file mode 100644 index 0000000000..c4261d574f --- /dev/null +++ b/docs/cloud/dbaas/mariadb.md @@ -0,0 +1,14 @@ +# MariaDB + + +* [How to access your MariaDB database](mariadb-accessing.md) +* [How to create database users and modify user permissions](mariadb-permissions.md) + +TODO I would like out documentation to manetion how the backup works in the background e.g. + * pg_basebackup is compressed encrypted and streamed to Allas + * mariadbbackup is compressed encyrpted and streamed to Allas + +TODO We should add some links to mariadb documentation from here maybe these: + * https://mariadb.com/kb/en/mariadb-client/ + * https://mariadb.com/kb/en/sql-statements-structure/ +TODO postgreql.md would also benefit of similar links From 2f3d78d50825b6730e62d5479cfcd62d96e2bf22 Mon Sep 17 00:00:00 2001 From: Oscar Kraemer Date: Wed, 30 Oct 2024 16:46:36 +0200 Subject: [PATCH 04/16] Added reminder to write something about Aria --- docs/cloud/dbaas/mariadb.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/cloud/dbaas/mariadb.md b/docs/cloud/dbaas/mariadb.md index c4261d574f..c6b687f9db 100644 --- a/docs/cloud/dbaas/mariadb.md +++ b/docs/cloud/dbaas/mariadb.md @@ -4,6 +4,12 @@ * [How to access your MariaDB database](mariadb-accessing.md) * [How to create database users and modify user permissions](mariadb-permissions.md) +# Database engine and backups +TOOD mention that Innodb is the default engine and that engine is the most test in Pukki and using +engine like Aria might cause usages issues while doing backups so one should think twice before +using anohter engine than InnoDB. +better infor: https://mariadb.com/kb/en/aria-storage-engine/ + TODO I would like out documentation to manetion how the backup works in the background e.g. * pg_basebackup is compressed encrypted and streamed to Allas * mariadbbackup is compressed encyrpted and streamed to Allas From 35a9a9cdf81be1c4c819d72c3c10b173091718ee Mon Sep 17 00:00:00 2001 From: Oscar Kraemer Date: Thu, 21 Nov 2024 11:27:58 +0200 Subject: [PATCH 05/16] Add warnings --- docs/cloud/dbaas/mariadb-accessing.md | 5 +++++ docs/cloud/dbaas/mariadb-permissions.md | 6 ++++++ docs/cloud/dbaas/mariadb.md | 5 +++++ mkdocs.yml | 5 +++++ 4 files changed, 21 insertions(+) diff --git a/docs/cloud/dbaas/mariadb-accessing.md b/docs/cloud/dbaas/mariadb-accessing.md index 02d5f12a82..938f7450f7 100644 --- a/docs/cloud/dbaas/mariadb-accessing.md +++ b/docs/cloud/dbaas/mariadb-accessing.md @@ -1,4 +1,9 @@ # Accessing your MariaDB instance +!!! warning "MariaDB in Pukki is still in beta" + This means that we have not tested as extensively that everything works correctly and there + might still be big changes how Pukki will manage MariaDB databases. We are hoping to be able + to get out of beta in March 2025. + ## Graphical user interface TODO add links to some graphical users interfaces ## Command-line diff --git a/docs/cloud/dbaas/mariadb-permissions.md b/docs/cloud/dbaas/mariadb-permissions.md index ae98b425e9..cca218c673 100644 --- a/docs/cloud/dbaas/mariadb-permissions.md +++ b/docs/cloud/dbaas/mariadb-permissions.md @@ -1,4 +1,10 @@ # MariaDB permissions and privilages +!!! warning "MariaDB in Pukki is still in beta" + This means that we have not tested as extensively that everything works correctly and there + might still be big changes how Pukki will manage MariaDB databases. We are hoping to be able + to get out of beta in March 2025. + + TOOD ## About privilages TOOD diff --git a/docs/cloud/dbaas/mariadb.md b/docs/cloud/dbaas/mariadb.md index c6b687f9db..c2c7e865f4 100644 --- a/docs/cloud/dbaas/mariadb.md +++ b/docs/cloud/dbaas/mariadb.md @@ -1,4 +1,9 @@ # MariaDB +!!! warning "MariaDB in Pukki is still in beta" + This means that we have not tested as extensively that everything works correctly and there + might still be big changes how Pukki will manage MariaDB databases. We are hoping to be able + to get out of beta in March 2025. + * [How to access your MariaDB database](mariadb-accessing.md) diff --git a/mkdocs.yml b/mkdocs.yml index a1a7c1173e..bad90a3e9d 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -304,6 +304,11 @@ nav: - PostgreSQL version differences: cloud/dbaas/postgres-versions.md - Extensions and parameters: cloud/dbaas/postgres-extensions.md - Permissions: cloud/dbaas/postgres-permissions.md + - MariaDB: + - cloud/dbaas/mariadb.md + - Accessing your database: cloud/dbaas/mariadb-accessing.md + - Permissions: cloud/dbaas/mariadb-permissions.md + - More advanced features: cloud/dbaas/advanced.md - Resizing database instance volume: cloud/dbaas/resize-volume.md - Rahti 2: From 0da84d3f5f8b01f624dba5be824769e57c95ddde Mon Sep 17 00:00:00 2001 From: Oscar Kraemer Date: Fri, 22 Nov 2024 11:39:05 +0200 Subject: [PATCH 06/16] Small additions --- docs/cloud/dbaas/backups.md | 1 + docs/cloud/dbaas/mariadb-accessing.md | 12 ++++++++++-- docs/cloud/dbaas/mariadb.md | 17 ++++++----------- 3 files changed, 17 insertions(+), 13 deletions(-) diff --git a/docs/cloud/dbaas/backups.md b/docs/cloud/dbaas/backups.md index 4c489fa55c..605fbe6928 100644 --- a/docs/cloud/dbaas/backups.md +++ b/docs/cloud/dbaas/backups.md @@ -6,6 +6,7 @@ after the last backup. All backups are stored for 90 days after which they get a deleted, backups can not be manually deleted by the user. The backups are stored encrypted in Allas. + The web interface is currently only showing the latest 20 backups. If you want to see all your backups you need to use the CLI-tool. diff --git a/docs/cloud/dbaas/mariadb-accessing.md b/docs/cloud/dbaas/mariadb-accessing.md index 938f7450f7..3dc995080f 100644 --- a/docs/cloud/dbaas/mariadb-accessing.md +++ b/docs/cloud/dbaas/mariadb-accessing.md @@ -5,8 +5,16 @@ to get out of beta in March 2025. ## Graphical user interface -TODO add links to some graphical users interfaces -## Command-line +You can find a non-comprehensive list of different graphical interfaces for using MariaDB on +[MariaDB's homepage](https://mariadb.com/kb/en/graphical-and-enhanced-clients/) + + +## Command-line client mariadb and mysql +[https://mariadb.com/kb/en/mariadb-command-line-client/] + +Nowadays the recommend client to use is `mariadb` , `mysql` does still work but it usually a +symbolic link to `mariadb` + TODO Some example how to use command line tool, mention --ssl ### Common issues with CLI connections ### Accessing your Pukki MariaDB database from Puhti diff --git a/docs/cloud/dbaas/mariadb.md b/docs/cloud/dbaas/mariadb.md index c2c7e865f4..9cfb50e190 100644 --- a/docs/cloud/dbaas/mariadb.md +++ b/docs/cloud/dbaas/mariadb.md @@ -10,16 +10,11 @@ * [How to create database users and modify user permissions](mariadb-permissions.md) # Database engine and backups -TOOD mention that Innodb is the default engine and that engine is the most test in Pukki and using -engine like Aria might cause usages issues while doing backups so one should think twice before +Innodb is the default engine and it is the most test in Pukki. Using other +engines like Aria might cause issues while doing backups so one should consider it twich before using anohter engine than InnoDB. -better infor: https://mariadb.com/kb/en/aria-storage-engine/ +More info can be found in the MariaDB documenation: https://mariadb.com/kb/en/aria-storage-engine/ -TODO I would like out documentation to manetion how the backup works in the background e.g. - * pg_basebackup is compressed encrypted and streamed to Allas - * mariadbbackup is compressed encyrpted and streamed to Allas - -TODO We should add some links to mariadb documentation from here maybe these: - * https://mariadb.com/kb/en/mariadb-client/ - * https://mariadb.com/kb/en/sql-statements-structure/ -TODO postgreql.md would also benefit of similar links +## Useful links when using MariaDB + * [MariaDB client](https://mariadb.com/kb/en/mariadb-client/) + * [MariaDB SQL statement structur](https://mariadb.com/kb/en/sql-statements-structure/) From 0cdc7d6621e070acfb0e34bb237b7a05e0e129ac Mon Sep 17 00:00:00 2001 From: Oscar Kraemer Date: Mon, 25 Nov 2024 13:11:48 +0200 Subject: [PATCH 07/16] Add mariadb access --- docs/cloud/dbaas/mariadb-accessing.md | 171 ++++++++++++++++++++++++-- 1 file changed, 158 insertions(+), 13 deletions(-) diff --git a/docs/cloud/dbaas/mariadb-accessing.md b/docs/cloud/dbaas/mariadb-accessing.md index 3dc995080f..a02df72e60 100644 --- a/docs/cloud/dbaas/mariadb-accessing.md +++ b/docs/cloud/dbaas/mariadb-accessing.md @@ -10,23 +10,168 @@ You can find a non-comprehensive list of different graphical interfaces for usin ## Command-line client mariadb and mysql -[https://mariadb.com/kb/en/mariadb-command-line-client/] +[MariaDB's documentation client](https://mariadb.com/kb/en/mariadb-command-line-client/) Nowadays the recommend client to use is `mariadb` , `mysql` does still work but it usually a -symbolic link to `mariadb` +symbolic link to `mariadb`. + +To access your database the command you probably want to use a command like this: + +``` +mariadb --ssl --password --host ${PUBLIC_IP} --user ${DATABASE_USER} ${DATABASE_NAME} +``` + +or + +``` +mysql --ssl --password --host ${PUBLIC_IP} --user ${DATABASE_USER} ${DATABASE_NAME} +``` + + * `--ssl` means that MariaDB client will connect with SSL because the database in Pukki is +enforcing encrypted connections. + * `--password` means that it will prompt for password + * `--host` means what public IP address the client will connect to + * `--user` means what database user you will connect as. + * `${DATABASE_NAME}` is to which database you want to connect to. + + +### Using command line with .my.cnf + +If you are frequently using the same database it might be easier to set up a `.my.cnf` file in +your home directory so you don't need to remember all flags when you connect to your database. + +1. Create the `.my.cnf` in your home directory + +``` +touch ~/.my.cnf; chmod 600 ~/.my.cnf +``` + +2. Open the configuration file with your favorite editor and add foll in the following +variables +``` +[client] +user = your_username +password = your_password +host = your_host +database = your_database +ssl +``` + +If you don't want to store the password in the file which is recommended you can enforce MariaDB +to prompt you for the password like this: + +``` +[client] +user = your_user +host = your_database_public_ip +database = your_database +ssl +password +``` + -TODO Some example how to use command line tool, mention --ssl ### Common issues with CLI connections + +``` +ERROR 2002 (HY000): Can't connect to MySQL server on '${PUBLIC_IP}' (115) +``` + +If the client asks for your password but the connection is stuck for a long time it probably means +that you have either provided the wrong public IP or you have not opened the firewalls to where +you are trying to connect from. + +``` +ERROR 3159 (08004): Connections using insecure transport are prohibited while --require_secure_transport=ON. +``` + +You tired to connect to the database without `--ssl` + +``` +ERROR 1045 (28000): Access denied for user 'username'@'yourhostname' (using password: YES) +``` +You password or username is wrong. + +``` +ERROR 1044 (42000): Access denied for user 'username'@'%' to database 'databasename' +``` + +This means that you are trying to connect to a database that does not exist or your user does not +have access to. + + ### Accessing your Pukki MariaDB database from Puhti -### Basic Puhti batch job example using mysql + +1. First you need to ensure that you allow [network traffic from Puhti.](firewalls.md#Puthi) +2. Once you have ssh into Puhti you need to load the `mariadb` module. +``` +module load mariadb +``` +3. Now you can connect the database with the mariadb-client + + + ## Some useful SQL commands - List databases - List tables - Show table descriptions - Change database - Example query - Show all database settings - Show all users - Extended display - Import database dump +List databases +```sql +SHOW DATABASES; +``` + +List tables +```sql +SHOW TABLES; +``` + +Show table descriptions +```sql +DESCRIBE $tables; +``` +Change database +```sql +USE DATABASE $databas_name; +``` +Example query +```sql +SELECT * FROM $table LIMIT 1; +``` +Show all database settings +```sql +SHOW VARIABLES; +``` +or if you want to show a subset you can use `LIKE` +```sql +SHOW VARIABLES LIKE 'innodb%'; +``` + +Import database dump +``` +cat your_database_dump.sql | mariadb +``` From 71709d800e7fe1f75ef601fdd8ccfb1bee76e23d Mon Sep 17 00:00:00 2001 From: Oscar Kraemer Date: Mon, 25 Nov 2024 15:02:16 +0200 Subject: [PATCH 08/16] Add priviliages --- docs/cloud/dbaas/mariadb-permissions.md | 68 +++++++++++++++++++++---- 1 file changed, 58 insertions(+), 10 deletions(-) diff --git a/docs/cloud/dbaas/mariadb-permissions.md b/docs/cloud/dbaas/mariadb-permissions.md index cca218c673..86899b7c86 100644 --- a/docs/cloud/dbaas/mariadb-permissions.md +++ b/docs/cloud/dbaas/mariadb-permissions.md @@ -1,17 +1,65 @@ -# MariaDB permissions and privilages +# MariaDB permissions and privileges !!! warning "MariaDB in Pukki is still in beta" This means that we have not tested as extensively that everything works correctly and there might still be big changes how Pukki will manage MariaDB databases. We are hoping to be able to get out of beta in March 2025. -TOOD -## About privilages -TOOD -## Example of giving a user read-only access to a table -TOOD -## Example of giving a user read-write access to a table -TOOD -## When to use root user -TOOD +## About privileges + +By default when creating a user in a MariaDB database the user does not have access to any +database, one need to specify to which database the user have access either from the web interface +or the openstack cli. + +When creating a new user: +```sql +openstack database user create $INSTNACE_ID myuser my_password --database my_database +``` + +When updating an existing user: + +```sql +openstack database user grant access $DATABASE_ID username database_name +``` + +When give user access to a database from the openstack cli or web interface the user gets +`ALL PRIVILEGES` to that database. + +If you want to crate a user with different privileges you need to use the +`openstack datbase enable root` command so that you can create a user manually. More information +in the next topic how to create a read-only user. + + +## Example of giving a user read-only access to a database +To be able to create a read-only user you will first need to enable root since read only users +are not able to be created from the Pukki interfaces. + +1. First create the root user +```sh +openstack database root enable 1de0dbe4-eed7-4291-b8ae-156c3d74473b +``` + +2. Then you can access the database with the root user and password: +```sql +GRANT SELECT ON database_name.* TO 'reader'@'%'; +FLUSH PRIVILEGES; +``` +You can show the result with: +``` +SHOW GRANTS FOR 'reader'@'%'; ++-------------------------------------------------------------------------------------------------------+ +| Grants for reader@% | ++-------------------------------------------------------------------------------------------------------+ +| GRANT USAGE ON *.* TO `reader`@`%` IDENTIFIED BY PASSWORD '*2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19' | +| GRANT SELECT ON `database_name`.* TO `reader`@`%` | ++-------------------------------------------------------------------------------------------------------+ +``` +If you would like to only give the `reader` user access to only a table you can do it by + +```sql +GRANT SELECT ON database_name.table_name TO 'reader'@'%'; +``` + +Be aware that when creating a read user the openstack cli tool will not show to what tables the user +have access to. From 57b55dc46658fd01247b9929db6075d134413b4f Mon Sep 17 00:00:00 2001 From: Oscar Kraemer Date: Mon, 25 Nov 2024 15:10:16 +0200 Subject: [PATCH 09/16] typo fixes --- docs/cloud/dbaas/mariadb-permissions.md | 6 +++--- docs/cloud/dbaas/mariadb.md | 8 ++++---- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/cloud/dbaas/mariadb-permissions.md b/docs/cloud/dbaas/mariadb-permissions.md index 86899b7c86..f421d1f616 100644 --- a/docs/cloud/dbaas/mariadb-permissions.md +++ b/docs/cloud/dbaas/mariadb-permissions.md @@ -26,7 +26,7 @@ When give user access to a database from the openstack cli or web interface the `ALL PRIVILEGES` to that database. If you want to crate a user with different privileges you need to use the -`openstack datbase enable root` command so that you can create a user manually. More information +`openstack database enable root` command so that you can create a user manually. More information in the next topic how to create a read-only user. @@ -36,7 +36,7 @@ are not able to be created from the Pukki interfaces. 1. First create the root user ```sh -openstack database root enable 1de0dbe4-eed7-4291-b8ae-156c3d74473b +openstack database root enable $DATABASE_ID ``` 2. Then you can access the database with the root user and password: @@ -50,7 +50,7 @@ SHOW GRANTS FOR 'reader'@'%'; +-------------------------------------------------------------------------------------------------------+ | Grants for reader@% | +-------------------------------------------------------------------------------------------------------+ -| GRANT USAGE ON *.* TO `reader`@`%` IDENTIFIED BY PASSWORD '*2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19' | +| GRANT USAGE ON *.* TO `reader`@`%` IDENTIFIED BY PASSWORD 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' | | GRANT SELECT ON `database_name`.* TO `reader`@`%` | +-------------------------------------------------------------------------------------------------------+ ``` diff --git a/docs/cloud/dbaas/mariadb.md b/docs/cloud/dbaas/mariadb.md index 9cfb50e190..8fd69e5123 100644 --- a/docs/cloud/dbaas/mariadb.md +++ b/docs/cloud/dbaas/mariadb.md @@ -11,10 +11,10 @@ # Database engine and backups Innodb is the default engine and it is the most test in Pukki. Using other -engines like Aria might cause issues while doing backups so one should consider it twich before -using anohter engine than InnoDB. -More info can be found in the MariaDB documenation: https://mariadb.com/kb/en/aria-storage-engine/ +engines like Aria might cause issues while doing backups so one should consider it twice before +using another engine than InnoDB. +More info can be found in the MariaDB documentation: https://mariadb.com/kb/en/aria-storage-engine/ ## Useful links when using MariaDB * [MariaDB client](https://mariadb.com/kb/en/mariadb-client/) - * [MariaDB SQL statement structur](https://mariadb.com/kb/en/sql-statements-structure/) + * [MariaDB SQL statement structure](https://mariadb.com/kb/en/sql-statements-structure/) From dea3601661a36abaab925302dcd70c30593d23c8 Mon Sep 17 00:00:00 2001 From: Oscar Kraemer Date: Mon, 25 Nov 2024 15:17:10 +0200 Subject: [PATCH 10/16] update broken link --- docs/cloud/dbaas/mariadb-accessing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/cloud/dbaas/mariadb-accessing.md b/docs/cloud/dbaas/mariadb-accessing.md index a02df72e60..7fe8f8f594 100644 --- a/docs/cloud/dbaas/mariadb-accessing.md +++ b/docs/cloud/dbaas/mariadb-accessing.md @@ -101,7 +101,7 @@ have access to. ### Accessing your Pukki MariaDB database from Puhti -1. First you need to ensure that you allow [network traffic from Puhti.](firewalls.md#Puthi) +1. First you need to ensure that you allow [network traffic from Puhti.](firewalls.md#puhti) 2. Once you have ssh into Puhti you need to load the `mariadb` module. ``` module load mariadb From b4b28aa1ad368037f396a32de8ef560de182346c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Matti=20K=C3=A4nninen?= Date: Tue, 3 Dec 2024 11:31:15 +0200 Subject: [PATCH 11/16] Typo fixes, recommend using default / latest datastore version --- docs/cloud/dbaas/backups.md | 2 +- docs/cloud/dbaas/cli.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/cloud/dbaas/backups.md b/docs/cloud/dbaas/backups.md index 605fbe6928..f4f960b2ae 100644 --- a/docs/cloud/dbaas/backups.md +++ b/docs/cloud/dbaas/backups.md @@ -6,7 +6,7 @@ after the last backup. All backups are stored for 90 days after which they get a deleted, backups can not be manually deleted by the user. The backups are stored encrypted in Allas. - + The web interface is currently only showing the latest 20 backups. If you want to see all your backups you need to use the CLI-tool. diff --git a/docs/cloud/dbaas/cli.md b/docs/cloud/dbaas/cli.md index 29d2dfc9d7..28487a7037 100644 --- a/docs/cloud/dbaas/cli.md +++ b/docs/cloud/dbaas/cli.md @@ -49,8 +49,8 @@ Remember that you can use the help command as `openstack help database` and the ```sh openstack datastore list ``` - - * `datastore version`. This depends on the datastore you have chosen and you should usually choose the latest. If you use PostgreSQL you can probably use `17.0`. You can find out the available datastore versions with: + + * `datastore version`. This optional flag depends on the datastore you have chosen. You can safely omit it if you're fine with the default datastore version, which should always be the latest one available. You can find out the available datastore versions with: ```sh openstack datastore version list postgresql From e88622c60c2cbfb26a27aaf33d072d094e11c9cf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Matti=20K=C3=A4nninen?= Date: Tue, 3 Dec 2024 13:29:06 +0200 Subject: [PATCH 12/16] Tidy the language in mariadb-accessing.md --- docs/cloud/dbaas/mariadb-accessing.md | 81 +++++++++++++++------------ 1 file changed, 44 insertions(+), 37 deletions(-) diff --git a/docs/cloud/dbaas/mariadb-accessing.md b/docs/cloud/dbaas/mariadb-accessing.md index 7fe8f8f594..b247154be9 100644 --- a/docs/cloud/dbaas/mariadb-accessing.md +++ b/docs/cloud/dbaas/mariadb-accessing.md @@ -1,53 +1,54 @@ # Accessing your MariaDB instance !!! warning "MariaDB in Pukki is still in beta" - This means that we have not tested as extensively that everything works correctly and there - might still be big changes how Pukki will manage MariaDB databases. We are hoping to be able - to get out of beta in March 2025. + This means that it hasn't been tested as extensively as PostgreSQL, and there might still be + large changes to how Pukki manages MariaDB database instances. We hope to move out of beta in + March 2025. ## Graphical user interface You can find a non-comprehensive list of different graphical interfaces for using MariaDB on -[MariaDB's homepage](https://mariadb.com/kb/en/graphical-and-enhanced-clients/) +[MariaDB's homepage](https://mariadb.com/kb/en/graphical-and-enhanced-clients/). ## Command-line client mariadb and mysql [MariaDB's documentation client](https://mariadb.com/kb/en/mariadb-command-line-client/) -Nowadays the recommend client to use is `mariadb` , `mysql` does still work but it usually a -symbolic link to `mariadb`. +The recommended CLI client to use is `mariadb`. The `mysql` client does still work, but is +usually a symbolic link to `mariadb`. -To access your database the command you probably want to use a command like this: +Example commands for accessing your database: ``` mariadb --ssl --password --host ${PUBLIC_IP} --user ${DATABASE_USER} ${DATABASE_NAME} ``` -or +or ``` mysql --ssl --password --host ${PUBLIC_IP} --user ${DATABASE_USER} ${DATABASE_NAME} ``` - * `--ssl` means that MariaDB client will connect with SSL because the database in Pukki is -enforcing encrypted connections. - * `--password` means that it will prompt for password - * `--host` means what public IP address the client will connect to - * `--user` means what database user you will connect as. - * `${DATABASE_NAME}` is to which database you want to connect to. - + * `--ssl` means the MariaDB client connects using SSL. This is necessary as +Pukki database instances enforce encrypted connections. + * `--password` means the client prompts for a password. You can specify one +on the command line (like `--password=password`), but that is considered insecure. + * `--host` specifies the host address to connect to. In Pukki this is almost +always your database instance's public IP address. + * `--user` specifies which user to connect to the database as. + * `${DATABASE_NAME}` specifies which database on the server to connect to. + ### Using command line with .my.cnf -If you are frequently using the same database it might be easier to set up a `.my.cnf` file in -your home directory so you don't need to remember all flags when you connect to your database. +If you are frequently connecting to the same database, it might be worthwhile to set up a +`.my.cnf` configuration file in your home directory to store the necessary flags and options. -1. Create the `.my.cnf` in your home directory +1. Create an empty `.my.cnf` file in your home directory, and restrict its access permissions: ``` touch ~/.my.cnf; chmod 600 ~/.my.cnf ``` -2. Open the configuration file with your favorite editor and add foll in the following -variables +2. Edit the configuration file with your favorite editor and add the following options: ``` [client] user = your_username @@ -57,8 +58,8 @@ database = your_database ssl ``` -If you don't want to store the password in the file which is recommended you can enforce MariaDB -to prompt you for the password like this: +As storing the password in a plaintext file isn't recommended, you can leave it empty to +always prompt for the password when connecting: ``` [client] @@ -76,40 +77,40 @@ password ERROR 2002 (HY000): Can't connect to MySQL server on '${PUBLIC_IP}' (115) ``` -If the client asks for your password but the connection is stuck for a long time it probably means -that you have either provided the wrong public IP or you have not opened the firewalls to where -you are trying to connect from. +If a password prompt appears, but the client is afterwards stuck connecting for a long time, you should +double-check that the `host` argument is correct, and that the firewall allows connections from your client's +address. ``` ERROR 3159 (08004): Connections using insecure transport are prohibited while --require_secure_transport=ON. ``` -You tired to connect to the database without `--ssl` +You tried to connect to the database without `--ssl`. ``` ERROR 1045 (28000): Access denied for user 'username'@'yourhostname' (using password: YES) ``` -You password or username is wrong. + +Either your password or your username is wrong. ``` ERROR 1044 (42000): Access denied for user 'username'@'%' to database 'databasename' ``` -This means that you are trying to connect to a database that does not exist or your user does not -have access to. +Either the database specified does not exist, or the username specified has no access to it. ### Accessing your Pukki MariaDB database from Puhti -1. First you need to ensure that you allow [network traffic from Puhti.](firewalls.md#puhti) -2. Once you have ssh into Puhti you need to load the `mariadb` module. +1. Ensure your database instance allows [network traffic from Puhti.](firewalls.md#puhti) +2. `ssh` onto Puhti and load the `mariadb` module ``` module load mariadb ``` -3. Now you can connect the database with the mariadb-client +3. Now you can connect to the database with the mariadb-client - ## Some useful SQL commands + List databases ```sql SHOW DATABASES; @@ -151,24 +153,29 @@ SHOW TABLES; Show table descriptions ```sql -DESCRIBE $tables; +DESCRIBE $table_name; ``` + Change database ```sql -USE DATABASE $databas_name; +USE DATABASE $database_name; ``` + Example query ```sql -SELECT * FROM $table LIMIT 1; +SELECT * FROM $table_name LIMIT 1; ``` + Show all database settings ```sql SHOW VARIABLES; ``` + or if you want to show a subset you can use `LIKE` ```sql SHOW VARIABLES LIKE 'innodb%'; ``` +Note that `%` here indicates a wildcard - this lists all variables that begin with `innodb`. Import database dump From a042cbe4c42fe34b81c31a1edad987c32d9bad8c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Matti=20K=C3=A4nninen?= Date: Tue, 3 Dec 2024 15:56:55 +0200 Subject: [PATCH 13/16] Tidy the language in mariadb-permissions --- docs/cloud/dbaas/mariadb-permissions.md | 43 ++++++++++++------------- 1 file changed, 21 insertions(+), 22 deletions(-) diff --git a/docs/cloud/dbaas/mariadb-permissions.md b/docs/cloud/dbaas/mariadb-permissions.md index f421d1f616..169f0a20b1 100644 --- a/docs/cloud/dbaas/mariadb-permissions.md +++ b/docs/cloud/dbaas/mariadb-permissions.md @@ -1,50 +1,51 @@ # MariaDB permissions and privileges !!! warning "MariaDB in Pukki is still in beta" - This means that we have not tested as extensively that everything works correctly and there - might still be big changes how Pukki will manage MariaDB databases. We are hoping to be able - to get out of beta in March 2025. + This means that it hasn't been tested as extensively as PostgreSQL, and there might still be + large changes to how Pukki manages MariaDB database instances. We hope to move out of beta in + March 2025. ## About privileges -By default when creating a user in a MariaDB database the user does not have access to any -database, one need to specify to which database the user have access either from the web interface -or the openstack cli. +When creating a user through the web interface or via openstack cli, you can define which databases +it has access to. By default, a freshly created user doesn't have access to any databases. When creating a new user: ```sql -openstack database user create $INSTNACE_ID myuser my_password --database my_database +openstack database user create $INSTANCE_ID my_user my_password --databases my_database ``` When updating an existing user: - ```sql -openstack database user grant access $DATABASE_ID username database_name +openstack database user grant access $INSTANCE_ID my_user my_database ``` +You can either specify a single database or a list of databases to these commands. The commands +also accept the database instance's name in place of the ID. -When give user access to a database from the openstack cli or web interface the user gets +Giving a user access to a database via openstack cli or the web interface means it gets `ALL PRIVILEGES` to that database. -If you want to crate a user with different privileges you need to use the -`openstack database enable root` command so that you can create a user manually. More information -in the next topic how to create a read-only user. +If you want more control over a user's privileges, you have to enable root access (through +the web interface, or with `openstack database enable root` with the CLI client) and manually +modify user privileges. ## Example of giving a user read-only access to a database -To be able to create a read-only user you will first need to enable root since read only users -are not able to be created from the Pukki interfaces. -1. First create the root user +1. Enable the root user: ```sh openstack database root enable $DATABASE_ID ``` -2. Then you can access the database with the root user and password: +2. Access the database using the root user and password. + +3. Grant `SELECT` privileges on a database to a user: ```sql GRANT SELECT ON database_name.* TO 'reader'@'%'; FLUSH PRIVILEGES; ``` -You can show the result with: + +You can view the grant with: ``` SHOW GRANTS FOR 'reader'@'%'; +-------------------------------------------------------------------------------------------------------+ @@ -54,12 +55,10 @@ SHOW GRANTS FOR 'reader'@'%'; | GRANT SELECT ON `database_name`.* TO `reader`@`%` | +-------------------------------------------------------------------------------------------------------+ ``` -If you would like to only give the `reader` user access to only a table you can do it by +You can also grant table-specific access: ```sql GRANT SELECT ON database_name.table_name TO 'reader'@'%'; ``` -Be aware that when creating a read user the openstack cli tool will not show to what tables the user -have access to. - +Be aware that the openstack cli tool or the the web interface will not display grants given through root access. For more information on MariaDB's grants, refer to [the official MariaDB documentation](https://mariadb.com/kb/en/grant/). From 253a0fd84fc189c20d3729ff4083d86523a57a79 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Matti=20K=C3=A4nninen?= Date: Tue, 3 Dec 2024 16:06:10 +0200 Subject: [PATCH 14/16] Touch up the language in mariadb.md --- docs/cloud/dbaas/mariadb.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/docs/cloud/dbaas/mariadb.md b/docs/cloud/dbaas/mariadb.md index 8fd69e5123..3f43188612 100644 --- a/docs/cloud/dbaas/mariadb.md +++ b/docs/cloud/dbaas/mariadb.md @@ -1,19 +1,21 @@ # MariaDB !!! warning "MariaDB in Pukki is still in beta" - This means that we have not tested as extensively that everything works correctly and there - might still be big changes how Pukki will manage MariaDB databases. We are hoping to be able - to get out of beta in March 2025. - + This means that it hasn't been tested as extensively as PostgreSQL, and there might still be + large changes to how Pukki manages MariaDB database instances. We hope to move out of beta in + March 2025. + * [How to access your MariaDB database](mariadb-accessing.md) * [How to create database users and modify user permissions](mariadb-permissions.md) # Database engine and backups -Innodb is the default engine and it is the most test in Pukki. Using other -engines like Aria might cause issues while doing backups so one should consider it twice before -using another engine than InnoDB. -More info can be found in the MariaDB documentation: https://mariadb.com/kb/en/aria-storage-engine/ + +MariaDB database instances on Pukki use InnoDB by default, as most testing has been done using it. +Changing to other engines such as Aria might cause issues with backups, so one should carefully +consider how necessary it is before switching from InnoDB. +More information on database engines can be found in the +[official MariaDB documentation](https://mariadb.com/kb/en/storage-engines/). ## Useful links when using MariaDB * [MariaDB client](https://mariadb.com/kb/en/mariadb-client/) From d8d9eb2f05481ace526fd621bb99ab2a6f307003 Mon Sep 17 00:00:00 2001 From: Oscar Kraemer Date: Tue, 7 Jan 2025 14:59:24 +0200 Subject: [PATCH 15/16] update what is new for Pukki --- docs/support/wn/cloud-new.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/support/wn/cloud-new.md b/docs/support/wn/cloud-new.md index 3e794b1f7e..b8bc08fcc1 100644 --- a/docs/support/wn/cloud-new.md +++ b/docs/support/wn/cloud-new.md @@ -1,5 +1,11 @@ # Cloud services +## Pukki now supports MariaDB, 7.1.2025 +Pukki now supports MariaDB as well as PostgreSQL. The MariaDB version we are support in Pukki is +MariaDB 11.4. More information can be found in the +[Pukki MariaDB documentation](../../cloud/dbaas/mariadb.md) + + ## Pukki now supports PostgreSQL 17, 9.10.2024 The default database in Pukki is now PostgreSQL 17 instead of the previous PostgreSQL 14. You can still use PostgreSQL 14 but we recommend that if you are creating a new database you start using From 52357dd0145975ab0c6cc6827ef70a26f5ed31b7 Mon Sep 17 00:00:00 2001 From: Oscar Kraemer Date: Tue, 7 Jan 2025 15:33:10 +0200 Subject: [PATCH 16/16] Update supported databases --- docs/cloud/dbaas/databases.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/cloud/dbaas/databases.md b/docs/cloud/dbaas/databases.md index 181a7217d9..229481457a 100644 --- a/docs/cloud/dbaas/databases.md +++ b/docs/cloud/dbaas/databases.md @@ -1,6 +1,6 @@ # Databases -Currently Pukki only supports PostgreSQL. If you think that Pukki should support some specific +Currently Pukki supports PostgreSQL and MariaDB. If you think that Pukki should support some specific database type please [contact us](../../support/contact.md) with your suggestions. * [PostgreSQL](postgresql.md)