Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 19 additions & 11 deletions docs/07-Datasources/MongoDB.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@ Humalect provides several databases that you can integrate and bring your data t

MongoDB is a *NoSQL*, document-based database management system. It uses JSON-like documents to store and retrieve data, offering high scalability, flexibility and performance. It also supports indexing, transactions, and rich querying capabilities.

:::info
For more information, see [MongoDB Docs.](https://www.mongodb.com/docs/)
:::

> For more information, see [MongoDB Docs.](https://www.mongodb.com/docs/)

> ## Prerequisites
## Prerequisites
1. Have a connected cloud provider
2. Have a connected source code provider
3. Have a cluster
Expand All @@ -19,8 +20,9 @@ MongoDB is a *NoSQL*, document-based database management system. It uses JSON-li

Let's create a Datasource with `MongoDB version 6.0.1`.


**Step 1:** Go to the DataSources Tab from the UI ([or click here to get redirected](https://console.humalect.com/user/datasources)) and click on `+ Create New Datasource.`


**Step 2:** Enter the following details:

> | Field | Desc |
Expand All @@ -29,6 +31,7 @@ Let's create a Datasource with `MongoDB version 6.0.1`.
> | **Select Cluster** | cluster name where you want your Datasource to be created |
> | **Select DataSource** | choose database/cache of your choice|
> | **Select Version** | `v6.0.1`, `v5.0.10` ([click for more details on MongoDB versions](https://www.mongodb.com/evolved))|
> | **Select Color Label**| choose b/w RED, GREEN, BLUE|


**Step 3:** After you enter the above details, you'll be prompted to enter additional details such as `persistence size`, `service port`, etc. Refer to the below table for more information.
Expand Down Expand Up @@ -56,17 +59,22 @@ Under the `Advanced Options`, enter the following details:

**Step 5:** Once the MongoDB Datasource is successfully created, you'll see them listed in the DataSources tab with details like:

a. Datasource Name
b. Cluster Name
c. Database used
d. Database version used
- Release Name - User entered name
- Related Cluster Name - The name of cluster which hosts the datasource
- Datasource name - [Redis | MongoDB | PostgreSQL | MySQL ]
- Database version used
- Color Label
- Status - Datasource creation Completed or Failed.
- Action - To fetch Datasource credentials

**Step 6:** If you wish to **delete** the datasource, click on the vertical 3 dots next to the datasource card and click on `Delete`. This will initiate the deletion process.

> **Caution:** *Do not delete any datasource if you are not sure about it as it can lead to data loss.*
**Step 6:** If you wish to **delete** the datasource, click on the vertical 3 dots next to the datasource card and click on `Delete`. This will initiate the deletion process.

:::caution
Do not delete any datasource if you are not sure about it as it can lead to dataloss.
:::

![datasource_mongo-2](./../../static/img/datasource_mongo-2.png)
![ds_delete](./../../static/img/datasources/ds_delete.png)


## How to use existing datasources in a pipeline?
Expand Down
96 changes: 96 additions & 0 deletions docs/07-Datasources/MySQL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
description: MySQL installation on the Humalect platform
---

Humalect provides several databases that you can integrate and bring your data to the platform with minimal permissions from Humalect's side, in this article, we will be focusing on **MySQL**.

MySQL is an open-source relational database management system (RDBMS) that provides a structured way to store, manage, and retrieve data. It uses Structured Query Language (SQL) to interact with databases and is widely used in various applications and industries for storing and manipulating data.

MySQL offers features such as data security, scalability, high availability, and support for various programming languages, making it a popular choice for building and maintaining databases in a wide range of software applications.

:::info
For more information, see [MySQL Docs.](https://dev.mysql.com/doc/)
:::

## Prerequisites
1. Have a connected cloud provider
2. Have a connected source code provider
3. Have a cluster

## Create a MySQL Datasource

Let's create a Datasource with `MySQL version 8.0.33`


**Step 1:** Go to the DataSources Tab from the UI ([or click here to get redirected](https://console.humalect.com/user/datasources)) and click on `+ Create New Datasource.`

**Step 2:** Enter the following details:

> | Field | Desc |
> | -- | -- |
> | **Name** | name of your Datasource|
> | **Select Cluster** | cluster name where you want your Datasource to be created |
> | **Select DataSource** | choose database/cache of your choice|
> | **Select Version** | `v8.0.33`, `v8.0.29` ([click for more details on MySQL versions](https://dev.mysql.com/doc/relnotes/mysql/8.0/en/))|
> | **Select Color Label**| choose b/w RED, GREEN, BLUE|


**Step 3:** After you enter the above details, you'll be prompted to enter additional details such as `persistence size`, `port number`, etc. Refer to the below table for more information.



### MySQL Advanced Options

Under the `Advanced Options`, enter the following details:

| Authentication | Desc |
| -- | -- |
| Password for Root User | The password assigned to the administrative "root" user, providing full control over the MySQL server |
| Auth user password |The password for an authorized user who can access and manage the specified database |
| DataBase name | The unique name identifying a specific database within MySQL where data is organized and stored |
| Auth User Username | The username of an authenticated user with defined privileges to interact with the designated database |

| Secondary | Value | Primary | Value |
| -- | -- | -- | -- |
|Enable persistence in Secondary datastore(Replica) | false* | Size for persistence in Primary datastore(Master) | 8Gi* |
|Size for persistence in Secondary datastore(Replica) | 8Gi* |Enable persistence in Primary datastore(Master) | true* |


`*default value`



**Step:4** After you successfully fill in all the required details, click on `Create` and the Datasource creation process will start.

![ds_mysql](./../../static/img/datasources/ds_mysql.png)

<center><i>Picture representing the state of UI when all the details are entered.</i></center>


**Step 5:** Once the MySQL Datasource is successfully created, you'll see them listed in the DataSources tab with details like:

- Release Name - User entered name
- Related Cluster Name - The name of cluster which hosts the datasource
- Datasource name - [Redis | MongoDB | PostgreSQL | MySQL ]
- Database version used
- Color Label
- Status - Datasource creation Completed or Failed.
- Action - To fetch Datasource credentials



**Step 6:** If you wish to **delete** the datasource, click on the vertical 3 dots next to the datasource card and click on `Delete`. This will initiate the deletion process.

:::caution
Do not delete any datasource if you are not sure about it as it can lead to dataloss.
:::

![ds_delete](./../../static/img/datasources/ds_delete.png)


## How to use existing datasources in a pipeline?
TBD

### Troubleshooting
Need help? [Contact](./../Contact-us/reach-out-to-us) us

35 changes: 29 additions & 6 deletions docs/07-Datasources/Overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ title: Datasources Overview
description: Databases and caches supported by the Humalect platform
---

import DocCardList from '@theme/DocCardList';

## What do we mean by data sources?

Every application needs some kind of database or cache to store data. There are multiple ways to use databases:
Expand All @@ -21,7 +23,13 @@ Ephermal databases on demand made easy.
> | -------- | ------------------------ | -- |
> | MongoDB | `v6.0.1`, `v5.0.10` | [LINK](https://www.mongodb.com/evolved) |
> | Redis | `v7.0.4`, `v6.2.7` |[LINK](https://redis.io/download/#:~:text=Stable%20(6.2.6)) |
> | Postgres | `v14.11.0`, `v11.14.0` |[LINK](https://www.postgresql.org/support/versioning/) |
> | Postgres | `v14.5.0`, `v11.14.0` |[LINK](https://www.postgresql.org/support/versioning/) |
> | MySQL | `v8.0.33`, `v8.0.29` | [LINK](https://dev.mysql.com/doc/relnotes/mysql/8.0/en/) |




<DocCardList />

## Steps to create a Datasource in Humalect

Expand All @@ -36,16 +44,30 @@ Follow these steps to create a new datastore:
<center><i>Picture representing the state of UI while creating Datasources.</i></center>


**Step 1:** Enter Datasource Name
**Step 1:** Enter Datasource Name.

**Step 2:** Select your Cluster in which you want to host your datasource
**Step 2:** Select your Cluster in which you want to host your datasource.

**Step 3:** Select datasource of your choice from the list
**Step 3:** Select datasource of your choice from the list.

**Step 4:** Select the Datasource version. You can also edit advanced options like username and replica count.

**Step 5:** Hit create button and Voila it's done.


![ds_view](./../../static/img/datasources/ds_view.png)


### How to fetch Datasource Credentials?
In order to fetch the datasource credentials, click on the eye button under `Actions` section for the respective datasource. Here, you'll be presented with informations like:

- Host IP - e.g: 10.0.53.83
- Port - e.g: 2635
- Password - e.g:nsky78reww3E
- Connection String - e.g: postgres://postgres:[email protected]:5432/db_name

![ds_creds](./../../static/img/datasources/ds_creds.png)

## Can these datasources be used in production?

These datasources are meant to be used in an ephermal way and are best suited for non-production workloads.
Expand All @@ -55,8 +77,9 @@ These datasources are meant to be used in an ephermal way and are best suited fo
All the data is safe as we use PVs and PVCs to store your data. All the data is stored in fault tolerant block storage with your cloud provider.


> **Tip:**
> *Did not find your favorite database in the list? Reach out to us to raise a request for new datasources.*
:::info
Did not find your favorite database in the list? Reach out to us to raise a request for new datasources.
:::

### Troubleshooting
Need help? [Contact](./../Contact-us/reach-out-to-us) us
Expand Down
34 changes: 21 additions & 13 deletions docs/07-Datasources/Postgres.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,28 +6,30 @@ Humalect provides several databases that you can integrate and bring your data t

Postgres is a free and open-source *relational database* management system that is used for managing large amounts of structured data. It offers robust support for SQL, as well as several advanced features, including multi-version concurrency control, point-in-time recovery, and native replication.

:::info
For more information, see [Postgres Docs.](https://www.postgresql.org/docs/)
:::

> For more information, see [Postgres Docs.](https://www.postgresql.org/docs/)

> ## Prerequisites
## Prerequisites
1. Have a connected cloud provider
2. Have a connected source code provider
3. Have a cluster

## Create a Postgres Datasource

Let's create a Datasource with `Postgres version 14.11.0`

Let's create a Datasource with `Postgres version 14.5.0`

**Step 1:** Go to the DataSources Tab from the UI ([or click here to get redirected](https://console.humalect.com/user/datasources)) and click on `+ Create New Datasource.`

**Step 2:** Enter the following details:

> | Field | Desc |
> | -- | -- |
> | **Name** | name of your Datasource|
> | **Select Cluster** | cluster name where you want your Datasource to be created |
> | **Select DataSource** | choose database/cache of your choice|
> | **Select Version** | `v14.11.0`, `v11.14.0` ([click for more details on Postgres versions](https://www.postgresql.org/docs/release/))|
> | **Select Version** | `v14.5.0`, `v11.14.0` ([click for more details on Postgres versions](https://www.postgresql.org/docs/release/))|
> | **Select Color Label**| choose b/w RED, GREEN, BLUE|


**Step 3:** After you enter the above details, you'll be prompted to enter additional details such as `persistence size`, `port number`, etc. Refer to the below table for more information.
Expand Down Expand Up @@ -64,17 +66,23 @@ Under the `Advanced Options`, enter the following details:

**Step 5:** Once the Postgres Datasource is successfully created, you'll see them listed in the DataSources tab with details like:

a. Datasource Name
b. Cluster Name
c. Database used
d. Database version used
- Release Name - User entered name
- Related Cluster Name - The name of cluster which hosts the datasource
- Datasource name - [Redis | MongoDB | PostgreSQL | MySQL ]
- Database version used
- Color Label
- Status - Datasource creation Completed or Failed.
- Action - To fetch Datasource credentials


**Step 6:** If you wish to **delete** the datasource, click on the vertical 3 dots next to the datasource card and click on `Delete`. This will initiate the deletion process.

> **Caution:** Do not delete any datasource if you are not sure about it as it can lead to dataloss.
**Step 6:** If you wish to **delete** the datasource, click on the vertical 3 dots next to the datasource card and click on `Delete`. This will initiate the deletion process.

:::caution
Do not delete any datasource if you are not sure about it as it can lead to dataloss.
:::

![datasource_mongo-2](./../../static/img/datasource_mongo-2.png)
![ds_delete](./../../static/img/datasources/ds_delete.png)


## How to use existing datasources in a pipeline?
Expand Down
38 changes: 22 additions & 16 deletions docs/07-Datasources/Redis.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,11 @@ Humalect provides several databases that you can integrate and bring your data t

Redis is an *in-memory data* structure store. It can be used as a database, cache, and message broker. It supports multiple data structures such as *strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, and geospatial indexes* with radius queries.

> For more information, see [Redis Docs.](https://redis.io/docs/about/)
:::info
For more information, see [Redis Docs.](https://redis.io/docs/about/)
:::

> ## Prerequisites
## Prerequisites
1. Have a connected cloud provider
2. Have a connected source code provider
3. Have a cluster
Expand All @@ -27,7 +29,8 @@ Let's create a Datasource with `Redis v7.0.4`.
> | **Name** | name of your Datasource|
> | **Select Cluster** | cluster name where you want your Datasource to be created |
> | **Select DataSource** | choose database/cache of your choice|
> | **Select Version** | `7.0.1`, `6.2.7` ([click for more details on redis versions](https://redis.io/download/))|
> | **Select Version** | `7.0.4`, `6.2.7` ([click for more details on redis versions](https://redis.io/download/))|
> | **Select Color Label**| choose b/w RED, GREEN, BLUE|


**Step 3:** After you enter the above details, you'll be prompted to enter additional details such as `persistence size`, `service port`, etc. Refer to the below table for more information.
Expand All @@ -44,8 +47,9 @@ Let's create a Datasource with `Redis v7.0.4`.

`*default value`

> **Note:** Redis uses asynchronous replication, with asynchronous replica-to-master acknowledges of the amount of data processed. **Master-slave replication** enables data from one database server (the master) to be replicated to one or more other database servers (the slaves). The master logs the updates, which then ripple through to the slaves.

:::info
Redis uses asynchronous replication, with asynchronous replica-to-master acknowledges of the amount of data processed. **Master-slave replication** enables data from one database server (the master) to be replicated to one or more other database servers (the slaves). The master logs the updates, which then ripple through to the slaves.
:::

**Step:4** After you successfully fill in all the required details, click on `Create` and the Datasource creation process will start.

Expand All @@ -55,21 +59,23 @@ Let's create a Datasource with `Redis v7.0.4`.


**Step 5:** Once the Redis Datasource is successfully created, you'll see them listed in the DataSources tab with details like:
a. Datasource Name
b. Cluster Name
c. Database used
d. Database version used

**Step 6:** If you wish to **delete** the datasource, click on the vertical 3 dots next to the datasource card and click on `Delete`. This will initiate the deletion process.

> **Caution:** Do not delete any datasource if you are not sure about it as it can lead to dataloss.


- Release Name - User entered name
- Related Cluster Name - The name of cluster which hosts the datasource
- Datasource name - [Redis | MongoDB | PostgreSQL | MySQL ]
- Database version used
- Color Label
- Status - Datasource creation Completed or Failed.
- Action - To fetch Datasource credentials


**Step 6:** If you wish to **delete** the datasource, click on the vertical 3 dots next to the datasource card and click on `Delete`. This will initiate the deletion process.

![redis-2](./../../static/img/redis-2.png)
:::caution
Do not delete any datasource if you are not sure about it as it can lead to dataloss.
:::

<center><i>Picture representing the state of UI when the Datasource has been created.</i></center>
![ds_delete](./../../static/img/datasources/ds_delete.png)



Expand Down
Binary file added static/img/datasources/ds_create.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/datasources/ds_creds.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/datasources/ds_delete.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/datasources/ds_mysql.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/datasources/ds_view.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.