Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DSIP-84][Feature] Add Caché datasource support (#16833) #16844

Open
wants to merge 4 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config/plugins_config
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ dolphinscheduler-datasource-databend
dolphinscheduler-datasource-db2
dolphinscheduler-datasource-doris
dolphinscheduler-datasource-hana
dolphinscheduler-datasource-cache
dolphinscheduler-datasource-hive
dolphinscheduler-datasource-k8s
dolphinscheduler-datasource-kyuubi
Expand Down
4 changes: 4 additions & 0 deletions docs/configs/docsdev.js
Original file line number Diff line number Diff line change
Expand Up @@ -355,6 +355,10 @@ export default {
title: 'doris',
link: '/en-us/docs/dev/user_doc/guide/datasource/doris.html',
},
{
title: 'Caché',
link: '/en-us/docs/dev/user_doc/guide/datasource/cache.html',
},
{
title: 'Vertica',
link: '/en-us/docs/dev/user_doc/guide/datasource/vertica.html',
Expand Down
23 changes: 23 additions & 0 deletions docs/docs/en/guide/datasource/cache.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# CACHE

![cache](../../../../img/new_ui/dev/datasource/cache.png)

## Datasource Parameters

| **Datasource** | **Description** |
|----------------------------|---------------------------------------------------------------------------------------|
| Datasource | Select CACHE. |
| Datasource name | Enter the name of the DataSource. |
| Description | Enter a description of the DataSource. |
| IP/Host Name | Enter the CACHE service IP.(If there are multiple IPs, please separate them with `,`) |
| Port | Enter the CACHE service port. |
| Username | Set the username for CACHE connection. |
| Password | Set the password for CACHE connection. |
| Database name | Enter the database name of the CACHE connection. |
| Jdbc connection parameters | Parameter settings for CACHE connection, in JSON format. |

## Native Supported

No, read section example in [datasource-setting](../installation/datasource-setting.md) `DataSource Center` section to activate
this datasource.

edclol marked this conversation as resolved.
Show resolved Hide resolved
17 changes: 17 additions & 0 deletions docs/docs/zh/guide/datasource/cache.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# CACHE 数据源

![CACHE](../../../../img/new_ui/dev/datasource/cache.png)

- 数据源:选择 CACHE
- 数据源名称:输入数据源的名称
- 描述:输入数据源的描述
- IP 主机名:输入连接 CACHE 的 IP(如有多个IP,请用`,`分隔)
- 端口:输入连接 CACHE 的端口
- 用户名:设置连接 CACHE 的用户名
- 密码:设置连接 CACHE 的密码
- 数据库名:输入连接 CACHE 的数据库名称
- Jdbc 连接参数:用于 CACHE 连接的参数设置,以 JSON 形式填写

## 是否原生支持

否,使用前需请参考 [数据源配置](../installation/datasource-setting.md) 中的 "数据源中心" 章节激活数据源。
Binary file added docs/img/new_ui/dev/datasource/cache.png
edclol marked this conversation as resolved.
Show resolved Hide resolved
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ public class DataSourceConstants {
public static final String NET_SNOWFLAKE_JDBC_DRIVER = "net.snowflake.client.jdbc.SnowflakeDriver";
public static final String COM_VERTICA_JDBC_DRIVER = "com.vertica.jdbc.Driver";
public static final String COM_HANA_DB_JDBC_DRIVER = "com.sap.db.jdbc.Driver";
public static final String COM_INTERSYS_JDBC_CACHE_DRIVER = "com.intersys.jdbc.CacheDriver";

/**
* validation Query
Expand All @@ -66,6 +67,7 @@ public class DataSourceConstants {
public static final String VERTICA_VALIDATION_QUERY = "select 1";

public static final String HANA_VALIDATION_QUERY = "select 1 from DUMMY";
public static final String CACHE_VALIDATION_QUERY = "select 1";

/**
* jdbc url
Expand All @@ -91,6 +93,7 @@ public class DataSourceConstants {
public static final String JDBC_SNOWFLAKE = "jdbc:snowflake://";
public static final String JDBC_VERTICA = "jdbc:vertica://";
public static final String JDBC_HANA = "jdbc:sap://";
public static final String JDBC_CACHE = "jdbc:Cache://";

/**
* database type
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -163,5 +163,10 @@
<artifactId>dolphinscheduler-datasource-aliyunserverlessspark</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-datasource-cache</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-datasource-plugin</artifactId>
<version>dev-SNAPSHOT</version>
</parent>

<artifactId>dolphinscheduler-datasource-cache</artifactId>
<packaging>jar</packaging>
<name>${project.artifactId}</name>

<dependencies>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-spi</artifactId>
<version>${project.version}</version>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-datasource-api</artifactId>
<version>${project.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
Comment on lines +44 to +45
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
</dependency>
<dependency>
</dependency>
<dependency>

<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-common</artifactId>
<version>${project.version}</version>
<scope>provided</scope>
</dependency>

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you forget to add the cache jdbc dependency?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I didn't forget. I intentionally didn't add the JDBC dependency because it wasn't available in the Maven repository.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It must be available in the maven repository and comply with apache protocol or approaching license since DS is an apache software foundation open source project.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry. Could you check if this one is okay? https://mvnrepository.com/artifact/com.intersystems/cache-jdbc/2.0.0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seem like this artifact haven't been updated more than 4 years. And it does not indicate compliance with any open source protocols.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that this cache database isn't really up to the mark. I don't want all my efforts to go to waste. So, I'll modify the code to make it compatible with the new version of the iris database. Is this okay? https://mvnrepository.com/artifact/com.intersystems/intersystems-jdbc

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cache is the predecessor of IRIS. However, we do indeed use this old version of the Cache database.😒

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cache is the predecessor of IRIS. However, we do indeed use this old version of the Cache database.😒

This doesn't comply with apache regulations.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to give up the merge.

</dependencies>
</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.dolphinscheduler.plugin.datasource.cache;

import org.apache.dolphinscheduler.plugin.datasource.api.client.BaseAdHocDataSourceClient;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;

public class CacheAdHocDataSourceClient extends BaseAdHocDataSourceClient {

public CacheAdHocDataSourceClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
super(baseConnectionParam, dbType);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.dolphinscheduler.plugin.datasource.cache;

import org.apache.dolphinscheduler.spi.datasource.AdHocDataSourceClient;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannel;
import org.apache.dolphinscheduler.spi.datasource.PooledDataSourceClient;
import org.apache.dolphinscheduler.spi.enums.DbType;

public class CacheDataSourceChannel implements DataSourceChannel {

@Override
public AdHocDataSourceClient createAdHocDataSourceClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
return new CacheAdHocDataSourceClient(baseConnectionParam, dbType);
}

@Override
public PooledDataSourceClient createPooledDataSourceClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
return new CachePooledDataSourceClient(baseConnectionParam, dbType);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.dolphinscheduler.plugin.datasource.cache;

import org.apache.dolphinscheduler.spi.datasource.DataSourceChannel;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannelFactory;

import com.google.auto.service.AutoService;

@AutoService(DataSourceChannelFactory.class)
public class CacheDataSourceChannelFactory implements DataSourceChannelFactory {

@Override
public String getName() {
return "cache";
}

@Override
public DataSourceChannel create() {
return new CacheDataSourceChannel();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.dolphinscheduler.plugin.datasource.cache;

import org.apache.dolphinscheduler.plugin.datasource.api.client.BasePooledDataSourceClient;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;

public class CachePooledDataSourceClient extends BasePooledDataSourceClient {

public CachePooledDataSourceClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
super(baseConnectionParam, dbType);
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.dolphinscheduler.plugin.datasource.cache.param;

import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;

public class CacheConnectionParam extends BaseConnectionParam {

@Override
public String toString() {
return "CacheConnectionParam{"
+ "user='" + user + '\''
+ ", password='" + password + '\''
+ ", address='" + address + '\''
+ ", database='" + database + '\''
+ ", jdbcUrl='" + jdbcUrl + '\''
+ ", driverLocation='" + driverLocation + '\''
+ ", driverClassName='" + driverClassName + '\''
+ ", validationQuery='" + validationQuery + '\''
+ ", other='" + other + '\''
+ '}';
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.dolphinscheduler.plugin.datasource.cache.param;

import org.apache.dolphinscheduler.plugin.datasource.api.datasource.BaseDataSourceParamDTO;
import org.apache.dolphinscheduler.spi.enums.DbType;

public class CacheDataSourceParamDTO extends BaseDataSourceParamDTO {

@Override
public String toString() {
return "CacheDataSourceParamDTO{"
+ "name='" + name + '\''
+ ", note='" + note + '\''
+ ", host='" + host + '\''
+ ", port=" + port
+ ", database='" + database + '\''
+ ", userName='" + userName + '\''
+ ", password='" + password + '\''
+ ", other='" + other + '\''
+ '}';
}

@Override
public DbType getType() {
return DbType.CACHE;
}
}
Loading
Loading