Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fateure](stream-load) Added direct load via BENODE #168

Closed
wants to merge 0 commits into from

Conversation

MYiYang
Copy link
Contributor

@MYiYang MYiYang commented Dec 5, 2023

changes

  1. Added the mode of displaying BENODES configuration for stream-load operation.
  2. Configure doris.benodes. The doris.fenodes configuration needs to be deleted.
  3. Such as 0.0.0.0:8040, X.X.X.X: 8040.

@@ -98,6 +98,8 @@ public class DorisStreamLoad implements Serializable {
private static final long cacheExpireTimeout = 4 * 60;
private final LoadingCache<String, List<BackendV2.BackendRowV2>> cache;
private final String fenodes;
//添加BENODES
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

English comments

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok , let me delete that

Comment on lines 417 to 415
if (autoRedirect && null == beNodes) {
return RestService.randomEndpoint(fenodes, LOG);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to use autoRedirect directly to judge here? There is no need to add benodes=null logic, otherwise the configuration will be confusing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok let me think about this one

import static org.apache.doris.spark.cfg.ConfigurationOptions.DORIS_TABLET_SIZE_DEFAULT;
import static org.apache.doris.spark.cfg.ConfigurationOptions.DORIS_TABLET_SIZE_MIN;
import static org.apache.doris.spark.cfg.ConfigurationOptions.DORIS_TABLE_IDENTIFIER;
import static org.apache.doris.spark.cfg.ConfigurationOptions.*;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove useless imports

@MYiYang MYiYang closed this Dec 14, 2023
@MYiYang MYiYang force-pushed the feature-yangmuyi-beNodes branch from 5588b7a to 3c723a5 Compare December 14, 2023 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants