Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][Flink] Add built -in Flink History Server to reduce the state of the Unknown, the final information of the Flink task is more accurate #3776

Closed
3 tasks done
zackyoungh opened this issue Sep 3, 2024 · 0 comments · Fixed by #3780
Assignees
Labels
New Feature New feature
Milestone

Comments

@zackyoungh
Copy link
Contributor

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

Chinese

大概思路如下:

  1. Dinky 内置 Flink History Server,并默认:rs:/tmp/flink-job-archive 路径进行任务归档
  2. 当任务关闭。状态进入reconnecting(重连)时,优先进入归档目录,查看此任务jobId是否已经归档,如果已经归档,就从Flink History Server 获取最新的任务状态信息,并写入数据库

English

The general thinking is as follows:

  1. Dinky built-in Flink History Server, and defaults: RS:/TMP/FLINK-JOB-MCHIVE
  2. When the task is closed.When the state enters the reconnecting, you will give priority to the archives directory to see if the task is archived. If it is archived, get the latest task status information from the Flink History Server, and write it to the database.

Use case

Built -in Flink History Server

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
New Feature New feature
Projects
Status: Done
1 participant