[Proposal] Get rid of external_fts pipeline

[avamingli]

Proposers

No response

Proposal Status

Under Discussion

Abstract

Remove external_fts CI.

Motivation

The external_fts CI is suggested by me to check a explain which have Entry locus.
It has the same cases with other CI except the gp_segment_configuration is stored in ETCD, only this one, other catalogs are in postgres.
Nowadays the #294 cases is changed to use pg_class instead of gp_segment_configuration.
We have no plan to check Entry locus using gp_segment_configuration.
And, the extra calls with ETCD makes our CI slower,
According to my observation, external_fts pipeine takes about extra 30min than normal ones(planner, ORCA). An example

exteranl_fts is definitely a short board of the barrel. It takes more time and cost.

Implementation

Remove CI.

Rollout/Adoption Plan

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[edespino]

@avamingli, are you referring to getting ride of the CI build_external_fts.yml (GitHub Action workflow) pipeline?

[avamingli]

on some interval

Do you mean release pipeline before each release version?

[edespino]

Yes, that's correct. As it stands, I am not aware of any release tests being performed on the open-source side specifically for external FTS. This presents a risk because contributors from the community will have no way of knowing if they impact important features like external FTS in the absence of a dedicated validation process. I would suggest considering some form of periodic or pre-release testing to ensure that external FTS functionality remains intact, especially since it touches critical components like gp_segment_configuration and involves ETCD, which may not be covered by standard CI pipelines.

[avamingli]

Yes, we have external_fts in internal release pipeline of HashData Lightning.
We could do the same, move it to Github Release pipeline to avoid overhead during dev.

[edespino]

@avamingli That is a great idea proposing we incorporate that to a Open Source release pipeline or possibly weekly run pipeline. I am in agreement the overhead to run it for each commit is too much.