[Improvement] Use new Apache 'closer.lua' syntax for kyuubi-download to obtain engine archives

[yaooqinn]

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before asking

• I have searched in the issues and found no similar issues.

What would you like to be improved?

The current closer.lua redirector, which redirects to download spark/flink etc, from a local mirror, has a new syntax. kyuubi-download module does not work properly with it currently.

How should we improve?

No response

Are you willing to submit PR?

• Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
• No. I cannot submit a PR at this time.

[yash-gupta167]

Hey @yaooqinn, I am willing to work upon this issue, could you please help me to understand this issue and the required technologies for the same

[yaooqinn]

Hi @yash-gupta167, you can refer to the closer.lua 's doc https://infra.apache.org/release-download-pages.html#closer

[yash-gupta167]

Thank you for providing this information. It seems like the closer.lua redirector has a new syntax that is causing issues with the kyuubi-download module. To use the new syntax, you need to alter your download page to point to closer.lua and pass in the relative path to the artifact as a parameter.

For example, if your artifact is located at bar/foo/foo-5.5.1.zip relative to downloads.apache.org, then the download link would be:

http://www.apache.org/dyn/closer.lua/bar/foo/foo-5.5.1.zip

Alternatively, you can generate a direct download link using the following syntax:

http://www.apache.org/dyn/closer.lua/bar/foo/foo-5.5.1.zip?action=download

Please Let me know, if I am going correct and please do provide further procedure!

[yaooqinn]

Using a direct download link looks good to me

[yash-gupta167]

The direct link site an error:

The requested file does not exist in our mirror system or in our archives.

What shall I do Further, Please guide me through it.

[yaooqinn]

please show us the link?

[yash-gupta167]

http://www.apache.org/dyn/closer.lua/bar/foo/foo-5.5.1.zip?action=download

[yaooqinn]

bar & foo are dummy, can you use the existing projects, which are also what we need?

[sincewhen]

@yaooqinn ，I understand, just need to modify the pom.xml of the Kyuubi-download module, right? The example code is as follows ：

<execution>
    <id>download-flink-release</id>
        <goals>
		<goal>curl</goal>
        </goals>
        <phase>compile</phase>
    <configuration>
	<skip>${flink.archive.download.skip}</skip>
        <url>***.zip?action=download)</url>
    </configuration>
</execution>

[yaooqinn]

Thank you @sincewhen.

You are probably right, but I'm not sure whether the Google Downloader maven plugin supports this feature.