-
Notifications
You must be signed in to change notification settings - Fork 89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: Use the bash container and a matrix strategy for bash versions #616
base: master
Are you sure you want to change the base?
Conversation
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
.github/workflows/rust.yml
Outdated
bash_version: | ||
- "5.3" | ||
- "5.2" | ||
- "5.1" | ||
- "5.0" | ||
- "4.4" | ||
- "4.3" | ||
- "4.2" | ||
- "4.1" | ||
- "4.0" | ||
sed: | ||
- "GNU" | ||
- "BusyBox" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All bash_versions should be ignored(skipped) when BusyBox is used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All bash_versions should be ignored(skipped) when BusyBox is used.
Sholudn't. But I will fix this when enabling BusyBox test.
src/compiler.rs
Outdated
Some(command) | ||
if env::var("GITHUB_ACTIONS_BASH_CONTAINER").is_ok() { | ||
let mut command = Command::new("/usr/bin/docker"); | ||
command.args(["exec", "--workdir", "/root", "--user", "405", "bash", "bash"]); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: A test requires the test run by a non-root user.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe it should detect if running inside docker so it will works always
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've inspected it, but I am sorry that it does not make sense for me. Let me explain this.
The test I mentioned is:
amber/src/tests/stdlib/is_root.ab
Lines 1 to 6 in cba1671
import { is_root } from "std/env" | |
main { | |
if not is_root() { | |
echo "Succeeded" | |
} | |
} |
and the implementation of is_root()
is:
Lines 100 to 107 in cba1671
/// Checks if the script is running with a user with root permission. | |
pub fun is_root(): Bool { | |
if trust $ id -u $ == "0" { | |
return true | |
} | |
return false | |
} |
I think the best way to know whether the test is run as root should be just the implement of is_root()
function. So If we use the best way, the test will be:
if trust $ id -u $ == "0" and is_root() {
echo "Succeeded"
}
And it seems it tests nothing, because it tests just redundant conditions.
I think the test should be moved out of an amber script, so the best result should be like:
$ amber is_not_root.ab
I am a not root user!
$ sudo amber is_root.ab
I am a root user!
And it is out of the current test process of stdlib
s. I am not sure what should I do, so I will leave --user 405
as it.
Seems that fix also #422 |
it seems also that a test fails for busybox |
I've excluded the busybox test now because #617 is not so simple to resolve. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see anything glaring, but just questions for now. Also, bear in mind that I know very little about GitHub or CI workflows; please take opinions on the YAML file with a pinch of salt.
.github/workflows/rust.yml
Outdated
sed: | ||
- GNU | ||
bash_version: | ||
- "5.3-alpha" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we actually want to support alpha Bash versions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, when I asked "Do we have a minimum supported version of Bash?", @Ph0enixKM replied "Minimum supported I think would be 3.0." Should probably include earlier versions here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Running tests on alpha version will be helpful, if it is allowed to failure. But, iirc, GitHub Actions does not support allow-failure which exists on Travis or other CI systems. So removing alpha version is a valid option.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The alpha removed! a6e56e1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
3.1 and 3.0 fail. I've created #621, but if bash array introduced at 3.2, it seems to be hard to support the elder versions. (I cannot easily find which bash version the array is introduced)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've removed 3.1 and 3.0 from the tests again because of failures.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I think that to support those version we should focus on them but having the tests to run for those version when we are not ready is just annoying.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Along with a builtin to return the Bash version (suggested in a comment on a different PR) we should also add an annotation specifying the minimum Bash version to support a given function or main
block.
There are some change between versions of bash. (See #592 (comment))
So makes a matrix strategy.