General issue to track failing tests in bash test suite

[IgorTodorovskiIBM]

Many tests are failing because there is a discrepancy in the error messages:
EDC5129I No such file or directory
vs
No such file or directory

We may need to create a platform specific *.right file for some tests

[MikeFultonDev]

hm. i did a patch for this in another repo. Now... which one?

[MikeFultonDev]

Similar patches in m4 and autoconf (see m4 and autoconf)

[MikeFultonDev]

As a temporary step, the following could be run as a 'post download' step to fix up the .right files:

find . -type f -name "*.right" -exec sed -i 's/No such file or directory/EDC5129I No such file or directory./' {} \;

An alternate would be to run the script and then generate patch files. If the comparison is 'smart' and accepts regular expressions then perhaps we can add some sort of [A-Z0-9]* type of addition (and then deal with the trailing period we decided to issue) so that we can make the 'right' file generic.

[MikeFultonDev]

Note the change 'find' above requires sedport (for the -i option)

[MikeFultonDev]

Running this change reduces the number of failures by 5 to 25.

tests/exp.output
tests/ifs.output
tests/heredoc.output
tests/posixpat.output
tests/read.output
tests/redir.output
tests/shopt.output
tests/test.output
tests/coproc.output
tests/errors.output
tests/posixexp.output
tests/printf.output
tests/procsub.output
tests/trap.output
tests/varenv.output
tests/vredir.output
tests/builtins.output
tests/exec.output
tests/history.output
tests/lastpipe.output
tests/nquote.output
tests/dynvar.output
tests/histexp.output
tests/intl.output
tests/new-exp.output

[MikeFultonDev]

heredoc.output has a similar issue with 'grep':

< grep: FSUM6003 input file "*.c": EDC5129I No such file or directory.
---
> grep: *.c: EDC5129I No such file or directory

[MikeFultonDev]

errors.output has a similar issue with 'Not a directory':

< ./errors.tests: line 193: cd: /bin/sh: EDC5135I Not a directory.
---
> ./errors.tests: line 193: cd: /bin/sh: Not a directory

[MikeFultonDev]

vredir.output has a similar issue with 'bad file descriptor':

103c101
< ./vredir8.sub: line 12: $fd: EDC5113I Bad file descriptor.
---
> ./vredir8.sub: line 12: $fd: Bad file descriptor

[MikeFultonDev]

exec.output has a similar issue with 'is a directory':

< ./execscript: line 40: /: EDC5123I Is a directory.
---
> ./execscript: line 40: /: Is a directory
16c16
< /: /: EDC5123I Is a directory.
---
> /: /: Is a directory

[MikeFultonDev]

Running the updates above to add message numbers only reduces the number of failures by 1:

tests/exp.output
tests/ifs.output
tests/heredoc.output
tests/posixpat.output
tests/read.output
tests/redir.output
tests/shopt.output
tests/test.output
tests/coproc.output
tests/posixexp.output
tests/printf.output
tests/procsub.output
tests/trap.output
tests/varenv.output
tests/vredir.output
tests/builtins.output
tests/exec.output
tests/history.output
tests/lastpipe.output
tests/nquote.output
tests/dynvar.output
tests/histexp.output
tests/intl.output
tests/new-exp.output

[MikeFultonDev]

adding diffutils, findutils, od (from coreutils), sed to set of tools and building from scratch reduces failures:

Issues opened:

• exp.tests exp.tests fails  #13
• ifs-posix.tests ifs.tests takes a long time to run (multiple minutes). validate if this is 'normal' #14
• heredoc.tests heredoc test fails with comparison #8
• read.tests read.tests : failure with exec to a named pipe #17
• redir.tests redir.tests: unable to set ulimit to required value (256) #19 redir.tests and coproc.tests expects a /etc/passwd file to exist #20
• shopt.tests shopt.tests does not print out the value of globskipopts #21
• test.tests test.tests sets up trap using numbers that aren't the same as on z/OS #22
• coproc.tests redir.tests and coproc.tests expects a /etc/passwd file to exist #20 coproc.tests no longer hanging  #23
• glob.tests glob.tests : locale zh_HK.big5hkscs not installed on z/OS system causes tests to fail #25
• exec.tests trap.tests loops through sigspec in a different order than expected #11
• procsub.tests procsub.tests prints extra blank after a number of lines causing the diff to not compare #26
• posixpat.tests posixpat.tests causes protection exception #15
• procsub.tests remove 'exit 1' from procsub.tests  #10
• trap.tests trap.tests loops through sigspec in a different order than expected #11
• history.tests history.test fails due to cat producing different results #9
• posixexp.tests posixexp.tests fails #6
• printf.tests printf.tests fails #7
• builtins.tests builtins.tests fails #12
• lastpipe.tests running lastpipe.tests and histexp.tests stand-alone works but causes broken pipe in test harness #30
• dynvar.tests dynvar.tests expects now1=$(date +%s 2>/dev/null) to work #27
• histexp.tests running lastpipe.tests and histexp.tests stand-alone works but causes broken pipe in test harness #30
• varenv.tests trap.tests loops through sigspec in a different order than expected #11 (possibly other issues, but fix this first)
• vredir.tests vredir.tests : wrong error message when setting ulimit under active descriptor #28 vredir.tests : grep is looking for '\$v: Bad' but needs to look for: '\$v: EDC5113I Bad' #29
• exec.tests trap.tests loops through sigspec in a different order than expected #11 (possibly other issues, but fix this first)
• intl.tests intl.tests: Length of UTF8 string is wrong #31 (probably other issues too)
• new-exp.tests new-exp.tests fails with string manipulation #16

tests/heredoc.output (8)
tests/read.output (17)
tests/redir.output (19, 20)
tests/test.output (22)
tests/coproc.output (20, 23) [no longer failing]
tests/glob.output (25)
tests/exec.output (11)
tests/procsub.output (26) [no longer failing]
tests/trap.output (11)
tests/varenv.output (11)
tests/vredir.output (28, 29)
tests/builtins.output (12)
tests/lastpipe.output (30)
tests/histexp.output (30)
tests/intl.output (31)
tests/new-exp.output (16)

[IgorTodorovskiIBM]

These are the new set of failures (as seen on zoscan2b):

tests/heredoc.output
tests/read.output
tests/redir.output
tests/glob.output
tests/printf.output
tests/trap.output
tests/varenv.output
tests/vredir.output
tests/exec.output
tests/builtins.output
tests/intl.output
tests/new-exp.output

Once it's reproduced on the CI machines, I will close off the issues for the newly passing tests