CONTRIBUTING.md

@@ -1,14 +1,39 @@
 # Contributing
 
+## Adding A Test
+
 A person who wants to contribute a test to the project should read
 this Github guide to
 issues and [pull requests](http://help.github.com/categories/collaborating-with-issues-and-pull-requests)
 (PRs) which describes in great detail the work flow for forks and
 submitting PRs to a project.
 
-Follow the same general steps for project `github.com/perl6/roast`:
+## Determine The Purpose
+
+There may be two different purposes to contribute to this project:
+
+1. Contributors may wish to fix an error or add a test for existing functionality. The
+   latter is often needed when a bug is discovered in a compiler and there is reason
+   to cover this bug with a test.
+2. Contributors may wish to propose a change to the Raku language itself. This kind
+   of contribution should be started by submitting an issue in the
+   [problem-solving](https://github.com/Raku/problem-solving) repository. The
+   issue must be opened using the _Report a language problem_ template. If the
+   proposed change is accepted then a test to cover the new behavior is to be
+   submitted to roast.
+
+Note that in the second case if the proposal is accepted but not implemented yet,
+the test must be fudged. See _Fudged tests_ below and
+[README](https://github.com/Raku/roast)) for more details. The preferable way is
+to use the `todo` verb because in this case a tester would get notified when a
+fudged test passes due to a feature being implemented or a bug fixed.
+
+## Submission Steps
+
+Follow the same general steps for [the Raku roast](https://github.com/Raku/roast)
+project:
 
-- fork project `roast`
+- fork the project 
 - clone your fork of `roast` to a local directory
 - set the origin and upstream remotes
 - checkout a branch to work on your issue or proposal
@@ -23,10 +48,10 @@ Follow the same general steps for project `github.com/perl6/roast`:
 
 New tests for existing features are usually accomplished by adding
 the test(s) to an existing test file. Then that file's `plan` count is
-updated.  The new test(s) are tested locally by executing
+updated. The new test(s) are tested locally by executing
 
-```perl6
-$ perl6 <test file(s)>
+```
+$ raku <test file(s)>
 ```
 
 When all is well, the commits are finalized, the branch is pushed
@@ -35,25 +60,26 @@ to the user's fork on Github, and there the PR is initiated.
 If a new test file has been created, one additional step has to be
 taken: the new test file has to be added to `spectest.data`.
 (This file used to live in the Rakudo repo at `github.com/rakudo/rakudo`
-but it is part of `roast` nowadays.)
+but it is part of `roast` nowadays.) The header of `spectest.data` is to be
+carefully examined as the test may require additional marking or other kinds of
+special care.
 
 #### Fudged tests
 
-Let's say you want to propose a new feature, method `printf` for
-IO::Handle, for `rakudo` and, being a believer in test-driven
-development, are submittng some test for something that can't yet be
-tested. Thus we will need to create the test but we will `fudge` it so
-it will be ignored.
+Let's say you want to propose a new feature, method `printf` for IO::Handle and,
+being a believer in test-driven development, are submittng some test for
+something that can't yet be tested. Thus we will need to create the test but we
+will `fudge` it so it will be ignored.
 
-We create a new test file named appropriately, say, `S16-io/printf.t` ,
+We create a new test file named appropriately. Say, `S16-io/printf.t`,
 the contents of which are:
 
-```perl6
+```raku
 use v6;
 use Test;
 plan 1;
 
-my $f = './.printf-tmpfil';
+my $f = $*TMPDIR.add('.printf-tmpfil');
 my $fh = open $f, :w;
 $fh.printf("Hi\n");
 $fh.close;
@@ -64,8 +90,8 @@ unlink $f;
 
 We know the test doesn't work yet:
 
-```perl6
-$ perl6 S16-io/printf.t
+```
+$ raku S16-io/printf.t
 1..1
 No such method 'printf' for invocant of type 'IO::Handle'
   in block <unit> at ./S16-io/printf.t line 9
@@ -75,14 +101,14 @@ No such method 'printf' for invocant of type 'IO::Handle'
 
 so we add the fudge to it to get the new contents:
 
-```perl6
+```raku
 use v6;
 use Test;
 plan 1;
 
-#?rakudo skip 'RT #999999 not yet implemented'
+#?rakudo skip 'GH #999999 not yet implemented'
 {
-    my $f = './.printf-tmpfil';
+    my $f = $*TMPDIR.add('.printf-tmpfil');
     my $fh = open $f, :w;
     $fh.printf("Hi\n");
     $fh.close;
@@ -98,23 +124,22 @@ are affected by the fudge line preceding it.
 We want to test the fudged file before we submit the PR so we have to
 manually create the fudged test file:
 
-```perl6
+```
 $ fudge rakudo S16-io/printf.t
 S16-io/printf.rakudo
 ```
 
 which, as we see by the return from the command, produces file
 `S16-io/printf.rakudo` whose contents are
 
-
-```perl6
+```raku
 use v6;
 use Test;
 plan 1;
 
-#?rakudo skip 'RT #999999 not yet implemented'
-skip('RT #999999 not yet implemented', 1);# {
-#     my $f = './.printf-tmpfil';
+#?rakudo skip 'GH #999999 not yet implemented'
+skip('GH #999999 not yet implemented', 1);# {
+#     my $f = $*TMPDIR.add('.printf-tmpfil');
 #     my $fh = open $f, :w;
 #     $fh.printf("Hi\n");
 #     $fh.close;
@@ -129,18 +154,17 @@ exit(1);
 
 We can then test it:
 
-```perl6
-$ perl6 S16-io/printf.rakudo
+```
+$ raku S16-io/printf.rakudo
 1..1
-ok 1 - \# SKIP RT \#999999 not yet implemented
+ok 1 - \# SKIP GH \#999999 not yet implemented
 # FUDGED!
 ```
 
 Success! Now we can commit the new test file, but **NOT** the generated fudge
-test file&mdash;that will be generated automatically by the test
-harness during the regular testing on the servers. As
-described earlier, the new test file will have to be added to the spectest.data
-file, either via a PR or a request to someone on IRC to add it.
+test file that will be generated automatically by the test harness during the
+regular testing on the servers. As described earlier, the new test file will
+have to be added to the spectest.data file via a PR.
 
 ### Suggestions
 
@@ -149,11 +173,35 @@ file, either via a PR or a request to someone on IRC to add it.
 In general, tests for the specific wording of error messages should go into the
 implementation's test suite and not become part of the specification.
 
-- Rakudo implementation has specific directory in its suite for messages tests:
+- the Rakudo implementation has a specific directory in its suite for messages tests:
     https://github.com/rakudo/rakudo/tree/master/t/05-messages
 
 #### Exception types
 
-If the test expects a generic exception, generally you should use `Exception` and not
-`X::Comp::AdHoc`. Otherwise, if we ever make those typed exceptions, the tests would start
-to fail.
+If the test expects a generic exception, generally you should use `Exception`
+and not `X::Comp::AdHoc` unless this is _exactly_ what is expected. This is to
+handle a situation when a generic `die` is replaced with a typed exception to
+produce a more specific and manageable error. For example, say, we had:
+
+```raku
+if $bad-data {
+    die "Bad data, expected better one"
+}
+```
+
+and replace it with
+
+```raku
+if $bad-data {
+    X::Data::Bad.new.throw
+}
+```
+
+Then a test like:
+
+```raku
+throws-like { test-sub() }, X::Comp::AdHoc;
+```
+
+will start failing because `X::Data::Bad` is unlikely to inherit from
+`X::Comp::AdHoc`.

README.md

@@ -99,7 +99,7 @@ fudge comments.
 The `fudgeall` program may be called to process all the needed fudging
 for a particular implementation:
 
-```perl6
+```
 $ fudgeall rakudo */*.t */*/*.t
 ```
 

spectest.data

@@ -1,24 +1,24 @@
-# This is a list of all spec tests that are expected to pass.
+# This is a list of all spec tests that are expected to pass. Each line in the
+# list is a path to a test file relative to the roast root directory.
 #
 # Empty lines and those beginning with a # are ignored
 #
-# We intend to include *all* tests from roast, even when we
-# skip most of the tests or the entire test file. To verify
-# we are running all tests, run:
+# Each file may have one or more markers following a '#' at the end of the line.
+# Currently the following markers are supported:
 #
-# perl tools/update-passing-test-data.pl
+#     long   - skip if quick test mode is on
+#     stress - run only if stress test mode is on
+#     moar   - run tests only for MoarVM backend (Rakudo specific)
 #
-# If a file appears in the output of the script, it is not run
-# by default. It may need to be fudged in order to run successfully.
-# Open an RT when necessary as part of the fudge process, using the
-# RT in the fudge message, and then add the test file to this file, sorted.
+# Please, visit https://github.com/Raku/roast to report bugs or submit pull
+# requests. Any kind of feedback is welcomed!
 #
-# Each file may have one or more markers that deselects the test:
-#     long   - run tests unless --quick
-#     stress - run tests only if --stress
-#     moar   - run tests only for MoarVM backend
-# See the "make quicktest" and "make stresstest" targets in
-# build/Makefile.in for examples of use.
+# For proposals affecting the Raku language itself please open a ticket in
+# the https://github.com/Raku/problem-solving repository. The proposal could be
+# about things like new syntax, change of semantics of existing constructs, or
+# extending the existing functionality. You can get a better impression of the
+# purpose of the repository by browsing the documents it contains, and the issues
+# it has, both opened and closed.
 
 APPENDICES/A01-limits/misc.t
 APPENDICES/A01-limits/overflow.t