Question: How to inject a node crash?

[willahmad]

First of all, thanks a lot for this amazing educational tool. It's really a joy to use this tool for quick prototyping and testing protocols when learning.

I would like to know, if there is a way to inject node crashes as part of test suite?

For instance, in the unique-ids workload, running the test with a single node that simply increments a global variable results in all tests passing. However, in practice, if the node restarts, the generated IDs are no longer unique.

Sample code

package main

import (
	"encoding/json"

	maelstrom "github.com/jepsen-io/maelstrom/demo/go"
)

func main() {
	counter := 1
	node := maelstrom.NewNode()

	node.Handle("generate", func(msg maelstrom.Message) error {
		body := map[string]any{}
		if err := json.Unmarshal(msg.Body, &body); err != nil {
			return err
		}

		body["type"] = "generate_ok"
		body["id"] = counter
		counter += 1
		return node.Reply(msg, body)
	})

	if err := node.Run(); err != nil {
		panic(err)
	}
}

How it was run

maelstrom test -w unique-ids --bin ./unique-ids --time-limit 5
      --rate 1000 --node-count 1 --availability total --nemesis partition

Thank you

[aphyr]

There's nothing for node crashes right now. I'd like to do it, just haven't had time.

If you'd like to experiment, I'd suggest adding a jepsen.nemesis/Nemesis which talks to maelstrom.process to kill/restart nodes, and adding a schedule for kill/start operations to the generator. You'll have to work out something for per-node persistent storage, and how to turn on, off, and schedule the faults.