Change to file tailing in pkg/resultstore

[applejag]

The freezing logic in pkg/resultstore is messy. If it works, that's great. But a rewrite is in order.

My idea is that we should be able to reuse the readers for the entire reading process, and let all messages get piped through the file. In other words:

• new log line comes in from pod
• write log line to file
• (wake up any sleeping readers)
• reader continues to read and publishes all log lines via pubsub, reads until reaches EOF, and then goes to sleep
• on freeze, wake up all readers and tell them to stop on EOF instead of go to sleep.

For wake up/sleep we could use channels. Ex:

func stuff() {
	sleepCh := make(chan struct{})

	// reader:
	go func() {
		for {
			// 1. read until EOF, and send all logs through pubsub
			// 2. check if should die or sleep
			<-sleepCh
		}
	}()

	// writer:
	go func() {
		for {
			// 1. wait for log line from pod
			// 2. write log line to file
			wakeUp(sleepCh)
		}
	}()
}

func wakeUp(ch chan struct{}) {
	select {
	case ch<-struct{}{}:
	default:
	}
}

Originally posted by @jilleJr in #77 (comment)