Ix{Int}.index: Index (23637) out of range ((0,164))

[maoe]

The eventlog file uploaded by @Unselfish at haskell/ThreadScope#108 (comment) causes ghc-events inc to fail:

ghc-events: Ix{Int}.index: Index (23637) out of range ((0,164))

It was produced with GHC 8.6.5 on Windows.

[mpickering]

@maoe What does ghc-events inc show? Does it show a lot of 0: create thread 0 events?

[maoe]

Does it show a lot of 0: create thread 0 events?

No, not really.

unselfish.ghc-events-inc.txt

Unfortunately I don't have time to debug this today. Please feel free to take a look if you're interested.

[maoe]

@Unselfish I looked into https://github.com/haskell/ThreadScope/files/4376429/log4.txt briefly and found that the header of the file seemed broken.

If you look at hexdump of log4.txt, the beginning of the file looks like this:

  Offset: 0001 0203 0405 0607 0809 0A0B 0C0D 0E0F 	
00000000: 6864 7262 6865 7462 6574 6200 0000 0004  hdrbhetbetb.....
00000010: 0000 000A 0D43 7265 6174 6520 7468 7265  .....Create.thre
00000020: 6164 0000 0000 6574 6500 6574 6200 0001  ad....ete.etb...

The first row (00000000) seems fine. The problem I think is the second row (00000010). The first 4 bytes of the row is a Word32 which represents the length of the next field, which is the string describing the event, "Create thread". The value is 0A here, which is 10 in decimal. This doesn't match the length of Create thread, which is 13! Actually if you truncate the 0A from the second row with a binary editor, ghc-events successfully parses the whole eventlog header. It still fails to parse the following events though.

While I haven't looked at the other eventlogs yet, I suspect your GHC or something (storage layer or memory modules?) is not working correctly. Can you try different versions of GHC or run the same program on a different machine, and see if the issue persists?

[ttuegel]

I see this error on Linux when the eventlog'd program is run with a large environment (about 69 kb, but I suspect the magic number is 2^16 bytes).

[maoe]

@ttuegel What do you mean by a large environment?

[ttuegel]

@maoe A large process environment, the environment variables. The ProgramEnv constructor of EventInfo.

[maoe]

@ttuegel I see. You're right that a payload of an eventlog message is limited to 2^16 bytes. See postPayloadSize in the RTS. This feature was removed as of GHC 8.8.1 though.

And I suppose this issue is something different because as I explained earlier the eventlog (https://github.com/haskell/ThreadScope/files/4376429/log4.txt) has a broken event header (rather than a payload).

[mpickering]

You can also run into this overflow if you have a command line longer than 2^16.