Fix utf8 state in regex

[coyove]

This PR proposes the following changes when dealing with invalid utf8 sequences:

1. Invalid utf8 code will now return 0xFFFD when regexing
2. Other utf8 operations will not crash but yield undefined results, so prior checking is required.

[tylov]

I'll merge this, but I will add recovery to back up one byte when in REJECT happens on the 2nd, 3rd or 4th byte in the codepage.