UnicodeDecodeError: 'charmap' codec can't decode byte 0x98 in position 2574: character maps to <undefined>

[wapebira]

YAML specification itself does not support encodings like CP-1252 or CP-1251.
When trying to read YAML files with Unicode characters using Windows' default encoding (cp1252).
Enforce utf-8 when reading yaml files

[e-p-armstrong]

Could you provide a traceback and maybe your config, and your input files if they're not private?

Thought we ironed out all the dang decoding issues, guess not

Either way if you're desperate all the issues have historically seemed to be on Windows, so if you have another machine you can use that and it should work

[wapebira]

I did not kept full traceback. This is a partial example I was still able to find.
file.write(json.dumps(item, ensure_ascii=False) + "\n") gives this error return codecs.charmap_encode(input,self.errors,encoding_table)[0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ UnicodeEncodeError: 'charmap' codec can't encode character '\u03c1' in position 2034: character maps to <undefined>

To fix it, all the file handling (read and write) got added the encoding. For example, with open(save_path_file, "r", encoding='utf-8')