More special characters

[jonthegeek]

I had this in a todo.R file just hanging out, and figured I should log it so I don't lose the thought:

# Something like this should go into remove_diacritics. Look for more examples
# in wiktionary.
dplyr::mutate(
  text,
  word = stringr::str_replace_all(.data$word, "æ", "ae"),
  word = stringr::str_replace_all(.data$word, "œ", "oe"),
  word = stringr::str_replace_all(.data$word, "ð", "th"),
  word = stringr::str_replace_all(.data$word, "þ", "th"),
  word = stringr::str_replace_all(.data$word, "ø", "o"),
  word = stringr::str_replace_all(.data$word, intToUtf8(305), "i"),
  word = stringr::str_replace_all(.data$word, intToUtf8(273), "d"),
  word = stringr::str_replace_all(.data$word, intToUtf8(322), "gh"),
  word = stringr::str_replace_all(.data$word, intToUtf8(945), "a"),
  word = stringr::str_replace_all(.data$word, intToUtf8(178), "2"),
  word = stringr::str_replace_all(.data$word, intToUtf8(8322), "2"),
  word = stringr::str_replace_all(.data$word, intToUtf8(8323), "3"),
  word = stringr::str_replace_all(.data$word, intToUtf8(8324), "4"),
  word = stringr::str_replace_all(.data$word, intToUtf8(8326), "6"),
  word = stringr::str_replace_all(.data$word, intToUtf8(8310), "6"),
  word = stringr::str_replace_all(.data$word, intToUtf8(8211), "-")

)

I'd love to find a more standardized way to deal with this.

[jonthegeek]

textclean does a lot of this stuff, but a) not all of it, and b) it has some big imports. But:

> rlang::set_names(
+   textclean::replace_non_ascii(to_replace),
+   to_replace
+ )
   æ    œ    ð    þ    ø    ı    đ    ł    α    ²    ₂    ₃    ₄    ₆    ⁶    – 
"ae" "oe"  "d" "th"  "o"  "i"  "d"  "l"   ""   ""   ""   ""   ""   ""   ""  "-"

The license is GPL-2, though, so I don't think I'll even look at their code, to avoid entanglements.