handle \u200d delimited emoji

[lsmith77]

Currently 👨🏽‍👩🏽‍👧🏽 is handled as multiple tokens. Note this likely relate to carpedm20/emoji#204

Ideally it would be handled as a single token.

[bdura]

Hello @lsmith77, the issue comes from the tokenizer defaults. It so happens that 👨 is one of the icons that are looked for during tokenization.

Given the root of the issue, I don't think we'll be able to much about it in the near future... However, you could customize your tokenizer to avoid splitting the emoji.

[lsmith77]

ah bummer .. thx for the feedback though.

[adrianeboyd]

I'll close this for now because it looks like it isn't a bug in spacymoji itself.