Could I please have some help with pyspelling in a CI pipeline?

[Alex-Bujorianu]

Hi there,

I am using pyspelling in Github Actions with the following .spellcheck.yaml file:

---
spellchecker: aspell
matrix:
  - name: markdown
    aspell:
      lang: en
    sources:
      - "Chapters/*.md"
    pipeline:
      - pyspelling.filters.markdown
    dictionary:
      wordlists:
        - wordlist.txt
      output: wordlist.dic
      encoding: utf-8
    default_encoding: utf-8
    expect_match: true

In the CI pipeline I see that it is running this command: Command: ['aspell', 'list', '--encoding', 'utf-8', '--add-extra-dicts', '/github/workspace/wordlist.dic', '--lang', 'en', '--mode', 'none']. The custom dictionary seems to have some effect, but it is still showing some of the words as mis-spelt, especially those with apostrophes. The dictionary looks something like this (truncated for brevity):

Battleborn
Carthago
Dacian
Dwarvish
Fangtooth

What happens if a word is duplicated inside of the dictionary? I think it might be related to that.

Also is there any way to get it to not print the contents of each file that it is checking?

[facelessuser]

I would need a minimal reproducible example so that I could troubleshoot the issue. If you can provide that, I might be able to help. I just don't have enough context about your environment to guess what your issue is.

What happens if a word is duplicated inside of the dictionary? I think it might be related to that.

Nothing, it should work fine.

What happens if a word is duplicated inside of the dictionary? I think it might be related to that.

There are a couple levels of verbose. You can add increasing -v. One v is verbose, two vs is more verbose, etc.

[Alex-Bujorianu]

I would need a minimal reproducible example so that I could troubleshoot the issue. If you can provide that, I might be able to help. I just don't have enough context about your environment to guess what your issue is.

What happens if a word is duplicated inside of the dictionary? I think it might be related to that.

Nothing, it should work fine.

Yes, it seems you're right, the problem is with the apostrophes. In the minimal example, attached, the word Rahash does not appear in the list of misspelled words, although Rahash’s (with an apostrophe) does, even though it's in the dictionary.

Edit: For some reason, the word Imuşay gets flagged up in as misspelled even when it's in the dictionary.

There are a couple levels of verbose. You can add increasing -v. One v is verbose, two vs is more verbose, etc.

Edit: it turns out that the HTML you see below was actually added by https://github.com/rojopolis/spellcheck-github-actions not pyspelling. I configured my CI to use pyspelling directly (installing all the dependencies and stuff) and now I get much more manageable output.

pyspell-repro.zip

[facelessuser]

Aspell doesn't actually have intelligent handling for possessive words. You basically have two options when dealing with possessive words.

1. Add the possessive form to the dictionary.

2. Create a filter that will ignore the possessive part of the word. By ignoring 's, the word will be treated as a non-possessive word.

     - pyspelling.filters.context:
         context_visible_first: true
         delimiters:
         # Ignore possessive endings
         - open: '(?<=\w)''s(?!\w)'
           close: '\b'

[facelessuser]

@Alex-Bujorianu Did I answer your questions? Do you have others?

[Alex-Bujorianu]

Hi Isaac, yes, thanks for answering my question. Adding the possessive form to the dictionary did not make any difference (at least not with ’ instead of '). I will try using the regex filter and get back to you. Edit: the following code is parsed but makes no difference, the possessive forms show up as mis-spelled.

- pyspelling.filters.context:
          context_visible_first: true
          delimiters:
          # Ignore possessive endings
          - open: '(?<=\w)’’s(?!\w)'
            close: '\b'

The difference seems to be the use of the ’ character instead of '.

[facelessuser]

Keep in mind that I used two ' only to escape it in a string that used '. If you are using curly quotes, you can use just one:

    - pyspelling.filters.context:
          context_visible_first: true
          delimiters:
          # Ignore possessive endings
          - open: '(?<=\w)’s(?!\w)'
            close: '\b'

[Alex-Bujorianu]

Thanks very much, that filtered out a lot of output and actually helped me catch a typo! I will close this PR now. It may be helpful to include the example in the documentation, as filtering the possessive form is a very common use-case, especially in a book.

[facelessuser]

It may be helpful to include the example in the documentation, as filtering the possessive form is a very common use-case, especially in a book.

Yeah, this may be a good idea. It is easier than duplicating a word with possessive forms in the dictionary, and this question surfaces in different places now and then.