[PATCH] Allow removing words from the personal dictionary, that weren't previously added

Fri Nov 4 11:07:32 UTC 2022

Am 25.10.2022 um 09:19 schrieb Jürgen Spitzmüller <jspitzm at gmail.com>:
> 
> I'd like this nice feature not to be forgotten.
> 
> Stephan, do you think the problems on Mac can be resolved? Could we add
> the patch with Mac support disabled for the time being?

I’ll review the patch again and post a modified version here.

Thanks for the reminder.

Stephan

> 
> Thanks,
> Jürgen
> 
> Am Montag, dem 30.05.2022 um 06:39 +0000 schrieb Isaac Oscar Gariano:
>> Dear Stephen,
>> Sorry for the delay, I had other pressing work to do.
>> Thank you so much for looking at my patch!
>> 
>> What exactly was the problem on MacOSX? Where you getting compilation
>> errors?
>> 
>> I just noticed the following line in src/AppleSpellChecker.cpp:
>>> status == SPELL_CHECK_LEARNED ? LEARNED_WORD : WORD_OK ;
>> Which I should've changed to:
>>           WORD_OK ;
>> 
>> If you can try and compile it and test it for me that would be great!
>> Although I'll try getting myself a Mac VM so I can do so myself.
>> 
>> The way the text is passed in chunks should hopefully not be a
>> problem,
>> the main problem I see is what happens when you call this
>> method https://developer.apple.com/documentation/appkit/nsspellchecke
>> r/1525147-unlearnword  when you haven't previously
>> called https://developer.apple.com/documentation/appkit/nsspellchecke
>> r/1534837-learnword
>> 
>> The documentation dosen't explain sadly.
>> If my code doesn't work, I could always write a post-processing loop
>> to check for unlearned words, (which will be slower).
>> In fact that's pretty much what I made the Hunspell/Aspell backends
>> do.
>> 
>> I should probably do some performance testing, but I've been using
>> this patch for a while now and haven't noticed any extra lag in spell
>> checking.
>> 
>> 
>> 
>> — Isaac Oscar Gariano
>> From: Stephan Witt <st.witt at gmx.net>
>> Sent: Monday, 16 May 2022 2:07 AM
>> To: Isaac Oscar Gariano <IsaacOscar at live.com.au>
>> Cc: lyx-devel at lists.lyx.org <lyx-devel at lists.lyx.org>
>> Subject: Re: [PATCH] Allow removing words from the personal
>> dictionary, that weren't previously added
>>  
>> Am 26.04.2022 um 09:27 schrieb Isaac Oscar Gariano
>> <IsaacOscar at live.com.au>:
>>> 
>>> I've made the "Remove from personal dictionary" function/context
>>> menu item work for words that haven't been previously added, but
>>> instead are in the system-wide dictionary.
>>> I've modified the behaviour so that it will cause the word to be
>>> marked as incorrectly spelt, regardless of whether it is in the
>>> user's personal dictionary, or the system-wide dictionary.
>>> I've attached two patches, one that can be applied on the latest
>>> release branch 2.3.6.1, and the other works on the master branch)
>>> 
>>> Previously, this only worked for words not in the system-wide
>>> dictionary, e.g.:
>>>         • write "ello", it will be red squiggly underlined
>>>         • right click and go "Add to personal dictionary"
>>>         • now the underline will disappear
>>>         • right click it and go "Remove from personal dictionary"
>>>         • it will now be red underlined again
>>> Now you can also do:
>>>         • write "hello"
>>>         • right click on it, and click "Remove from personal
>>> dictionary"
>>>         • there should now be a red squiggly underline under
>>> "hello", i.e. it is no longer considered a word
>>> (The above also works with the corresponding lyx-functions
>>> spelling-add and spelling-remove)
>>> It might be better to change the text of the context menu button,
>>> e.g., "Add to personal bad words list" or something, but then it'd
>>> need to be translated
>>> 
>>> 
>>> The main use cases I have for this feature are:
>>>         • removing rare words that are common misspellings, e.g.,
>>> if you often write "whet" instead of "wet", you can mark the former
>>> as invalid.
>>>         • ensure you consistently use the same variant of a word ,
>>> e.g., make "spelled" an error if you prefer "spelt" (the latter
>>> being a word in the en_GB dictionary)
>>> This change is backwards compatible: any words you had previously
>>> added to the personal dictionary will still be recognised.
>>> 
>>> I have fully tested this on Linux (specifically OpenSUSE
>>> Tumbleweed, with Aspell v0.60.8, Enchant v2.2.15, and, Hunspell
>>> 1.7.0), and Windows 11 (using the included Hunspell v1.6.2), I
>>> don't have a Mac so I can't test the AppleSpeller/Native backend.
>>> 
>>> How it works:
>>> 
>>>         • Enchant already supports this feature out of the box, so
>>> I didn't need to change the backend at all, all I needed to do was
>>> to make the "Remove from personal dictionary" context menu button
>>> show up for all correctly spelt words (and not just those in the
>>> personal dictionary).
>>> Specifically, enchant uses two files ~/.config./enchant/<lang>.dic
>>> and ~/.config/enchant/<lang>.exc to store the personal dictionary.
>>> The former contains all words that you have clicked "Add to
>>> personal dictionary", and the latter uses all that you have clicked
>>> "Remove from personal dictionary" for.
>>> Note that words are added to .dic and .exc even if unnecessary
>>> (because the word is in the system-wide dictionary, or not in it,
>>> respectively).
>>> The ".exc" file appears to take precedence over .dic, so if a word
>>> is in both, it is considered misspelt.
>>>         • Aspell and Hunspell currently uses a file
>>> $LYX_USERDIR/pwl_<lang>.dict to store words you have clicked "Add
>>> to personal dictionary" for. LyX now also uses
>>> the$LYX_USERDIR/pwl_<lang>.excl file for words you have "Remove
>>> from personal dictionary". For consistency, my code treats these
>>> files like the enchant .dic and .exc files above, specifically
>>> the .excl file takes precedence of the .dict file, and words may
>>> be added to these files even if redundant/unnecessary.
>>> The Hunspell backend already natively supports removing words from
>>> the dictionary at runtime, however Aspell does not, so after spell
>>> checking a word, my code manually checks for it's presence in the
>>> .excl list, which I have not optimised at all, and so it is an
>>> O(n) operation, where n is the number of words in the .excl file.
>>> AppleSpeller/Native: this may work out of the box like Enchant, or
>>> not. I have no idea, as I can't test it and the documentation is
>>> unhelpful
>>> (e.g.https://developer.apple.com/documentation/appkit/nsspellchecke
>>> r/1525147-unlearnword)
>>> Note: I have deleted LEARNED_WORD from the SpellChecker::Result
>>> enum, as the code no longer distinguishes between words in the
>>> personal dictionary and the base dictionary. I also removed the
>>> ROOT_FOUND, COMPOUND_WORD, and IGNORED_WORD variants as they
>>> were never used.
>>> 
>>> For the master branch, I haven't modified the "Remove from document
>>> dictionary" option to also work with words not in said dictionary;
>>> I'd have to modify the LyX file format to support a
>>> \spellchecker_reject or something like, but I can probably work
>>> out how to do that if you're happy with the idea.
>> 
>> Hi Isaac,
>> 
>> thank you for your work on LyX.
>> 
>> I’ve tried your patch and unfortunately it doesn’t work on Mac.
>> 
>> Despite the fact that it relies on the existence of the LEARNED_WORD
>> in the SpellChecker::Result enum there is IMO a problem with the
>> general idea of the solution with the Apple speller. Here the whole
>> paragraph is passed as a big chunk to the speller engine to get a
>> reasonable performance.
>> 
>> Best regards,
>> Stephan
> 
> -- 
> Jürgen
> -- 
> lyx-devel mailing list
> lyx-devel at lists.lyx.org
> http://lists.lyx.org/mailman/listinfo/lyx-devel