Handle morph-type tokens: filtering / persistence

[myieye]

Describe the feature
Morph-Types have leading and trailing tokens that make an entry's morph type more visible.
E.g. The headword of a prefix should typically end with a dash.

In FieldWorks

The tokens:

• are not part of either the lexeme form or the citation form. They are only part of the "calculated" headword.
• are only applied to the headword if there is no Citation form. I.e. a Citation form overrides those tokens:

Filtering:

• The basic search dialog seems to just strip the tokens off of the entered query and ignores them.
  That can be seen here where the dialog is showing the search results of "=test" and complains when I enter a "-" on the end, which is a token, but results in an invalid combination:

• Filtering on the headword column respects the tokens, but doesn't validate them (like the dialog is doing). I.e. it seems to do a simple string comparison:

Sorting:

• morph-tokens do not affect sorting
• Sorting is based on:
  • First: CitationForm ?? LexemeForm (without morph-tokens)
  • Then: MorphType.SecondaryOrder

Citation form quirks:

• Although sorting and homograph number grouping is based on headwords without morph-tokens, if an entry has a citation form and the user has manually added morph-tokens to the citation form, those morph-tokens do affect sorting and homograph grouping. I assume that's an intentional discrepancy, because it's not essential.

FieldWorks Lite

API payload

• Does not need to include calculated headwords with morph-tokens
• The UI needs to know how to calculate headwords as well, so we don't need to round-trip through the API in order to update the UI
• We could include headwords for the benefit of third-party callers / just for fun 🤷

Filtering:

• How FieldWorks' default search dialog strips morph-tokens feels quirky to me.
• For starters I'd say we strictly respect/include any morph-tokens the user filters by.

Persisted headwords:

• We already have calculated headwords in the FTS table. Those should be updated to include morph-tokens
• They also need to be updated when: (1) a entry's morph-type changes or (2) a morph-type's tokens change
• Sort of in preparation for custom-views (which could exclude the primary vernacular WS) I see no reason not to include all WSs in the headword column (space-separated like lexeme-form and citation-form)
• I think that we might be able to calculate headwords on the fly outside of FTS, because we have more powerful JSON features there. So, I think we should try hard to avoid persisting headwords in the Entry table