Class: TokenGraphemeSplitter
libs/cursorless-engine/tokenGraphemeSplitter/tokenGraphemeSplitter.TokenGraphemeSplitter
Constructors
constructor
• new TokenGraphemeSplitter()
Defined in
libs/cursorless-engine/tokenGraphemeSplitter/tokenGraphemeSplitter.ts:81
Properties
algorithmChangeNotifier
• Private
algorithmChangeNotifier: Notifier
<[]>
Defined in
libs/cursorless-engine/tokenGraphemeSplitter/tokenGraphemeSplitter.ts:78
disposables
• Private
disposables: Disposable
[] = []
Defined in
libs/cursorless-engine/tokenGraphemeSplitter/tokenGraphemeSplitter.ts:77
registerAlgorithmChangeListener
• registerAlgorithmChangeListener: (listener
: Listener
<[]>) => Disposable
Type declaration
▸ (listener
): Disposable
Register to be notified when notifyListeners is called
Parameters
Name | Type | Description |
---|---|---|
listener | Listener <[]> | A function to be called when notifyListeners is called |
Returns
A function that can be called to unsubscribe from notifications
Defined in
libs/cursorless-engine/tokenGraphemeSplitter/tokenGraphemeSplitter.ts:187
tokenHatSplittingMode
• Private
tokenHatSplittingMode: TokenHatSplittingMode
Defined in
libs/cursorless-engine/tokenGraphemeSplitter/tokenGraphemeSplitter.ts:79
Methods
dispose
▸ dispose(): void
Returns
void
Defined in
libs/cursorless-engine/tokenGraphemeSplitter/tokenGraphemeSplitter.ts:190
getTokenGraphemes
▸ getTokenGraphemes(token
): Grapheme
[]
Splits {@link token} into a list of graphemes, normalised as per normalizeGrapheme.
Parameters
Name | Type | Description |
---|---|---|
token | string | The token to split |
Returns
Grapheme
[]
A list of normalised graphemes in {@link token}
Defined in
libs/cursorless-engine/tokenGraphemeSplitter/tokenGraphemeSplitter.ts:122
normalizeGrapheme
▸ normalizeGrapheme(rawGraphemeText
): string
Normalizes the grapheme {@link rawGraphemeText} based on user configuration. Proceeds as follows:
- Runs text through Unicode NFC normalization to ensure that characters that look identical are handled the same (eg whether they use combining mark or single codepoint for diacritics).
- If the grapheme is a known grapheme, returns it.
- Transforms grapheme to lowercase if
TokenHatSplittingMode.preserveCase is
false
- Returns the (possibly case-normalised) grapheme if it appears in TokenHatSplittingMode.lettersToPreserve
- Strips diacritics from the grapheme
- If the grapheme doesn't match {@link KNOWN_GRAPHEME_MATCHER}, maps the grapheme to the constant UNKNOWN, so that it can be referred to using "special", "red special", etc.
- Returns the grapheme.
Parameters
Name | Type | Description |
---|---|---|
rawGraphemeText | string | The raw grapheme text to normalise |
Returns
string
The normalised grapheme
Defined in
libs/cursorless-engine/tokenGraphemeSplitter/tokenGraphemeSplitter.ts:150
updateTokenHatSplittingMode
▸ Private
updateTokenHatSplittingMode(): void
Returns
void
Defined in
libs/cursorless-engine/tokenGraphemeSplitter/tokenGraphemeSplitter.ts:99