Tokenizers
Tokenizers for Japanese text analysis provided by the kuromoji plugin.
Installation
bash
npm install @dynamosearch/plugin-analysis-kuromojiAvailable Tokenizers
KuromojiTokenizer
Japanese morphological analyzer using Kuromoji.
Best for: Japanese text, proper word segmentation for Japanese, Japanese search applications