Skip to content

Tokenizers

Tokenizers for Japanese text analysis provided by the kuromoji plugin.

Installation

bash
npm install @dynamosearch/plugin-analysis-kuromoji

Available Tokenizers

KuromojiTokenizer

Japanese morphological analyzer using Kuromoji.

Best for: Japanese text, proper word segmentation for Japanese, Japanese search applications

Released under the MIT License.