Word Explorer

Tokenize

/ˈtoʊ.kən.aɪz/

To reduce to a set of tokens by lexical analysis.

Full Dictionary Entry

📖 Definitions of "Tokenize"

verb
  1. 1

    To reduce to a set of tokens by lexical analysis.

  2. 2

    To substitute sensitive data with meaningless placeholders.

  3. 3

    To treat as a token minority.

Translate “Tokenize” into Another Language

Pick a language — the word will be pre-filled in the translator.

SpanishEspañolFrenchFrançaisGermanDeutschChinese中文ArabicالعربيةPortuguesePortuguêsRussianРусскийJapanese日本語Korean한국어ItalianItalianoHindiहिन्दीTurkishTürkçeDutchNederlandsIndonesianBahasa IndonesiaVietnameseTiếng Việt
Open translator with all 254+ languages →
Home/Dictionary/Tokenize