dictionary· e01.org

dictionary · English Wiktionary

tokenize

Pronunciation T OW1 K AH0 N AY0 Z3 definition(s) across 1 part(s) of speech · fetched 2026-11-19

  • Source Wiktionary
  • Retrieved 2026-11-19
  • Licence CC BY-SA 4.0 (Wiktionary dump)
  • Cache cache
  • Language English
Verb3 senses
  1. To reduce to a token or set of tokens by lexical analysis.

  2. To substitute sensitive data with meaningless placeholders.

  3. To treat as a token minority.

Source
English Wiktionary
Record
tokenize
Retrieved
2026-11-19
Licence
CC BY-SA 4.0 (Wiktionary dump)
Completeness
Cached copy of the source entry; not a rewrite or a generated definition.