MohammedPreview
Open in TheBrain

To Tokenize

  1. (transitive, computing) To reduce to a set of tokens by lexical analysis.
  2. (transitive, computing) To substitute sensitive data with meaningless placeholders.
  3. (transitive) To treat as a token minority.