Text Tokenization Algorithm
(Redirected from Tokenization algorithm)
A Text Tokenization System is a text segmentation algorithm/string segmentation algorithm that can be applied by a text tokenization system (to solve a text tokenization task).