Changed tokenization to be compatible with Mailvisa 1.
commit34d3046d0d460993c17becd832fc780874445a84
authoringlorion <mailvisa@inglorion.net>
Sun, 5 Oct 2008 09:04:13 +0000 (5 11:04 +0200)
committeringlorion <mailvisa@inglorion.net>
Sun, 5 Oct 2008 09:04:13 +0000 (5 11:04 +0200)
tree5c945827a6d533c1345ba91b49360ec4765f8820
parentc57c72ef29aaad7372a174501d396d584d05e491
Changed tokenization to be compatible with Mailvisa 1.

Fixed score to actually use tokenization algorithm from tokenize.ml.
src/score.ml
src/tokenize.ml