follow PLRM p. 27 and accept various characters as indicators for a new token