Creating a word-to-token dictionary for inference