feat: Make tokenizer add_special_tokens option configurable#54

njhill wants to merge 1 commit intomainIBM/text-generation-inference:mainfrom

omit_special_tokensIBM/text-generation-inference:omit_special_tokensCopy head branch name to clipboard