Skip to content

fix(perplexity): use all_special_tokens for GPT-2 tokenizer compatibility - #789

Open
akaashsa wants to merge 1 commit into
huggingface:mainfrom
akaashsa:fix/perplexity-gpt2-tokenizer
Open

fix(perplexity): use all_special_tokens for GPT-2 tokenizer compatibility#789
akaashsa wants to merge 1 commit into
huggingface:mainfrom
akaashsa:fix/perplexity-gpt2-tokenizer

fix(perplexity): use all_special_tokens for GPT-2 tokenizer compatibi…

1e217e6
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs