msqark-tokenizer / tokenizer.json
ThingsAI's picture
BPE tokenizer 32768 vocab with thinking/tool tokens
f862712 verified
Raw
History Contribute Delete
2.31 MB
File too large to display, you can check the raw version instead.