26 Jul
2022
26 Jul
'22
5:22 p.m.
at the same time, i'll want to figure out how to make the scripts use te tokenizer i trained. they're still using the extended vanilla tokenizer. i actually trained a tokenizer on source code.