23 Jul
2022
23 Jul
'22
8:30 p.m.
having trouble focusing on combining the files into input. probably hesitating due to lack of knowledge of how much input the vm's gpu ram can hold. makes sense to separate the file data from the commit message data, so that an arbitrary number of files can be included similarly, it would be possible for the script that combines them to do so automatically and reliably if they were delimited in some way. this would also help the model have this needed information too. so maybe i'll change how they're generated to include delimiters. i'll look up common tokenizer delimiters.