2022-10-06 2003+0000 I’m working on writing a test training loop. I’ve written a basic data generator, copied in some of the hyperparameters from the paper, and linked the algorithm to a toy gpt2 architecture from transformers. I’m running into strong psychological issues, which is normal for me when trying to write a training loop. 2020 i’ve drafted a training loop. i know it has missing parts i’ve glossed over from how i get when i write them. an exception is throwing in code i thought i had avoided. i’m surprised to still be working. very tangible, the harsh multidirectional internal experiences. i usually take a break when things are this intense, to help retain my memories and thoughts and feelings a little. 2027 i’ve fixed the unexpected exception, which was due to mistakes within the mutations i made to library code that look like they trigger when it is loaded. i’m confused that the tests did not catch them, but at this point the training loop will. i’m now engaging bugs in my test code. i’m doing somewhat better. 2045 i’ve fixed a number of mistakes in my training loop, and am now engaging bugs in the forward function of the algorithm. [i’m taking a break to at least find water, but still feel the multidirectional intensity and energy. i’ve pushed the current state to git.] 2100 i have water. i was working on a remote system so as to run bigger tests than my raspberry pi can hold. the internet has stopped working on the device, interrupting this. i am taking a break, i guess. 2020-10-07 0550 I’m having a sense of stronger cognitive issues than I want to provide for, and am planning to take a break for a few days. The next bug was that the causal mask did not have the right shape for the sequence length.