[spam] [personal] perceiver model notes

[Undiscussed Horrific Abuse, One Victim & Survivor of]

i'm working on the matrix permutations inside huggingface perceiver
and the major torch implementation of efficient attention.  i have two
scripts to call them, to step through and map the offsets. it's very
hard for me to think about the axis permutations.

scripts are attached.

perceiver_loader.py also functions as an interactive model loader for
the model generated by the line in the previous email.  using it on a
trained model, one can see that the training fits to many thousands of
numbers but still fails on rare numbers especially small numbers which
only have so many examples in the data. it also fails if the data
input format changes, such as adding the word 'and' or a hyphen.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: perceiver_loader_2.py
Type: text/x-python
Size: 5356 bytes
Desc: not available
URL: <https://lists.cpunks.org/pipermail/cypherpunks/attachments/20220126/4f9b2945/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: perceiver_loader.py
Type: text/x-python
Size: 5354 bytes
Desc: not available
URL: <https://lists.cpunks.org/pipermail/cypherpunks/attachments/20220126/4f9b2945/attachment-0001.py>