[spam] [personal] perceiver model notes

Undiscussed Horrific Abuse, One Victim & Survivor of gmkarl at gmail.com
Wed Jan 26 15:20:50 PST 2022


these are my incomplete model permutation notes, for inside the
attention implementations.  each axis is labeled with an einsum
letter.

        chunked:
                queries: ...qhd
                keys: ...khd
                values: ...vhd
                mask: ...hqk -> ...qhk
                scores: ...qhk
        unchunked:
                queries: .hqd
                keys: .hkd
                values: .hvd
                mask: .
                scores: .


More information about the cypherpunks mailing list