[spam] [personal] perceiver model notes
k
gmkarl at gmail.com
Thu Jan 20 14:52:33 PST 2022
I wrote up some about reviewing google's masked language modeling
implementation, but lost the writing during a spasm, despite saving
drafts.
Anyway, I think the most useful approach will be to port google's
example pretrained model to pytorch, and see how it runs. I glanced
at huggingface's test for this use case, and the test doesn't actually
verify the output data yet, so they could have a bug.
Google's example model is at
https://storage.googleapis.com/perceiver_io/language_perceiver_io_bytes.pickle
I guess I'll build haiku for arm64, since the pickle uses haiku objects.
More information about the cypherpunks
mailing list