[spam] [personal] perceiver model notes

k gmkarl at gmail.com
Tue Jan 18 09:40:18 PST 2022


## num_self_attends_per_block ##

- number of self attention layers

configuration object is
transformers.models.perceiver.configuration_perceiver.PerceiverConfig
and it briefly documents the parameters (and likely a few more).

configs deriving from perceiverconfig can simply set attribute to add
more configs

the perceiver classes are aliased into transformers at the top level

import transformers
config = transformers.PerceiverConfig()
model = transformers.PerceiverModel(config)


More information about the cypherpunks mailing list