[crazy][hobby][spam] Automated Reverse Engineering
Undiscussed Horrific Abuse, One Victim of Many
gmkarl at gmail.com
Wed Mar 16 07:11:12 PDT 2022
so maybe
pip3 install https://github.com/xloem/GPTb
from GPTB import GPTBLMHeadModel
from transformers.models.gpt2.configuration_gpt2 import GPT2Config
config = GPT2Config() # pass settings or you can pull the config from some
pretrained model and tweak it
config.rebias = True # additional parameter for GPTB
model = GPTBLMHeadModel(config)
model.train()
model.zero_grad()
optimizer.zero_grad()
past_hidden_states = None
past_logits = None
for batch_of_tokens in data: # shape of batch_of_tokens is (batchsize, 1)
if past_logits is not None:
loss = torch.nn.functional.cross_entropy(past_logits.view(-1,
vocab_size), batch_of_tokens.view(-1))
loss.backward()
optimizer.step()
model.zero_grad()
optimizer.zero_grad()
past_logits, past_hidden_states, extra = model(batch_of_tokens,
past_hidden_states=past_hidden_states)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/html
Size: 1486 bytes
Desc: not available
URL: <https://lists.cpunks.org/pipermail/cypherpunks/attachments/20220316/997a7106/attachment.txt>
More information about the cypherpunks
mailing list