[crazy][hobby][spam] Automated Reverse Engineering

Undiscussed Horrific Abuse, One Victim of Many gmkarl at gmail.com
Wed Mar 16 07:11:12 PDT 2022


so maybe

pip3 install https://github.com/xloem/GPTb

from GPTB import GPTBLMHeadModel
from transformers.models.gpt2.configuration_gpt2 import GPT2Config
config = GPT2Config() # pass settings or you can pull the config from some
pretrained model and tweak it
config.rebias = True # additional parameter for GPTB
model = GPTBLMHeadModel(config)

model.train()
model.zero_grad()
optimizer.zero_grad()
past_hidden_states = None
past_logits = None
for batch_of_tokens in data: # shape of batch_of_tokens is (batchsize, 1)
    if past_logits is not None:
        loss = torch.nn.functional.cross_entropy(past_logits.view(-1,
vocab_size), batch_of_tokens.view(-1))
        loss.backward()
        optimizer.step()
        model.zero_grad()
        optimizer.zero_grad()
    past_logits, past_hidden_states, extra = model(batch_of_tokens,
past_hidden_states=past_hidden_states)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/html
Size: 1486 bytes
Desc: not available
URL: <https://lists.cpunks.org/pipermail/cypherpunks/attachments/20220316/997a7106/attachment.txt>


More information about the cypherpunks mailing list