[ot][spam][crazy] automated influence/etc steps
a simple step pattern seems it would be to train a model on human data i have a lot of terminal logs. i could run these through a model training script to get a part. it’s just a part, just a symbol. it’s still helpful. it would probably be helpful to write a script to do this. maybe i could use an adapter with bloomz, although the extra complexity of the adapter seems difficult. i could use a shell on this ipad to reach a server i am burning money on, to do this.
bloom has inhibition in adapter transformers regardless a further step is data. the huggingface training scripts with their reduced inhibition likely take data as line or jsonlines format (not sure). if this is true, likely the thing to do is convert the data to jsonlines so as to preserve linebreaks. for encoder/decoder models the format is input/output whereas decoder only models have only input. reasonable next step is to learn the data format of these scripts, and maybe come up with a way to provide data either architecture could read.
big inhib! the nice thing with t5 is there are mt5 and t0pp and flan-t5 pretrained models that have been trained on a lot of tokens, not sure if they are in adapterhub. i found the newer hf scripts can train decoder-only models on pure text with linebreaks whereas t5 models may take jsonlines of the format {“translation”:{“key1”:”text1”,”key2”:”text2”}} where key1 and key2 are specified at launch time
i’m thinking i could go through terminal logs using one as training and the next as validation finetuning the same model or adapter
participants (1)
-
Undescribed Horrific Abuse, One Victim & Survivor of Many