i saw somebody else with a large prosthesis and again engaged some prosthesis project energy :D
i've learned more about dissociation and i took it very slowly and made very tiny moving-forward progress :}
i made 11 practices of very simple metamodel behavior (holding idea of a prosthetic limb could use a low-end chip to learn to adapt to an owner's changing signals some day) [metamodel work is the hardest part of the project design for me]
hey i'm sorry i'm not including an example yet i was just practicing some of them are very poor and some of them perform astoundingly and it's kind of random and i was navigating harsh dissociations through them all the time so i'm taking it gentle, i'm very very sorry but i'm learning to do it and the basic form is something like (PLEASE DON'T HURT IT ! it's soooo nascent !!!!) import pr_dep, torch, tqdm def main(): s = pr_dep.make_sin_arch().make_settable() # student model s_len = len(s.get()) t = pr_dep.[TBasic or TGroup](...named hyperparameters maybe scaled from s, plus output for weights and anything else of interest optim = torch.optim.AdamW(t.parameters(), lr=[something between 1e-4 and 1e-8]) test_data = torch.linspace(0, 3, [# of items], device=s.device, dtype=s.dtype) test_out = torch.sin(test_data) with tqdm.tqdm(range(num_loops)) as pbar: for it in pbar: # optional, replace a random value in test_data and test_out wts = t([optional parameters, it can synthesize with no input too if configured to]) s.set(wts) test_loss = torch.nn.function.mse_loss(s(test_data), test_out) # if the model is configured to predict its own loss, which makes it perform better, mse_loss that too and sum it with test_loss if pbar.last_print_n == it: pbar.desc = str(loss.item()) loss.backward() optim.step() optim.zero_grad() if __name__ == '__main__': main() ... so, yeah, i'm still practicing it, and i'm not sure which of my approaches were successsful and i'd want to share one that succeeds not one that fails ;p library closes in 7 minutes !! encourage me to keep trying and to have positive warm safe energy around it :) i really need the ability to continue well here <expressing some of this didn't go well :s> be well !!
attached is the dependency file i made toward the end. grad output untested but i've figured out how to make it work if it doesn't. the grid search was just to figure out what hyperparameters could make a successful sin(x) model (it was my synthesis test), i didn't do any other grid search yet
this uses a high-level class from torch.nn but i have since learned there is more now in the newer torchtune package. but its one way to abstract the idea away.
the make_settable() etc methods let one use the weights of one model in the training graph of another without crashing pytorch by letting the user specify when this is or isn't happening and what the role is
it's just been very hard for me to try any of this at all up until now