
{"id": "pZqfAnMEZxmIeGNFRwvmVBm216uoDBa6S8ALscl2NkE", "timestamp": oops, that was the first version. i meant to paste version 4 which has tweaks and contains embeddings with only 5.7 loss (mostly because i dropped extra tokens to 2) that are possibly useful for recurrence and backfeeding. the hardest part for me was training the embeddings O_O i find this qlora stuff pretty hard. atm my torch doesn't even have cuda enabled.
colab closes me out for the day after just like half an hour or something. i think it would make sense [to make/use better optimizers, but this alternative seems very hard to energize]. (one issue with better optim parts is preserving them when not really associated with this work) here is version 4 (i hope): {"id": "q8P5ypMR4-2VXxjg6qeIDY3EazLG9WYkAE8b-zk_YOQ", "timestamp": 1708381843639, "version": "1.0.0", "public": "pGFsvdSB9sxbwU5L4HD2v12DK40kzZ5N69s6WlI3Uw9pFdHMHei3n1Tv4jvZqU9yeIMGsS60MQRvfJK1AEoNYsQqk4Rciajw0_IemZdwlt4u4voDALRalrQ3NV4knOlHRY11anqV0fNhikWCsiRPukIRZrdcFfqzFr0boH8bou7DgESNvWxROOxSC149oKxJ06FQsBDaIeElBsR8qTddybvXqMagXCM9y_HNrtAoz_8LgPjQtK5LFEbXhh9PyI_GOuoHyzJUc9Sm-V9kCB4kTm-SHrPbETQnvejZBcqEHxNcDNWBv6CWjj3-0V3dFMhjM1cy14d0Lm4j0IyRLm9bHM3s0ssVDd20gjWyar-D0o6guJIrteEC7UGR-w1yvXoGuIwdfZeoSAZ_CU9FrOJfQCTDs2aLgdCNeYKXg0Rt8YZL_elZnG7utCkO78TwxbGqear_I-1dlO39CUlo13YSS6pPonioWqkzXcXh93G7BYjgUxcPJ31kLyr2wBRA4OObAYRvh-5V3TkULlmwR4Q0pV3cUeOLI94b4WhaDZDI_RIJiCXQvtGy190NqTBeVogPrrAXLFkK0E013GByHrmzZoELfSUorjK-bDk4wXxdbVqzY7KXP-NEt3Bu-woinbUf56i3DXLrYlwINYK39VUydGpcQLZ5EDCL4u_IL_iFPt0", "signature": "KSRC64vLx9Ah5eo8w9-PJRzm9dBhsyoDRR59MwPrU-NvVW0wSN0_egyOmLrVUN2Si_sxCnREbIV9M-j4PIpNLcp1tItbhney6uQIQ8QpFA-0ZjTqsoydJ-DkTUhVSsnGmALXjGBH4-BO4dGBbsi_2R5pRZzn7NerP8C9DBB6RGCZIAg_O5VAedhzb2K_WMCHVdUJ34fg5DygIeQcYaJkh9cGQvyvCQBvZjqRgHp5azYjORdNHyCGOZ7C9jMeGiMRH0Oyv_JZhcdcDxG9h7IKELlRmi_ray65fFGNic5FHSZrUMmH342wBkRGuNdv9UEs3ziQRdHjrfA3lU2ptkxfAuC_kKw09eq1V_iYPexlD5nc-4QMSW_3mTcxUzjD8j6lxIqrXkx6u_nrxNPK6ghDgtxeRsG9ffKgVg1UXZ4Ip8ZDQclhdci12LDGEvsjlG9BFLSbzkyUqomPxn-2FX7FiA2vWOSxgEJ3P2vLiFvjYK-ZvplePUks9JwXeFjf7Npr4xuwKPFl9YzxWMfowemT8EUjY2UrbWq2hg1Xv1dSLQ50DrrjmUztHkDJMdnK2J_j1j0lmG6ov6ZBqkKpRt8Jtc1_WfxU1rcNejl3MMCmm-9Zx-CMBvu1z3qnqu40Gbi6jZNlY38v2zLEep-mdT5AECkuMQFtEA3D6fzX4XPf22A", "deadlineHeight": 1371463, "block": 1371463, "validatorSignatures": [], "name": "2024-02-17-04.tar", "size": 188293120, "start_height": 1367463, "start_block": "oKOoodzrHfwZNUn10GQOck3WiEfmQjFYRcDhOrh-mGRnMNSvbyMSelGSRNpP8WqS", "depth": 3}