19 Jun
2023
19 Jun
'23
9:21 a.m.
ggml provides an opt function that performs training inside it based on the number of iterations set in the parameters. this is why the example code takes so many minutes to complete, it’s running the opt pass 10k times on a single core. at a glance it looks like the performance stats are near zero because they are disabled.
the example drops the loss of a single layer with params on the order of 1k large to something 40% of its starting value in a few minutes. to clarify the module initialization params are otoo 1k so the total
On 6/18/23, Karl Semich <0xloem@gmail.com> wrote: params might be say on the order of 10m because matrices and multiplicity and i have an i7 2.75GHz
a smidge frustrating that it blackboxes the training loop