[ot][spam][crazy][data] transformer model 'attention' improvement

Undiscussed Horrific Abuse, One Victim & Survivor of Many gmkarl at gmail.com
Thu Feb 3 04:14:41 PST 2022


Appts today.

Below includes implementation of the mask and bias features as the
owner requested. Latest at top.
I still have two changes to make:
- modify if conditions i added to handle all cases, including unworkable ones
- fix bug when chunk size is not a factor of total size. existing code
is written to handlee this but has an error.lacking test coverage.

commit a3c18ed22088272bbf09bc6f5308e4fd3c1e1add (HEAD -> callbacks,
origin/callbacks)
Author: xloem <0xloem at gmail.com>
Date:   Thu Feb 3 11:52:19 2022 +0000

    add mask_calc_fn, bias_calc_fn, added and moved things to support this

commit c20b1a3ea22528acd4620445400718c399fed51c
Author: xloem <0xloem at gmail.com>
Date:   Thu Feb 3 10:35:09 2022 +0000

    rename chunk_callback to weights_calc_fn for similarity to pr comment

commit c4c72b592ab6d2aa0d04456705b5abd1cd83b7a0
Author: xloem <0xloem at gmail.com>
Date:   Thu Feb 3 10:15:11 2022 +0000

    consolidate datasets() into data() as mentioned in PR comment.
also reduced count.

commit acc0a8afb5208eec14f090c3008790b21971bd85
Author: xloem <0xloem at gmail.com>
Date:   Thu Feb 3 01:56:52 2022 +0000

    bugfix: callback was not being tested. it now passes.

commit e36dd43bb1efd1f5e650f50c9573e39b7d0027a7
Author: xloem <0xloem at gmail.com>
Date:   Thu Feb 3 00:22:38 2022 +0000

    tests pass when random data is generated with the same distribution


More information about the cypherpunks mailing list