June 2022 - cypherpunks - lists.cpunks.org

[data science] EleutherAI
by Undiscussed Horrific Abuse, One Victim of Many 22 Jun '22

22 Jun '22

1 1

[data science] EleutherAI Model Access
by Undiscussed Horrific Abuse, One Victim of Many 22 Jun '22

22 Jun '22

Announcement in eleutherai's discord from today. Since it's topical with my other recent posts. -- We believe that providing widespread and public access to language models like GPT-NeoX-20B is an important step to take to promote research on both the capabilities and the limitations of language models. We know that many people lack the hardware to run a 20B model, but we are excited to announce that CoreWeave and Anlatan are powering a free online demo at https://20b.eleuther.ai/ that supports queries to GPT-NeoX-20B (and GPT-J-6B) with up to 256 tokens of context through their GooseAI platform. As a reminder, if you would like to be doing open source research using our models but lack the computational resources to do so we would love to talk to you about it! Support for both inference and limited finetuning is available. If this interests you, please fill out the following form https://forms.gle/W7AKuP2WzNVVjfFj9.

1 0

The Feds
by Gunnar Larson 22 Jun '22

22 Jun '22

Can't the paparazzi (the feds) just let me innovative?

4 5

This 1.5TB microSD is surely witchcraft
by jim bell 22 Jun '22

22 Jun '22

This 1.5TB microSD is surely witchcraft https://share.newsbreak.com/1bhubzb6

2 1

Timing Attacks
by zeynep＠keemail.me 21 Jun '22

21 Jun '22

https://timing.attacks.cr.yp.to/ Timing attacks extract your confidential data from observations of the time used by your computer. Timing attacks have been demonstrated again and again to successfully extract complete cryptographic keys and other secrets. This web site, timing.attacks.cr.yp.to, is aimed at end users, operating-system distributors, CPU manufacturers, and software engineers who would like to know what they can do to protect data against timing attacks. AuthorsDaniel J. Bernstein has coauthored more than 50 attack papers on a wide range of cryptographic topics, including the influential paper "Cache-timing attacks on AES", but he works primarily on protecting systems against attack. His fast high-security Curve25519 and ChaCha20 cryptographic algorithms are now used every day by billions of people. These algorithms were designed from the outset to help systems engineers stop timing attacks. Billy Bob Brumley works primarily on attacking systems using timing as a side channel, protection being a by-product. On the offense side, he coinvented / codiscovered / codeveloped the first cache-timing attack on OpenSSL ECCthe first cache-timing attack on OpenSSL DSAthe first remote timing attack on OpenSSL ECC, CVE-2011-1945other side-channel vulnerabilities in OpenSSL, e.g., CVE-2018-0737side-channel vulnerabilities in Google's BoringSSL (and Chrome), e.g., CVE-2016-2178side-channel vulnerabilities in LibreSSL, e.g., CVE-2016-7056side-channel vulnerabilities in ARM's Mbed TLS, e.g., CVE-2019-18222, CVE-2020-10932side-channel vulnerabilities in Mozilla's NSS (and Firefox, Thunderbird), e.g., CVE-2020-6829, CVE-2020-12399, CVE-2020-12400, CVE-2020-12401, CVE-2020-12402side-channel vulnerabilities in wolfSSL, e.g., CVE-2020-11735the first bug attack in the wild, CVE-2011-4354various side-channel attack vectors, e.g., Cache Storage, Certified Side Channels, CVE-2019-1547various microarchitecture attack techniques, e.g., PortSmash, CVE-2018-5407, Degrade, HyperDegradeOn the defense side, his implementations of cryptographic algorithms are now used every day by billions of people, including (but not limited to) his free and open-source software (FOSS) contributions to OpenSSL and Mozilla's NSS / Firefox / Thunderbird.

1 0

Hertzbleed Is A New CPU Hack Affecting Just About Everybody
by jim bell 21 Jun '22

21 Jun '22

Jim Bell's note: In late 1977, I built a single board microprocessor trainer called a Dyna-Micro. https://www.google.com/search?q=%22dyna+micro%22&oq=%22dyna+micro%22&aqs=ch… At the time, I noticed that if I turned on the AM radio, running the computer caused varying bleeps and bloops to be received. Probably then and before, owners of primitive PCs like Altair noticed the same thing. A few years later, I first heard of the idea called tempest, the practice of shielding computers to avoid transmitting information by radio. Even later, in the early 2000s, I read a (even then, old) book by ex MI-5 person Peter wright, called Spycatcher, that described how they could remotely determine what radio station a radio was receiving, by detecting its local oscillator's frequency. So-called heterodyne radios work by generating a local oscillator frequency, then nonlinearly mixing that with the signal from the antenna, and then filtering the difference through an IF (intermediate frequency) filter. https://www.google.com/search?q=%22dyna+micro%22&oq=%22dyna+micro%22&aqs=ch… >From that: 'Wright examines the techniques of intelligence services, exposes their ethics, notably their "eleventh commandment", "Thou shalt not get caught." He described many MI5 electronic technologies (some of which he developed), for instance, allowing clever spying into rooms, and identifying the frequency to which a superhet receiver is tuned. " -----------HertzBleed Is A New CPU Hack Affecting Just About Everybody https://share.newsbreak.com/1ay58kwf Unlike more traditional ways to hack information, side-channel attacks rely on these signatures to try to infer what information was being processed. You can think of it kind of like guessing your presents before your actual birthday: a stereotypical “hacker” would think of ever-more sneaky ways to simply open the wrapping paper, but someone using a side-channel attack would be giving it a shake, feeling the edges, and estimating the weight. Hertzbleed is not by any means the first such attack to be discovered – side-channel attacks have been around for more than two decades at this point – it has a few extra capabilities that haven’t been seen before. It can be deployed remotely, making it much easier to use than previous side-channel attacks, and it also works on “constant time” mechanisms – that is, code specifically designed to eliminate one of the biggest clues for a would-be hacker, the length of time a process takes to complete. And the really bad news is, you’re almost certainly affected. Certainly, all Intel processors are susceptible to Hertzbleed, as are dozens of AMD chips. And even if your personal computer, laptop, tablet or phone doesn’t use those affected processors, thousands of servers across the planet do – servers which, as a matter of course, store your data, process your information, and run the services we depend on every day.

2 1

Collapse of SOL governance forecast
by professor rat 21 Jun '22

21 Jun '22

Time for $SOL to die. Its now or in a few years when the fallout will be much much worse https://twitter.com/FatManTerra/status/1538448035885240321 Repunts not horsesnorts

2 1

[ot][spam][journal] last few days of ML work for me
by Undiscussed Horrific Abuse, One Victim of Many 21 Jun '22

21 Jun '22

this is a snapshot of a channel I made in a discord called 'Land of AI'. I'm an honorary moderator in the discord, but never really come in. I used to spam the chatbots to help myself move through my issues. I made this channel in my subsection so as to work on a machine learning model I saw in online feeds. I've matured a little in my spamming. Rather than sending flurries of stuff, I edited my posts throughout hours, shortening and consolidating the information. Maybe that won't stay my behavior, who knows. But others sometimes express value in what I say, so I tried to make it more organised. This is what it looks like now: baffo32 — 06/18/2022 an independent researcher from china shared a model in #research in eleutherai that downscales and has unlimited context. it's an rnn that reuses state like gptb. they made a demo that infers clientside in a web browser on a phone. i'm thinking of seeing what would be needed to make it into a text adventure engine. it could retain consistent room and map state and save/load games, across unlimited game size, if finetuned to. it of course like all these models would be useful for many other wonderful and scary things, and likely will be, as others are. it's also small enough i can do more on my low-end systems. the system i was building, i broke the motherboard before i got it to work. the old version of the model is at https://github.com/BlinkDL/RWKV-LM and the newer version is at https://github.com/BlinkDL/RWKV-v2-RNN-Pile . the main repo is more organised for reusing but the internal layout is a little different, and appears to be missing a number of optimizations, not certain, and sugar for things like tokenization. the pile-trained model has two modes, an RNN mode and a GPT-like mode which I believe processes context faster because it doesn't have to predict the next state, not sure. They developed with the RNN mode, so it is more put-together, but it's slower to read a prompt. You can cache prompts by storing the state generated after. This is the run step of the RNN model from the second repo (src.model.RWKV_RNN.run). It produces the next state from the preceding one. I'm adding comments to describe each step. # ctx is provided as a list of tokens def run(self, ctx): # self.w is just an object to hold all the weights w = self.w # ignore all tokens but the last, and convert it to embeddings x = w.emb.weight[ctx[-1]] # pass the values through every layer # every step updates the state, held as self.xx, self.aa, and self.bb for i in range(n_layer): x = self.LN(x, w.blocks[i].ln1) x = x + self.SA(x, w.blocks[i].att, f'att.{i}') x = self.LN(x, w.blocks[i].ln2) x = x + self.FF(x, w.blocks[i].ffn, f'ffn.{i}') # i'm guessing these final weights are used only for producing vocab logits # this doesn't appear to update the state, the conversion to words x = self.LN(x, w.ln_out) x = w.head.weight @ x # looking at the example, it appears these are log probs that can be softmaxed or # argmaxed to pick the most likely next token # convert from tensor to python list before returning x = x.tolist() return x baffo32 — 06/18/2022 he states that the GPT model can be used to quickly produce the states, but given they are held in a different manner it isn't immediately clear to me how to do that. looking at the training code, it appears the GPT model is used for training (providing for more batched processing), and that explains its increased complexity. baffo32 — 06/18/2022 draft rnn demo script git clone https://github.com/BlinkDL/RWKV-v2-RNN-Pile cd RWKV-v2-RNN-Pile wget https://github.com/BlinkDL/RWKV-v2-RNN-Pile/releases/download/20220615-1080… unzip 20220615-10803.zip # all-10803.pth python3 test.py #!/usr/bin/env python3 import sys from src.model import RWKV_RNN model = RWKV_RNN(MODEL_NAME='all-10803') # loads contents of all-10803.pth tokenizer = model.tokenizer def process(token_id): # output token token = tokenizer.decode(token_id) sys.stdout.write(token) sys.stdout.flush() # pass through the model logits_list = model.run([token_id]) # use max(range,key) as an argmax for python list to do greedy sampling token_id = max(range(tokenizer.vocab_size), key=lambda token_id: logits_list[token_id]) return token_id # read context (prompt) print('Enter the context, terminate with EOF (^D):') token_id = 0 for next_token_id in tokenizer.encode(sys.stdin.read()): process(token_id) token_id = next_token_id # generate while True: token_id = process(token_id) example use, or hit ctrl-D to terminate context input echo -n 'Once upon a time,' | python3 test.py baffo32 — 06/18/2022 So, something you could do with this, is have it e.g. process an entire encyclopedia, and then save the state (.xx, .aa, .bb) for reuse later. It would also need to be finetuned to care about information that far apart: i dunno, but i'm guessing it would work to finetune it while it reads it if it's long enough. the model would also need to be large enough to comprehend the kind and variety of information you want. i tried this locally on a file and it tends to just repeat its last completion still, with basic pattern changes. i don't really know what i'm doing. i think the researcher says in their repo that you need to finetune it for longer contexts. anyway, next i imagine is gpt, maybe training [:party:] i made a model run <discord owner> — 06/18/2022 @baffo32 thank you that is awesome thanks for sharing your findings [ed: i'm kind of awkward here, these are the first things said in months] baffo32 — 06/18/2022 notes on finding the states in the faster GPT form: - states are stored like torch parameters, indexed by name associated with "att" (TimeMix) and "ffn" (ChannelMix). - xx is the input logits passed to each named layer - a and b in the RNN form are wkv and wk in the GPT form. aa and bb, the token-to-token-state, roughly coincide with A[t] and B[t] in the math docs, but the RNN code is a better guide; it is consistent with the GPT code. they're the time-weighted coefficients without the first_time coefficient which is only for output. - the aa and bb state outputs are only used internally in TimeMix's conv1d, and must be separately calculated if output, and summed into the conv1d if input. when this is done, the GPT model can be used as output or input with the recurrent model, and can train rapidly for infinite contexts in a recurrent manner (might need time decay tweaking to really be infinite, dunno). - gpt->rnn, in TimeMix.forward: aa = (torch.exp(torch.exp(self.time_decay) * self.time_curve)[None,:,-(T):-1] * kv[:,:,:-1]).sum(dim=-1) + kv[:,:,-1] bb = (torch.exp(torch.exp(self.time_decay) * self.time_curve)[None,:,-(T):-1] * k[:,:,:-1]).sum(dim=-1) + k[:,:,-1] - rnn->gpt, not quite figured yet. i found this before but have not reconstructed it. above took all day. i had to reread this stuff over and over and over again. - modified the time_shifts so as to shift in xx baffo32 — 06/19/2022 - simplification of gpt->rnn: bb = (torch.exp(torch.exp(self.time_decay) * self.time_curve)[None,:,-(T):-1] * k[:,:,:-1]).sum(dim=-1) + k[:,:,-1] bb = (wk[...,-1] - torch.exp(self.time_first[:,0]) * k[...,-1]) * torch.exp(-torch.exp(self.time_decay[:,0])) + k[...,-1] bb = (wk[...,-1] - w[...,0,-1] * k[...,-1]) * w[...,0,-3] + k[...,-1] aa = (wkv[...,-1] - w[...,0,-1] * kv[...,-1]) * w[...,0,-3] + kv[...,-1] - rnn->gpt: # replace both time_shifts with: torch.cat([xx[...,None,:], x], dim=-2)[...,:-1,:] # then after kv is calculated k = torch.cat([bb[...,None],k], dim=-1) kv = torch.cat([aa[...,None],kv], dim=-1) # uses of T need to be engaged to handle the increased kv sizes, and wk and wkv shrunk to remove the extra value from the start - when doing this, it was quite helpful to set torch.set_printoptions(precision=16); this works even in the debugger, and shows inaccuracies much earlier - fork draft: https://github.com/xloem/RWKV-v2-RNN-Pile/commit/0e3e8a4c262855c7381a4678f8… - it turns out that there are a number of parallel implementations of the model. the one i updated is the one used for inference, not training. oops. - the model has a context length limit on it that is pending applying some tricks to the cuda kernel used in training. the documentation of the tricks is vague, although derivable; says contact author to contribute. comments in src/model_train.py - another work was linked in #research, https://magenta.tensorflow.org/perceiver-ar baffo32 — Yesterday at 7:18 AM - I'm chatting a little with the model dev and the other researchers a little bit in eleutherai #research. there's an existing script for validating model output; i had made my own, and should switch to the existing one. - other works: Transformer-XL (recurrent gpt sequence training), ERNIE (recursive layer to develop complex state), sentence-transformers (memory lookup, has a function to do this, mentioned months ago on this discord). integreting the concepts of the first two could be more familiar to me, since i've already been thinking on the topics. - I'm working a little on implementing the removal of the clamp() call from this rwkv model i've been journaling about, the presence of which is limiting context length. The optimization in question appears to be a usual one done in the implementation of softmax (haven't verified): k = exp(k - max(k)) instead of k = exp(k) to prevent floating point overflow in exp. this becomes equivalent to rwkv = r*(wkv*exp(-max(k)))/(wk*exp(-max(k)) which factors out on its own. - I tried preventing floating point errors by setting the mantissa of the maximum to 1/2 (binary 1000000...) as such: # break into mantissa, exponent max_k = torch.frexp(max_k) # reconstruct with mantissa of 0.5 so no need of rounding when used max_k = torch.ldexp(torch.tensor(0.5,device=k.device), max_k.exponent) to get same output as original model when the torch.exp(k - max_k) change is made; EDIT: not sure this is doing anything yet, another error remains. - I've found I can preserve accuracy across recurrent runs enough to see success by calculating this_aa = last_aa * torch.exp(last_max_k - this_max_k) to shift max_k's; there might be a better solution here that preserves more accuracy. - with those two it seems to work correctly for a single gpt recurrence. next: bugs, optimization. baffo32 — Yesterday at 8:27 AM [ - i'm losing it and may choose to not pursue completion of this work, unsure. i don't want to start spamming #research in a psychotic state of mind. - the frexp snippet above might help other models and projects if they aren't aware of it yet; the calculation it affects is within every softmax operation. - i have a multi-hour appointment today ] - i somehow resolved some errors via fudging while troubleshooting issues, and found a cool thing around 15:00UTC; the model retains memories much better with the max_k fix, and can form longer/more-accurate completions from its training data it could not form before the change, generalising a little as the dev expected it to in their claims of value to it - shared my max_k solution in #research around 17:25 UTC - spent a long time trying to push it through normalised tests after response in #research to that effect. i'm relatievly sure it gets the exact same scores as the original model - regarding divide-by-zero, i'm thinking on how the space is a little comparable to the max_k space; both numerator and denominator have been already scaled by max_k. hence adding the epsilon to the denominator, which it is currently doing, changes the results a little - torch has a nan_to_num function that can replace nan with zero: https://pytorch.org/docs/stable/generated/torch.nan_to_num.html - 01:00 UTC, i've made a branch in my fork called 'longctx' that contains a draft of code for training on data with unbounded length, using gpt-recurrence and the max_k/clamp and epsilon removal improvements baffo32 — Today at 8:24 AM - petering off. the dev invited me to their telegram chat. not much activity there. i asked about the huggingface integration work, which would be more productive for me to engage. no reply yet. earlier, they said the branch i am working on is the one to share changes in. - when i set up telegram, it thought i was somebody from taiwan, maybe a reused phone number. i didn't understand and people were talking to me like my name was something else. (i have visual cortex issues and didn't notice all the chat rooms were different from mine). i messaged telegram support (no reply, although they read the message), and switched accounts, a bit of scuffling, deleted a message i posted from the other account. left things in a bit of a confused state. - i got my branch running inference correctly using the cuda training accelerations, and added scripts. haven't tested training itself yet, nor decided on how to test it. [they have a handmade cuda kernel they use for training with] [could put shell example here of cloning my branch, compressing a prompt to a short state file, and completing from the state file using gpt, gpt-cuda, and rnn, all with same results] - maybe first work on verifying numerical accuracy of gpt-cuda, if following through with the plan to

1 1

Collapse of UK govt
by professor rat 21 Jun '22

21 Jun '22

Dying downward from the head - Northern Ireland again - the King is dead - long live the Queen - bad news for Conservatives comes in battalions https://cointelegraph.com/news/hm-treasury-changes-course-on-collecting-dat… Reposts not endorsements

1 0

CryptoPunk Sale Raises $100K to Support Ukraine War Effort
by professor rat 21 Jun '22

21 Jun '22

And in related news there is a deadpool on Nazbol-skunks that keep attacking this list. https://decrypt.co/103396/cryptopunk-sale-raises-100k-in-ethereum-to-suppor… The only good Argenti...Russian, at the moment, is a dead one

1 0