[user@DESKTOP-E3P7TC0 apifudge]$ pwd /home/user/src/apifudge [user@DESKTOP-E3P7TC0 apifudge]$ ls -l total 0 drwxrwxr-x. 2 user user 25 Jan 7 01:44 apifudge1 [user@DESKTOP-E3P7TC0 apifudge]$ ls -l apifudge1 total 4 -rw-rw-r--. 1 user user 213 Jan 7 01:44 __init__.py My big dream is to have code that pastes user input onto a language model prompt. The prompt is collected from previous user input.

Rather than writing general code, I've made a specific interface for the use, to make it emotionally easier and appease the great oppressions. However, I've tried to offset and normalize it a little, to make it easier to collect and generalize if the great oppressions lift. class API: def __init__(self, name, HOME = os.path.expanduser('~/.apifudge')): self.name = name self.path = os.path.join(HOME, self.name) self.examples_path = os.path.join(self.path, 'examples') os.makedirs(self.examples_path, exist_ok=True) def get_examples(self): return { fn: open(os.path.join(self.examples_path, fn)).read() for fn in os.listdir(self.examples_path) } def set_example(self, data, name = None): fn = os.path.join(self.examples_path, name) try: with open(fn + '.new', 'w') as f: f.write(data) os.rename(fn + '.new', fn) except: os.unlink(fn + '.new') raise

I tried writing a test and realized that examples have two parts, a user prompt, and the correct output. Maybe one big json or jsonlines file would be simpler.

Here's the prolog of set_example now: def set_example(self, prompt, result, name=None): if name is None: name = ''.join([ char for char in prompt.replace(' ','_').lower() if char in 'abcdefghijklmnopqrstuvwxyz0123456789_-' ]) + '.txt' Here's how far I am with a test. I thought I'd write an example that used the np.tile function: if __name__ == '__main__': import apifudge1 as af1 npapi = af1('numpy') for name, text in npapi.get_examples().items(): print(f'{name}: {repr(text)}') npapi.set_example( '''Repeat the array Y ''') I'll make a github repo to handle my urge to paste all my code in here a little.

I'm noting that the approach of using the first linebreak as a delimiter makes it hard for a user to provide relevant api data that a model should be able to infer the result from, as there is no organization associated with the file structure to expand.

I daydreamed a little about how prompt-based data augmentation might be used for many other very useful purposes. This triggered me. I likely did this habitually to prevent my implementation of the test.

I'm thinking that maybe Jay Hack has a little similarity to me. Maybe they worked hard to make simple examples with good demos, and are hoping to sneak advanced technology out to the world. Of course everybody knows it is on the tip of everyone's tongue, somebody has to do it.

We, as the community, get to step up and fill in the gap a little! If we don't, people like me get to make baby steps ;p

i pushed now i'm handling my satisfaction and excitement. i have a lot of microtriggers that respond to satisfaction in projects, roughly, by trying to terminate activity on the project. often associated with relaxation, amnesia, and building of further associated triggers during the time away.

gibberish! gibbering gibberish

We stored a single example! This single example is likely enough for many language models to demonstrate contextual utility: _if prompted with it_! They would, actually, need to be _prompted_ with the example, to produce associated output. One of the steps of prompting a language model. Giving the prompt.

Cognitive thing? Yes the one where you .. um .. avoid the task? no, there must be .. um ... here just keep going!

i'm looking into using an 8-bit quantized model, but i'm not sure if the bitsandbytes library dependency runs for me

ok ummmmmmmmmmmm i have about 2G gpu ram. so i'll want a model that fits in a little under that. if i use 16 bit floats, that means ummm 2b / 2 = <1b params if i use 32 bit floats, that means ummm 2b / 4 = <500m params maybe i'll see if bloomz-560m loads to start with

this tiny model won't perform well with only 1 example, but it will let me test my code I can also use online interfaces to test my example and generate more. Most recently I've been spamming the rwkv dev's chatbot (the model is only halfway trained), rather than using chatgpt. I could see what it thinks of my prompt.

I tried a few prompt forms that failed and this is the first one I found that worked: xloem — Today at 2:48 AM +rwkv_gen > Construct an array B that repeats array Y 4 times along the last dimension. B = np.tile(Y, 4)

Story-Bot BOT — Today at 2:48 AM Select only elements of B that correspond to elements in A where the last

index is zero and the second index is a negative number. B = B[np.arange(B.shape[0]) == A[:,:-1, :] == 0] Select only elements of B that correspond to elements of A where the last index is a positive number. B = B[np.arange(B.shape[0]) == A[:,

I prompted it to continue and it finished the line ( :,-1] == 0] ) but then lost the structure. Sometimes with rwkv changes like that are due to numerical stability across the runs. I haven't reviewed the output for correctness; I process information poorly nowadays.

i running into this error trying to run the model on my gpu (after converting to a 16-bit format so it fit): RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling `cublasCreate(handle)` I've hit this error before, and recall it's very frustrating, but not what it is or how to resolve it.

i'm getting that weird error when i run my test code, but not from within the repl. it looks as if i've made a logic error implementing it, with the error as a side effect.

the error seems to be triggered only when i provide the examples i am using the bos/eos tokens to separate them, which is not normal. it seems the most likely thing to make sense causing it. the model does not usually complete text containing eos tokens.

i think it happens when it processes more than 16 tokens

i've committed and pushed code where test.py generates a new completion. the completion is incorrect. a second example may help. i might try reviewing and pasting the example rwkv generated.

i used colab to run it thru bloomz-3b. it still needs another example or something, but produces new content now Open In Colab In [ ]: !pip3 install adapter-transformers accelerate bitsandbytes !git clone https://github.com/xloem/apifudge !nvidia-smi !ln -vsf apifudge/* . Cloning into 'apifudge'... remote: Enumerating objects: 15, done. remote: Counting objects: 100% (15/15), done. remote: Compressing objects: 100% (9/9), done. remote: Total 15 (delta 4), reused 14 (delta 3), pack-reused 0 Unpacking objects: 100% (15/15), done. Sat Jan 7 15:11:06 2023 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 460.32.03 Driver Version: 460.32.03 CUDA Version: 11.2 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 | | N/A 52C P0 27W / 70W | 0MiB / 15109MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+ './apifudge1' -> 'apifudge/apifudge1' './test.py' -> 'apifudge/test.py' In [ ]: # use drive to cache downloads from google.colab import drive drive.mount('/content/drive') !mkdir -p /content/drive/MyDrive/colab/.cache !rm -rf In [1]: import apifudge1 as af1 import transformers npapi = af1.API('numpy') npapi.set_example( '''Construct an array B that repeats array Y 4 times along the last dimension.''', '''B = np.tile(Y, 4)''') for name, text in npapi.get_examples().items(): print(f'{name}: {repr(text)}') print('loading ...') pipeline = transformers.pipeline( 'text-generation', 'bigscience/bloomz-3b', model_kwargs=dict(device_map='auto')) generator = af1.Generator(pipeline) prompt = 'Reverse an array C.' output = generator(npapi, prompt) print(f'{prompt}: {repr(output)}') Construct an array B that repeats array Y 4 times along the last dimension.: 'B = np.tile(Y, 4)' loading ... Reverse an array C.: 'C = np.rollaxis(C, 1, axis=0)' rollaxis isn’t how to reverse an array

some parts of the caching code werelost inpasting, it still runs

it's still a useful step, if this usecase collapses for now.

with 3 examples (i added one for sorting) it gives a correct answer on colab npapi.set_example( '''Construct an array B that repeats array Y 4 times along the last dimension.''', '''B = np.tile(Y, 4)''' ) npapi.set_example( '''Swap the last two axes of an array D.''', '''D = D.swapaxes(-2, -1)''' ) npapi.set_example( '''Sort an array along the third axis.''', '''array.sort(3)''' ) Reverse an array C.: 'C = np.flipud(C)' I didn't even know about the flipud function until now, which appears to default to selecting the outermost axis rather than all of them. This is likely the meaning of the word reverse.

notes: using model_kwargs=dict(device_map='auto') you can squeeze slightly larger models into a system npapi.set_example( '''Construct an array B that repeats array Y 4 times along the last dimension.''', '''B = np.tile(Y, 4)''' ) npapi.set_example( '''Swap the last two axes of an array D.''', '''D = D.swapaxes(-2, -1)''' ) npapi.set_example( '''Sort an array along the third axis.''', '''array.sort(3)''' )

here is the failed implementation i tried to make today, and the working implementation i had made some time ago. i haven't looked at them side-by-side. simple_beam_search is the one i tried to do today try_multigen is the one from some time ago, my state of mind was trying out parts for synthetic text adventure games i believe both of them use a batch size of 1. i consider it not complex to stabilize it with a batch size of 1, and then batch the top batch_size items in the priority queue.

On 1/7/23, Undescribed Horrific Abuse, One Victim & Survivor of Many <gmkarl@gmail.com> wrote:

with 3 examples (i added one for sorting) it gives a correct answer on colab

npapi.set_example( '''Construct an array B that repeats array Y 4 times along the last dimension.''', '''B = np.tile(Y, 4)''' ) npapi.set_example( '''Swap the last two axes of an array D.''', '''D = D.swapaxes(-2, -1)''' ) npapi.set_example( '''Sort an array along the third axis.''', '''array.sort(3)''' )

Reverse an array C.: 'C = np.flipud(C)'

I didn't even know about the flipud function until now, which appears to default to selecting the outermost axis rather than all of them. This is likely the meaning of the word reverse.

it is no longer producing this output, even when returning to the old commit, although other state changes remain untried. the time of the email is Sat, Jan 7, 2023 at 11:23 AM for me.

oh of course to update a dependency library you need to reset things. it has old version cached.

whew this left me confusedish! unsure if i want to continue immediately. really cool project. certainly more productive than other things. i bet finetuning is far more helpful than i downplayed it as.

doing this associated with reviewing the demo of numpy api usage, and considering making one’s own. it would come together from small parts so there are multiple small parts that could be developed note the demo uses finetuning, showing it could learn nee apis than numpy, and can perform inference more efficiently and accurately.