i used colab to run it thru bloomz-3b. it still needs another example or something, but produces new content now Open In Colab In [ ]: !pip3 install adapter-transformers accelerate bitsandbytes !git clone https://github.com/xloem/apifudge !nvidia-smi !ln -vsf apifudge/* . Cloning into 'apifudge'... remote: Enumerating objects: 15, done. remote: Counting objects: 100% (15/15), done. remote: Compressing objects: 100% (9/9), done. remote: Total 15 (delta 4), reused 14 (delta 3), pack-reused 0 Unpacking objects: 100% (15/15), done. Sat Jan 7 15:11:06 2023 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 460.32.03 Driver Version: 460.32.03 CUDA Version: 11.2 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 | | N/A 52C P0 27W / 70W | 0MiB / 15109MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+ './apifudge1' -> 'apifudge/apifudge1' './test.py' -> 'apifudge/test.py' In [ ]: # use drive to cache downloads from google.colab import drive drive.mount('/content/drive') !mkdir -p /content/drive/MyDrive/colab/.cache !rm -rf In [1]: import apifudge1 as af1 import transformers npapi = af1.API('numpy') npapi.set_example( '''Construct an array B that repeats array Y 4 times along the last dimension.''', '''B = np.tile(Y, 4)''') for name, text in npapi.get_examples().items(): print(f'{name}: {repr(text)}') print('loading ...') pipeline = transformers.pipeline( 'text-generation', 'bigscience/bloomz-3b', model_kwargs=dict(device_map='auto')) generator = af1.Generator(pipeline) prompt = 'Reverse an array C.' output = generator(npapi, prompt) print(f'{prompt}: {repr(output)}') Construct an array B that repeats array Y 4 times along the last dimension.: 'B = np.tile(Y, 4)' loading ... Reverse an array C.: 'C = np.rollaxis(C, 1, axis=0)' rollaxis isn’t how to reverse an array