--- two.cpp 2024-08-06 20:28:36.848916229 -0400 +++ output.txt 2024-08-07 13:24:28.349998464 -0400 @@ -2,5 +2,5 @@ int main() { - std::cout << "Hi hi !" << std::endl; + std::cout << "Hi hi!"; } I had rented a server for a different reason and tried running this on it. It took 200GB of gpu ram to do this with the raw llama 70b bfloat16 weights via huggingface transformers (not sure why quite so much, but the context is huge so it could be that). You can run llama 405B at 4bit quantization in the same size. I rented a 10x 24GB gpu system on vast.ai for $3/hour. There are other providers now that may be cheaper and/or blockchain-based. Sadly presently needs a huggingface login and application to download the weights, there are also liberated copies of the model in various forms I believe. [... i'd like to upload it to arweave but my uploader is buggy and it would land corrupt atm...]