[ml] langchain runs local model officially

efc at swisscows.email efc at swisscows.email
Thu Apr 6 10:13:32 PDT 2023

Thank you for the pointer. I will look it up! =)

On Thu, 6 Apr 2023, Undescribed Horrific Abuse, One Victim & Survivor of Many wrote:

> On 4/6/23, efc at swisscows.email <efc at swisscows.email> wrote:
>> I run alpaca.cpp on a laptop with 8 GB ram and the 7B model. Works pretty
>> well.
>> I would love to find a project that would enable me to go to the 13B
>> model, but have not yet found one that enables me to run that on only 8 GB
>> ram.
> 4 bit quantization and mmap?
> i’m not using them myself yet but people are doing these things

More information about the cypherpunks mailing list