6 Apr
2023
6 Apr
'23
5:13 p.m.
Thank you for the pointer. I will look it up! =) On Thu, 6 Apr 2023, Undescribed Horrific Abuse, One Victim & Survivor of Many wrote:
On 4/6/23, efc@swisscows.email <efc@swisscows.email> wrote:
I run alpaca.cpp on a laptop with 8 GB ram and the 7B model. Works pretty well.
I would love to find a project that would enable me to go to the 13B model, but have not yet found one that enables me to run that on only 8 GB ram.
4 bit quantization and mmap?
i’m not using them myself yet but people are doing these things