lucidrains’ most recent commit is for a medical chatgpt. note: openai’s chatgpt rlhf training approach looks incredibly slow to me. i haven’t checked if the same thing is being pursued here, https://github.com/lucidrains/medical-chatgpt Explorations into training a ChatGPT, but tailored towards primary care medicine, with the reward being able to collect patient histories in a thorough and efficient manner and come up with a differential diagnosis. May also explore to see if it can be further fine-tuned on pirated copies of Up-To-Date for specialist knowledge

https://github.com/hwchase17/langchain appears to be a python package for using and managing pretrained language models that supports both local and cloud (most of the examples are cloud) backends in a pluggable manner. (github told me jay hack forked this. it is quite popular and something similar had not previously turned up for me.) looking at projects that use this is one way to find other useful things, for example: - https://github.com/namuan/dr-doc-search uses langchain and the paid openai api to process books into trained systems that can answer questions about their content - https://github.com/hwchase17/chat-langchain uses the paid openai api to produce a chatbot that answers questions about langchain’s documentation - https://github.com/jagilley/fact-checker is script that feeds language model output back on itself so as to correct false information …

https://github.com/openai/shap-e openai released text-to-3d-mesh model publicly. size is about 5GB, code is somewhat obscure, reimplementations usually expected over the week.

TinyStories: How Small Can Language Models Be and Still Speak Coherent English? via reddit https://www.reddit.com/r/MachineLearning/comments/13j0spj/r_tiny_language_mo... https://arxiv.org/pdf/2305.07759 Language models (LMs) are powerful tools for natural language processing, but they often struggle to produce coherent and fluent text when they are small. Models with around 125M parameters such as GPT-Neo (small) or GPT-2 (small) can rarely generate coherent and consistent English text beyond a few words even after extensive training. This raises the question of whether the emergence of the ability to produce coherent English text only occurs at larger scales (with hundreds of millions of parameters or more) and complex architectures (with many layers of global attention). In this work, we introduce TinyStories, a synthetic dataset of short stories that only contain words that a typical 3 to 4-year-olds usually understand, generated by GPT-3.5 and GPT-4. We show that TinyStories can be used to train and evaluate LMs that are much smaller than the state-of-the-art models (below 10 million total parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate reasoning capabilities.