Finetuning LLMs with Axolotl

1 · Lawrence Wu · May 24, 2024, 2:45 a.m.
Summary
I started Hamel Husain’s fine-tuning LLM course Mastering LLM course last week. I don’t have a ton of experience fine-tuning LLMs so I thought this would be a good way to learn. One of the examples he is using throughout the course is fine-tuning an LLM to generate Honeycomb queries. So you can turn natural language into a domain specific language. My goal was to reproduce the model he trained here. Here are the steps I took to reproduce what Hamel did: The class gave us $200 of Jarvislabs credi...