Novel Buds Agent
A QLoRA fine-tune of Google's Gemma 4 31B into a dedicated novel-writing agent for Novel Buds. It drives the same project tools — create characters, build outlines, draft and patch prose, search the project — and was trained end to end on a synthetic dataset distilled from public-domain novels by a multi-agent data pipeline.
Built for fiction, not for STEM
Modern frontier models are tuned for science and coding, and it shows when they write fiction: they slip on continuity — a character with no glasses suddenly pushing them up — and they can't convincingly play a character who doesn't know something. A novel agent has a harder brief than a chatbot: stay perfectly consistent with everything established so far while still inventing what comes next, and use a toolbox of project actions to do it.
So rather than lean on a general-purpose model, I fine-tuned one for the job. Gemma 4 31B is the sweet spot — large enough to wield the agent's tools well, small enough that an individual can actually fine-tune and run it on rented cloud GPUs.
Turning novels into agent training data
Public-domain corpus
The raw material is a curated Project Gutenberg fiction collection (organised by Despina), so every novel is copyright-free and safe to train on.
Structured extraction
OpenAI and Gemini, held behind typed Pydantic schemas, read each novel and pull out its characters, locations, lore, outline and chapter structure as a single structured project.
Orchestrator + sub-agents
Processing novels one by one would take forever, so an orchestrator dispatches parallel sub-agents — one novel each — coordinated through a shared [ ] / [-] / [x] progress checklist so hundreds run concurrently.
Render, then fill
Over forty Jinja2 scenario templates render the projects into conversation skeletons, but templates can't cover every case — so a second orchestrator pass sends sub-agents to fill the placeholders with long thinking blocks, in-voice prose and realistic tool results.
Fine-tuning a 31B model on one rented GPU
QLoRA on Gemma 4 31B
An Unsloth 4-bit base with LoRA adapters (rank 16, alpha 16) on every attention and MLP projection, across both language and vision layers — trained for a single epoch with train-on-completions and 8-bit AdamW.
A custom chat template
Google's official template is built for serving and throws away the thinking content, and Unsloth's mis-renders function calls — so I wrote a renderer from scratch that emits Gemma 4's turn / channel / tool token stream with thinking, tool calls and tool responses intact, expanding multi-turn samples one per user turn.
Long context, heavy VRAM
Novels are long, so training used a large context window — the main driver of its appetite for VRAM. It ran on an RTX PRO 6000 (about $2/hour on RunPod), roughly 45 hours for a single epoch.
Unsloth Core over the GUI
Unsloth Studio's beta GUI lacked fine-grained control over the tuning parameters, so I dropped to Unsloth's Python Core library. One epoch was enough — the loss curve stayed stable the whole way.
What it's built on
- Base model
- Gemma 4 31B (instruction-tuned)Unsloth 4-bit (bnb)
- Fine-tuning
- QLoRA / PEFTLoRA r16 · α16Unsloth Coreadamw_8bit1 epoch
- Data synthesis
- Project Gutenberg (Despina)OpenAI + GeminiPydantic schemasJinja2 templatesorchestrator + sub-agents
- Tooling
- custom Gemma 4 chat templateHuggingFace HubGGUF (llama.cpp)
- Infra
- RunPodRTX PRO 6000 Blackwell
Built solo end to end — the public-domain corpus, the multi-agent data factory, a custom Gemma 4 training format, the QLoRA fine-tune on rented GPUs, and the published model in three forms.