kylebrodeur/microfactory-node-lora

Microfactory Node: 3D Printer (LoRA v1 — historical)

This was the first fine-tune attempt. It failed, and that failure taught me what not to do. I keep it here as a historical artifact and a reminder.

What went wrong

I trained a LoRA on google/gemma-3-1b-it with rank 16 for three epochs on deterministic targets. The result parroted the same settings template for every input — it memorized, it did not judge.

Training (for the record)

Parameter	Value
Base model	google/gemma-3-1b-it
Method	LoRA (PEFT)
Rank	r=16, α=32
Epochs	3
Learning rate	2e-4
Dataset	Deterministic targets (single template)
GPU	NVIDIA A10G (24GB)
Framework	TRL SFTTrainer + transformers

Lessons learned

1. High rank + many epochs + deterministic targets = parrot. The model had too much capacity and too little variety. It learned one answer and repeated it.
2. Noisy targets force judgment. v2 switched to temperature=0.7, top_p=0.95 during dataset generation so the model cannot memorize a single template.
3. Low rank, single epoch. v2 used r=4 for one epoch. Less capacity, less memorization, more attention to the actual job.
4. Base model matters. gemma-3-1b was too small for the task. v2 moved to gemma-4-E4B-it (~4B effective).

Do not use this adapter

Use microfactory-node-lora-v2 or microfactory-node-lora-v3-qat instead. This one is here for the paper trail.

License

This adapter inherits the Gemma license from its base model.