i personally do finetuning in FP16 because then the LoRA merge at the end has less rounding error.
for most models it's lossless as well (when coming from a BF16 source), it's very quick to scan the weights and make sure they're all in range if you're concerned.
Yea that's pretty related. In this case, I'm looking at full finetunes (although we don't end up actually finetuning the whole model, just the small weights!)
Those are still numerical scores and, as far as I know, essentially based on “does it solve the task correctly or not”. By “qualitative”, I meant for example some analysis of how it succeeds (e.g. quality of prose or code) or its failure modes (lack of persistence, etc.).
size doesn't really have a correlation if u look at the hallucination benchmarks. this is mostly mitigated thru post training imo. also, deepseek 4.1 is notorious for hallucinations, so kinda ironic to mention it as an example here =)
I have found it to be OK on sort of general purpose overview code stuff. At the moment it's too slow in llama.cpp on Mac for me to bother testing further.
reply