Method note
Evaluate LLMs against your codebook first (Halterman & Keith, Political Analysis)
A five-stage framework: prepare the codebook, test basic capabilities, evaluate zero-shot accuracy, analyze errors and consider supervised tuning, demonstrated on three political-science datasets.
What is reported
Open-weight models often failed to follow real codebooks zero-shot, while supervised instruction tuning improved them substantially.
Before you use it
Fine-tuning needs labeled examples and compute; the zero-shot results apply to the models tested.
Sources and review
Source checked · Not tested by this site
Checked the public abstract and bibliographic record. Results describe the authors' study, not a replication by this site.
License and reuse
No license has been recorded. Check the source’s license before reusing code, models or data.
A license note is not a license audit. Open source does not guarantee free hosting, model access or commercial reuse.