Set Ollama Modelfile Parameters for a Repeatable Local Evaluation
A model evaluation is easier to interpret when the base model, prompt, and sampling settings stay fixed. Use a dedicated Ollama variant with explicit parameters for the comparison.
Worked example
Sampling defaults for one evaluation variant
FROM llama3.2:latest PARAMETER seed 48 PARAMETER temperature 0.1 PARAMETER top_p 0.8 SYSTEM Summarize each test shipment in one sentence without changing the ID or item count.
Steps
- Choose an installed base model and record the tag and model ID from ollama list. Put the sample settings in a Modelfile, replacing the tag if needed.
- Run ollama create warehouse-eval -f Modelfile. Run the same synthetic prompt twice with warehouse-eval from fresh sessions and save the results.
- Change one parameter for a second named variant. Use the same prompt and compare the results using a fixed rubric, such as whether output preserves the shipment identifier and count.
What to check
- Record the Ollama version and exact input alongside the output. The same seed can help repeat a run with a fixed setup, while a model update or runtime change can alter results.
- A per-request option can override a model default. Inspect the API request during the comparison if you are calling the variant from an app.