Skip to main content

Modelfile Reference

A Modelfile is a blueprint document containing instructions to compile, configure, and customize local models running inside GoingMerry.

1. Syntax Overview​

Instructions are case-insensitive and follow a simple layout:

# Comment
INSTRUCTION arguments

2. Instructions Reference​

FROM (Mandatory)​

Defines the base model configuration or path to a local GGUF/safetensors weight file.

  • Usage:
    • FROM <model_name> (Pull from registry)
    • FROM <path_to_gguf_file> (Read local binary)
  • Examples:
    • FROM llama4:latest
    • FROM ./models/custom-model.gguf

PARAMETER​

Sets parameters that control runtime inference behavior or model context bounds.

ParameterTypeDescription
num_ctxintMaximum context length size (e.g. num_ctx 8192).
temperaturefloatControls completion creativity (higher is more creative, e.g. temperature 0.7).
top_kintRestricts token selection to top K highest probabilities (e.g. top_k 40).
top_pfloatRestricts token selection using cumulative probabilities (e.g. top_p 0.9).
num_predictintMaximum number of output tokens to generate (e.g. num_predict 256).
stopstringDefines sequences that trigger stop conditions (e.g. stop "<│im_end│>").
num_gpuintSpecifies target GPU layers to allocate (0 forces CPU execution).
  • Example: 
    PARAMETER temperature 0.2
    PARAMETER num_ctx 16384
    PARAMETER stop "<|im_end|>"

SYSTEM​

Overrides the global system prompt to set custom instructions, persona profiles, or guardrails.

  • Usage: SYSTEM "<prompt_text>"
  • Example: 
    SYSTEM "You are a senior site reliability engineer. Be direct, technical, and precise."

TEMPLATE​

Defines the conversation structure and prompt wrapping logic to ensure model formats align.

  • Usage: TEMPLATE "<template_structure>"
  • Example: 
    TEMPLATE """
    {{- if .System }}<|im_start|>system
    {{ .System }}<|im_end|>
    {{- end }}
    {{- if .Prompt }}<|im_start|>user
    {{ .Prompt }}<|im_end|>
    {{- end }}
    <|im_start|>assistant
    """

ADAPTER​

Injects additional custom LoRA adapter weight matrices into the base model weights at load-time.

  • Usage: ADAPTER <path_to_adapter_file>
  • Example: 
    ADAPTER ./adapters/lora-finetune.bin

LICENSE​

Attaches licensing agreements or usage restrictions to the model.

  • Usage: LICENSE "<license_text>"

3. Creating a Custom Model: Step-by-Step​

Step A: Write the Modelfile​

Create a file named Modelfile on your working directory:

FROM llama4:8b

# Tune inference parameters
PARAMETER temperature 0.85
PARAMETER num_ctx 4096

# Override the system behavior
SYSTEM "You are an assistant that replies exclusively in concise, clean markdown bullet points."

Step B: Compile the Model​

Run the compile step via the CLI, choosing a name for your custom model configuration tag:

merry create bullet-assistant -f ./Modelfile

Step C: Execute​

Test the customized assistant immediately:

merry run bullet-assistant "Why is Rust memory-safe?"