Skip to main content

Importing Custom Models

GoingMerry allows you to import and compile custom weights (such as GGUF files or Safetensors directories) directly into local model configurations.

1. Importing GGUF Files​

If you have a pre-quantized GGUF model file:

Step A: Create the Modelfile​

Create a file named Modelfile and use the FROM instruction to point to your local GGUF weight path:

FROM ./my-model-q4.gguf

Step B: Compile the Model tag​

Run the compile step:

merry create custom-gguf-model -f ./Modelfile

Step C: Execute​

Test the new local configuration tag:

merry run custom-gguf-model

2. Importing Safetensors Directories​

If you have a base model in Hugging Face Safetensors format, you must convert it to GGUF format before compilation. GoingMerry provides helper utilities to perform conversion locally:

Step A: Run Conversion Tool​

Specify the source safetensors folder path and the target GGUF file path:

merry convert ./hf-model-dir/ ./target-model.gguf

Step B: Build from Modelfile​

Set your Modelfile source target:

FROM ./target-model.gguf
SYSTEM "You are a custom AI node."

Compile the tag:

merry create custom-hf-model -f ./Modelfile