Skip to main content

Troubleshooting & Diagnostics

If you encounter issues with model execution, GPU allocation, or network bindings, follow this debugging guide.

1. Accessing Application Logs​

Logs are vital for diagnosing kernel compiling issues or driver mismatches.

Linux​

If running as a systemd service:

journalctl -u merry.service -e --no-pager

If running manually, logs are printed directly to stdout/stderr.

macOS​

Logs are captured by the OS service manager. View them at:

tail -n 100 ~/.merry/logs/server.log

Windows​

Check the application logs directory:

%LOCALAPPDATA%\GoingMerry\server.log

2. Port Conflict: "address already in use"​

By default, the server binds to port 11434. If this port is occupied by another local service:

listen tcp 127.0.0.1:11434: bind: address already in use

Solution​

  1. Identify the process occupying the port: 
    sudo lsof -i :11434
  2. Stop the conflicting service, or tell GoingMerry to listen on an alternate port: 
    export MERRY_HOST=127.0.0.1:11435
    merry serve

3. GPU Allocation Fallback (Running on CPU only)​

If a model runs slow or the log indicates a fallback:

falling back to CPU execution: no compatible graphics cards detected

Diagnostics Checklist​

  1. Driver Validation: Ensure your system graphics drivers match the library expectations:
    • For NVIDIA: Run nvidia-smi to check driver and CUDA compatibility.
    • For AMD: Check rocm-smi to confirm library configuration.
  2. Container Boundaries: If running in Docker, verify runtime hooks are passed:
    • Must run with --gpus all (for NVIDIA) or container device passes (for AMD).
  3. Environment Flags: Make sure you haven't explicitly disabled GPU layers inside a custom Modelfile: 
    # Ensure num_gpu is not explicitly set to 0
    PARAMETER num_gpu 99

4. VRAM Overflows and Crashes​

If your machine crashes or hangs when loading large model parameters (e.g., deepseek-v3 or 70B models), you are likely running out of physical GPU memory:

Solutions​

  • Quantization: Load a smaller quantization variant (e.g., llama4:8b-instruct-q4_K_M instead of the FP16 weights).
  • Offload Control: Force a subset of layer tensors to run on standard system CPU/RAM, leaving the rest on the GPU:
    • Create a custom Modelfile and manually specify the parameter: 
      # Map only 15 layers to GPU, standardizing the rest on system RAM
      PARAMETER num_gpu 15