GPU Acceleration & Environment Configuration

GoingMerry is built to execute bare-metal compilation targets that offload mathematical computations directly to available system GPU hardware interfaces.

GPU Platforms

1. NVIDIA (CUDA)

On systems with CUDA support, GoingMerry automatically detects compute capabilities and compiles GPU execution kernels. If memory constraints permit, GoingMerry loads the complete layer stack into VRAM.

To verify CUDA utilization:
```
nvidia-smi
```

2. Apple Silicon (Metal / MPS)

GoingMerry communicates directly with Apple Metal interfaces on macOS, ensuring that weights are sharded across unified VRAM boundaries without memory copy overhead.

Allocations are optimized automatically based on available system memory headroom.

3. AMD (ROCm)

For AMD graphics targets, ROCm drivers are loaded dynamically to map inference operations.

Environment Variables Reference

Configure the behavior of the GoingMerry server daemon using the following system-level variables:

Variable	Default	Description
`MERRY_HOST`	`127.0.0.1:11434`	The IP address and port the daemon binds to. Set to `0.0.0.0:11434` to enable remote access.
`MERRY_MODELS`	Path to `~/.merry/models`	The path where model weight blobs and configurations are stored.
`MERRY_NUM_PARALLEL`	`1`	Controls the number of parallel requests allowed to execute concurrently.
`MERRY_FLASH_ATTENTION`	`true`	Toggles FlashAttention hooks. Disable (`false`) only for legacy GPUs experiencing driver crashes.
`MERRY_KEEP_ALIVE`	`5m`	The duration models remain loaded in VRAM after the last request finishes.

Advanced Configurations

Setting Environment Variables in Linux (systemd)

If you manage the daemon using systemd, configure environment overrides:

Edit the service override:
```
sudo systemctl edit merry.service
```

Add the variables under the [Service] block:

[Service]
Environment="MERRY_HOST=0.0.0.0:11434"
Environment="MERRY_NUM_PARALLEL=4"
Environment="MERRY_MODELS=/mnt/fast_nvme/models"

Restart the daemon:

sudo systemctl daemon-reload
sudo systemctl restart merry

Running Multiple Models Concurrently

To serve multiple model configurations concurrently, increase both the parallel request bounds and keep-alive durations:

export MERRY_NUM_PARALLEL=4
export MERRY_KEEP_ALIVE=2h
merry serve

When you make API calls to different model tags, GoingMerry splits your GPU layers to accommodate multiple models simultaneously, provided there is enough physical VRAM headroom.

GPU Platforms​

1. NVIDIA (CUDA)​

2. Apple Silicon (Metal / MPS)​

3. AMD (ROCm)​

Environment Variables Reference​

Advanced Configurations​

Setting Environment Variables in Linux (systemd)​

Running Multiple Models Concurrently​