mac_ai remote_ramalama_run_bench

Benchmark a model with ramalama, on a remote host

Parameters

base_work_dir

  • The base directory where to store things

path

  • The path to the llama.cpp bin directory

model_name

  • The name of the model to use

env

  • The env values to set before running ramalama

ngl

  • Number of layers to store in VRAM

  • default value: 99

device

  • Name of the device to pass to the container