Claude Prompt Caching: When to Use the 1-Hour TTL Instead of the 5-Minute Default
When your code sends repeated requests to Claude through the Anthropic Messages API, prompt caching lets Claude reuse the processed portion of…
If you want to run a local large language model on an Intel Arc GPU, llama.cpp offers three software paths, called backends: SYCL, Vulkan, and OpenVINO. Each backend uses a different software stack to run model calculations on the GPU. Choose a starting point based on what matters most to you: Intel-specific support, portability, or Intel’s broader inference toolkit.
SYCL is Intel’s compute programming model and the most direct starting point for Arc. Vulkan is a portable graphics and compute interface, worth testing if you already use Vulkan. OpenVINO is Intel’s inference toolkit. It runs llama.cpp models on Intel CPUs, GPUs, and neural processing units (NPUs), with more limits on supported operations and models. No comparable benchmark establishes a universal speed winner among the three.
llama.cpp uses GGUF, a model-file format designed for local inference. A quantized model stores its weights in a lower-precision format to reduce memory use. All three backends can run GGUF models, but the backend alone does not determine whether a model fits: GPU memory, shared system memory, and the amount of context you request also matter.
| Backend | A good starting point when… | Check before relying on it |
|---|---|---|
| SYCL | You want the Intel-oriented path for Arc and are comfortable installing Intel oneAPI tools. | Confirm that SYCL detects the GPU and that llama.cpp offloads the expected layers. |
| Vulkan | You want a portable compute backend or already have a working Vulkan stack. | GPU detection does not establish how fast a model will run. |
| OpenVINO | You want Intel’s inference runtime to target a CPU, GPU, or NPU. | Check the current model and operation support, and expect the first inference to take longer. |
For a discrete Arc card, start with SYCL if you want the backend with the clearest Intel-specific guidance. Try Vulkan when its simpler build path or portability fits your setup. Choose OpenVINO when its runtime and device options suit your deployment, after checking its current limitations. The llama.cpp build guide covers the Vulkan and SYCL build options; the OpenVINO backend guide documents its device and model constraints.
Pick a GGUF file with a quantization level that fits your available memory and intended context length. The context length sets how many tokens, or pieces of text, the model can consider at once. A larger context uses more memory, so begin with a modest value and raise it only when the task requires more room.
Discrete Arc GPUs have their own video memory. Integrated graphics share system memory with the operating system and other programs. For a Llama 2 7B Q4 model on an integrated GPU, Intel’s example guidance recommends at least 4.5 GB of shared memory and 16 GB of system RAM. These figures apply to that configuration and are not general requirements. If you are choosing among models, see which local LLMs fit on a 16 GB GPU.
The build commands below are for Linux. Package names and installation steps vary by distribution. llama.cpp’s master documentation changes over time, so record the exact llama.cpp commit and driver and toolkit versions you use if you need repeatable results.
SYCL is a C++ programming model for running computations across different kinds of processors. llama.cpp’s SYCL backend uses Intel’s oneAPI compiler and runtime. Its documented support includes Intel Arc discrete and integrated graphics, alongside other Intel GPUs. The SYCL backend guide describes its dependencies and device selection.
First, make sure Linux can access the GPU. The SYCL guide recommends adding the user to the render and video groups, then checking that the device appears in the SYCL device listing. After installing oneAPI, source its environment script and run sycl-ls. If the Arc GPU does not appear there, resolve the driver or device-access issue before troubleshooting llama.cpp.
Build llama.cpp with a separate directory for this backend:
source /opt/intel/oneapi/setvars.sh
cmake -B build-sycl \
-DGGML_SYCL=ON \
-DCMAKE_C_COMPILER=icx \
-DCMAKE_CXX_COMPILER=icpx \
-DGGML_SYCL_F16=ON
cmake --build build-sycl --config Release -j
GGML_SYCL_F16=ON enables the backend’s FP16 code path. FP16 can affect speed and results differently depending on the GPU and model, so compare it with a build or run that uses FP32 when accuracy or performance matters.
List the devices visible to the build, then run a model:
./build-sycl/bin/llama-ls-sycl-device
./build-sycl/bin/llama-completion \
-m models/model.gguf \
-p "Write a short introduction to local inference." \
-n 100 -ngl 99 -sm none -mg 0
Here, -ngl 99 asks llama.cpp to offload up to 99 model layers to the GPU. It does not guarantee that every layer fits in GPU memory. Check the startup log to see which device llama.cpp selected and how many layers it actually offloaded. The guide also documents ONEAPI_DEVICE_SELECTOR for selecting a Level Zero device, Intel’s low-level GPU interface. For multi-GPU setups, it describes layer and tensor splitting; its current notes say tensor-parallel optimization targets two GPUs.
Vulkan is a graphics and compute interface with a shared set of commands that lets software use supported GPUs. llama.cpp enables its Vulkan backend with GGML_VULKAN. On Debian or Ubuntu, the build guide lists libvulkan-dev, glslc, and spirv-headers as dependencies. Run vulkaninfo first to check whether the Vulkan stack sees the GPU.
Build in a separate directory:
cmake -B build-vulkan -DGGML_VULKAN=ON
cmake --build build-vulkan --config Release
Then run a GGUF model:
./build-vulkan/bin/llama-cli \
-m /path/to/model.gguf \
-p "Hello" \
-ngl 99
Check the runtime log for Intel GPU detection and layer offload. Enabling Vulkan in CMake confirms that llama.cpp was built with the backend; it does not confirm that the run used the GPU. The documented build path includes Intel graphics detection, but it does not establish how Vulkan performance compares with SYCL or OpenVINO on Arc. Measure it on your own hardware.
OpenVINO is Intel’s inference toolkit. llama.cpp represents the model’s calculations as a computation graph, translates that graph into OpenVINO’s format, and runs it on a selected device. You can use the same GGUF file on supported Intel CPUs, GPUs, and NPUs.
Install OpenVINO Runtime and an appropriate Intel GPU driver, then source the OpenVINO environment script before building. The project’s Linux instructions also call for build tools and OpenCL components. Build in a separate directory:
source /opt/intel/openvino/setupvars.sh
cmake -B build-openvino -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_OPENVINO=ON
cmake --build build-openvino --parallel
List available devices and select the GPU:
./build-openvino/bin/llama-cli --list-devices
export GGML_OPENVINO_DEVICE=GPU
export GGML_OPENVINO_STATEFUL_EXECUTION=1
./build-openvino/bin/llama-cli \
-m /path/to/model.gguf \
-c 1024
Use a listed device name that matches your hardware. On a machine with multiple GPUs, the OpenVINO guide documents selectors such as GPU.0 and GPU.1. An explicitly requested device that is unavailable produces an error; do not assume llama.cpp will switch to the CPU.
OpenVINO’s first inference can take longer because llama.cpp converts the computation graph at runtime. The backend also supports only a subset of GGML operations, and unsupported operations or shapes can fail during graph translation. Its documented quantization support includes Q8_0, Q4_0, and Q4_1. The list also includes Q4_K and Q4_K_M. The project says quantized-model accuracy validation and performance work remain in progress. The guide’s model validation results focus on specified test hardware and configurations, so check the current support notes before choosing a model or device.
Benchmark the same model file and llama.cpp revision across backends. Keep the context length, prompt, and output length the same. Record your Arc model and available memory. Also note your operating system, driver versions, and backend tool versions. The runtime logs should confirm that each backend selected the intended GPU and show the extent of offload.
Measure prompt processing separately from text generation, and record both the first run and later runs. That distinction matters for OpenVINO because its first inference includes graph conversion. Also record memory use and stability. Note the output quality and whether the run falls back to system memory. A comparison that changes model quantization or context length between backends cannot show which backend performed better under the same conditions.
The reviewed documentation does not provide an apples-to-apples speed comparison for SYCL, Vulkan, and OpenVINO on the same Arc GPU and model. Treat any speed ranking as specific to the hardware, software versions, and test settings behind it.
Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.
Take a look at vroni.com