Rebase b10069 - #190
Open
makaveli10 wants to merge 281 commits into
Open
Conversation
Introduce a CMAKE option for disabling Adreno-specific shaders if needed, this improves build time, but should not be used when targeting Adreno devices.
Signed-off-by: Simon Zeni <simon.zeni@collabora.com>
* Update backend filename prefix for Windows and Linux to use 'qvac-ggml-' * fix cmake exports * add macro guards to prevent dlopen when dynamic backends disabled
rebase: FLOAT_TYPE_VEC2 -> FLOAT_TYPEV2. Co-authored-by: Lubosz Sarnecki <lubosz.sarnecki@collabora.com>
Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
TODO: Repick this patch from `infinitalo:italo/tq1_0` and don't fork mul_mat_vecq.glsl. rebase: FLOAT_TYPE_VEC2 -> FLOAT_TYPEV2 Co-authored-by: Lubosz Sarnecki <lubosz.sarnecki@collabora.com>
rebase: FLOAT_TYPE_VEC2 -> FLOAT_TYPEV2. Co-authored-by: Lubosz Sarnecki <lubosz.sarnecki@collabora.com>
Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
… there is no effect on the output gradients. Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
llama-context: Align call sites with API changes. Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
…a is provided Rebase on b8828: * Fix build. Co-authored-by: Lubosz Sarnecki <lubosz.sarnecki@collabora.com>
Align call sites with API changes. Co-authored-by: Marcus Edel <marcus.edel@collabora.com> Co-authored-by: Lubosz Sarnecki <lubosz.sarnecki@collabora.com> Signed-off-by: vineet <vineet.suryan@collabora.com> Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
Signed-off-by: vineet <vineet.suryan@collabora.com>
Signed-off-by: vineet <vineet.suryan@collabora.com>
Rebase on b8828: * Fix build. Co-authored-by: Lubosz Sarnecki <lubosz.sarnecki@collabora.com> Signed-off-by: vineet <vineet.suryan@collabora.com>
Signed-off-by: vineet <vineet.suryan@collabora.com>
Signed-off-by: vineet <vineet.suryan@collabora.com>
---
b9840 rebase:
Convert F16 operands to f32 too, not just quantized ones.
The kernel funnels src0/src1 through cublasSgemm, which is f32-only. The
original predicate src0_is_quantized = (type != F32 && type != F16) let
F16 tensors pass straight through, reinterpreting their bytes as f32:
this corrupts the values and halves the leading dimension
(ldb = nb11/sizeof(float) = ne10/2).
rocBLAS rejects the bad ld with CUBLAS_STATUS_INVALID_VALUE; NVIDIA's
cuBLAS is laxer but computes garbage. Renaming to src0_needs_f32 = (type
!= F32) forces the dequantize/copy path for F16 as well, giving ne-based
f32 strides.
OUT_PROD is the MUL_MAT backward op driven by the branch's LoRA/training
path, where F16 base weights routinely reach this kernel; on AMD/RADV
(rocBLAS) that previously aborted the finetune.
Co-authored-by: Guilherme Gallo <guilherme.gallo@collabora.com>
supports_op advertised OUT_PROD for any src type once dst was F32, but the kernel
only handles src types with a ggml_get_to_fp32_cuda() dequant fn — others hit
GGML_ABORT("Unsupported quant type for src0/src1") at runtime. test-backend-ops
feeds such quant types on HIP/rocBLAS, and the abort takes down the whole process
(cascading into test-opt/test-gguf). Gate supports_op on the same predicate the
kernel actually uses so an unsupported quant OUT_PROD is reported n/a instead.
Assisted-by: Claude Opus 4.8
Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
…r issue. Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
This commit adds checkpointing for fine-tuning: - Add checkpoint saving every N steps with --checkpoint-save-steps - Save complete training state: model weights, optimizer state, metadata - Implement two-phase optimizer state loading to avoid memory issues - Add --resume-from and --auto-resume functionality - Store optimizer momentum/variance tensors in GGUF format - Add checkpoint validation for rank, alpha, and target modules - Update README.md with checkpointing documentation The optimizer state loading: iteration count is loaded during initialization, while tensor data (grad_m, grad_v) is loaded after ggml_opt_alloc creates the proper tensor structures. Rebase on b8828: - Drop .get() calls for context access. - `--auto-resume` reuses the existing `remove_arg_single(i)` helper instead of re-implementing the same argv shift-and-decrement inline. - Collapse the identical `train_callback` and `eval_callback` definitions (both the same `checkpoint_save_steps <= 0` ternary) into a single `epoch_callback`, passed twice to `llama_opt_epoch_resume`. Rebase on b9952: - check tensor type when loading optimizer state Signed-off-by: makaveli10 <vineet.suryan@collabora.com> Co-authored-by: Lubosz Sarnecki <lubosz.sarnecki@collabora.com> Co-authored-by: Guilherme Gallo <guilherme.gallo@collabora.com>
…UT_PROD datatype implementation. Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
…rieval Add get_k_lora() and get_v_lora() methods that use concatenation instead of ggml_view_4d to maintain gradient connectivity during training. This ensures LoRA K/V parameters receive proper gradients while preserving causal attention behavior. --- b9952 rebase: - simplify lora's gradient-connected KV retrieval to return k_cur/v_cur Signed-off-by: makaveli10 <vineet.suryan@collabora.com> Co-authored-by: Guilherme Gallo <guilherme.gallo@collabora.com>
added 4 commits
July 23, 2026 03:31
… cleanup) Remove the unused `metadata`, `set_tensor_data`, `set_tensor_data_ud` and `file` parameters (and their b9310-rebase `(void)` silencing casts) from `llama_model_load` and update its call site; mark the now-unused `set_tensor_data` / `set_tensor_data_ud` parameters of `llama_model_load_from_file_impl` `[[maybe_unused]]` to keep -Werror happy. Downstream fixup on top of upstream PR ggml-org#22004 ("model: move load_hparams and load_tensors to per-model definition", landed at b9019). Its target is upstream, so it can never autosquash into a downstream commit; carry it as a standalone downstream patch instead of a dangling squash!/fixup! pair.
Move common_opt_sft_dataset_init out of common.cpp into a dedicated finetune translation unit, dropping the nlohmann include from common.cpp. Unify the per-template assistant span scanners into one tag-driven loop and write samples directly into the dataset instead of staging vectors. Assisted-by: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S28Qf144zq87KN8AdbJf5z
Guard the per-device FA maps and shared kernel argument state at both entry points that can access them. Keep compilation serialization in the existing helper locks. Assisted-by: OpenAI Codex
Assisted-by: OpenAI Codex
github-actions
Bot
added
documentation
Upstream master's ubuntu-wasm job is red with the same error: github.com/ggml-org/llama.cpp/actions/runs/29361861822/job/87183687962 Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
…hing its shader layout Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
…'t read repacked layouts Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
Enable training loss scaling for mixture-of-experts architectures, also correct SGD weight decay under loss scaling, dropping the loss scale env var Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
GPUs have a limited number of fast registers (VGPRs) CI checks fail when any HIP kernel that uses more than 256 of them, unless that kernel is on a known-ok list Two kernels were over the limit (rwkv_wkv_f32 and a Q2_K matmul tile) Similar kernels were already on that list, so this commit just adds these two names too. So we also don't fail the build because of these two Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
Signed-off-by: Marcus Edel <marcus.edel@collabora.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebase tag: https://github.com/ggml-org/llama.cpp/releases/tag/b10069