Poor metrics on a new training run using same dataset as an older run using same parameters

[billdraper]

Search before asking

• I have searched the HUB issues and discussions and found no similar questions.

Question

I trained a dataset today and the results were unexpectedly very poor. As an experiment to see if there is an issue with my dataset I re-trained an older dataset using the same parameters as I used when I trained from that same dataset in November, 2025 and receive dramatically different results. So, same dataset, same parameters, producing dramatically different model metrics from November 2025 and January 2026. I've attached screen snapshots of the 2 training session metrics and it includes the parameters used.

Has something significant changed with the training tools on UL Hub recently? Or am I missing something obvious? Thank you for your help.

Additional

No response

[UltralyticsAssistant]

👋 Hello @billdraper, thank you for raising an issue about Ultralytics HUB 🚀! Please visit our HUB Docs to learn more:

• Quickstart. Start training and deploying YOLO models with HUB in seconds.
• Datasets: Preparing and Uploading. Learn how to prepare and upload your datasets to HUB in YOLO format.
• Projects: Creating and Managing. Group your models into projects for improved organization.
• Models: Training and Exporting. Train YOLOv5 and YOLOv8 models on your custom datasets and export them to various formats for deployment.
• Integrations. Explore different integration options for your trained models, such as TensorFlow, ONNX, OpenVINO, CoreML, and PaddlePaddle.
• Ultralytics HUB App. Learn about the Ultralytics App for iOS and Android, which allows you to run models directly on your mobile device.
  • iOS. Learn about YOLO CoreML models accelerated on Apple's Neural Engine on iPhones and iPads.
  • Android. Explore TFLite acceleration on mobile devices.
• Inference API. Understand how to use the Inference API for running your trained models in the cloud to generate predictions.
  Thanks for the detailed context and the screenshots 📸—this is helpful. This is an automated response 🤖, and an Ultralytics engineer will also assist soon.
  If this is a 🐛 Bug Report, please provide screenshots and steps to reproduce your problem to help us get started working on a fix. In particular, since this involves unexpectedly different metrics across runs, please include a minimum reproducible example (MRE) if possible (the smallest dataset + exact settings needed to reproduce the behavior).
  If this is a ❓ Question, please provide as much information as possible, including dataset, model, environment details etc. so that we might provide the most helpful response.
  We try to respond to all issues as promptly as possible. Thank you for your patience! 🙏

[pderrenger]

Thanks for the detailed screenshots @billdraper — with the same dataset + “same parameters”, you generally shouldn’t see a dramatic metric swing between a November 2025 run and your Jan 16, 2026 run unless something material changed in one of these buckets:

1) Training environment/version changed (most common across months)

Ultralytics HUB Cloud Training is backed by the Ultralytics YOLO training stack, which is updated periodically. Even if your UI parameters look identical, underlying defaults/behavior can shift across releases (optimizer defaults, augmentation internals, determinism defaults, etc.).

Action: please paste the first ~20 lines of the training logs from both runs (they usually include the ultralytics version, torch version, device info, etc.). That will immediately tell us if the training engine changed.

2) Seed / determinism / split differences

If the run is not pinned to a fixed seed (or if determinism differs), you can absolutely get noticeably different results, especially on smaller datasets.

Ultralytics YOLO exposes both seed and deterministic for reproducibility — see the arguments table in the Train mode docs (look for seed and deterministic). If one run used different values (or defaults changed between versions), that alone can explain a big swing.

Action: confirm whether your HUB training “Advanced” settings included a fixed seed, and whether both runs used the same train/val split.

3) “Same dataset” but not the same dataset version

A very common gotcha is that the dataset name is the same, but the underlying artifact changed (re-upload, regenerated labels, different split, different preprocessing, etc.). Even a small label/export difference can crater mAP.

Action: in Ultralytics HUB, verify:

• train/val image counts match exactly
• class list/order matches exactly
• the dataset “version”/artifact shown in the run details is identical

4) Pretrained vs from-scratch (or different starting weights)

If one run started from pretrained weights and the other effectively trained from scratch (or from different base weights), metrics can diverge a lot.

Action: confirm whether pretrained was enabled and what the starting weights were. (Ultralytics generally recommends pretrained for best results; see pretrained in the Train mode docs.)

5) Your new January dataset may simply be harder (data drift)

For the dataset you trained “today” (Jan 16, 2026) that performed poorly: if its image distribution shifted (lighting, camera, backgrounds, object scale), it can behave like classic data drift relative to what your previous setup generalized to. See the Data Drift glossary for a concrete CV-oriented explanation and mitigation ideas (collect representative data + retrain).

---

What we need from you to pinpoint this quickly

1. Links to the two HUB training runs (if they’re private, paste the log header from both runs).
2. Dataset ID/version + train/val counts for both runs.
3. Model used (e.g., Ultralytics YOLO26n vs another variant) and whether it was pretrained.

Once you share those, we can tell you whether this is (a) expected reproducibility variance, (b) a dataset/split/version mismatch, or (c) something we need to fix on the HUB side.

[billdraper]

Thank you for the quick response. My replies in [] below: 1. Links to the two HUB training runs (if they’re private, paste the *log header* from both runs).Sorry [Sorry, can you please describe where to find these? I do not see them in the UI. These are private] 2. Dataset ID/version + train/val counts for both runs. [below are snapshots of the datasets as shown on each model page. Both model training sessions were run from exactly the same dataset with no changes. The first one produced good results in November 2025, the 2nd, using same dataset, same parameters was produced January 16, 2026. The metrics were included in my original question submission] 3. Model used (e.g., Ultralytics YOLO26n vs another variant) and whether it was *pretrained*. [both models were trained using Ultralytics YOLO11m. Config images below] [image: image.png][image: image.png] [image: image.png] [image: image.png] Did something change on the Hub side between November and today?

…

On Sat, Jan 17, 2026 at 11:05 AM Paula Derrenger ***@***.***> wrote: *pderrenger* left a comment (ultralytics/hub#1247) <#1247 (comment)> Thanks for the detailed screenshots @billdraper <https://github.com/billdraper> — with the *same dataset + “same parameters”*, you generally shouldn’t see a *dramatic* metric swing between a November 2025 run and your Jan 16, 2026 run unless something material changed in one of these buckets: 1) Training environment/version changed (most common across months) Ultralytics HUB Cloud Training is backed by the Ultralytics YOLO training stack, which is updated periodically. Even if your UI parameters look identical, underlying defaults/behavior can shift across releases (optimizer defaults, augmentation internals, determinism defaults, etc.). *Action:* please paste the first ~20 lines of the training logs from *both* runs (they usually include the ultralytics version, torch version, device info, etc.). That will immediately tell us if the training engine changed. 2) Seed / determinism / split differences If the run is not pinned to a fixed seed (or if determinism differs), you can absolutely get noticeably different results, especially on smaller datasets. Ultralytics YOLO exposes both seed and deterministic for reproducibility — see the arguments table in the Train mode docs <https://docs.ultralytics.com/modes/train/> (look for seed and deterministic). If one run used different values (or defaults changed between versions), that alone can explain a big swing. *Action:* confirm whether your HUB training “Advanced” settings included a fixed seed, and whether both runs used the same train/val split. 3) “Same dataset” but not the same *dataset version* A very common gotcha is that the dataset name is the same, but the underlying artifact changed (re-upload, regenerated labels, different split, different preprocessing, etc.). Even a small label/export difference can crater mAP. *Action:* in Ultralytics HUB, verify: - train/val image counts match exactly - class list/order matches exactly - the dataset “version”/artifact shown in the run details is identical 4) Pretrained vs from-scratch (or different starting weights) If one run started from pretrained weights and the other effectively trained from scratch (or from different base weights), metrics can diverge a lot. *Action:* confirm whether pretrained was enabled and what the starting weights were. (Ultralytics generally recommends pretrained for best results; see pretrained in the Train mode docs <https://docs.ultralytics.com/modes/train/>.) 5) Your *new* January dataset may simply be harder (data drift) For the dataset you trained “today” (Jan 16, 2026) that performed poorly: if its image distribution shifted (lighting, camera, backgrounds, object scale), it can behave like classic *data drift* relative to what your previous setup generalized to. See the Data Drift glossary <https://www.ultralytics.com/glossary/data-drift> for a concrete CV-oriented explanation and mitigation ideas (collect representative data + retrain). ------------------------------ What we need from you to pinpoint this quickly 1. Links to the two HUB training runs (if they’re private, paste the *log header* from both runs). 2. Dataset ID/version + train/val counts for both runs. 3. Model used (e.g., Ultralytics YOLO26n vs another variant) and whether it was *pretrained*. Once you share those, we can tell you whether this is (a) expected reproducibility variance, (b) a dataset/split/version mismatch, or (c) something we need to fix on the HUB side. — Reply to this email directly, view it on GitHub <#1247 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKFLBHFCDT5LZ7UKH3HXCKD4HJMSZAVCNFSM6AAAAACR66Q76KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTONRUGA2TSMRXGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[pderrenger]

Thanks for clarifying @billdraper — and yes, it’s possible something changed between your Nov 2025 run and the Jan 16, 2026 run, but the fastest way to confirm is to compare the run log header (it will show the exact training stack versions and GPU type used).

Where to find the “run link” + log header in Ultralytics HUB (private runs)

In Ultralytics HUB, each training job has its own details page:

1. Open the Model in HUB (the one you trained in Nov 2025 or Jan 2026)
2. Go to the model’s Training / Runs area (where you see the training history + charts)
3. Click the specific run (usually the row with a timestamp/status like Completed)
4. In the run details view, look for a Logs (or Console) panel/tab

From there, please copy/paste the first ~20 lines (you can redact any dataset/model names if you prefer). Those lines typically include:

• ultralytics package version
• torch version + CUDA
• GPU type
• Key args summary

If you truly don’t see any logs UI for that run, share the Model URL (just the .../models/<id> part is enough), plus the exact run timestamps, and we can help you locate what to extract.

Why the results can diverge even when “parameters look the same”

Even if the UI settings match, there are a few common “silent differences” that can change metrics a lot:

1) Backend training stack updates

HUB Cloud Training is periodically updated, so a Nov 2025 job and a Jan 2026 job may be running different ultralytics/torch versions and different internal defaults.

2) Randomness / split membership (even if counts match)

Even if the train/val counts are identical, the exact images in val can differ if:

• the dataset was auto-split at some point, or
• a split wasn’t “locked” deterministically

This is especially impactful on smaller datasets.

3) “auto” choices can change across versions/hardware

If you used anything like optimizer=auto or batch=auto, those can legitimately pick different values depending on the training stack and GPU, which can move metrics.

Quick reproducibility check to try (no dataset changes needed)

For the next re-run, I’d recommend explicitly pinning:

• seed (e.g. seed=0)
• deterministic=True

These are standard train args in Ultralytics YOLO (see the Train mode parameters for details). (docs.ultralytics.com)

If you’re using Cloud Training in the UI, you can set these in the “advanced” training arguments. (If you don’t see them, tell me what UI you’re seeing and I’ll point you to the exact spot.)

One more note (your Jan 16 “new dataset” case)

If that new dataset is coming from a different environment/camera/lighting than what your old model saw, it may simply be a distribution shift (data drift), which can drop metrics sharply even when training “looks fine”. See the Data Drift glossary for a CV-specific explanation + mitigation ideas. (docs.ultralytics.com)

---

What to paste here (minimum info)

For both runs (Nov 2025 and Jan 16, 2026):

1. First ~20 lines of the run logs (redacted is fine)
2. Whether seed/deterministic were explicitly set, or left default
3. Whether optimizer / batch were auto

Once we have that, we can say definitively whether this is expected variance vs. a backend change/regression.

[billdraper]

Two updates: 1. I looked and looked on the Model page as you described for the Training Run/Logs and just don't see it. Perhaps I'm looking right at it. It would be useful in the future to have access to these logs 2. Re-running these test models again yesterday indicates that everything is fine now. Same exact datasets, same parameters, same hardware, indicating that something had changed on the Hub side and is not fixed. So we no longer are seeing any problems Thanks again for your support on this. Bill

…

On Sat, Jan 17, 2026 at 11:30 PM Paula Derrenger ***@***.***> wrote: *pderrenger* left a comment (ultralytics/hub#1247) <#1247 (comment)> Thanks for clarifying @billdraper <https://github.com/billdraper> — and yes, it’s *possible* something changed between your *Nov 2025* run and the *Jan 16, 2026* run, but the fastest way to confirm is to compare the *run log header* (it will show the exact training stack versions and GPU type used). Where to find the “run link” + log header in Ultralytics HUB (private runs) In Ultralytics HUB, each training job has its own details page: 1. Open the *Model* in HUB (the one you trained in Nov 2025 or Jan 2026) 2. Go to the model’s *Training / Runs* area (where you see the training history + charts) 3. Click the specific *run* (usually the row with a timestamp/status like *Completed*) 4. In the run details view, look for a *Logs* (or *Console*) panel/tab From there, please copy/paste the *first ~20 lines* (you can redact any dataset/model names if you prefer). Those lines typically include: - ultralytics package version - torch version + CUDA - GPU type - Key args summary If you truly don’t see any logs UI for that run, share the *Model URL* (just the .../models/<id> part is enough), plus the *exact run timestamps*, and we can help you locate what to extract. Why the results can diverge even when “parameters look the same” Even if the UI settings match, there are a few common “silent differences” that can change metrics a lot: 1) Backend training stack updates HUB Cloud Training is periodically updated, so a Nov 2025 job and a Jan 2026 job may be running different ultralytics/torch versions and different internal defaults. 2) Randomness / split membership (even if counts match) Even if the *train/val counts* are identical, the *exact* images in val can differ if: - the dataset was auto-split at some point, or - a split wasn’t “locked” deterministically This is especially impactful on smaller datasets. 3) “auto” choices can change across versions/hardware If you used anything like optimizer=auto or batch=auto, those can legitimately pick different values depending on the training stack and GPU, which can move metrics. Quick reproducibility check to try (no dataset changes needed) For the next re-run, I’d recommend explicitly pinning: - seed (e.g. seed=0) - deterministic=True These are standard train args in Ultralytics YOLO (see the Train mode parameters <https://docs.ultralytics.com/modes/train/> for details). ( docs.ultralytics.com <https://docs.ultralytics.com/modes/train/>) If you’re using Cloud Training in the UI, you can set these in the “advanced” training arguments. (If you don’t see them, tell me what UI you’re seeing and I’ll point you to the exact spot.) One more note (your Jan 16 “new dataset” case) If that *new* dataset is coming from a different environment/camera/lighting than what your old model saw, it may simply be a distribution shift (data drift), which can drop metrics sharply even when training “looks fine”. See the Data Drift glossary <https://www.ultralytics.com/glossary/data-drift> for a CV-specific explanation + mitigation ideas. (docs.ultralytics.com <https://docs.ultralytics.com/platform/>) ------------------------------ What to paste here (minimum info) For *both* runs (Nov 2025 and Jan 16, 2026): 1. First ~20 lines of the *run logs* (redacted is fine) 2. Whether seed/deterministic were explicitly set, or left default 3. Whether optimizer / batch were auto Once we have that, we can say definitively whether this is expected variance vs. a backend change/regression. — Reply to this email directly, view it on GitHub <#1247 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKFLBHCNC3N46F53YEX75X34HMD6BAVCNFSM6AAAAACR66Q76KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTONRUHA4TMNZUGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[pderrenger]

Thanks for the updates @billdraper — really glad to hear the re-runs on Jan 18, 2026 are back to expected metrics. ✅

1) About “something changed on HUB”

It’s definitely possible a transient backend change (training stack update, runner/GPU image, etc.) caused a one-off regression on Jan 16, 2026, and then was corrected by the time you re-ran. Without the run log headers / run IDs we can’t do a precise postmortem, but if you’re willing to share just the model IDs + the exact run timestamps, we can still check internally what training environment those jobs executed on.

2) Where to find Cloud Training logs (and why you might not see them)

You should be able to view a run’s console output from the run details in HUB, but the exact placement can vary a bit depending on UI updates and the type of training job (cloud vs local). If you can post a quick screenshot of what you see on the run details page (redacting anything sensitive), I can point to the exact spot in your UI.

Also, for reproducibility/debugging, the key info we look for is the run “header” (Ultralytics version, torch/CUDA, GPU type, args summary). That’s the fastest way to confirm whether two runs truly used the same underlying stack.

In the meantime, for general training troubleshooting + reproducibility tips, see:

• Troubleshooting Common YOLO Issues
• Ultralytics YOLO Train mode settings (especially seed / deterministic)

3) Preventing this kind of surprise in the future

Even when everything “looks identical” in the UI, you’ll get the most stable comparisons by pinning:

• seed (e.g. 0)
• deterministic=True

And avoiding “auto” settings when you’re trying to do A/B comparisons (e.g. optimizer/batch), since those can legitimately resolve differently across environments/hardware.

---

If you can drop the model URL(s) (just the /models/<id> part) + the problematic run’s timestamp on Jan 16, I’ll take one more look on our side to see what changed.

[github-actions]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐