jailmaker still working on 25.10!

[Moose-Gaming]

Hi guys,

I was recently in the process of migrating from jailmaker to TrueNAS Apps after dreading to update TrueNAS past 24.10 since forever but was surprised to find that jailmaker was working flawlessly on the latest 25.10 release. I know its no longer maintained etc. but it really helped me with the migration process. Besides, having another option is always great :)

[Moose-Gaming]

Just a quick performance update - with jailmaker my average CPU load was 3.7% and now with apps its hovering around 5%... But oh well I guess the apps system is more complex and therefore has more overhead.

[kuya1284]

I'm actually about to switch back to Jailmaker from Incus, myself. I really wish I never switched to Incus to begin with, especially since its integration was only a short-lived experiment. I don't want to keep playing whack-a-mole because who knows what ixSystems will end up doing beyond TrueNAS 26. Let's just hope they won't remove systemd-nspawn.

The main motivation for switching back is because they left Incus stuck at v6.04. This version is stuck with a version of LXC that breaks Docker using containerd.io v1.7.28-2 or newer. The issue was fixed in Incus v6.19, but since ixSystems dropped support for Incus in favor of libvirt, options are limited. The two workarounds are to either pin containerd.io to v1.7.28-1, or disable AppArmor for the LXC container: lxc.apparmor.profile = unconfined. Each has their pros and cons, but for a homelab, I think disabling AppArmor just to get the latest version of containerd.io and Docker to take advantage of security updates has more benefits.

But why make silly sacrifices? Why not just switch back to something that just works without all the fuss and hassle? That's why I'm going back to Jailmaker. If ixSystem decides to get rid of systemd-nspawn, then I'm gonna just dedicate a weekend to migrating my setup to Proxmox and just run TrueNAS as a VM. That way, ix can do WTF ever they want without screwing with people's precious time.

I'm glad to hear your experience had gone well for the most part. I'm hoping my switch back won't take too long, because time is a luxury I don't have a whole lot of. I'm hoping for a similar positive experience...

[Lockszmith-GH]

If you're using GPU with docker, I would not recommend the native Docker - unless it works flawlessly with the latest nvidia drivers - found it crashed my system.
Moving everything into an Incus container resolved the issue - I'm curious where did you see Incus is being dropped?
The dropped Incus for VMs, but containers are still there.

[kuya1284]

There have been discussions about it. Here are some:

• https://forums.truenas.com/t/incus-vms-containers/49237
• https://forums.truenas.com/t/update-incus-to-at-least-6-19-or-newer/63552
• https://forums.truenas.com/t/containers-not-showing-stats/63824
• https://forums.truenas.com/t/truenas-virtualization-solutions-docker-lxc-and-kvm-which-to-use-truenas-tech-talk-t3-e051/63113

The last one links to a tech talk where they state at the beginning that they're switching to libvirt. They don't explicitly state or make mention about it in the TrueNAS 26 release notes, but this bug report seems to coraborate the switch as well, which describe a container plugin: truenas/middleware#17764

[Lockszmith-GH]

Wow, thank you so much @kuya1284, I've been out of the loop.

Well Jip-Hop, this project here might just what will save everyone's sanity in the long run.
Looks like I'm figuring out how to move back into a Jail....

[kuya1284]

@Lockszmith-GH no prob. I concur with what you said. I opted to continue using Jailmaker over native install of Docker. It just makes things easier for me for various reasons:

• I need a (Debian) container for my custom scripts
• I also need specific packages for those scripts
• Other packages I need:
  • logrotate
  • crond
  • curl
• I can mount my datasets to /opt, which makes it easier for me (i.e. less to type) to navigate to and reference my Docker stacks and volumes than something like /mnt/tank/docker/stack or /mnt/tank/docker/volumes
• Since I have two instances of TrueNAS, /opt would be consistently named in the Debian container; I use different names for the parent datasets on each instance so I can easily identify the server I'm on

I have other reasons that I won't get into, but Jailmaker is just a Godsend. Not to mention, it's much easier to configure, provision, and deploy. My template is so much simpler than the Incus template.

[Lockszmith-GH]

I did something similar (at first with Jailmaker, then with Incus, which I hope they won't nuke) - I run runtipi inside it, with the PCI device for the GPU mapped into it. I have control over the drivers, I have control over the system

Will revert to jailmaker if Incus goes bust

[kuya1284]

I just wanted to share that I just tagged a new Jailmaker v2.2.0 release in a new fork of Jip-Hop's awesome Jailmaker. It makes various improvements for TrueNAS Goldeye, most notably support for the NVIDIA Open Kernel driver, and the ability to install the NVIDIA Proprietary driver for legacy GPUs. It also adds a new jlmkr nvidia command to manually install/uninstall the NVIDIA Proprietary driver provided by @zzzhouuu.

Feel free to check it out and test. I hope nothing breaks and that everything still works properly.

@Jip-Hop, if you like, I can submit a PR so that it can be merged in this original repo. Otherwise, I can continue helping to maintain your legacy in my fork as time permits. Thanks for all you've done!