Can not acces outside container Mikrotik 7.21 on x86

[syahj0-pjbac]

any help will appreciate. thanks

Just mikrotik using hotspot and nat, for access internet

[root@mikrotik] > ip route pr
Flags: D - DYNAMIC; A - ACTIVE; c - CONNECT, d - DHCP
Columns: DST-ADDRESS, GATEWAY, ROUTING-TABLE, DISTANCE
DST-ADDRESS GATEWAY ROUTING-TABLE DISTANCE
DAd 0.0.0.0/0 192.168.18.1 main 1
DAc 10.0.0.0/24 Container main 0
DAc 192.168.3.0/24 bridge1 main 0
DAc 192.168.18.0/24 ether2 main 0
[root@mikrotik] > ping count=3 172.27.5.31
SEQ HOST SIZE TTL TIME STATUS
0 172.27.5.31 tim...
1 172.27.5.31 tim...
2 172.27.5.31 tim...
sent=3 received=0 packet-loss=100%

[root@mikrotik] > ping count=3 10.0.0.1
SEQ HOST SIZE TTL TIME STATUS
0 10.0.0.1 56 64 47us
1 10.0.0.1 56 64 38us
2 10.0.0.1 56 64 38us
sent=3 received=3 packet-loss=0% min-rtt=38us avg-rtt=41us
max-rtt=47us

container shell 0
/ # ip route list
default via 10.0.0.1 dev veth1-zt
10.0.0.0/24 dev veth1-zt scope link src 10.0.0.100
172.27.0.0/16 dev ztiv5nn233 scope link src 172.27.5.31
192.168.5.0/24 via 172.27.169.13 dev ztiv5nn233 metric 5000
192.168.8.0/24 via 172.27.243.15 dev ztiv5nn233 metric 5000
192.168.88.0/24 via 172.27.243.15 dev ztiv5nn233 metric 5000
192.168.253.0/24 via 172.27.169.13 dev ztiv5nn233 metric 5000
192.168.254.0/24 via 172.27.169.13 dev ztiv5nn233 metric 5000

/ # ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1): 56 data bytes
64 bytes from 10.0.0.1: seq=0 ttl=64 time=0.063 ms
64 bytes from 10.0.0.1: seq=1 ttl=64 time=0.047 ms
64 bytes from 10.0.0.1: seq=2 ttl=64 time=0.048 ms
64 bytes from 10.0.0.1: seq=3 ttl=64 time=0.044 ms
^C
--- 10.0.0.1 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.044/0.050/0.063 ms

[Paraphraser]

I don't want to tread on anyone's toes here but isn't this a problem that should be reported to MicroTik? From looking at their web site, it looks like their own implementation of ZeroTier. I can't find anything which suggests they are using a Zyclonite container (which is this repo).

I'm also not entirely sure what you mean by "can not acces outside container". I assume you mean network reachability. It should work but, other than for debugging, I can't actually think of a reason why you might want to do that (start from inside the container).

As I said, it should work. Here's a traceroute started from inside my Zyclonite/zerotier-router container running on a Raspberry Pi:

traceroute to www.apple.com (184.31.252.37), 30 hops max, 46 byte packets
 1  router.your.domain.com (192.168.132.1)  0.237 ms  0.331 ms  0.262 ms
 2  100.96.0.1 (100.96.0.1)  10.926 ms  13.502 ms  10.789 ms
 3  180.150.0.235 (180.150.0.235)  9.863 ms  10.279 ms  10.379 ms
 4  be50-3999.cfl3.8gif.nsw.aussiebb.net (180.150.2.96)  11.096 ms  10.820 ms  10.746 ms
 5  10.241.15.5 (10.241.15.5)  13.029 ms  12.811 ms  14.752 ms
 6  ae15.r01.syd01.ien.netarch.akamai.com (23.56.138.142)  10.812 ms  10.856 ms  10.356 ms
 7  *  *  *
 8  lo1.r04.stem01.syd01.fab.netarch.akamai.com (23.39.110.25)  10.870 ms  11.158 ms  lo1.r03.stem01.syd01.fab.netarch.akamai.com (23.39.110.24)  10.705 ms
 9  lo1.r03.spine02.syd01.fab.netarch.akamai.com (23.39.110.12)  10.462 ms  lo1.r01.spine02.syd01.fab.netarch.akamai.com (23.39.110.10)  10.621 ms  lo1.r04.spine03.syd01.fab.netarch.akamai.com (23.39.110.17)  11.131 ms
10  lo1.r04.leaf01.syd01.fab.netarch.akamai.com (23.39.110.5)  11.225 ms  lo1.r03.leaf01.syd01.fab.netarch.akamai.com (23.39.110.4)  11.059 ms  lo1.r01.leaf01.syd01.fab.netarch.akamai.com (23.39.110.2)  11.266 ms
11  vlan100.r16.tor01.syd01.fab.netarch.akamai.com (23.40.103.193)  11.744 ms  11.439 ms  10.707 ms
12^C

It's a host-mode Docker container so it's really no different to starting the traceroute on the host.

Does that help?

[syahj0-pjbac]

G'day,

Many thanks mate @Paraphraser for reply.

only happened on my system,
other x86 system just works fine.
been reported to mikrotik support, they just inform to update the ROS only.

and same issue with official zerotier/zerotier:latest with Error : Cant fine tune/tap directory .... port 9993 .. something, but with this distro, no error, but have same unreachable issue.

in mikrotik

` [/: > tool/traceroute apple.com

ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV
0 192.168.18.1 0% 3 0.4ms 0.4 0.4 0.5 0
1 10.108.64.1 0% 3 1.8ms 2.1 1.8 2.2 0.2
2 116.199.196.134 0% 3 2.3ms 2.2 2.1 2.3 0.1
3 112.78.172.65 0% 3 31.7ms 35.7 31.7 38.9 3
4 112.78.190.1 0% 3 31.6ms 35.6 31.6 39 3
5 112.78.190.5 0% 3 31.4ms 31.2 30.8 31.4 0.3
6 17.1.116.66 0% 3 158.3ms 150.2 143.1 158.3 6.2
7 17.253.144.10 50% 3 timeout 161.1 161.1 161.1 0 `

in zerotier-cli

/ # traceroute apple.com
traceroute: bad address 'apple.com'

/ # traceroute 17.253.144.10
traceroute to 17.253.144.10 (17.253.144.10), 30 hops max, 46 byte packets
1 10.0.0.1 (10.0.0.1) 0.005 ms 0.004 ms 0.004 ms
2 192.168.18.1 (192.168.18.1) 0.450 ms 0.483 ms 0.396 ms
3 10.108.64.1 (10.108.64.1) 2.422 ms 2.731 ms 2.223 ms
4 116.199.196.134 (116.199.196.134) 2.111 ms 2.592 ms 2.310 ms
5 112.78.172.65 (112.78.172.65) 39.501 ms 32.774 ms 32.781 ms
6 112.78.190.1 (112.78.190.1) 31.930 ms 31.407 ms 32.110 ms
7 112.78.190.5 (112.78.190.5) 31.734 ms 30.930 ms 31.698 ms
8 * * 17.1.116.66 (17.1.116.66) 151.042 ms
9 * 17.253.144.10 (17.253.144.10) 156.786 ms !C 163.641 ms !C

`

[Paraphraser]

Please compare and contrast these two snippets from your post:

• I assume this is from the Microtik command line:

   [/: > tool/traceroute apple.com
   ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV
   0 192.168.18.1 0% 3 0.4ms 0.4 0.4 0.5 0
   1 10.108.64.1 0% 3 1.8ms 2.1 1.8 2.2 0.2
  

• while I assume this is from inside the ZeroTier container:

   / # traceroute 17.253.144.10
   traceroute to 17.253.144.10 (17.253.144.10), 30 hops max, 46 byte packets
   1 10.0.0.1 (10.0.0.1) 0.005 ms 0.004 ms 0.004 ms
   2 192.168.18.1 (192.168.18.1) 0.450 ms 0.483 ms 0.396 ms
   3 10.108.64.1 (10.108.64.1) 2.422 ms 2.731 ms 2.223 ms
  

The second traceroute has an extra hop at 10.0.0.1.

What that suggests to me is that your ZeroTier container is running in non-host mode.

If you don't understand what I mean, a brief explanation is that Docker containers in:

• host mode attach directly to the host's network (where "host" means "the Microtik router"). A process running inside a host mode container binds to the host's network ports in the same way as a standalone process would; while
• non-host-mode are closer to small independent computers. The containers attach to an internal bridged network which is set up and managed by Docker. A process running inside a non-host-mode container binds to the container's ports. Docker sets up a software router (in your case it's at 10.0.0.1) to forward packets between the internal bridged network and the host network. When you declare port mappings, Docker sets up NAT rules to map between the external (host ports) and internal (container ports).

Now please take a look at these URLs which are all links to parts of this GitHub repo:

• Zyclonite README

• client service definition

• router service definition

Thedocker run from the README and the two service definitions all specify host mode.

What I recommend is:

1. IF the Microtik environment allows you to run containers in host mode then you should try that; however,
2. If the Microtik won't let you do that then I suspect you will at least need to declare a port mapping for UDP port 9993 so the appropriate NAT rule gets set up.

Googling suggests Microtik doesn't support running containers in host mode but, as I said, I lack the knowledge and experience to know whether the answer Google has given me is correct.

Now I will turn to:

/ # traceroute apple.com
traceroute: bad address 'apple.com'

That's a DNS failure.

In general, host mode containers follow the host's /etc/resolv.conf while non-host-mode containers direct queries to 127.0.0.11 which is a docker-supplied resolver.

The usual pattern for 127.0.0.11 is to first consult container name to IP mappings in the style of /etc/hosts so that non-host-mode containers can reach each other at their internal ports across the internal bridged network, after which containers follow the host's /etc/resolv.conf.

The fact that apple.com isn't resolving suggests either something is wrong with the router's DNS setup (its own /etc/resolv.conf if that exists) or something else about the Microtik environment is blocking DNS requests (eg firewall rules) from non-host-mode containers. It's therefore possible that placing the container in host mode will cure this problem too. If it doesn't then 🤷 and you're on your own to figure that bit out.

I have not drilled into the internals of ZeroTier but I kinda doubt that it does everything with hard-coded IP addresses. I think it's far more likely that it also relies on the DNS working from inside the container. The fact that apple.com won't resolve is really just the canary in the coal mine.

To put that another way, if the Microtik UI won't let you use host mode and you're forced to use non-host mode, you'll have to figure out both the port forwarding and why you can't resolve DNS names within the container. Once you solve those issues, the container will probably start working.

Something else I stumbled across is this. It might help.

---

Now let me move the goalposts slightly. I am going to ask the kind of irritating question we all encounter a lot on the Internet. I'm hoping you will see that I'm asking this question in a serious fashion, and that you will give it some thought rather than reject it outright:

Why do you want to run ZeroTier in your Microtik router?

I agree that it's a nice idea. Any router has a fair bit of information in its routing tables. Adding ZeroTier will extend those routing tables to include your ZeroTier Cloud networks. All devices inside your home network can just go on using the router as the default gateway and it should all just work.

I'm pretty sure it will all just work if you are able to run ZeroTier in host mode. I can be fairly confident in making that claim because, at one point, I was using a SEEED mini router as my home router. That's just a Raspberry Pi CM4 module with a couple of Ethernet interfaces. It runs standard Raspberry Pi OS so there were no "gotchas" about running Docker and host mode containers. It all just worked.

For reasons I won't get into, I needed to switch to an ISP-supplied router of the extremely dumb variety.

The only things I needed to do to keep ZeroTier running were:

1. Move the ZeroTier container to a Raspberry Pi 4 running inside my home network; and
2. Add a static route to the ISP-supplied router to set the next hop of my ZeroTier network (in my case that's 10.244.0.0/16) to the IP address of the Raspberry Pi.

You could sidestep all the Microtik issues by doing the same thing.

It doesn't have to be a Raspberry Pi, of course. Those just happen to be fairly cheap and they definitely do the job. It really can be any bit of hardware, including ZeroTier running in a Proxmox-VE guest. You might get some ideas for an appropriate topology here. That's documentation I wrote for IOTstack on the topic.

If you decide to try this approach, perhaps also take a look at PiBuilder. What you get out of that is a rock solid platform with Docker and IOTstack installed, meaning you can immediately move to installing either the ZeroTier client or ZeroTier router container (the two flavours from this Zyclonite repo). By the time you've read the documentation link above you'll likely know whether you want the client or router (in essence, the client attaches the host on which it is running to the Zerotier network, while the router is the client plus the addition of NAT rules such that the Raspberry Pi can "route" traffic for other devices on your home network).

PiBuilder is not constrained to the Raspberry Pi. Anything that will run Debian will do.

Hope this helps.