Illumos, Zones and DNS
Thomas Büttner
Why illumos
My offsite Backup-Server running Fedora CoreOS on a Btrfs RAID-5 died recently due to Btrfs Shooting its own foot of, since a single disk-failure with Btrfs RAID-5 obviously means that the whole array is unrecoverable…
So in what was i thought was better than Btrfs RAID-5 i decided to install OmniOS which is a illumos distribution stemming from the OpenSolaris days at SUN Microsystems.
I always thought of Solaris as this elusive highly advanced UNIX since it had so many really advanced features way before any other *NIX ZFS, DTrace, Crossbow and Zones so with this impressive repertoir i decided to give OmniOS a shot.
Getting in the ZONE
Since Zones are such a core feature of any illumos distribution, i simply had to use them. And OmniOS also has quite a number of different zone ‘brands’ which allow a zone to run foreign Software like Solaris 10, emulated aarch64 illumos or even Linux userland within a LX-branded Zone!
My Networking Problem
Now to my problem with zones, they (as far as i was able to find out) prefer/require to setup their IP-Addressing manually in advance. Since i was used to Linux and how the libvirt stack does it’s internal NAT-Networking via the default virbr0
bridge i tried to do something equivalent in OmniOS.
Stubbing my Ethernet
After looking in the Oracle Solaris 10 (1) documentation i found something maybe just right etherstub
which looks like it functions akin to a empty bridge interface on Linux, provides a detached L2 Interface to which one can attach VNICs to.
So this is how i set up my virbr0
like interface in OmniOS:
1dladm create-etherstub virbr0
2dladm create-vnic -l virbr0 global0
3dladm create-vnic -l virbr0 zone1
4ipadm create-if global0
5ipadm create-addr -T static -a 192.168.0.1 global0/v4
The above creates a etherstub
with a couple VNICs one for the “Global Zone” aka the Host and for the “Guest” Zone, assigns a static IP to the global0
“Interface” but there is still further setup to do…
Since this is the land of the SUN managing Link and IP configuration is done quite different to the way i know of even from the BSDs like FreeBSD and OpenBSD.
The dladm
command manages the “Ethernet” Layer 2 Interfaces eg. DataLinkADM and ipadm
manages the IP addressing be it static or DHCP.
Networking the Masquerade ipfilter Edition
Now we have a etherstub
“bridge” a VNIC for the Host with a static IP but we still need to setup the Firewall to allow the in and out traffic from the Zones.
For that we have to tell the ipf
that it should allow inbound traffic on global0
with this minimal and most permissive Firewall config for the /etc/ipf/ipf.conf
file:
1### no restrictions on loopback interface
2pass in quick on lo0 all
3pass out quick on lo0 all
4### allow traffic on our etherstub interface
5pass in on global0 from any to any
6pass out on global0 from any to any
7### allow any other traffic like default without IPF
8pass in all
9pass out all
To configure NAT with the Solaris ipfilter we add to the /etc/ipf/ipnat.conf
file:
1### Masquerade the outbound traffic on bge1 IP
2map bge1 192.168.0.0/24 -> 0/32 portmap tcp/udp auto
3map bge1 192.168.0.0/24 -> 0/32
Where bge1
is network connected “upstream” Interface.
With the Firewall configured we just need to actually start the ipfilter SMF service:
1svcadm enable ipv4-forwarding
2svcadm enable ipv6-forwarding
3svcadm enable network/ipfilter
4ipf -Fa -f /etc/ipf/ipf.conf
The last ipf -Fa -f
command flushes and reloads the filter.
And yes Solaris had a system service manager before systemd even was a thing with its SMF.
Creating the Zone
With the Networking stuff prepared now its time to get in the Zone:
1zonecfg -z zone1
2zonecfg:zone1> create
3zonecfg:zone1> set zonepath=/zones/zone1
4zonecfg:zone1> brand=sparse
5zonecfg:zone1> autoboot=false
6zonecfg:zone1> ip-type=exclusive
7zonecfg:zone1> add net
8zonecfg:zone1:net> set physical=zone1
9zonecfg:zone1:net> set allowed-address=192.168.0.2
10zonecfg:zone1:net> set defrouter=192.168.0.1
11zonecfg:zone1:net> end
12zonecfg:zone1> verify
13zonecfg:zone1> commit
14zonecfg:zone1> exit
With the Zone defined it is now ready to install and start:
1zoneadm -z zone1 install
2zoneadm -z zone1 boot
Enter the Zone
Now to configure the Zone itself we have to enter the zone:
1zlogin zone1
Running ifconfig
confirms that the Zone has the configured IP address:
1lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
2 inet 127.0.0.1 netmask ff000000
3zone1: flags=40001000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,L3PROTECT> mtu 9000 index 2
4 inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255
5 ether 2:8:20:7c:9e:37
6lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
7 inet6 ::1/128
Solaris ifconfig
has separate entries v4 and v6 addresses which show the Address.
So now the Zone has a IP and can ping the internet so everything works now, right? haha no:
1% ping 8.8.8.8
28.8.8.8 is alive
Ping works but SunOS/Solaris dates back to a time before DNS was even a thing so its not really in the default configuration of Zones…
For that we to configure the Zone to use DNS via NSS:
1% cp /etc/nsswitch.{dns,conf}
2% echo nameserver 1.1.1.1 > /etc/resolv.conf
3% svcadm enable svc:/network/dns/client:default
4% ping google.de
5google.de is alive
Now the Zone can talk to the Internet.
Notes & Resources
- [1] Solaris 11 was released after moving Solaris back to a closed Software license and has significant differences to the last OpenSolaris release of which illumos was forked of from.
- A extremely helpful cheat sheet if found later: https://github.com/jpdasma/unix-cheat-sheet