How to Set Up FreeBSD Jails with VNET Networking
Standard FreeBSD jails share the host's network stack. They use IP aliases on the host's interfaces, which means the host and all jails see each other's traffic and share routing tables. This works for simple setups but breaks down when you need isolated network stacks, per-jail firewalls, or jails that run their own routing daemons.
VNET jails solve this. Each VNET jail gets its own complete network stack -- its own interfaces, routing table, ARP cache, and firewall rules. VNET jails can run PF or IPFW independently from the host. They behave like separate machines connected by virtual Ethernet cables.
This guide covers everything: VNET concepts, epair interfaces, bridge networking, static and DHCP IP assignment, running PF inside jails, inter-jail communication, and production configurations.
For background on FreeBSD jails, see our FreeBSD jails guide.
How VNET Works
VNET (Virtual Network Stack) is a FreeBSD kernel feature that virtualizes the entire network subsystem. When a jail is created with vnet, it gets:
- Its own network interfaces (visible only inside the jail)
- Its own routing table
- Its own ARP/NDP cache
- Its own firewall state (PF or IPFW)
- Its own TCP/UDP socket namespace
The jail cannot see or interact with the host's network stack, and the host cannot directly see the jail's internal networking.
Connection to the Host: epair
Jails need a way to communicate with the outside world. FreeBSD provides epair -- virtual Ethernet pairs. An epair creates two connected interfaces (like a crossover cable). One end goes into the jail, the other stays on the host or connects to a bridge.
Bridges: Connecting Multiple Jails
A bridge interface connects multiple epair endpoints together, creating a virtual switch. All jails attached to the same bridge can communicate at Layer 2, and the bridge can be connected to a physical interface for external access.
Prerequisites
Ensure your FreeBSD kernel supports VNET (it does by default on FreeBSD 13+):
shsysctl kern.features.vimage
Should return 1.
Enable required kernel modules:
shkldload if_bridge if_epair sysrc kld_list+="if_bridge if_epair"
Basic VNET Jail Setup
Step 1: Create the Jail Filesystem
shzfs create zroot/jails zfs create zroot/jails/templates zfs create zroot/jails/running fetch https://download.freebsd.org/releases/amd64/14.2-RELEASE/base.txz -o /tmp/base.txz mkdir -p /jails/templates/base-14.2 tar -xf /tmp/base.txz -C /jails/templates/base-14.2 zfs snapshot zroot/jails/templates/base-14.2@clean zfs clone zroot/jails/templates/base-14.2@clean zroot/jails/running/web01
Step 2: Configure the Bridge and epair
Create a bridge on the host:
shsysrc cloned_interfaces="bridge0" sysrc ifconfig_bridge0="inet 10.0.0.1/24 up" service netif cloneup
The bridge acts as the gateway for all VNET jails. It gets IP 10.0.0.1.
Step 3: Write the Jail Configuration
Edit /etc/jail.conf:
sh# /etc/jail.conf # Global defaults exec.start = "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown"; exec.clean; mount.devfs; allow.raw_sockets; web01 { host.hostname = "web01.jail"; path = "/jails/running/web01"; vnet; vnet.interface = "epair0b"; exec.prestart = "ifconfig epair0 create up"; exec.prestart += "ifconfig bridge0 addm epair0a up"; exec.start += "/sbin/ifconfig epair0b inet 10.0.0.10/24 up"; exec.start += "/sbin/route add default 10.0.0.1"; exec.poststop = "ifconfig bridge0 deletem epair0a"; exec.poststop += "ifconfig epair0a destroy"; }
Step 4: Configure DNS in the Jail
shecho "nameserver 1.1.1.1" > /jails/running/web01/etc/resolv.conf
Step 5: Enable IP Forwarding on the Host
The host needs to forward packets between the bridge and physical interfaces:
shsysrc gateway_enable="YES" sysctl net.inet.ip.forwarding=1
Step 6: NAT with PF on the Host
If jails need internet access, configure PF on the host for NAT:
sh# /etc/pf.conf ext_if = "em0" jail_net = "10.0.0.0/24" nat on $ext_if from $jail_net to any -> ($ext_if) pass from $jail_net to any pass in on bridge0
Enable and start PF:
shsysrc pf_enable="YES" service pf start pfctl -f /etc/pf.conf
Step 7: Start the Jail
shservice jail start web01
Verify from inside the jail:
shjexec web01 ifconfig epair0b jexec web01 ping -c 3 1.1.1.1 jexec web01 ping -c 3 google.com
Multiple Jails on the Same Bridge
To add more jails, create additional epair interfaces. Each jail gets a unique epair number and IP:
sh# /etc/jail.conf -- additional jail web02 { host.hostname = "web02.jail"; path = "/jails/running/web02"; vnet; vnet.interface = "epair1b"; exec.prestart = "ifconfig epair1 create up"; exec.prestart += "ifconfig bridge0 addm epair1a up"; exec.start += "/sbin/ifconfig epair1b inet 10.0.0.11/24 up"; exec.start += "/sbin/route add default 10.0.0.1"; exec.poststop = "ifconfig bridge0 deletem epair1a"; exec.poststop += "ifconfig epair1a destroy"; } db01 { host.hostname = "db01.jail"; path = "/jails/running/db01"; vnet; vnet.interface = "epair2b"; exec.prestart = "ifconfig epair2 create up"; exec.prestart += "ifconfig bridge0 addm epair2a up"; exec.start += "/sbin/ifconfig epair2b inet 10.0.0.20/24 up"; exec.start += "/sbin/route add default 10.0.0.1"; exec.poststop = "ifconfig bridge0 deletem epair2a"; exec.poststop += "ifconfig epair2a destroy"; }
All three jails (web01, web02, db01) are on the 10.0.0.0/24 network and can communicate directly via the bridge.
Isolated Networks: Multiple Bridges
For network segmentation, create separate bridges for different jail groups:
shsysrc cloned_interfaces="bridge0 bridge1" sysrc ifconfig_bridge0="inet 10.0.0.1/24 up" sysrc ifconfig_bridge1="inet 10.0.1.1/24 up" service netif cloneup
Put web jails on bridge0 (10.0.0.0/24) and database jails on bridge1 (10.0.1.0/24). The host routes between them, but you can use PF rules to restrict which bridges can communicate:
sh# /etc/pf.conf -- network isolation web_net = "10.0.0.0/24" db_net = "10.0.1.0/24" # Web jails can reach database jails on specific ports only pass from $web_net to $db_net port { 5432, 3306 } block from $web_net to $db_net # Database jails cannot initiate connections to web jails block from $db_net to $web_net
DHCP Inside VNET Jails
Instead of static IPs, you can run a DHCP server on the bridge and let jails acquire addresses dynamically:
Install ISC DHCP on the Host
shpkg install isc-dhcp44-server
Configure /usr/local/etc/dhcpd.conf:
shsubnet 10.0.0.0 netmask 255.255.255.0 { range 10.0.0.100 10.0.0.200; option routers 10.0.0.1; option domain-name-servers 1.1.1.1, 1.0.0.1; }
Start the DHCP server on the bridge:
shsysrc dhcpd_enable="YES" sysrc dhcpd_ifaces="bridge0" service isc-dhcpd start
In the jail configuration, replace the static IP assignment with DHCP:
shexec.start += "/sbin/dhclient epair0b";
Running PF Inside VNET Jails
One of VNET's most powerful features is per-jail firewalls. Each jail can run its own PF instance.
Enable PF in the Jail
In /etc/jail.conf, add:
shweb01 { # ... existing config ... allow.raw_sockets; allow.set_hostname; # Allow PF inside the jail enforce_statfs = 1; }
Configure PF Inside the Jail
Create /jails/running/web01/etc/pf.conf:
sh# /etc/pf.conf inside web01 jail jail_if = "epair0b" set skip on lo0 block in all pass out all keep state # Allow SSH pass in on $jail_if proto tcp from any to any port 22 # Allow HTTP/HTTPS pass in on $jail_if proto tcp from any to any port { 80, 443 } # Allow ICMP pass in on $jail_if inet proto icmp icmp-type echoreq
Enable PF in the jail:
shecho 'pf_enable="YES"' >> /jails/running/web01/etc/rc.conf
Restart the jail:
shservice jail restart web01
Verify PF is running inside:
shjexec web01 pfctl -sr
Production Configuration Template
Here is a complete, production-ready jail.conf with VNET for a typical web application stack:
sh# /etc/jail.conf -- Production VNET Configuration # Global settings exec.start = "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown jail"; exec.clean; mount.devfs; # VNET defaults $bridge = "bridge0"; $gateway = "10.0.0.1"; $netmask = "24"; # Web server nginx { host.hostname = "nginx.local"; path = "/jails/running/nginx"; vnet; vnet.interface = "epair10b"; $ip = "10.0.0.10"; $epair = "epair10"; exec.prestart = "ifconfig $epair create up"; exec.prestart += "ifconfig $bridge addm ${epair}a up"; exec.start += "/sbin/ifconfig ${epair}b inet $ip/$netmask up"; exec.start += "/sbin/route add default $gateway"; exec.poststop = "ifconfig $bridge deletem ${epair}a"; exec.poststop += "ifconfig ${epair}a destroy"; allow.raw_sockets; } # Application server appserver { host.hostname = "app.local"; path = "/jails/running/appserver"; vnet; vnet.interface = "epair11b"; $ip = "10.0.0.11"; $epair = "epair11"; exec.prestart = "ifconfig $epair create up"; exec.prestart += "ifconfig $bridge addm ${epair}a up"; exec.start += "/sbin/ifconfig ${epair}b inet $ip/$netmask up"; exec.start += "/sbin/route add default $gateway"; exec.poststop = "ifconfig $bridge deletem ${epair}a"; exec.poststop += "ifconfig ${epair}a destroy"; allow.raw_sockets; } # Database server postgres { host.hostname = "db.local"; path = "/jails/running/postgres"; vnet; vnet.interface = "epair12b"; $ip = "10.0.0.12"; $epair = "epair12"; exec.prestart = "ifconfig $epair create up"; exec.prestart += "ifconfig $bridge addm ${epair}a up"; exec.start += "/sbin/ifconfig ${epair}b inet $ip/$netmask up"; exec.start += "/sbin/route add default $gateway"; exec.poststop = "ifconfig $bridge deletem ${epair}a"; exec.poststop += "ifconfig ${epair}a destroy"; allow.raw_sockets; }
Corresponding Host PF Rules
sh# /etc/pf.conf ext_if = "em0" jail_net = "10.0.0.0/24" nginx_ip = "10.0.0.10" app_ip = "10.0.0.11" db_ip = "10.0.0.12" # NAT for outbound jail traffic nat on $ext_if from $jail_net to any -> ($ext_if) # Port forwarding to nginx rdr on $ext_if proto tcp from any to ($ext_if) port { 80, 443 } -> $nginx_ip # Allow forwarded traffic pass in on $ext_if proto tcp to $nginx_ip port { 80, 443 } # Allow inter-jail on bridge pass on bridge0 # Block database from external access block in on $ext_if to $db_ip
Troubleshooting
Jail Cannot Reach the Internet
Check IP forwarding:
shsysctl net.inet.ip.forwarding
Must be 1. Check PF NAT rules:
shpfctl -sn
Verify the jail's routing:
shjexec web01 netstat -rn
Epair Not Created
Check that the if_epair module is loaded:
shkldstat | grep epair
Bridge Not Forwarding
Ensure the bridge interface is up and has members:
shifconfig bridge0
Check for bridge members:
shifconfig bridge0 | grep member
PF Not Working Inside Jail
VNET is required. Standard (non-VNET) jails cannot run their own PF. Verify the jail has vnet in its configuration.
FAQ
What is the performance overhead of VNET jails?
Minimal. The epair and bridge add a small amount of latency (microseconds) compared to IP-alias jails. For most workloads, the difference is unmeasurable. Network throughput between VNET jails on the same bridge exceeds 10 Gbps on modern hardware.
Can VNET jails get public IP addresses?
Yes. Instead of NAT, bridge the epair to your physical interface and assign public IPs directly to the jail's epair interface. The jail then appears as a separate machine on the physical network.
How many VNET jails can I run?
There is no hard limit. Each jail uses one epair (two interfaces) and one bridge member. FreeBSD handles hundreds of jails on a single host. Memory is the practical limit -- each jail's network stack uses approximately 2-4 MB.
Can I use IPv6 with VNET jails?
Yes. VNET fully supports IPv6. Add IPv6 addresses to the epair interface inside the jail and configure routing accordingly.
Should I use VNET or IP-alias jails?
Use IP-alias jails for simple setups where you trust all jails and do not need per-jail firewalls. Use VNET jails when you need full network isolation, per-jail PF rules, or jails that run network services requiring raw sockets.
Can VNET jails run their own DHCP server?
Yes. A VNET jail has a full network stack and can run any network service, including DHCP server, DNS server, or routing daemons.