FreeBSD Jail Networking: Complete Architecture Guide

FreeBSD jails support two fundamentally different networking models: inherited IP and VNET. The choice between them determines your security boundary, performance characteristics, and operational complexity. This guide covers both models in depth, plus the infrastructure that ties them together -- bridges, epair interfaces, PF firewall rules, NAT, and IPv6.

Networking Model Overview

Inherited IP (IP-Based Jails)

The jail shares the host's network stack. The jail gets one or more IP addresses assigned from the host's interfaces, but cannot configure interfaces, change routes, or run its own firewall.

Advantages:

Zero overhead (no virtual interfaces)
Simpler configuration
Lower resource usage

Disadvantages:

No network isolation between jails (they share the host stack)
Cannot run services that need raw sockets (ping, traceroute)
Cannot run per-jail firewalls

VNET (Virtual Network Stack)

Each jail gets its own complete network stack -- interfaces, routing table, ARP table, and firewall. The jail sees its own network environment, completely independent of the host and other jails.

Advantages:

Full network isolation
Jail can run its own firewall, routing, DHCP
Can run any network service without restriction

Disadvantages:

Higher resource usage (each jail has its own network stack)
More complex configuration
Slight performance overhead from virtual interfaces

Inherited IP Configuration

Basic Single-IP Jail

The simplest configuration. Add an alias IP to the host interface and assign it to the jail:

sh
ifconfig em0 alias 10.0.0.101/32

Create the jail in /etc/jail.conf:

sh
cat > /etc/jail.conf << 'EOF'
web {
    host.hostname = "web.jail";
    path = "/jails/web";
    ip4.addr = 10.0.0.101;
    interface = em0;
    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
    mount.devfs;
    allow.raw_sockets;
}
EOF

Make the alias persistent in /etc/rc.conf:

sh
ifconfig_em0_alias0="inet 10.0.0.101/32"

Start the jail:

sh
service jail start web

Multiple IP Jails

Assign multiple IPs to a single jail (for example, a web server that also runs on a management network):

sh
cat >> /etc/jail.conf << 'EOF'

multi {
    host.hostname = "multi.jail";
    path = "/jails/multi";
    ip4.addr = 10.0.0.102, 192.168.1.102;
    interface = em0, em1;
    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
    mount.devfs;
}
EOF

Inherited IP Limitations

Inside an inherited IP jail, network operations are restricted:

sh
jexec web ifconfig
# Only shows assigned IPs, cannot add/remove interfaces

jexec web route add default 10.0.0.1
# route: writing to routing socket: Operation not permitted

The jail uses the host's routing table and DNS configuration. Copy /etc/resolv.conf into the jail or mount it.

VNET Configuration

VNET gives each jail its own network stack. This requires virtual interfaces to connect the jail to the host's network.

epair Interfaces

An epair is a virtual Ethernet cable -- two interfaces connected back-to-back. One end goes in the jail, the other stays on the host (usually attached to a bridge).

Create an epair:

sh
ifconfig epair0 create

This creates epair0a and epair0b. By convention, the a side stays on the host and the b side goes into the jail.

Bridge Setup

A bridge connects virtual interfaces to a physical interface, putting all jails on the same network segment:

sh
ifconfig bridge0 create
ifconfig bridge0 addm em0
ifconfig bridge0 addm epair0a
ifconfig bridge0 up
ifconfig epair0a up

VNET Jail Configuration

sh
cat > /etc/jail.conf << 'EOF'
web {
    host.hostname = "web.jail";
    path = "/jails/web";
    vnet;
    vnet.interface = "epair0b";
    exec.prestart = "ifconfig epair0 create";
    exec.prestart += "ifconfig bridge0 addm epair0a";
    exec.prestart += "ifconfig epair0a up";
    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
    exec.poststop = "ifconfig epair0a destroy";
    mount.devfs;
    allow.raw_sockets;
    enforce_statfs = 2;
}
EOF

Inside the jail, configure networking via its /etc/rc.conf:

sh
cat > /jails/web/etc/rc.conf << 'EOF'
ifconfig_epair0b="inet 10.0.0.101/24"
defaultrouter="10.0.0.1"
EOF

Persistent Bridge in rc.conf

Make the bridge survive reboots:

sh
cat >> /etc/rc.conf << 'EOF'
cloned_interfaces="bridge0"
ifconfig_bridge0="addm em0 up"
EOF

Multi-Jail Architecture with Bridges

For a production setup with multiple jails, use a single bridge and create epair interfaces dynamically:

jail.conf with Multiple VNET Jails

sh
cat > /etc/jail.conf << 'EOF'
# Global defaults
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
mount.devfs;
allow.raw_sockets;
vnet;
enforce_statfs = 2;

web {
    host.hostname = "web.jail";
    path = "/jails/web";
    vnet.interface = "epair0b";
    exec.prestart  = "ifconfig epair0 create";
    exec.prestart += "ifconfig bridge0 addm epair0a";
    exec.prestart += "ifconfig epair0a up";
    exec.poststop  = "ifconfig epair0a destroy";
}

db {
    host.hostname = "db.jail";
    path = "/jails/db";
    vnet.interface = "epair1b";
    exec.prestart  = "ifconfig epair1 create";
    exec.prestart += "ifconfig bridge0 addm epair1a";
    exec.prestart += "ifconfig epair1a up";
    exec.poststop  = "ifconfig epair1a destroy";
}

app {
    host.hostname = "app.jail";
    path = "/jails/app";
    vnet.interface = "epair2b";
    exec.prestart  = "ifconfig epair2 create";
    exec.prestart += "ifconfig bridge0 addm epair2a";
    exec.prestart += "ifconfig epair2a up";
    exec.poststop  = "ifconfig epair2a destroy";
}
EOF

Each jail gets its own epair, all bridged to the physical network. Jails can communicate with each other and the outside world through the bridge.

PF Firewall Per-Jail

Firewall on the Host (Inherited IP)

With inherited IP jails, the host's PF controls all traffic. Filter by jail IP:

sh
cat > /etc/pf.conf << 'EOF'
ext_if = "em0"
jail_web = "10.0.0.101"
jail_db  = "10.0.0.102"
jail_app = "10.0.0.103"

set skip on lo0
block in all
pass out all keep state

# Web jail: allow HTTP/HTTPS from anywhere
pass in on $ext_if proto tcp to $jail_web port { 80, 443 } keep state

# DB jail: allow PostgreSQL only from app jail
pass in on $ext_if proto tcp from $jail_app to $jail_db port 5432 keep state

# App jail: allow from web jail only
pass in on $ext_if proto tcp from $jail_web to $jail_app port 8080 keep state

# SSH to host only
pass in on $ext_if proto tcp to ($ext_if) port 22 keep state
EOF

sysrc pf_enable="YES"
service pf start

Firewall Inside VNET Jails

VNET jails can run their own PF instance. Inside the jail:

sh
jexec web sh -c 'cat > /etc/pf.conf << PFEOF
block in all
pass out all keep state
pass in proto tcp to port { 80, 443 } keep state
PFEOF'

jexec web sysrc pf_enable="YES"
jexec web service pf start

Each VNET jail has an independent firewall. The host can also filter traffic on the bridge for defense-in-depth.

NAT for Jails

When jails use private IP addresses and need internet access, configure NAT on the host.

NAT with PF

sh
cat > /etc/pf.conf << 'EOF'
ext_if = "em0"
jail_net = "10.0.0.0/24"

set skip on lo0

# NAT jail traffic
nat on $ext_if from $jail_net to any -> ($ext_if)

block in all
pass out all keep state

# Allow incoming to specific jails
pass in on $ext_if proto tcp to 10.0.0.101 port { 80, 443 } keep state
pass in on $ext_if proto tcp to ($ext_if) port 22 keep state
EOF

Enable IP forwarding:

sh
sysrc gateway_enable="YES"
sysctl net.inet.ip.forwarding=1

Port Forwarding (RDR)

Redirect incoming traffic on the host's IP to a jail:

sh
rdr on $ext_if proto tcp from any to ($ext_if) port 80 -> 10.0.0.101 port 80
rdr on $ext_if proto tcp from any to ($ext_if) port 443 -> 10.0.0.101 port 443

This lets you run jails with private IPs and expose services through the host's public IP.

IPv6 in Jails

Inherited IP with IPv6

sh
ifconfig em0 inet6 alias 2001:db8::101/64

cat >> /etc/jail.conf << 'EOF'
web6 {
    host.hostname = "web6.jail";
    path = "/jails/web6";
    ip4.addr = 10.0.0.101;
    ip6.addr = 2001:db8::101;
    interface = em0;
    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
    mount.devfs;
}
EOF

VNET with IPv6

VNET jails configure IPv6 inside their own stack:

sh
cat > /jails/web/etc/rc.conf << 'EOF'
ifconfig_epair0b="inet 10.0.0.101/24"
ifconfig_epair0b_ipv6="inet6 2001:db8::101 prefixlen 64"
defaultrouter="10.0.0.1"
ipv6_defaultrouter="2001:db8::1"
EOF

IPv6 Router Advertisement

For VNET jails that should auto-configure IPv6 via SLAAC:

sh
pkg install rtadvd

On the host, run rtadvd on the bridge interface to advertise the IPv6 prefix to jails.

Multi-Homed Jail Architectures

DMZ Pattern: Two Bridges

Separate public-facing and internal networks:

sh
# Public bridge (connected to external NIC)
ifconfig bridge0 create
ifconfig bridge0 addm em0 up

# Internal bridge (no physical NIC, jail-to-jail only)
ifconfig bridge1 create
ifconfig bridge1 up

A web jail connects to both bridges:

sh
web {
    host.hostname = "web.jail";
    path = "/jails/web";
    vnet;
    vnet.interface = "epair0b";
    vnet.interface += "epair1b";
    exec.prestart  = "ifconfig epair0 create";
    exec.prestart += "ifconfig epair1 create";
    exec.prestart += "ifconfig bridge0 addm epair0a";
    exec.prestart += "ifconfig bridge1 addm epair1a";
    exec.prestart += "ifconfig epair0a up";
    exec.prestart += "ifconfig epair1a up";
    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
    exec.poststop  = "ifconfig epair0a destroy";
    exec.poststop += "ifconfig epair1a destroy";
    mount.devfs;
}

Inside the web jail:

sh
cat > /jails/web/etc/rc.conf << 'EOF'
ifconfig_epair0b="inet 203.0.113.10/24"
ifconfig_epair1b="inet 10.0.1.10/24"
defaultrouter="203.0.113.1"
EOF

The database jail connects only to the internal bridge:

sh
db {
    vnet.interface = "epair2b";
    exec.prestart  = "ifconfig epair2 create";
    exec.prestart += "ifconfig bridge1 addm epair2a";
    exec.prestart += "ifconfig epair2a up";
    exec.poststop  = "ifconfig epair2a destroy";
    # ...
}

The database is unreachable from the public network. The web jail acts as the boundary.

Isolated Jail Networks

For jails that must only talk to each other (e.g., a microservice mesh):

sh
ifconfig bridge2 create
ifconfig bridge2 up
# No physical interface added -- this bridge is isolated

Traffic on bridge2 cannot leave the host. Jails on this bridge can communicate only with each other.

Performance Considerations

Inherited IP vs VNET Throughput

Inherited IP has zero overhead -- it is the host's network stack. VNET adds the cost of virtual interface processing:

|------|-------------|---------------|----------|

| HTTP requests/sec | 112K | 98K | ~12% |

| Latency (ping) | 0.1 ms | 0.15 ms | +50 usec |

For most workloads, the VNET overhead is negligible. For extremely latency-sensitive applications, inherited IP avoids the virtual interface hop.

Bridge Tuning

For high-throughput bridges:

sh
sysctl net.link.bridge.pfil_member=0
sysctl net.link.bridge.pfil_bridge=1

This runs PF only once per packet (on the bridge) instead of twice (on the bridge and the member interface).

FAQ

Q: Should I use inherited IP or VNET for my jails?

A: Use VNET for production workloads that need network isolation, per-jail firewalls, or complex routing. Use inherited IP for simple services where performance matters more than isolation.

Q: Can I mix inherited IP and VNET jails on the same host?

A: Yes. Each jail independently chooses its networking model. You can run some jails with inherited IP and others with VNET on the same host.

Q: How do VNET jails communicate with the host?

A: Through the bridge. The host can have an IP on the bridge interface: ifconfig bridge0 inet 10.0.0.1/24. Jails reach the host at that IP.

Q: Can I use DHCP inside a VNET jail?

A: Yes. Run dhclient epair0b inside the jail or set ifconfig_epair0b="DHCP" in the jail's /etc/rc.conf. The DHCP request goes through the bridge to your network's DHCP server.

Q: How do I limit jail network bandwidth?

A: Use dummynet (IPFW) or PF queue rules on the host. Create a pipe with bandwidth limits and attach it to the jail's traffic.

Q: What happens to jail networking during a host reboot?

A: The jail rc script recreates epair interfaces and bridge memberships based on the exec.prestart commands in /etc/jail.conf. Jails come back up with the same network configuration.

Q: Can I use VLANs with jail bridges?

A: Yes. Create a VLAN interface and add it to a bridge: ifconfig vlan100 create vlan 100 vlandev em0, then ifconfig bridge0 addm vlan100. Jails on that bridge are in VLAN 100.

Q: How do I debug jail network connectivity issues?

A: Start from the inside out. Inside the jail: ifconfig, route -n, ping gateway. On the host: ifconfig bridge0, check bridge membership, check PF rules with pfctl -sr. Use tcpdump -i epair0a on the host to capture jail traffic.