Really understanding and debugging iptables (or better these days, nftables) is an art. Of course a sysadmin should know what s/he’s doing, but there are also the moments where a non-sysadmin might need to set up, say, a laptop or a Raspberry as a very temporary wireless gateway. There are plenty of longer or shorter tutorials around the ubiquitous lines
echo 1 > /proc/sys/net/ipv4/ip_forward iptables -t nat -A POSTROUTING -o $EXTERNAL_IF -j MASQUERADE
where $EXTERNAL_IF could be your wi-fi device, such as “wlan0” or “wlp2s0” if you are sharing your wi-fi connection to other non-wireless systems; or it could be your Ethernet interface (such as “eth0” or “eno1”) if that’s where your Internet connection is.
If you happen to have Docker on that machine, and possibly firewalld, Docker might be configured to add a lot of firewall rules. As I want my containers to keep their connectivity both among each other and towards the net, I’d rather not touch those rules so much. Docker changed the “policy” for the FORWARD chain – that is, what happens inside a chain if no rules match – from ACCEPT to DROP. This means that without extra rules, packets that before passed your firewall are now blocked. The Docker documentation explains how to add rules to the DOCKER-USER chain to remedy this, and they give as an example:
iptables -I DOCKER-USER -i src_if -o dst_if -j ACCEPT
It’s an excellent hint that they talk about adding “rules” in the plural, unfortunately the example above is a single line, and after that single iptables line above for masquerading, you might say: OK, I got the source interface where my internal network is, and the destination interface is the external one where the Internet connection comes from, and I write one line. But it won’t work. It may be enough to send out packets, but in order to let the responses come back, you also need to add the other direction! Very obvious once you have it working.
iptables -I DOCKER-USER -i $INTERNAL_IF -o $EXTERNAL_IF -j ACCEPT iptables -I DOCKER-USER -i $EXTERNAL_IF -o $INTERNAL_IF -j ACCEPT
Lest I forget, I note down what helped me debugging, here on an AlmaLinux / Rocky Linux / CentOS / RHEL 8:
- To see what’s going on in iptables, “-L” is not enough. “iptables -v -x -n -L“, or for another table “iptables -t nat -v -x -n -L” is much better.
- To see accepted or blocked packages, you can mark them to be traced, like this:
iptables -I DOCKER-USER -j TRACE
TRACE does not stop the rule evaluation, but ACCEPT would. So make sure your TRACE rule is at the right position (e.g. above, not below, others that would match, too).
But how do you see the traces? I did not succeed in getting them in the kernel log, but with “nft monitor trace” I could see them. (Yes, Red Hat says you shouldn’t mix firewalld, iptables, and nftables, but read-only usage of another tool won’t hurt.) - firewalld adds another level of complexity here with its “zones”. I suspected it to be involved in my Docker/NAT router problem, but in the end, I did not need to change anything there.