iptables rules deleted after reboot on Kubernetes nodes

After manually adding some iptables rules and rebooting the machine, all of the rules are gone (no matter the type of rule ).

ex.

$ iptables -A FUGA-INPUT -p tcp --dport 23 -j DROP
$ iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
KUBE-EXTERNAL-SERVICES  all  --  anywhere             anywhere             ctstate NEW /* kubernetes externally-visible service portals */
KUBE-FIREWALL  all  --  anywhere             anywhere
DROP       tcp  --  anywhere             anywhere             tcp dpt:telnet

After the reboot:

$ iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
KUBE-EXTERNAL-SERVICES  all  --  anywhere             anywhere             ctstate NEW /* kubernetes externally-visible service portals */
KUBE-FIREWALL  all  --  anywhere             anywhere

If I am not mistaken, kube-proxy running on every node is dynamically modifying the iptables. If that is correct how can I add rules that are permanent but still enable kubernetes/kube-proxy to do it's magic and not delete all the INPUT, FORWARD and OUTPUT rules that both Kubernetes and Weave plugin network dynamically generate?

Solution

Running iptables on any system is not a persistent action and would be forgotten on reboot, a k8s node is not an exception. I doubt that k8s will erase the IPTABLES rules when it starts, so you could try this:

create your rules (do this starting with empty iptables, with iptables -A commands, as you need them)
run iptables-save >/etc/my-iptables-rules (NOTE you could create a rules file manually, too).
create a system service script that runs on boot (or use /etc/rc.local) and add iptables-restore -n </etc/my-iptables-rules to it. This would load your rules on reboot. Note if you use rc.local, your 'iptables-restore' command may well run after k8s starts, check that your iptables -A commands are not sensitive to being loaded after those of k8s; if needed replace the -A commands in the file with -I (to place your commands first in the tables).

(be aware that some OS installations might include a boot-time service that loads iptables as well; there are some firewall packages that install such a service - if you have one on your server, the best approach is to add your rules to that firewall's config, not write and load your own custom config).