%include "default.mgp" %default 1 bgrad %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page %nodefault %back "blue" %center %size 7 First steps towards next generation netfilter March 31, 2006 NLUUG @ CWI Amsterdam %center %size 4 by Harald Welte %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter Contents rustynat nfnetlink conntrack event API nfnetlink_conntrack conntrack program nf_queue / nfnetlink_queue / libnetfilter_queue nf_log / nfnetlink_log / libnetfilter_log nf_conntrack nf-hipac x_tables pkt_tables (nf_tables?) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter History New filter every new major kernel release 1.2: ipfw 2.0: ipfwadm 2.2: ipchains 2.4: iptables 2.6: iptables What went wrong? Was iptables good enough? No, it just takes more time to rewrite all design mistakes ;) We don't have a development kernel anymore, changes are gradually %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter "rustynat" rustynat Part of kernel >= 2.6.12 What is it? Revision to "newnat" infrastructure of 2.4.18 Simpler data structures for conntrack / expectations NAT helpers are now called directly from conntrack helpers Benefits Reduces size of "struct ip_conntrack" Reduces memory usage by freeing expect when confirmed Backdaws Complex helpers (like PPTP) need lots of porting %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter nfnetlink nfnetlink netlink-based kernel/userspace transport layer used by all 'new' netfilter subsystems kernel module: nfnetlink.ko endian / type / alignment safe communication protocol extensible because of TLV / netlink attribute usage will in the future be transported over real network present in 2.6.14 libnfnetlink userspace library to communicate via nfnetlink not to be used by application programs, just by other libs released on netfilter.org homepage %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter conntrack-event API conntrack event notifiers tells other kernel code about conntrack state updates used by (out of tree) conntrack sync code (ct_sync) used by nfnetlink_conntrack (see later) implemented as "struct notifier_chain" present in 2.6.14 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter nfnetlink_conntrack nfnetlink_conntrack (aka ctnetlink) nfnetlink based userspace interface for conntrack state table reading dumping manipulating (adding, removing, flushing) present in 2.6.14 deprecates stupid /proc/net/ip_conntrack are a first step in the direction of userspace conntrack helpers also supports event messages to userspace libnetfilter_conntrack previously known as libnfnetlink_conntrack even more previously known as libctnetlink userspace API for conntrack interaction abstracts all (nf)netlink details %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter conntrack program conntrack Command line program for administrator access to conntrack Uses libnetfilter_conntrack, libnfnetlink and nfnetlink_conntrack Has plugins for conntrack protocols and conntrack helpers Implements feature-complete command line interface for listing conntracks / expects deleting conntracks / expects creating conntracks / expects reading / resetting conntrack accounting counters monitoring conntrack events %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter nf_queue / nfnetlink_queue nf_queue queue handler registration interface every layer 3 protocol has its own queue handler is where ip_queue already registers itself today had no other users than ip_queue for AF_INET 2.6.14: /proc/net/netfilter/nf_queue nfnetlink_queue layer 3 independent packet queueing to userspace deprecates ip_queue module allows 65535 different queues (ip_queue: 1) uses nfnetlink sends more anciliary information to userspace phys{in,out}dev MARK / CONNMARK allows setting of MARK from userspace %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter libnetfilter_queue libnetfilter_queue previously known as libnfnetlink_queue userspace library for queue-attaching processes NuFW group has already ported their App also provides API (but not ABI) compatible replacement for libipq %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter nf_log / nfnetlink_log nf_log log handler registration interface every layer 3 protocol has its own log handler is where ipt_LOG backend is now registered for syslog default /proc/net/netfilter/nf_log all targets (ip6t_LOG,ipt_LOG,...) just call nf_log_packet() nfnetlink_log layer 3 independent packet logging to userspace deprecates ipt_ULOG module allows 65535 different logging groups (ULOG: 32) uses nfnetlink sends more anciliary information to userspace phys{in,out}dev MARK / CONNMARK present in 2.6.14 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter libnetfilter_log libnetfilter_log previously known as libnfnetlink_log userspace library for queue-attaching processes NuFW group has already ported their App also provides API (but not ABI) compatible replacement for libipq %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter ulogd2 ulogd existing userspace logging daemon for ipt_ULOG ulogd2 generalized framework for logging per-packet information (nfnetlink_log, ipt_ULOG) per-flow information (nfnetlink_conntrack) export data to MySQL, PostgreSQL, SYSLOG, text file Planned support for import/export of IPFIX (NETFLOW) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter nf_conntrack nf_conntrack generic layer 3 independent connection tracking designed by netfilter.org, implemented by USAGI present in 2.6.15 obsoletes ip_conntrack (but ip_conntrack will remain for some time) has layer 3 plugins for ipv4 and ipv6 I'm intending to write 802.2 LLC conntrack as a cool hack has all features of ip_conntrack, but iptable_nat cannot run on top of it nfnetlink_conntrack has not yet been ported once we're feature complete, ip_conntrack will be phased out %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter x_tables ip_tables was first ip6_tables started as copy+paste arp_tables is another of those copies eb_tables is a copy that was heavily modified why don't people know that copying is bad? x_tables (2.6.16) tries to unify common kernel code between {ip6,ip,arp}_tables has to stay binary compatible with userspace cannot include eb_tables support, too many differences transparent to the users, just kernel module names change especially for matches such as limit,mark,connmark,state,owner,... will be gradually extended to make matches/targets available for nf-hipac, pkt_tables %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter nf-hipac nf-hipac high-performance packet classification algorithm reduces packet classification to range location problem uses dimension tree for solving range location problem developed by two students at Saarbruecken University is extremely efficient when compared with any kind of linear lookup history was originally developed for 2.4.x project stalled because of time constraints now funded by MARA Systems new 0.9.0 release for 2.6.x, uses RCU netfilter + nf-hipac project working together for nfnetlink based userspace interface expected to be merged in a couple of months %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter pkt_tables pkt_tables (ongoing work) tries to replace {ip,ip6,arp,eb}tables also uses nfnetlink based communication structure nfnetlink_pkttables uses same protocol as nf-hipac shared userspace tool sticks to existing linear list traversal mainly interesting for embedded systems with memory/flash constraints non-ipv4 matching that is currently lacking in nf-hipac libpkttables pkttables (ongoing work) new userspace library and tool %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter conntrack optimizations conntrack hash table resizing (2.6.15) how many people know about the "hashsize" module parameter? problem: what is a reasonable size? servers? personal firewall? dedicated packet filter? users end up with extremely bad performance we used to say it's a configuration problem %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter userspace conntrack helpers userspace contrack helpers based on nfnetlink_conntrack and nfnetlink_queue library provided called 'libnetfilter_cthelper' allows development and production use of conntrack+nat helpers in userspace ongoing development effort, not useable yet (2.6.18?) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter conntrack optimizations conntrack hash trie (ongoing work) Martin Josefsson working on this replaces existing linar hash table by dynamic hash trie drastically reduces number of cache misses in large installations proof-of-concept code exists, will be merged with nf_conntrack submission date unclear %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Next-generation netfilter Thanks Thanks to the BBS scene, Z-Netz, FIDO, ... for heavily increasing my computer usage in 1991 KNF (http://www.franken.de/) for bringing me in touch with the internet as early as 1994 for providing a playground for technical people for telling me about the existance of Linux! Alan Cox, Alexey Kuznetsov, David Miller, Andi Kleen for implementing (one of?) the world's best TCP/IP stacks Paul 'Rusty' Russell for starting the netfilter/iptables project for trusting me to maintain it today Astaro AG for sponsoring parts of my netfilter work %size 3 The slides of this presentation are available at http://svn.gnumonks.org/ %size 3