path: root/2006/netfilter_curdevel-cwi2006/netfilter_nextgen-lk2005.mgp

%include "default.mgp"
%default 1 bgrad
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
%nodefault
%back "blue"

%center
%size 7

First steps towards next
generation netfilter

March 31, 2006
NLUUG @ CWI Amsterdam

%center
%size 4
by

Harald Welte <laforge@netfilter.org>


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
Contents

	rustynat
	nfnetlink
	conntrack event API
	nfnetlink_conntrack
	conntrack program
	nf_queue / nfnetlink_queue / libnetfilter_queue
	nf_log / nfnetlink_log / libnetfilter_log
	nf_conntrack
	nf-hipac
	x_tables
	pkt_tables (nf_tables?)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
History

	New filter every new major kernel release
		1.2: ipfw
		2.0: ipfwadm
		2.2: ipchains
		2.4: iptables
		2.6: iptables

	What went wrong?
		Was iptables good enough? 
		No, it just takes more time to rewrite all design mistakes ;)
		We don't have a development kernel anymore, changes are gradually


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
"rustynat"

	rustynat
		Part of kernel >= 2.6.12
	What is it?
		Revision to "newnat" infrastructure of 2.4.18
		Simpler data structures for conntrack / expectations
		NAT helpers are now called directly from conntrack helpers
	Benefits
		Reduces size of "struct ip_conntrack"
		Reduces memory usage by freeing expect when confirmed
	Backdaws
		Complex helpers (like PPTP) need lots of porting


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
nfnetlink

	nfnetlink
		netlink-based kernel/userspace transport layer
		used by all 'new' netfilter subsystems
		kernel module: nfnetlink.ko
		endian / type / alignment safe communication protocol
		extensible because of TLV / netlink attribute usage
		will in the future be transported over real network
		present in 2.6.14

	libnfnetlink
		userspace library to communicate via nfnetlink
		not to be used by application programs, just by other libs
		released on netfilter.org homepage


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
conntrack-event API

	conntrack event notifiers
		tells other kernel code about conntrack state updates
		used by (out of tree) conntrack sync code (ct_sync)
		used by nfnetlink_conntrack (see later)
		implemented as "struct notifier_chain"
		present in 2.6.14

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
nfnetlink_conntrack

	nfnetlink_conntrack (aka ctnetlink)
		nfnetlink based userspace interface for conntrack state table
			reading
			dumping
			manipulating (adding, removing, flushing)
		present in 2.6.14
		deprecates stupid /proc/net/ip_conntrack
		are a first step in the direction of userspace conntrack helpers
		also supports event messages to userspace

	libnetfilter_conntrack
		previously known as libnfnetlink_conntrack
		even more previously known as libctnetlink
		userspace API for conntrack interaction
		abstracts all (nf)netlink details

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
conntrack program

	conntrack
		Command line program for administrator access to conntrack
		Uses libnetfilter_conntrack, libnfnetlink and nfnetlink_conntrack
		Has plugins for conntrack protocols and conntrack helpers
		Implements feature-complete command line interface for
			listing conntracks / expects
			deleting conntracks / expects
			creating conntracks / expects
			reading / resetting conntrack accounting counters
			monitoring conntrack events

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
nf_queue / nfnetlink_queue

	nf_queue 
		queue handler registration interface
		every layer 3 protocol has its own queue handler
		is where ip_queue already registers itself today
		had no other users than ip_queue for AF_INET
		2.6.14: /proc/net/netfilter/nf_queue

	nfnetlink_queue
		layer 3 independent packet queueing to userspace
		deprecates ip_queue module
		allows 65535 different queues (ip_queue: 1)
		uses nfnetlink
		sends more anciliary information to userspace
			phys{in,out}dev
			MARK / CONNMARK
		allows setting of MARK from userspace

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
libnetfilter_queue

	libnetfilter_queue
		previously known as libnfnetlink_queue
		userspace library for queue-attaching processes
		NuFW group has already ported their App
		also provides API (but not ABI) compatible replacement for libipq

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
nf_log / nfnetlink_log

	nf_log 
		log handler registration interface
		every layer 3 protocol has its own log handler
		is where ipt_LOG backend is now registered for syslog default
		/proc/net/netfilter/nf_log
		all targets (ip6t_LOG,ipt_LOG,...) just call nf_log_packet()

	nfnetlink_log
		layer 3 independent packet logging to userspace
		deprecates ipt_ULOG module
		allows 65535 different logging groups (ULOG: 32)
		uses nfnetlink
		sends more anciliary information to userspace
			phys{in,out}dev
			MARK / CONNMARK
		present in 2.6.14

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
libnetfilter_log

	libnetfilter_log
		previously known as libnfnetlink_log
		userspace library for queue-attaching processes
		NuFW group has already ported their App
		also provides API (but not ABI) compatible replacement for libipq

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
ulogd2

	ulogd
		existing userspace logging daemon for ipt_ULOG

	ulogd2
		generalized framework for logging
			per-packet information (nfnetlink_log, ipt_ULOG)
			per-flow information (nfnetlink_conntrack)
			export data to MySQL, PostgreSQL, SYSLOG, text file
			Planned support for import/export of IPFIX (NETFLOW)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
nf_conntrack

	nf_conntrack
		generic layer 3 independent connection tracking
		designed by netfilter.org, implemented by USAGI
		present in 2.6.15
		obsoletes ip_conntrack (but ip_conntrack will remain for some time)
		has layer 3 plugins for ipv4 and ipv6
		I'm intending to write 802.2 LLC conntrack as a cool hack
		has all features of ip_conntrack, but
			iptable_nat cannot run on top of it
			nfnetlink_conntrack has not yet been ported
		once we're feature complete, ip_conntrack will be phased out

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
x_tables

	ip_tables was first
		ip6_tables started as copy+paste
		arp_tables is another of those copies
		eb_tables is a copy that was heavily modified

	why don't people know that copying is bad?

	x_tables (2.6.16)
		tries to unify common kernel code between {ip6,ip,arp}_tables
		has to stay binary compatible with userspace
		cannot include eb_tables support, too many differences
		transparent to the users, just kernel module names change
		especially for matches such as limit,mark,connmark,state,owner,...
		will be gradually extended to make matches/targets available for nf-hipac, pkt_tables

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
nf-hipac

	nf-hipac
		high-performance packet classification algorithm
		reduces packet classification to range location problem
		uses dimension tree for solving range location problem
		developed by two students at Saarbruecken University
		is extremely efficient when compared with any kind of linear lookup
	history
		was originally developed for 2.4.x
		project stalled because of time constraints
		now funded by MARA Systems
		new 0.9.0 release for 2.6.x, uses RCU
		netfilter + nf-hipac project working together for nfnetlink based userspace interface
		expected to be merged in a couple of months
 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
pkt_tables

	pkt_tables (ongoing work)
		tries to replace {ip,ip6,arp,eb}tables
		also uses nfnetlink based communication structure
		nfnetlink_pkttables uses same protocol as nf-hipac
			shared userspace tool
		sticks to existing linear list traversal
		mainly interesting for
			embedded systems with memory/flash constraints
			non-ipv4 matching that is currently lacking in nf-hipac
	libpkttables
	pkttables (ongoing work)
		new userspace library and tool

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
conntrack optimizations

	conntrack hash table resizing (2.6.15)
		how many people know about the "hashsize" module parameter?
		problem: what is a reasonable size?
			servers?
			personal firewall?
			dedicated packet filter?
		users end up with extremely bad performance
			we used to say it's a configuration problem

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
userspace conntrack helpers

	userspace contrack helpers
		based on nfnetlink_conntrack and nfnetlink_queue
		library provided called 'libnetfilter_cthelper' 
		allows development and production use of conntrack+nat helpers in userspace
		ongoing development effort, not useable yet (2.6.18?)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
conntrack optimizations

	conntrack hash trie (ongoing work)
		Martin Josefsson working on this
		replaces existing linar hash table by dynamic hash trie
		drastically reduces number of cache misses in large installations
		proof-of-concept code exists, will be merged with nf_conntrack
		submission date unclear
	
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Next-generation netfilter
Thanks

	Thanks to
		the BBS scene, Z-Netz, FIDO, ...
			for heavily increasing my computer usage in 1991
		KNF (http://www.franken.de/)
			for bringing me in touch with the internet as early as 1994
			for providing a playground for technical people
			for telling me about the existance of Linux!
		Alan Cox, Alexey Kuznetsov, David Miller, Andi Kleen
			for implementing (one of?) the world's best TCP/IP stacks
		Paul 'Rusty' Russell
			for starting the netfilter/iptables project
			for trusting me to maintain it today
		Astaro AG
			for sponsoring parts of my netfilter work
%size 3
	The slides of this presentation are available at http://svn.gnumonks.org/
%size 3