path: root/2005/flow-accounting-lt2005/flow-accounting-lt2005.mgp

%include "default.mgp"
%default 1 bgrad
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
%nodefault
%back "blue"

%center
%size 7


Flow-based network accounting with Linux
OLS 2005 (July 22, 2005)

%center
%size 4
by

Harald Welte <hwelte@hmw-consulting.de>


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
Contents

	Introduction
	Network Acounting
	Existing Tools
	ip_conntrack_acct
	ctnetlink / conntrack tool
	ulogd2

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page 
Flow-based network accounting with Linux
Introduction

	Who is speaking to you?
		an independent Free Software developer
		who earns his living off Free Software since 1997
		who is one of the authors of the linux kernel firewall system called netfilter/iptables
		who has recently given lots of non-technical presentations about GPL enforcement
		who is happy to again speak about a technical subject today


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
Network Accounting


	Counting of metadata of network traffic
	Optionally Summarizing
	Kind of metadate dependant on application
		number of packets
		number of bytes
	Scope
		per timeframe
		per connection
		per flow


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
Network Accounting


	Reasons for network accounting
		volume or bandwith based billing
		monitoring of network utilization / disstribution
		research on network usage patterns, ...


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
Existing accounting solutions


	Existing accounting solutions for Linux
		nacctd (net-acct)
		ipt_LOG based
		ipt_ULOG based
		iptables-based (ipac-ng)
		ipt_ACCOUNT
		ntop


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
nacctd / net-acct


	nacctd
		Oldest tool available, at least since 1995
		Originally developed by Ulrich Callmeier
		Later unmaintained, multiple forks
		Principle of operation:
			Capture all packets using libpcap (AF_PACKET)
			try to aggregate packets into flows
			log to ASCII file (some branches provide SQL backends)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
ipt_LOG based


	ipt_LOG
		iptables "LOG" target, available in all 2.4.x and 2.6.x kernels
		Designed to log policy violations, not accounting data
		Not intended for logging of high data volumes
		Principle of Operation
			Iptables rule with "LOG" target for to-be-logged packets
			syslogd writes one line for each packet
			Perl scripts (or similar) used to parse syslog files
		Summary
			Doesn't scale since it abuses ipt_LOG for unintended purpose

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
ipt_ULOG based


	ipt_ULOG
		iptables "ULOG" target, available in almost all 2.4.x and 2.6.x kernels
		Designed to efficiently log policy violations, not accounting data
		Principle of Operation
			Copy header of packets into buffer
			Flush buffer to userspace
			Have a daemon parse packet headers in buffer
			Write information to some form of storage
		Summary
			Scales way better than ipt_LOG
			I still abusing an interface for a different purpose
			Still needs to transfer all packets to userspace

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
ip_tables counter based


	Accounting based on ip_tables
		Every ip_tables ruleset has per-rule packet and byte counters
		A number of ready-built tools exist to parse and summarize
		Most commonly used is "ipac-ng", supports storage in SQL DB
		Principle of Operation
			Careful placement of fallthrough-rules
			Executing "iptables -L -vn" or "iptables-save -c" displays counters
			Counters can be reset by "iptables -Z"
		Summary
			Scales well with high traffic
			Scales badly for lots of different accounting groups (which require lots of rules)


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
ipt_ACCOUNT


	ipt_ACCOUNT
		http://www.intra2net.com/opensource/ipt_account/
		A special purpose iptables target, requires kernel patch
		Principle of Operation
			Keeps byte counters per IP address in a given subnet (/24, eg.)
			Counters can be read by special "iptaccount" commandline tool
		Summary
			Is limited to networks up to /8
			Granularity only down to per-ip level
			Highly optimized, but special-purpose

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
ip_conntrack_acct


	ip_conntrack based accounting
		The netfilter connection tracking subsystem runs on almost any firewall
		Accounting is usually done at the edge of a network, where a firewall is placed
		ip_conntrack already maintains some ~ 350 bytes of state per connection
		Principle of Operation
			Add per-connection, per-direction packet and byte counters
			Read the counters from userspace (/proc/net/ip_conntrack or ctnetlink-based)
		Summary
			adds little extra overhead if ip_conntrack is used already
			Not recommended for non-firewall systems


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
ip_conntrack_acct


Userpace interfaces
	/proc/net/ip_conntrack
		shows one line per connection
		if CONFIG_IP_NF_CT_ACCT is enabled, "packets=5749 bytes=423453" is added for each direction
		Pro:
			Easy to use
		Con:
			Not always accurate
			No way to reset counters
			Inefficient

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
ip_conntrack_acct


	ctnetlink based interface
		What is ctnetlink?
			it's a netlink-based interface to ip_conntrack
			allows reading/deleting/updating/creating conntrack entries from usrspace
			exists as out-of-kernel patch for many years
		Extending ctnetlink with ip_conntrack_acct
			Simple: Add counter information to TLV's passed from kernel to userspace
		Additional features
			Add new IPCTNL_MSG_CT_GET_CTRZERO command request for atomic get-counters-and-zero


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
ip_conntrack_acct


Possible ctnetlink based implementations
	polling-based
		use GET_CTRZERO in a regular sampling interval
		add up counters with every call
		Pro: configurable granularity
		Con: overhead increases with short samling interval
	event-based
		listen for ctnetlink DELETE event messages
		store flow-based information only once at the end of every connection
		Pro: Very easy to implement
		Con: Data only available after connection finishes

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
ip_conntrack_acct


Programs to use ip_conntrack_acct
	'conntrack' tool
		http://svn.netfilter.org/trunk/conntrack
		Try "conntrack -E conntrack" for event-based output
		Try "conntrack -L conntrack" for polling
		Try "conntrack -L conntrack -z" for poll with zeroing counters

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
ip_conntrack_acct


Programs to use ip_conntrack_acct
	ulogd2
		http://svn.netfilter.org/branches/ulogd2
		next-gerneration of 'ulogd'
		can log per-packet and per-flow information
		can aggregate per-packet to per-flow information
		can run multiple 'plugin stacks' for multiple outputs
		can export per-flow data in IPFIX format
		is not fully implemented yet, but pretty far ahead

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
Flow-based network accounting with Linux
Thanks

	Thanks to
		Alan Cox, Alexey Kuznetsov, David Miller, Andi Kleen
			for implementing (one of?) the world's best TCP/IP stacks
		Paul 'Rusty' Russell
			for starting the netfilter/iptables project
			for trusting me to maintain it today
		Astaro AG
			for sponsoring parts of my netfilter work
		Free Software Foundation
			for the GNU Project 
			for the GNU General Public License
%size 3
	http://gnumonks.org/
%size 3
	http://netfilter.org/
%size 3
	http://svn.netfilter.org/