diff options
Diffstat (limited to '2017')
| -rw-r--r-- | 2017/ttcn-netdev2.2/paper/isea.tex | 57 |
1 files changed, 44 insertions, 13 deletions
diff --git a/2017/ttcn-netdev2.2/paper/isea.tex b/2017/ttcn-netdev2.2/paper/isea.tex index d329f49..fc83e00 100644 --- a/2017/ttcn-netdev2.2/paper/isea.tex +++ b/2017/ttcn-netdev2.2/paper/isea.tex @@ -20,7 +20,7 @@ /Author (Harald Welte)} % The file isea.sty is the style file for ISEA 2015 proceedings. % -\title{TTCN-3 and Eclipse TITAN for testing protocol stacks [draft]} +\title{TTCN-3 and Eclipse TITAN for testing protocol stacks} \author{Harald Welte\\ sysmocom - systems for mobile communications GmbH\\ Berlin, Germany\\ @@ -181,7 +181,7 @@ as {\tt universal charstring} (UCS-4). Using the {\tt record}, {\tt set}, {\tt union}, {\tt record of} and {\tt set of} TTCN-3 structured types, programmers can create abstract container types. -FIXME: example +% FIXME: example TTCN-3 also knows a {\tt enumberated} type, like many other programming languages. @@ -240,7 +240,7 @@ template MyMessageType tr_MyTemplate Templates can be hierarchical, so from the most generic to the most specific case, you can specify ever more concrete templates, depending on the need. -FIXME: Example +% FIXME: Example Templates can further be specified as {\tt subset}, {\tt superset}, {\tt permutation} (any order of elements). @@ -249,13 +249,13 @@ against one or multiple receive templates. Matching against a fixed template wo over e.g. comparing with a 'const struct' in a C-language test case. However, TTCN-3 templates can have wild-cards and pattern matching. -FIXME: Example +% FIXME: Example The built-in {\tt match()} function can be used to determine if a received message (or actually any value) matches the template. Even more so, the fundamental {\tt receive()} function, which is used to receive any inbound message from a test port, has built-in matching capability, so explicit -calls of {\tt match()} are rarely required. See FIXME later. +calls of {\tt match()} are rarely required. See Section \ref{receive} below. \subsection{TTCN-3 + TITAN encoders/decoders} @@ -331,10 +331,11 @@ type record GRE_Header { Much more complex constructs are possible in the TTIAN binary codec, e.g. the extension octet concept found in many telecom protocols. -FIXME: examples for a text based protocol +% FIXME: examples for a text based protocol \subsection{Abstract Communications Operations} +\label{receive} Abstract communications happens on the {\em test ports} which connect the test case of the abstract test suite (ATS) with the implementation under test (IUT). TTCN-3 supports abstract communications operations for both @@ -413,21 +414,51 @@ interleave { } \end{lstlisting} +\subsection{Fuzzing extensions} -FIXME: synchronous API calls example +As the TTCN-3 type notation contains stringent information about what values are permitted, it is normally not +possible to create illegal values. TITAN introduces the {\tt erroneous} extension which can be usd to +generate invalid messages, e.g. those missing mandatory fields, with invalid values, etc. -FIXME: diagram about netfilter conntrack test suite +\section{Test execution} -FIXME: test execution +The test suite, comprised of any number of individual test cases is executed by the TITAN executor. The +executor takes care of starting any of the parallel test components (on the local host or even on remote +hosts), performing the test cases as indicated by either the command line, or by the configuration file, or in +absence of that by the compile-time default list of testcases in the ({\tt control}) section of the TTCN-3 +code. It also manages opening all configured log files / plugins, as described below. The test executor +is typically started using the {\tt ttcn3\_start} program. -FIXME: configuration file / moduleparam +\subsection{Configuration File} -FIXME: logfile, including match error +Every TITAN test suite has a configuration file, where many aspects of test execution can be specified. Among +others, you can configure the log verbosity of the TITAN-internal logging. There are different log masks +for the log file ({\tt FileMask}) and for the console ({\tt FileMask}). One extremely useful feature is +the {\tt TTCN\_MATCHING} logging, which will provide detailed information about the exact (even nested) field +of a value that did not match in any explicit or implicit matching against a template. Another useful example +is {\tt TTCN\_ENCDEC}, which will automatically log the input and output of every encode/decode function, +which is very useful during debugging of codec problems. -FIXME: Junit-XML output plugin +\subsection{Log Files} +TITAN writes log files in a structured format, which makes them easy to parse by downstream log-processing +tools. Every parallel test component will write its own log file. As logs are timestamped, the {\tt +ttcn3\_logmerge} tool can be used to {\em splice} the individual log files into each other, providing one +log file of all components, sorted by timestamps. + +The {\tt ttcn3\_logformat} tool can be used to convert the single-line nested value representation into a +multi-line format with proper indenting which is more easy to understand as a human. Compare this to e.g. +JSON or XML pretty-printing. + +\subsection{Junit-XML output plugin} + +The TITAN runtime understands the concept of log output plugins. Such plugins can be custom-developed to +generate whatever output format from test case logs / results as needed. A number of standard plugins are +included, among them the capability to write Junit-XML output, which is a standard format for reporting test +results in a machine-readable format. This format is understood by the popular Jenkins continuous integration +software, so that one can easily feed per-testcase results into the Jenkins test results analyzer as part of a +CI test suite. -FIXME: fuzzing extensions \section{Acknowledgments} The author acknowledges that like everyone in Free Software, he is merely standing on the shoulders of |