Network Protocols Illuminated, LFNW 2008

This year I gave two presentations af Linux Fest Northwest. The first was Network Protocols Illuminated, the second was Shell Scripting from Scratch.

The description of Network Protocols Illuminated was as follows:

What 'language' do clients and servers use for communications like HTTP, FTP, and SMTP? Bri will show you how to watch and recreate the sessions that common clients use, show you how to interact at a lower level with remote machines, and show the security concerns with cleartext transmissions and measures you can take to increase network security.

It seemed that we got sidetracked and enthralled in SSL/TLS more than I had anticipated, and as such we did not get to FTP (and I was dying to show a pizza thief attack!) at all, but we were able to cover HTTP, HTTPS, SMTP, SSH and how encryption is insecure without authentication. SSL/TLS and STARTTLS fit into the whole security scheme.

The tools I demoed included:

Embarassing Occurances

Embarrassing things that happened included:

  • I'd made a brand new Linux user, without any of my handy dandy scripts, in hopes I wouldn't be hand waving anything that's magically set up based on my home directory environment. Naturally, I was missing some tools that would have been useful.
  • Was unable to talk HTTP to anal retentive^W^W strict servers that required proper \\r\\n linefeeds when using netcat. Boy, would have been nice to have my nccrlf script available and not resort to telnet.
  • Did I forget to mention that the perfectly set up VMWare image I'd prepped for this class, complete with four servers, a MITM attack, and other goodies was on the laptop that died a few months ago, and I forgot to check until the night before? Oops.


    I used script to make a log of the two windows (Foo and Bar) that I was using during the presentation. script will output each character that goes to the screen and, optionally, a timing file that lists how much time elapses between characters. Thus you are able to download the files below and watch exactly, typos and all, what went on overhead.

    Here are the available files:

    ContentTiming file (real time)Timing file (accelerated)
    protocols-illuminated-window1 protocols-illuminated-window1.timing.realtime protocols-illuminated-window1.timing
    protocols-illuminated-window2 protocols-illuminated-window2.timing.realtime protocols-illuminated-window2.timing

    Or, you can just download this handy-dandy tarball: bri-lfnw-presentations.tgz.

    To watch them, run one of the following commands:

    # To play the condensed versions:
    $ scriptreplay protocols-illuminated-window1.timing.realtime protocols-illuminated-window1
    $ scriptreplay protocols-illuminated-window2.timing.realtime protocols-illuminated-window2
    # To play the real-time versions:
    $ scriptreplay protocols-illuminated-window1.timing protocols-illuminated-window1
    $ scriptreplay protocols-illuminated-window2.timing protocols-illuminated-window1
    Make sure your screen is set to 24x80 so things look right when in vi, etc.

    I have a local copy of scriptreplay if you don't have it available on your system already.

    I am considering making a 'video' of these as well, and will post those here if I do so.

    Here are some of the more useful command lines that were run:

    # Command line http request
    $ curl
    # Command line https request
    $ curl
    # Command line https request, ignoring X509 certificate CN mismatch
    $ curl -k
    # Command line HTTP request, forging a specific HTTP "Host:" header,
    # necessary when hitting an IP against a webserver configured to use
    # virtual hosts
    $ curl -H "Host:"
    # The really low level way to interact with a webserver:
    $ nc 80
    # Get an SSH fingerprint based on the server's public key file directly,
    # to verify the fingerprint you blindly accepted when connecting to the
    # it the first time is, likely, correct.
    $ ssh-keygen -f /etc/ssh/
    # Connect to a mail server that supports STARTTLS, negotiate
    # TLS and let you interact.  Note: this is old stunnel3 syntax:
    $ stunnel -f -c -r -n smtp
    # Connect to an SSL/TLS port, showing lots of gory certificate
    # details.
    $ openssl s_client -connect host:port
    # Start sniffing our network interface to see traffic
    # in rather ugly hex dumps.  -X == hex dump, -n == don't
    # resolve dns, -i == interface
    $ sudo tcpdump -n -X -i eth0
    # Start sniffing our network interface but only for our
    # IP - we don't want to see all this netbios broadcast
    # stuff.  Or worse...
    $ sudo tcpdump -n -X -i eth0 host IP.AD.DR.ES
    # Start sniffing our network interface to show the content of
    # the stream in a user-friendly way, rather than lower level
    # packet capture that has ethernet fields, etc.
    $ sudo ssldump -d -n -i eth0
    # Same, but show SSL/TLS handshake information, when present.
    $ sudo ssldump -A -d -n -i eth0

    The presentation was created using /usr/bin/script -t 2>timingfile, and the timing files modified with this quick and ugly perl hack:

    my($timing, $chars) = split;
    if ( $chars == 1 ) {
            $timing = "0.000500";
    } elsif ( $timing > 1 ) {
    	$timing = '1.000000';
    print "$timing $chars\n";

    Everything herein is Copyright 2008, Bri Hatch of Onsight, Inc.