This repository contains a full-featured demo environment of a simple server network with a few servers, which is constantly accessed (and attacked) by automated clients.
Two networks:
- local network representing the cloud network → contains services
- virtual network representing the external network (Internet) → contains simulated clients
Running services (in internal network):
- nginx web server (SSL offloader) for :443
- static HTML web site (simple nginx or apache2 httpd) :80
- API (nginx) :8080
- API (php) :9000/fcgi
- FTP server :22 with weak passwords and plain text
- SSH server with root user without password :22 (→ invisible to IDS because encrypted)
- C&C "botnet" :12345 (→ intrusion)
Simulated clients (in external network):
- several that access random parts of the website+API (https) (→ benign)
- one that tries to exploit a vulnerability (→ sql injection)
- one that accesses API directly via http (!) and not via reverse proxy (→ configuration mistake)
- one that accesses FTP (→ benign but plain text)
- one that tries to guess FTP password (→ password bruteforce)
- one that accesses SSH (→ benign) with root/. (→ invisible to IDS)
- one that accesses c&c server
- several that try DDoS
apt install ca-certificates curl gnupg lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
apt update
apt install docker-ce docker-ce-cli containerd.io
curl -L "https://github.com/docker/compose/releases/download/v2.2.3/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-composeapt install wireshark
# say 'Yes' to allow non-root users to use Wireshark
adduser student wiresharkadd-apt-repository ppa:oisf/suricata-stable
apt update
apt install suricata jqCopy suricata-config/suricata.yml to /etc/suricata/.
Copy suricata-config/rules/demo.rules to /etc/suricata/rules/ (move any other .rules files out of that folder).
Make sure that you have executed:
git submodule init
git submodule updateCopy the contents of docker-idsdemo/ to /root/idsdemo/. Run as root:
cd /root/idsdemo
docker-compose up -dcd /root
git pull https://github.com/deviantony/docker-elk/
cd docker-elk
./dc up -d
# once ELK stack is running (this takes a while):
./setupNote: for ELK, additional configuration is necessary.
See the README file in the docker-elk repository.
The training consists of 3 parts:
- Brainstorming: what kind of attacks exist?
- Manually finding attacks with the help of Wireshark
- Automating the search for attacks with Suricata
Ask students to name any hacking attacks that they know, e.g.:
- DDoS
- sniffing
- virus
- botnet
- vulnerability exploit
- phishing
- credentials stealing/bruteforcing
- ... Keep note of them on a blackboard.
Let students briefly brainstorm how each of them can be detected. Keep note of this on the blackboard.
Conclusion: Some can be detected on the host of victim, some on the server only, some in the server network. Some need unencrypted network traffic, some don't.
Today we will focus on those that can be detected in the network.
For that we use IDS = Intrusion Detect System
They can be combined with e.g. a firewall or a switch that is able to block → IPS = Intrusion Protection System
Goal: detect any of the previously listed attacks using Wireshark
(guided workshop, in plenary)
Explain how to read network communication in Wireshark (in Wireshark itself).
Open Wireshark, select
eth0/enp0s0(or similar).
Visit http://example.org, see it in Wireshark.
UI is divided into 5 parts:
- menu
- filter bar (we'll see soon)
- packet list
- packet contents (for selected packet)
- select fields and see the corresponding raw bytes highlighted
- raw bytes
Important part of the menu:
- start, stop, restart
- open, save, close capture files
- to come back to main screen: stop capturing, then close capture file
Wireshark filters crash course:
- can type protocol name
http→ see only http
- can filter on field values
ip.src == 10.10.2.1http.host == "site.local"
- can use IP network masks
ip.src == 10.10.0.0/16
- can use matching
http.request.uri contains ".jpg"
- aliases:
ip.addr=ip.srcorip.dsttcp.port=tpc.srcportortcp.dstport
Analysis features:
- Statistics → Protocol hierarchy: see all protocols used
- Statistics → Conversations: see all hosts
Useful tricks:
- right-click a packet → Follow → TCP stream
- note: Wireshark automatically sets filter
tcp.stream eq ###
- note: Wireshark automatically sets filter
- can right-click any field → Prepare as a filter → Selected
On your computer you'll find a simulated environment of a network
In your home folder (see Desktop) you will find a network.png, open it
In Wireshark, you will find two network adapters which correspond to the two networks in the graphics
→ vnet_internal and vnet_external
Ask students to find any of the attacks on their own (prepared) machine. Let them individually experiment for 15-30 min or so. They should keep note of them and think about how attacks can be identified automatically.
(in plenary)
- What attacks have you found?
- What is the criteria that allows us to identify these attacks (i.e., what are the filters you put into Wireshark)?
Goal: automate the previously identified criteria
Briefly explain what Suricata is and how it works:
≈ Wireshark + configured rules that match network packets
Normally IDS is located on network, and listens passively (→ explain) to the network traffic
This is normally done using a special switch with a so-called monitoring port, which receives a copy of the whole network traffic
(guided workshop, in plenary)
- "Suricata does not have a UI, we will work with the terminal"
- "Open terminal, become root (
sudo -s). Your password isstudent." cd /etc/suricataandlsrules/folder is where our rules live (see in a moment)suricata.yamlconfigures Suricata →less suricata.yaml(use arrow keys,/to search,qto quit)Step 1: Inform Suricata about your networkCan be used in rules later to distinguish incoming and outgoing packetsStep 2: Select outputs to enableWhat output should Suricata make? By default alerts and statistics, but can also dump entire network traffic or extract sent files.Step 3: Configure common capture settingsWhere should we listen? → select proper network interfaceStep 4: App Layer Protocol configurationWhat protocols should be decoded? This makes Suricata slower, so enable only what you need.
- "No need to make changes, they have been done already for you"
cd rulesandls→ see only suricata.rulesnano suricata.rules→ see a dummy rule
Rule format: ACTION PROTOCOL SRC SRCPORT -> DST DSTPORT ( OPTIONS )
→ https://suricata.readthedocs.io/en/suricata-6.0.5/rules/intro.html
- ACTION is
alert(99% of cases) orpass(to flag something as good; but caution!); can also bedropin IPS mode. - PROTOCOL is
ip(= any),tcp, or a higher level protocol (enabled in suricata.yaml!) such ashttp,tls,smb... - SRC and DST can be
anyor$HOME_NETor10.10.0.0/16. Can use negation!and grouping[xxx, xxx, xxx]. - SRCPORT and DSTPORT can be
anyor80or6500:6599. Can use negation!and grouping[xxx, xxx, xxx]. - direction can be specified using
->(unidirectional) or<>(bidirectional). - OPTIONS are of the form
keyword: value;
Some options should always be there (best practises):
→ https://suricata.readthedocs.io/en/suricata-6.0.5/rules/meta.html
msg:"xxx"the alert messageclasstype:xxxthe alert category (seeclassification.confin Suricata configuration directory)sid:12345the rule IDrev:1version number of the rule
By convention, msg should be the first keyword and classtype, sid, rev the last ones.
Options are processed one after the other (order matters!).
→ https://suricata.readthedocs.io/en/suricata-6.0.0/rules/index.html
These options behave on two different ways (depending on the keyword): → https://suricata.readthedocs.io/en/suricata-6.0.0/rules/intro.html#modifier-keywords
- some keywords are content modifiers; they modify a content that has been previously selected by another keyword in the rule
for example:
nocase;after acontent:"";modifies thecontentkeyword to make it case-insensitive - some keywords are sticky buffers; they announce how a future keyword should behave
for example:
http.uri;announces that the upcomingcontent:"";should match the HTTP URI
Show how to detect these features by writing corresponding rules (joint exercise):
- unusual ports from unusual networks
→ https://docs.suricata.io/en/suricata-6.0.5/rules/intro.html#source-and-destination
for example, no services except rproxy, ftp and ssh should be exposed on Internet (172.16.x.x)
Corresponding rule:
alert ip any any -> [172.16.0.0/16, !172.16.32.11, !172.16.32.41, !172.16.32.12] any (msg:"Direct access to internal network"; classtype:intrusion; sid:1; rev:1;) - general plaintext
→ https://suricata.readthedocs.io/en/suricata-6.0.0/rules/payload-keywords.htmlcontent:"";to search for text (use|AF|to search for byte 0xAF)- need to escape
"=|22|,;=|3B|,:=|3A|,|=|7C| - can use negation:
content:!"";
- need to escape
- can use another
content:"";aftercontent:"";to search within the previous content
- content search options
→ https://suricata.readthedocs.io/en/suricata-6.0.0/rules/payload-keywords.htmlnocase;modifies content search to be case-insensitivedepth:N;modifies content to search for firstNbytes from the start (given byoffset)offset:N;modifies content to start searching at theN-th byte from the startdistance:N;modifies content to search at offsetNafter end of previouscontent(likeoffset)within:N;modifies content to search withinNbytes after end of previouscontent(likedepth)startswith,endswith
- keywords specific to protocols
→ e.g. http, dns, ...
protocol keywords:http.uri,http.request_body,http.user_agent,http.host, ... and many morefilename:"";,fileext:"";,filesize:>100MBand more (file extraction module)dns.queryfor example, detect.exedownloads:
alert http any any -> any any (msg:"Exe download"; http.uri; content:".exe"; endswith; classtype:malware; sid:1; rev:1;)
alert http any any -> any any (msg:"Exe download"; fileext:"exe"; classtype:malware; sid:1; rev:1;)
- threshold (→ DDoS)
→ https://docs.suricata.io/en/suricata-6.0.0/rules/thresholding.html
→ https://medium.com/@mshulkhan/detection-attack-using-suricata-1-5ea7b2f62551
control alerts using:threshold: type threshold, track <by_src|by_dst|by_both>, count N, seconds T;to set a minimum threshold (Ntriggers withinTseconds) after which an alert is raisedthreshold: type limit, track <by_src|by_dst|by_both>, count N, seconds T;to make the rule alert at mostNtimes withinTseconds
- GeoIP, IP reputation
→ https://docs.suricata.io/en/suricata-6.0.0/rules/header-keywords.html#geoip → https://docs.suricata.io/en/suricata-6.0.0/rules/ip-reputation-rules.html
Reload rules using
suricatasc -c reload-rules
# if the above fails (see below), use the following, slower command
systemctl restart suricataIf you get No such file or directory for the former, you might need to open /etc/suricata/suricata.yaml and edit the following (search for .socket with Ctrl+W) to match the real path to the Suricata command socket:
unix-command:
enabled: yes
filename: /var/run/suricata/suricata-command.socketSuricata logs to a file at /var/log/suricata/fast.log by default.
Examples:
alert ip !$HOME_NET any -> any ![80,443] (msg:"Unusual ports";)
alert tcp any any -> any any (msg:"Adult content"; content:"porn";)
alert tcp any any -> any any (msg:"Adult content"; content:"porn"; nocase;)
# "sex" is bad, but "wirtschaftsexperte" is ok
alert tcp any any -> any any (msg:"Adult content"; content:"sex"; nocase; content:!"perte"; nocase; distance:0; within:5;)
alert http any any -> any any (msg:"Unusual user agent"; http.user_agent; content:!"Mozilla/"; startswith;)
alert ip any any -> any any (msg:"New client"; threshold: type limit, track by_src, seconds 3600, count 1;)
alert tcp any any -> any any (msg:"DDoS"; threshold: type threshold, track by_dst, count 100, seconds 60;)
Let students experiment on their own and write rules to detect (raise alert) the previously identified attacks. Go around the class and help the students.
For help, do one of the exercices in class.
(guided workshop, in plenary)
Up until now, alerts are logged in a local log file. How can they be visualized?
- log format is JSON-based → use
tail -f | jq - use with GUI: Kibana
- malware rule set: EMET - https://rules.emergingthreatspro.com/OPEN_download_instructions.html
- rule naming convention
Ask students if they can think of any issues/limitations when using an IDS:
- the purpose of an IDS is not to replace security; it is an additional security measure! (detection ≠ prevention)
- IDS should be passive – otherwise it can become a bottleneck! (attacks against IDS exist)
- IDS cannot read encrypted data – need to put it behind an SSL offloader
- Suricata documentation https://suricata.readthedocs.io/
- Presentation on some Suricata techniques https://resources.sei.cmu.edu/asset_files/presentation/2016_017_001_449890.pdf
- Workshop with exercices https://nsrc.org/workshops/2015/pacnog17-ws/raw-attachment/wiki/
- How-to Suricata with ELK https://www.howtoforge.com/tutorial/suricata-with-elk-and-web-front-ends-on-ubuntu-bionic-beaver-1804-lts/
- How-to Simple DDoS detection with Suricata https://medium.com/@mshulkhan/detection-attack-using-suricata-1-5ea7b2f62551
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent