labeling not finds any label

[hoboristi]

I have executed ssh-brute-force attack against my victim linux host, and it pops up in suricata's
fast.log, but after all, label command on my trace file shows nothing (even fast.log copied by label command is empty..)

fast.log:
09/20/2019-10:27:18.123609 [] [1:2001219:20] ET SCAN Potential SSH Scan [] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 172.xx.xx.xx:44552 -> 192.xx.xx.xx:22
09/20/2019-10:27:20.198623 [] [1:10000001:1] Possible SSH brute forcing! [] [Classification: An attempted login using a suspicious username was detected] [Priority: 2] {TCP} 172.xx.xx.xx:44560 -> 192.xx.xx.xx:22

labeling:
label -debug -r ../../sshbrute-attack.pcap
checking log dir: ../../sshbrute-attack
removing suricata logfiles from previous runs
scanning ../../sshbrute-attack.pcap with suricata...
done. reading logs from ../../sshbrute-attack/fast.log
parsing suricata fast.log
0 alerts ignored in labelMap
no labels found.

Could you please help, what I am missing?

[dreadl0ck]

Hi Istvan,

glad to help!

1. What version of Netcap are you using? (v0.4 contains bugfixes related to labelling)

2. What OS are you running netcap on?

3. Netcap will create a folder in the current directory (named like the ncap file with the pcap extension), and tells suricata to create the fast.log file there. This file will only be removed if you execute net.label again, so the first step would be checking if the freshly generated fast.log file contains alerts.

4. if that is the case, you would see something like this:

$ net.label -r 2019-03-19-traffic-analysis-exercise.pcap
scanning 2019-03-19-traffic-analysis-exercise.pcap with suricata...
done. reading logs from 2019-03-19-traffic-analysis-exercise/fast.log
parsing suricata fast.log
9 alerts ignored in labelMap
got 442 labels

┌───────────────────────────────────────┬───────┐
│            Classification             │ Count │
├───────────────────────────────────────┼───────┤
│ Potentially Bad Traffic               │ 1     │
│ Potential Corporate Privacy Violation │ 2     │
│ A Network Trojan was detected         │ 439   │
└───────────────────────────────────────┴───────┘


done in 3.866265319s

In case records have been found, these will be displayed in a table view.
In that case, nothing further happened, because I did not generate the netcap audit records before executing net.label.

To generate the audit records use net.capture:

$ net.capture -r 2019-03-19-traffic-analysis-exercise.pcap
                       / |
 _______    ______   _10 |_     _______   ______    ______
/     / \  /    / \ / 01/  |   /     / | /    / \  /    / \
0010100 /|/011010 /|101010/   /0101010/  001010  |/100110  |
01 |  00 |00    00 |  10 | __ 00 |       /    10 |00 |  01 |
10 |  01 |01001010/   00 |/  |01 \_____ /0101000 |00 |__10/|
10 |  00 |00/    / |  10  00/ 00/    / |00    00 |00/   00/
00/   10/  0101000/    0010/   0010010/  0010100/ 1010100/
                                                  00 |
Network Protocol Analysis Framework               00 |
created by Philipp Mieden, 2018                   00/
v0.4.0

┌──────────────┬───────┐
│   Setting    │ Value │
├──────────────┼───────┤
│ Workers      │ 1000  │
│ MemBuffer    │ true  │
│ Compression  │ true  │
│ PacketBuffer │ 100   │
└──────────────┴───────┘

opening 2019-03-19-traffic-analysis-exercise.pcap | size: 7.8 MB
counting packets... done. 13186 packets found in 13.882355ms
spawned 1000 workers
initialized 50 layer encoders
initialized 4 custom encoders
done.

HTTPEncoder: Processed 13186 packets (7563934 bytes) in 1.396919659s (errors: 42, type:2)
Final flush: 43 closed

TCP stats:

┌───────────────────────┬─────────┐
│      Description      │  Value  │
├───────────────────────┼─────────┤
│ IPdefrag              │ 0       │
│ missed bytes          │ 2918    │
│ total packets         │ 8592    │
│ rejected FSM          │ 40      │
│ rejected Options      │ 2       │
│ reassembled bytes     │ 6823550 │
│ total TCP bytes       │ 6826593 │
│ conn rejected FSM     │ 9       │
│ reassembled chunks    │ 2       │
│ out-of-order packets  │ 446     │
│ out-of-order bytes    │ 629674  │
│ biggest-chunk packets │ 272     │
│ biggest-chunk bytes   │ 391573  │
│ overlap packets       │ 92      │
│ overlap bytes         │ 92      │
└───────────────────────┴─────────┘

Errors: 42

┌───────────────┬───────┐
│     Error     │ Count │
├───────────────┼───────┤
│ FSM           │ 40    │
│ OptionChecker │ 2     │
└───────────────┴───────┘

flushed 0 http events. requests 4 responses 4

┌──────────┬────────────┬────────────┐
│  Layer   │ NumRecords │   Share    │
├──────────┼────────────┼────────────┤
│ Ethernet │ 13186      │ 100.00000% │
│ IPv4     │ 13186      │ 100.00000% │
│ UDP      │ 221        │ 1.67602%   │
│ NTP      │ 24         │ 0.18201%   │
│ Payload  │ 8278       │ 62.77870%  │
│ DNS      │ 82         │ 0.62187%   │
│ IGMP     │ 6          │ 0.04550%   │
│ TCP      │ 12959      │ 98.27848%  │
│ DHCPv4   │ 16         │ 0.12134%   │
└──────────┴────────────┴────────────┘

┌───────────────┬────────────┬──────────┐
│ CustomEncoder │ NumRecords │  Share   │
├───────────────┼────────────┼──────────┤
│ TLS           │ 26         │ 0.19718% │
│ HTTP          │ 3          │ 0.02275% │
│ Flow          │ 438        │ 3.32170% │
│ Connection    │ 224        │ 1.69877% │
└───────────────┴────────────┴──────────┘

-> total bytes of data written to disk: 656 kB

done in 1.426693827s

Now your working directory should look like this:

$ tree .
.
├── 2019-03-19-traffic-analysis-exercise
│   ├── eve.json
│   ├── fast.log
│   └── stats.log
├── 2019-03-19-traffic-analysis-exercise.pcap
├── Connection.ncap.gz
├── DHCPv4.ncap.gz
├── DNS.ncap.gz
├── Ethernet.ncap.gz
├── Flow.ncap.gz
├── HTTP.ncap.gz
├── IGMP.ncap.gz
├── IPv4.ncap.gz
├── NTP.ncap.gz
├── TCP.ncap.gz
├── TLS.ncap.gz
├── UDP.ncap.gz
└── errors.log

1 directory, 17 files

If you do now execute net.label and pass the pcap file for scanning, the alerts that were found will be mapped to the audit records, resulting in several CSV files with the _labeled extension:

$ net.label -r 2019-03-19-traffic-analysis-exercise.pcap
scanning 2019-03-19-traffic-analysis-exercise.pcap with suricata...
done. reading logs from 2019-03-19-traffic-analysis-exercise/fast.log
parsing suricata fast.log
9 alerts ignored in labelMap
got 442 labels

┌───────────────────────────────────────┬───────┐
│            Classification             │ Count │
├───────────────────────────────────────┼───────┤
│ Potential Corporate Privacy Violation │ 2     │
│ Potentially Bad Traffic               │ 1     │
│ A Network Trojan was detected         │ 439   │
└───────────────────────────────────────┴───────┘

 + HTTP_labeled.csv                        labels: 2
 + Connection_labeled.csv                  labels: 27
 + Flow_labeled.csv                        labels: 29
 + Ethernet_labeled.csv                    labels: 433
 + TCP_labeled.csv                         labels: 433

done in 4.930203878s

5. If net.label does not find alerts in the generated fast.log file although it contains some, let me know!

In that case most likely there is an error in the regular expressions used for parsing, or there was a format update for the output in the latest suricate version.

I will refactor netcap soon to use the eve.json log file instead, then the old regular expression based parsing logic will be become obsolete anyway and the alert parsing will be much more robust.

Cheers,
Philipp

[dreadl0ck]

Seems like this was resolved, if not ping me via mail and I will reopen the issue.