# Ingest audit logs in Kafka

### 1.0 Add new entries to hosts file
```
sudo vim /etc/hosts
```
```
127.0.0.1       broker-1
127.0.0.1       broker-2
127.0.0.1       broker-3
127.0.0.1       zookeeper-1
127.0.0.1       zookeeper-2
127.0.0.1       zookeeper-3
```
```
Unix: /etc/hosts
Windows: C:\Windows\System32\drivers\etc\hosts
```

### 1.1 Generate secrets
```
cd secrets
./create_secrets.sh
cd ..
```
### 1.2 Start the Kafka Cluster
```
docker compose up -d
```

### 1.3 Check if everything is up
```
docker ps
```
```
zookeeper-1
zookeeper-2
zookeeper-3
broker-1
broker-2
broker-3
schema-registry
connect
ksqldb-cli
ksqldb-server
audit-logger
```

### 1.4 Make a few requests
```
curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" -H "Authorization: Basic Ym9nZGFuOm5vdHNlY3JldA==" http://localhost:8080/resources -d '
{
	"resourceType": "vm",
	"name": "server-1"
}'
```
```
curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" -H "Authorization: Basic Ym9nZGFuOm5vdHNlY3JldA==" http://localhost:8080/resources -d '
{
	"resourceType": "vm",
	"name": "server-2"
}'
```

### 1.5 Deploy the Kafka Connect FileSource Connector
```
curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" -k https://localhost:8083/connectors/ -d '
{
  "name": "audit-logs",
  "config": {
    "connector.class": "FileStreamSource",
    "tasks.max": "1",
    "file": "/data/logs/audit.log",
    "topic": "audit-logs"
  }
}'
```

### 1.6 Launch ksqlDB CLI
```
docker exec -ti ksqldb-cli ksql http://ksqldb-server:8088
```

### 1.7 Inspect audit records produced
```
PRINT 'audit-logs' FROM BEGINNING;
```

# Analyze audit logs for SPAM resource creation

### 2.0 Create Audit Logs Stream
```
CREATE STREAM AUDIT_LOGS (dateTime VARCHAR, identifier VARCHAR, resourceType VARCHAR, action VARCHAR, resourceName VARCHAR)
WITH (KAFKA_TOPIC='audit-logs', VALUE_FORMAT='DELIMITED', VALUE_DELIMITER=',');
```

### 2.1 Identify SPAM attacks

```
CREATE TABLE AUDIT_LOGS_SPAM
    WITH (kafka_topic='audit-logs-spam') AS
    SELECT identifier ,
           COUNT(*) AS identifier_count
    FROM AUDIT_LOGS
    WINDOW TUMBLING (SIZE 60 SECONDS)
    WHERE action='create'
    GROUP BY identifier HAVING COUNT(*) > 2
    EMIT CHANGES;
```

### 2.3 Inspect the results
```
PRINT 'audit-logs-spam' FROM BEGINNING;
```

# Analyze network logs for Port Scan Attacks

### 3.0 Create [network.log](logs/network.log) file

```
touch ./logs/network.log
```

### 3.1 Insert value into [network.log](logs/network.log)
```
19:48:23.550407 IP bogdan.45099 > 172.28.0.4.987: Flags [S], seq 3601187300, win 1024, options [mss 1460], length 0
```

### 3.2 Ingest Network Logs

```
curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" -k https://localhost:8083/connectors/ -d '
{
  "name": "network-logs",
  "config": {
    "connector.class": "FileStreamSource",
    "tasks.max": "1",
    "file": "/data/logs/network.log",
    "topic": "network-logs"
  }
}'
```

### 3.3 Start the Network Logs Analyzer application
```
docker run -d --rm --name network-logs-analyzer --network protective-technology-with-apache-kafka_default -v /path/to/project/secrets:/etc/kafka/secrets bsucaciu/network-logs-analyzer:1.0.0
```

### 3.4 Analyze network traffic
```
docker inspect audit-logger
sudo tcpdump -i en0 host IP_ADDRESS > ./logs/network.log
```

### 3.5 Perform port scan attack
```
sudo nmap -sS -Pn -T4 -vv --reason IP_ADDRESS
```

### 3.6 Inspect the results using ksqlDB
```
PRINT 'alerts.port-scan-attack' FROM BEGINNING;
```

### 3.7 Create a structured stream
```
CREATE STREAM port_scan_attacks_alerts (
    identifier VARCHAR KEY,
    nbPortsScanned BIGINT
  ) WITH (
    KAFKA_TOPIC = 'alerts.port-scan-attack',
    VALUE_FORMAT = 'DELIMITED',
    VALUE_DELIMITER=','
  );
```

### 3.8 View the results
```
SELECT * FROM port_scan_attacks_alerts
EMIT CHANGES;
```

# Cleanup
```
docker stop network-logs-analyzer
docker compose down
```
