Add training documentation

e14e625 verified 5 months ago

12.2 kB

license: mit
language:
  - en
library_name: pytorch
tags:
  - security
  - cybersecurity
  - network-security
  - c2-detection
  - beacon-detection
  - threat-detection
  - malware-detection
  - logbert
  - transformer
  - safetensors
pipeline_tag: other

C2Sentinel

A machine learning model for detecting Command and Control (C2) beacon communications in network traffic. Built on a fine-tuned LogBERT transformer architecture.

Author: Daniel Ostrow Website: neuralintellect.com Release Date: January 18, 2026

Base Model

This model is fine-tuned from the LogBERT architecture for log anomaly detection.

Paper: LogBERT: Log Anomaly Detection via BERT (Guo, Yuan, Wu - IJCNN 2021)
Original Implementation: github.com/HelenGuohx/logbert

Overview

C2Sentinel analyzes network connection patterns to identify C2 beacon activity. The model uses behavioral analysis rather than port-based filtering, enabling detection of C2 communications on any port. This approach catches C2 activity regardless of whether attackers use expected ports (4444) or attempt to blend in on common ports (443, 80, 53).

Capabilities

Detection of 34+ C2 framework behavioral patterns across all ports
Slow beacon detection (intervals from seconds to hours)
Legitimate traffic pattern recognition (SSH keepalive, health checks, database connections)
Optional context enrichment (process information, reputation scores, threat intelligence)
IP reconnaissance and IOC generation
Safetensors format for secure model loading

Installation

pip install torch numpy safetensors huggingface_hub

Usage

Loading from HuggingFace Hub

from c2sentinel import C2Sentinel

sentinel = C2Sentinel.from_pretrained('danielostrow/c2sentinel')

Loading from Local Files

from c2sentinel import C2Sentinel

sentinel = C2Sentinel.load('c2_sentinel')

Analyzing Connections

connections = [
    {
        'timestamp': 1000000,
        'dst_ip': '10.0.0.1',
        'dst_port': 443,
        'bytes_sent': 200,
        'bytes_recv': 500
    },
    {
        'timestamp': 1000060,
        'dst_ip': '10.0.0.1',
        'dst_port': 443,
        'bytes_sent': 200,
        'bytes_recv': 500
    },
]

result = sentinel.analyze(connections)

if result.is_c2:
    print(f"C2 detected: {result.c2_type}")
    print(f"Probability: {result.c2_probability}")
else:
    print("No C2 detected")

Connection Record Format

Field	Type	Required	Description
`timestamp`	float	Yes	Unix timestamp
`dst_ip`	str	Yes	Destination IP address
`dst_port`	int	Yes	Destination port
`bytes_sent`	int	Yes	Bytes sent
`bytes_recv`	int	Yes	Bytes received
`src_ip`	str	No	Source IP address
`src_port`	int	No	Source port
`protocol`	str	No	Protocol (tcp/udp)
`duration`	float	No	Connection duration in seconds

Analysis Options

Threshold

# Default threshold (0.5)
result = sentinel.analyze(connections)

# Lower threshold for higher sensitivity
result = sentinel.analyze(connections, threshold=0.3)

# Higher threshold for higher precision
result = sentinel.analyze(connections, threshold=0.7)

# Strict mode enforces minimum 0.7 threshold
result = sentinel.analyze(connections, strict_mode=True)

Context

from c2sentinel import ConnectionContext

context = ConnectionContext(
    process_name='sshd',
    known_good=True,
    ip_reputation=0.95,
    dns_queries=['api.example.com']
)

result = sentinel.analyze(connections, context=context)

Whitelist and Blacklist

sentinel.add_whitelist(ips=['8.8.8.8'], domains=['google.com'])
sentinel.add_blacklist(ips=['10.10.10.10'], domains=['malware.example'])

Result Object

The AnalysisResult object contains:

Attribute	Type	Description
`is_c2`	bool	True if C2 detected
`c2_probability`	float	Probability score (0.0-1.0)
`c2_type`	str	Detected C2 framework type
`confidence`	float	Model confidence
`detection_method`	str	Method used (signature/ml/context/whitelist)
`immediate_detection`	bool	True if signature-based
`risk_factors`	list	Factors supporting C2 classification
`mitigating_factors`	list	Factors against C2 classification
`matched_legitimate_pattern`	str	Matched legitimate pattern name
`service_type`	str	Detected service type
`recommendations`	list	Suggested actions

Batch Analysis

connection_groups = [
    [conn1, conn2, conn3],
    [conn4, conn5, conn6],
]

results = sentinel.analyze_batch(connection_groups)

Log File Parsing

with open('conn.log', 'r') as f:
    log_lines = f.readlines()

results = sentinel.analyze_logs(log_lines, group_by_dst=True)

Supported formats: JSON, Zeek conn.log, syslog

Reconnaissance

IP Analysis

info = sentinel.recon.analyze_ip('104.16.132.229')
# Returns: is_valid, is_private, is_cdn, cdn_provider, reverse_dns

Pattern Analysis

patterns = sentinel.recon.analyze_connection_patterns(connections)
# Returns: timing stats, volume stats, behavioral indicators

IOC Generation

if result.is_c2:
    iocs = sentinel.recon.generate_iocs(connections, result.to_dict())
    # Returns: ips, ports, timing_signatures, behavioral_indicators

Detection Methodology

C2 Indicators

Consistent beacon intervals (low timing variance)
Consistent packet sizes (low size variance)
Single persistent destination
Balanced request/response ratio

Signature Detection

Immediate detection for high-confidence C2 ports with matching behavioral patterns:

Port 4444 (Metasploit default)
Port 5555 (Metasploit alternative)
Port 31337 (Sliver)
Port 40056 (Havoc)

Legitimate Traffic Indicators

High response size variance
Asymmetric traffic patterns (small requests, large responses)
Multiple destinations
SSH keepalive patterns (small symmetric packets on port 22)
Health check patterns (regular intervals, variable response sizes)

Model Specifications

Specification	Value
Architecture	LogBERT Transformer
Parameters	4.9 million
Feature Dimensions	40
Encoder Layers	6
Attention Heads	8
Hidden Dimension	256
Format	Safetensors
Size	20 MB

Training Your Own Model

C2Sentinel supports training custom weights on your own data. This is useful for:

Fine-tuning on your network's specific traffic patterns
Adding detection for new C2 frameworks
Reducing false positives in your environment

Prerequisites

pip install torch numpy safetensors tqdm packaging

Using Pre-trained Weights

The released weights are trained on synthetic C2 beacon patterns covering 10+ framework types:

from c2sentinel import C2Sentinel

# Load pre-trained weights from HuggingFace
sentinel = C2Sentinel.from_pretrained('danielostrow/c2sentinel')

# Or load from local files
sentinel = C2Sentinel.load('c2_sentinel')

Training From Scratch

Use the provided training script to train on synthetic data:

# Basic training (20,000 samples, 100 epochs)
python train_model.py --epochs 100 --samples 20000

# Faster training with fewer samples
python train_model.py --epochs 50 --samples 10000

# Custom learning rate
python train_model.py --epochs 100 --samples 25000 --lr 0.0001

Training on Custom Data

Create a custom dataset class that returns connection records:

from torch.utils.data import Dataset
from c2sentinel import FeatureExtractor

class CustomC2Dataset(Dataset):
    def __init__(self, labeled_connections):
        self.feature_extractor = FeatureExtractor()
        self.samples = []
        self.labels = []

        for connections, is_c2 in labeled_connections:
            features = self.feature_extractor.extract_features(connections)
            self.samples.append(features)
            self.labels.append(1 if is_c2 else 0)

        # Normalize features (critical for training stability)
        self.samples = np.array(self.samples, dtype=np.float32)
        self.mean = np.mean(self.samples, axis=0)
        self.std = np.std(self.samples, axis=0) + 1e-8
        self.samples = (self.samples - self.mean) / self.std

    def __len__(self):
        return len(self.samples)

    def __getitem__(self, idx):
        return {
            'features': torch.tensor(self.samples[idx]),
            'label': torch.tensor(self.labels[idx], dtype=torch.float32)
        }

Fine-tuning Pre-trained Weights

Start from pre-trained weights and fine-tune on your data:

from c2sentinel import LogBERTC2Sentinel, C2SentinelConfig
from safetensors.torch import load_file, save_file
import torch.optim as optim

# Load pre-trained model
config = C2SentinelConfig()
model = LogBERTC2Sentinel(config)
state_dict = load_file('c2_sentinel.safetensors')
model.load_state_dict(state_dict)

# Fine-tune with lower learning rate
optimizer = optim.AdamW(model.parameters(), lr=0.00005, weight_decay=0.01)

# Train on your data...

# Save fine-tuned weights
save_file(model.state_dict(), 'c2_sentinel_finetuned.safetensors')

Training Tips

Feature Normalization: Always normalize input features. Save the mean/std for inference:
```
np.savez('normalization_params.npz', mean=mean, std=std)
```
Learning Rate: Use 0.0001 for training from scratch, 0.00005 for fine-tuning

Gradient Clipping: Prevent exploding gradients:

torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)

Early Stopping: Monitor validation accuracy and stop when it plateaus
Balanced Data: Use roughly equal C2 and benign samples

Model Output Files

After training, you'll have:

c2_sentinel.safetensors - Model weights
normalization_params.npz - Feature normalization parameters
c2_sentinel.json - Model configuration

Files

c2sentinel/
    c2sentinel.py              # Main module
    c2_sentinel.safetensors    # Model weights
    c2_sentinel.json           # Model configuration
    README.md                  # Documentation
    API_REFERENCE.md           # API reference
    examples/
        basic_usage.py
        advanced_usage.py

License

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.