Generic SIEM Integration
Receiving logs using our pre-built, generic docker container based on FluentD
Overview
The general and most generic way of receiving logs from Control D's SIEM output integration is by deploying our pre-built docker container.
This container securely terminates log data sent from our network to yours, and forwards it on to a SIEM tool within your network.
Pre-Requisites
Customer Side
- Hardware
- Virtual Machine, VPS, or Baremetal Server (X86 architecture)
- Operating System
- Linux Distribution (capable of running Docker Containers / Docker CE)
- Networking
- Public IP address (For securely receiving logs from our SIEM integration output over the internet)
If this IP is dynamic, you must provide us with a DNS record below!
- Internal network access to your SIEM system (For forwarding logs received into your system)
- Public IP address (For securely receiving logs from our SIEM integration output over the internet)
- Firewall / Container Port Mappings
- Permit inbound traffic on
tcp/24224
to the docker container
- Permit inbound traffic on
Submitting Details to Support
Our system will generate a docker config bundle file for you. Just send an e-mail to Control D Support asking for the SIEM Integration config bundle with your:
- Org ID
- Public IP address
- [Optional] DNS record (if the public IP is prone to changing / dynamic)
Installation
-
Install Docker Engine on the host. Instructions Link
-
Upload the config bundle to the host
scp yourbundle.tar.gz your-linux-host.example.com:/tmp/
-
Extract the config + container bundle:
mkdir logs-receiver tar -xvf /tmp/yourbundle.tar.gz -C logs-receiver
-
Start the container (interactively at first for testing)
cd logs-receiver docker compose up
-
Confirm that logs are arriving (they won't be going anywhere yet!)
-
Modify the configuration
logs-receiver/fluent-bit.yaml
and update theoutputs:
section to add your desired logs destination.See https://docs.fluentbit.io/manual/pipeline/outputs for instructions
fluent-bit supports a wide variety of log collectors, including ELK, Datadog and Splunk
Example: Splunk
outputs: - name: splunk host: splunk.example.com # Modify splunk_token: 55555555-5555-5555-5555-555555555555 # Modify port: 8088 # Modify tls: on # Modify tls.verify: on # Modify
-
Start the container in
daemon
mode so it continues running when you close your shell:docker compose up --detach
-
Please notify the Control D support team to confirm that you are receiving logs successfully!
Troubleshooting
- If your destination system does not receive logs, check the running fluent-bit instance for errors:
docker compose logs --follow
- If needed,
restart
thecontainer
with:docker compose up --force-recreate --detach
Example Config
Here is a sample docker compose file and fluent-bit config that you can use to start a fluentbit receiver on your end. Be mindful this not a working sample. It lists templated components that you will need to comment out or provide.
More importantly it will not actually receive anything from Control D until we create a custom configuration. Once you're ready to proceed, provide us with the org IDs you wish to receive status for and we'll send you a final config bundles.
version: "3.7"
services:
fluent-bit:
image: fluent/fluent-bit
command: ["/fluent-bit/bin/fluent-bit", "-c", "/fluent-bit/etc/fluent-bit.yaml"]
volumes:
- ./fluent-bit.yaml:/fluent-bit/etc/fluent-bit.yaml
- ./certs:/etc/ssl/fluent-bit-certs
ports:
- "24224:24224"
---
pipeline:
inputs:
- name: forward
tls: on
tls.verify: "on"
tls.ca_file: /etc/ssl/fluent-bit-certs/${CN}-ca.pem
tls.crt_file: /etc/ssl/fluent-bit-certs/${CN}-cert.pem
tls.key_file: /etc/ssl/fluent-bit-certs/${CN}-privkey.pem
outputs:
# Edit this!
# See https://docs.fluentbit.io/manual/pipeline/outputs for a full list of supported outputs
# Output logs in JSON format to the container's standard output
# Can be used to validate logs are being received, but typically should be disabled in production
- name: stdout
match: '*'
format: json
# Splunk Example
# https://docs.fluentbit.io/manual/pipeline/outputs/splunk
# - name: splunk
# host: splunk.example.com # change me
# splunk_token: 55555555-5555-5555-5555-555555555555 # change me
# match: '*'
# port: 8088
# tls: on
# tls.verify: on
Updated about 2 months ago