With the launch of Safety Engine 1.0.x, we enabled the Safety Engine to operate as an HTTP REST API, permitting it to collect alerts from different Safety Engines.
I’ll information you thru the steps to arrange the CrowdSec Safety Engine throughout a number of servers, the place one server will function the mum or dad and two further machines will ahead alerts to it.
Advantages
Sharing cybersecurity incidents throughout machines utilizing the CrowdSec Safety Engine is a extremely efficient technique to reinforce collective safety defenses. By leveraging CrowdSec’s functionality to distribute remediations amongst linked machines, every machine advantages from real-time updates about new threats detected elsewhere within the community.
Structure
Within the diagram above, the mum or dad Safety Engine, designated as server-1, might be arrange because the HTTP REST API, generally often called the LAPI (Native API). This engine might be accountable for storing and distributing the gathered alerts. Remediation is managed by means of the Remediation Parts, which depend upon the LAPI provided by server-1. It is essential to know that mitigation can happen independently from detection.
Server-2 and server-3 are designated as internet-facing machines that can host companies accessible to the general public and might be often called the kid Log Processors. On these servers, we are going to set up CrowdSec Safety Engine and Remediation Parts, which can work together with the server-1 LAPI.
Word: The phrase little one Log Processors refers to a CrowdSec Safety Engine that operates with its LAPI turned off. For extra info on this, seek the advice of our Taxonomy Replace Article.
We strongly encourage you to discover the CrowdSec Hub to study in regards to the in depth vary of companies the Safety Engine can defend. This platform showcases the varied capabilities of the Engine in securing every little thing from net purposes to databases in opposition to cyber threats.
Structure Choices
I selected a postgresql backend for the server-1 LAPI to realize better stability in database learn and write operations. Nonetheless, relying in your operational scale, you may uncover that the default SQLite with WAL (Write-Forward Logging) enabled meets your wants, in that case you may skip part 1b.
Stipulations
To comply with this tutorial, you will want the next:
Two internet-facing Ubuntu 22.04 machines internet hosting companies.
One Ubuntu 22.04 machine.
A neighborhood community connection between the Mother or father and Little one machines.
Step 1: Setup and Configure Mother or father LAPI server-1
Step 1a: Set up CrowdSec Safety Engine
Let’s set up the Safety Engine, following the set up information.
bash
curl -s https:/packagecloud.io/set up/repositories/crowdsec/crowdsec/script.deb.sh | sudo bash
sudo apt set up crowdsec
Step 1b (Elective): Utilizing postgresql on Mother or father server-1
Set up the PostgreSQL bundle utilizing the apt bundle supervisor.
bash
sudo apt set up postgresql
Subsequent, transition to the ‘postgres’ Linux consumer after which join by executing the psql command.
bash
sudo -i -u postgres
psql
You’ll be able to arrange the database and create a certified consumer utilizing the instructions under. Exchange <PASSWORD> with a password you choose, you MUST preserve it inside the single quotes.
bash
postgres=# CREATE DATABASE crowdsec;
CREATE DATABASE
postgres=# CREATE USER crowdsec WITH PASSWORD ‘<PASSWORD>‘; CREATE ROLE
postgres=# GRANT ALL PRIVILEGES ON DATABASE crowdsec TO crowdsec;
GRANT
Now, we’ll arrange the Safety Engine to make the most of this newly created database as its backend. This requires updating the db_config part within the /and so forth/crowdsec/config.yaml file.
yaml
db_config:
log_level: data
kind: postgres
consumer: crowdsec
password: “”
db_name: crowdsec
host: 127.0.0.1
port: 5432
Through the set up of the Safety Engine, the native machine was configured to make use of the SQLite database. To modify to the newly arrange postgres database, you will want to regenerate the credentials after which proceed to restart the Safety Engine.
bash
sudo cscli machines add -a –power
sudo systemctl restart crowdsec
Step 1c: Expose LAPI port
To allow communication between the LAPI and the kid Log Processors/Remediation Parts, it’s needed to regulate the LAPI’s settings to simply accept connections from exterior sources, since its default configuration binds it to the machine’s loopback tackle (127.0.0.1). This adjustment will be made by enhancing the /and so forth/crowdsec/config.yaml configuration file and altering the required settings.
yaml
api:
server:
listen_uri: 10.0.0.1:8080
Within the talked about setup, we modify the settings to hear on the ten.0.0.1 interface on port 8080. Must you want to hear on a number of interfaces, you may change this to 0.0.0.0 and implement firewall guidelines to allow particular connections.
Step 2: Setup and Configure Little one Log Processors
Step 2a: Set up CrowdSec Safety Engine
Let’s set up the Safety Engine, following the set up information.
bash
curl -s https:/packagecloud.io/set up/repositories/crowdsec/crowdsec/script.deb.sh | sudo bash
sudo apt set up crowdsec
Step 2b: Configure to make use of LAPI server
First, lets register the Log Processor to the LAPI server utilizing the next command
bash
sudo cscli lapi register -u http://10.0.0.1:8080
Make sure you modify the -u flag to fit your community. Make the most of the IP tackle if it is static, or go for the hostname in case your community permits it.
Subsequent, we’ll flip off the native API on the Safety Engine, turning it right into a Log Processor. This motion is taken as a result of the API will not be utilized, which can preserve system sources and keep away from occupying a TCP port unnecessarily.
To attain this, we will disable the API within the configuration with:
yaml
api:
server:
allow: false
Step 2c: Validate the registration request on LAPI
Since we used the cscli lapi register on the kid Log Processor we should validate the request on server-1 through the next instructions:
bash
sudo cscli machines record
NAME VERSION
IP ADDRESS
LAST UPDATE
STATUS
dc6f34b3a4994700a2e333df43728701D0iARTSQ6dxiwyMR
10.0.0.1
2021-04-13T12:16:11Z
✔️
v1.0.9-4-debian-pragmatic-a8b16a66b110ebe03bb330cda2600226a3a862d7
9f3602d1c9244f02b0d6fd2e92933e75zLVg8zSRkyANxHbC
10.0.0.3
2021-04-13T12:24:12Z
🚫
From this output, it is evident there is a new machine that hasn’t been validated but by the 🚫 inside the standing column. We have to manually validate this machine to make sure the LAPI acknowledges which machines are approved to transmit alerts.
Word: In case you do not see a brand new machine marked with a 🚫 within the standing column, be sure you are executing the command on the LAPI server.
bash
sudo cscli machines validate 9f3602d1c9244f02b0d6fd2e92933e75zLVg8zSRkyANxHbC
Ensure that to alter the argument following “validate” to correspond with the brand new machine title displayed within the record output.
Step 2nd: Restart the kid Log Processor service
On the kid Log Processor machine you may run the next command to restart the service:
bash
sudo systemctl restart crowdsec
Then, for every machine you want to join, repeat step 2. In our case, we are going to carry out this motion twice, as soon as for every Ubuntu machine.
Step 3: Organising Remediation
Now, it is essential to configure remediation measures in your internet-facing servers since merely working the Log Processor doesn’t implement enforcement actions. On this article, we’ll concentrate on establishing the Linux firewall Remediation Part. For added remediation choices, you’ll want to discover the in depth record accessible within the CrowdSec Documentation.
Step 3a: Producing API key on LAPI
First, we’ll create API token on the LAPI server by executing the next command:
bash
sudo cscli bouncers add server-2-firewall
Api key for ‘server-2-firewall’:
02954e85c72cf442a4dee357f0ca5a7c
Please preserve this key because you will be unable to retrieve it!
I used server-2-firewall because the title for the important thing, however you may select any title you favor. It is essential to pick a descriptive title for the important thing to facilitate future administration, particularly if you must revoke a key because of a token compromise.
Step 3b: Set up the Remediation Part
IPtables firewall is among the many mostly used on Linux, so we’ll proceed to put in the Part that interacts with it, utilizing the apt bundle supervisor.
bash
sudo apt set up cs-firewall-bouncer-iptables
As soon as the Part is put in, we are going to edit the configuration underneath /and so forth/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml to level in direction of the LAPI
yaml
api_url: http://10.0.0.1:8080/
api_key: 02954e85c72cf442a4dee357f0ca5a7c
Make sure you modify the api_url to align along with your LAPI tackle and replace the api_key to the one generated by the earlier command. Keep in mind you need to use both the IP tackle or the hostname. After getting altered the configuration let’s restart the firewall Remediation Part.
bash
sudo systemctl restart crowdsec-firewall-bouncer
Then, for every Remediation Part you want to join, repeat step 3. In our case, we are going to carry out this motion twice, as soon as for every firewall on the Ubuntu machines. Ensure that to change the naming scheme of the api key.
A number of closing ideas
This information illustrated the method for establishing a multi-server Safety Engine setup. Whereas this instance utilized three servers, the structure permits for simple enlargement. The useful resource consumption on server-2 and server-3 stays minimal because the majority of operations are directed in direction of server-1, facilitating easy scalability of the system:
Register and validate further Safety Engines on the LAPI server
Add any further Remediation Parts
As beforehand said, there is no requirement for the Remediation Parts and Safety Engines to be put in on the identical server. This means that the Safety Engine ought to be put in on the location the place logs are produced, whereas the Remediation Part will be deployed at any desired location.
It is essential to notice that this configuration comes with sure limitations:
The communication between Safety Engines happens through unencrypted HTTP, which is appropriate for an area community however not safe for web use. Nonetheless, the CrowdSec Safety Engine helps the usage of HTTPS for these interactions.
This text doesn’t delve into monitoring or alerting. Nonetheless, the Safety Engine helps complete monitoring capabilities through Prometheus, and yow will discover extra detailed details about it on this article.
Having each the CrowdSec LAPI and PostgreSQL on server-1 creates a single level of failure, doubtlessly resulting in delays in risk response ought to any points come up with the server.
Now you might be questioning — how do I construct a extremely accessible multi-server CrowdSec setup? We can have a devoted article on that within the coming weeks, so keep tuned!
We’re at all times more than pleased to obtain your suggestions! Don’t hesitate to succeed in out to us on our neighborhood platforms on Discord and Discourse.