## The Open Source Rebel: The ELK Stack (Elastic Stack)

Not every company has the budget for Splunk or Sentinel. Enter the **ELK Stack**, the world's most popular open-source log management platform.

ELK stands for **E**lasticsearch, **L**ogstash, and **K**ibana. (Though now it includes **B**eats, so sometimes it's called the *Elastic Stack*).

### The Components

#### 1. Beats (The Shippers)
Lightweight data shippers that you install on your servers (Agents).
*   **Filebeat**: Tails log files. Use this for Apache, Nginx, or custom app logs.
*   **Winlogbeat**: Ships Windows Event Logs.
*   **Packetbeat**: Sniffs network traffic (like Wireshark) and sends metadata.
*   **Auditbeat**: Linux audit framework data (File Integrity Monitoring).
*   *Role*: "Go get the data and push it."

#### 2. Logstash (The Plumber)
The heavy-duty processing pipeline. Logstash receives data from Beats (or Syslog), parses it, filters it, and transforms it.
*   **Input**: Receive data on TCP 5044.
*   **Filter**: This is where the magic happens.
    *   *Grok*: A plugin to parse unstructured text into fields using Regex comparisons.
    *   *GeoIP*: Adds Latitude/Longitude based on IP address.
    *   *Drop*: Discard noise (e.g., "Don't store debug logs").
*   **Output**: Send to Elasticsearch.
*   *Role*: "Clean, format, and enrich the data."

#### 3. Elasticsearch (The Brain / Database)
A distributed, RESTful search and analytics engine. It is a NoSQL database that stores data in JSON format.
*   It is built for **speed**. You can search through billions of documents in milliseconds.
*   It uses an "Inverted Index" (like the index at the back of a textbook). It doesn't scan every row; it looks up the word "Error" and instantly knows which rows contain it.
*   *Role*: "Store the data and find it fast."

#### 4. Kibana (The Face)
The visualization dashboard.
*   This is where the Analyst lives. You query Elasticsearch and visualize the results.
*   Create pie charts, line graphs, maps, and data tables.
*   **Canvas**: A feature to create pixel-perfect infographic presentations for management.

### Elastic Security
In recent years, Elastic has added a dedicated "Security" app on top of ELK.
*   **Detection Engine**: Runs automated rules (like Splunk correlation matches).
*   **Prebuilt Rules**: Elastic provides hundreds of open detection rules mapped to MITRE ATT&CK.
*   **Endpoint Security**: Elastic Agent can now prevent malware, not just log it.

### Why use ELK?
1.  **Cost**: The core is open source (Free). You pay for hardware/cloud hosting, but no licensing fees for data volume.
2.  **Flexibility**: You have total control over every configuration file. You can build exactly what you need.
3.  **Community**: Massive community support.

### The Trade-off
ELK is "Free as in Puppies," not "Free as in Beer."
*   Managing an Elasticsearch cluster is **hard**.
*   Sharding, replication, heap memory management, and upgrades require specialized DevOps skills.
*   If your cluster crashes during an attack, you are blind, and there is no 1-800 number to call (unless you pay for Enterprise support).

### Analyst Workflow in Kibana
1.  **Discover Tab**: Review raw logs. Filter by `event.category: "authentication"`.
2.  **Visualize**: Create a "Vertical Bar" chart of `user.name` where `event.outcome: "failure"`.
3.  **Dashboard**: Pin that chart to your "Morning Coffee" dashboard to spot anomalies instantly.