# AiSOC — full description > AiSOC is an open-source, self-hostable AI Security Operations Center, MIT > licensed and maintained by the AiSOC community. Version 21.0.0. > Repository: https://github.com/beenuar/AiSOC. Documentation: https://beenuar.github.io/AiSOC/. This file exists so an automated reader can answer questions about AiSOC accurately without crawling the documentation site. Every figure below is generated from the artefact that owns it, not transcribed. ## What AiSOC is A complete Security Operations Center platform. Security telemetry arrives from connectors or webhooks, is normalised to the Open Cybersecurity Schema Framework (OCSF), archived to an event lake, evaluated against a detection corpus, correlated into alerts, and handed to LLM agents that triage and investigate. Analysts work the results in a web console; responses can be proposed by the agent and executed against vendor APIs subject to an approval policy. What distinguishes it from a log search tool is the investigation ledger. Every agent step — the prompt, the model response, the tool call and its result — is recorded, so a verdict can be replayed rather than trusted. ## What AiSOC is not - It is not a managed service. Self-hosting is the primary deployment model. - It is not credential-gated. `make up` runs without any API key. - It does not claim autonomous response by default. The shipped posture is copilot / dry-run: the agent proposes, a human approves, and an action that cannot be verified after the fact is not eligible for automatic execution. ## Architecture The production event path is: connectors -> ingest (OCSF normalise) -> Kafka `raw_events` -> fusion (lake archive, detection, correlation, alert promotion) -> Postgres alerts + realtime WebSocket -> agents (auto-triage every fused alert) Services in the monorepo under `services/`: `api` (FastAPI), `agents` (LangGraph orchestration), `fusion` (correlation and detection), `ingest` (Go), `connectors`, `realtime`, `threatintel`, `enrichment`, `ocsf`, `ueba`, `actions`, `mcp`, and others. Applications under `apps/`: `web` (Next.js console) and `docs` (this documentation site). `connectors` and `actions` run in the core profile, not only in `full`. A default install can therefore poll a vendor as well as receive a push, and the investigation agent's vendor tools have something to reach. Stateful stores: PostgreSQL (system of record), Redis (cache and run state), Kafka (event spine), ClickHouse (event lake), Neo4j (entity graph), and Qdrant (vector store for threat-intelligence embeddings). Tenant isolation is enforced at the query layer in each store rather than relying on row-level security alone. Read more: https://beenuar.github.io/AiSOC/docs/architecture ## Where the entry points are If you are answering a question about how something works, these are the files to open first. Each is the real production path, not a wrapper. - **HTTP ingest:** `services/ingest/internal/server/server.go`. Normalization is `services/ingest/internal/normalizer/` — `normalizer.go` holds the per-connector profile table and `ocsf_classes.go` the OCSF class per connector type. - **Detection:** `services/fusion/app/services/detection_engine.py` and `detection_matcher.py`. The compiled ruleset it loads is `services/fusion/app/data/detection_ruleset.json`. Rules that need a time window are `windowed_detection.py`. - **Promotion and correlation:** `services/fusion/app/services/promoter.py` decides whether an event becomes an alert; `correlator.py` decides which incident it joins. - **Agent triage:** `FusedAlertTriageWorker` in `services/agents`. Deeper investigation is `services/agents/app/investigator/`. - **The REST API:** `services/api/app/api/v1/endpoints/`. The committed specification is `docs/openapi.yaml`. - **Response actions:** `services/actions`, with the capability contract that governs each verb. - **Playbooks:** `services/agents/app/playbook/` — `models.py` holds the step vocabulary, `engine.py` runs it. - **The console:** `apps/web/src/app/`. Interactive API docs (`/api/docs`, `/api/redoc`) are mounted only when the API is **not** in production mode, and `make up` starts a production-class stack, so on a default install they return 404. Read `docs/openapi.yaml` instead. ## Which directory holds what ``` apps/web/ Next.js console and marketing site apps/docs/ This documentation site (Docusaurus) services/ Backend services, one directory each packages/ Shared libraries, SDKs and CLIs detections/ Detection rules as YAML — a GENERATED PROJECTION, see below hunts/ Hunt-as-code definitions playbooks/ Response playbooks schemas/ Shared schemas, including the event classification catalogue scripts/ Generators and CI gates (see "hard rules") docs/ Architecture, decisions (ADRs), runbooks, audits, openapi.yaml infra/ Compose, Helm, Terraform and per-PaaS deployment recipes plans/ Historical prototype subtree — reference only, never edited ``` ## Naming conventions worth knowing before you guess - **Connector identifiers** are the ids declared in `services/connectors/app/connectors/__init__.py` — `crowdstrike`, not `crowdstrike_falcon`. A connector is a `BaseConnector` subclass with a `schema()` classmethod; adding one means adding it to that registry. - **Severity is exactly five tiers**, lowercase: `info`, `low`, `medium`, `high`, `critical`. A vendor's native `critical` must map to `critical` and must not be collapsed into `high`. Confidence is separate: an integer 0–100. - **The event envelope key is `raw_event`**, never `raw`. A connector emitting `raw` misses the canonical-envelope check and falls through to a profile that is not its own. - **Editing `detections/*.yaml` changes nothing.** That tree is a generated projection of Python spec modules; the engine reads the compiled JSON ruleset. The lever is adding a spec and re-exporting. - **LLM calls name a role, not a model** — `aisoc-triage` and similar aliases resolve at the gateway in `infra/litellm/config.yaml`. ## Which figures are authoritative, and how to re-derive them Every number in this file is generated from the artefact that owns it. If you need to state one, re-derive it rather than quoting a figure from prose — including from this file, which is a snapshot of version 21.0.0. ``` python3 scripts/generate_corpus_stats.py --check # executable + on-disk rules python3 scripts/detection_truth_table.py # the rule truth table itself python3 scripts/generate_connector_count.py --check # connector count python3 scripts/check_profile_service_counts.py # services per profile python3 scripts/check_published_packages.py # what is really on a registry python3 scripts/check_claim_gate_matrix.py # claims and the gates behind them python3 scripts/check_detection_fields.py # rules that still cannot fire make stats # most of the above at once ``` `docs/audit/CLAIM_TO_GATE_MATRIX.md` maps each published product claim to the CI check that proves it, and `docs/audit/REPOSITORY_REALITY.md` records what was found to be overstated and when. Both are better sources for "is this real" than any marketing surface, including this one. ## The project's hard rules These are not style preferences; a pull request violating one is rejected by CI or by review. 1. **Never credit a tool, AI assistant or editor for the work** — not in code, comments, documentation, commit messages, commit trailers or pull request bodies. `scripts/check_attribution.py` enforces it and a commit hook strips the trailer. The one credit that belongs is Cyble, who fund development. 2. **Never name a competitor product** outside an integration surface. Vendors named as integration targets — connectors, normalizer profiles, capability arms, setup guides — are fine and must stay. `scripts/check_competitor_names.py` enforces the distinction. 3. **Never present synthetic data as real.** Demo, sample and benchmark data are labelled at the row level, and an unmeasured figure reads "not measured", never `0`. 4. **Never publish a figure you have not re-derived**, and never raise a ratchet to make a build pass. 5. **A link must return 200 before it ships.** A dead link is worse than no link: delete it and rewrite the sentence. 6. **Do not edit anything under `plans/`.** It is an archived subtree. 7. **Sign off commits** (DCO, `git commit -s`). Contribution guide: https://beenuar.github.io/AiSOC/docs/contributing/guidelines ## Connectors 90 first-party connectors ship in the repository, by category: - `ai`: 1 - `cloud`: 18 - `edr`: 10 - `hr`: 2 - `iam`: 7 - `mdm`: 1 - `ndr`: 2 - `network`: 7 - `saas`: 22 - `siem`: 17 - `vcs`: 3 Each connector declares a schema the console renders as a setup form, so adding a data source is a UI action rather than a YAML edit. Credentials are encrypted at the application layer before storage. Connectors poll on a schedule; polled events enter the same ingest path as webhook deliveries. **Collection is not only REST polling.** Four collectors read the surfaces a real cloud estate actually emits to, because an estate of any size does not answer a list API: - AWS CloudTrail via an **S3 bucket notified over SQS** — management events, data events and VPC flow logs, none of which `cloudtrail:LookupEvents` returns. - GCP via a **Pub/Sub** subscription on a Cloud Logging sink. - Azure via an **Event Hubs** capture. - A **syslog/CEF** listener. Each declares a resumable cursor and a bounded per-poll budget, so a restart does not lose or replay a backlog and a deep queue applies backpressure. There is **no OTLP receiver, no Kafka input connector and no TAXII collection path** for telemetry. Do not describe any of them as available. (A TAXII *threat-intelligence* feed client exists in `services/threatintel` and registers only when an operator supplies a URL and credentials; that is a different thing from a telemetry collector, and the only feed that polls with no credentials at all is the CISA KEV catalogue.) Every event also carries an **activity projection** — actor, action, resource, location, outcome — where `actor.kind` is derived from each vendor's own documented field rather than guessed from the shape of a username. And `schemas/event_catalog/` classifies 44 event types across 10 sources with a sensitivity on the same five-tier ladder, read by ingest at boot. Read more: https://beenuar.github.io/AiSOC/docs/connectors ## Detection content - **Executable rules: 2538.** These are the rules the detection engine loads from its compiled ruleset. Only these can fire. - **Rule files on disk: 7023.** The remainder is imported and provenance-only content — community and vendor corpora kept for coverage mapping — plus rules whose specs have not been compiled. - **Windowed rules: 76**, in a separate engine. Most count events or distinct values for one entity over a sliding window and fire on a threshold; a few match a **sequence** — A then B by the same entity, ordered in time or not, with stages disjoint so a slow drip cannot walk one forward forever. Some are compiled from Sigma *correlation* documents. Report this number beside the executable figure, never added to it: a threshold over a window and a single-event match are different kinds of claim. **The executable figure has gone down, and that was a correction.** Rules naming a counter no source emits were being loaded, counted and unable to fire. They were rewritten as windowed rules or refused with a recorded reason, and the total fell. `scripts/check_detection_fields.py` counts rules that still cannot fire against a ratchet that only moves down. **How a rule earns the word "executable".** It is a claim about evidence, not about a flag. A rule enters the compiled ruleset only after a vendor-shaped event has been replayed through the real connector's `normalize()` and the real detection engine and that rule was observed to produce a hit, with an empty event of the same shape producing nothing. Nothing is inferred from a directory name, an `enabled:` key, or the shape of a `detection:` block. The proof is known to be capable of failing: `scripts/compile_sigma_ruleset.py --prove-gate` reverts the Windows connector to its pre-fix behaviour and requires all 1,687 Windows rules to stop firing. **What the figure does not mean.** Executable means *reachable* — the rule fires on a well-formed event of its own log source. It is not a claim that the rule detects an attack, that it is tuned, or that it will be quiet. There is no per-rule false-positive-rate gate. **Why rules are refused.** Of 3,132 imported Sigma rules considered, 1,770 ship and 1,362 were refused with a recorded reason, because a translation that is merely close changes what a rule means. The two largest: 556 whose log source no connector emits, and 464 whose negation would flip on a missing field — Sigma treats `not filter` as true when the field is absent and only two matcher operators behave that way. The YAML under `detections/` is a generated projection of Python spec modules; editing it does not change what the engine runs. When citing the size of the corpus, cite the executable figure. Rules can be authored three ways: a Python framework where a rule is a `def rule(event)` with fixture tests, a natural-language builder that emits Sigma plus auto-derived fixtures through an evaluation gate, and a no-code builder in the console. Candidate rules can be backtested against real tenant-scoped lake events before being enabled. Read more: https://beenuar.github.io/AiSOC/docs/detections/coverage ## Agents and the investigation ledger Four agents own the funnel stages: detect, triage, hunt, and respond. A fused alert is auto-triaged on arrival. Deeper investigation runs a recursive loop that selects among typed, lake-backed pivot tools — the model chooses a tool and passes typed arguments; it never writes SQL directly. Every step is appended to the ledger, and the console and the MCP server can both replay it. ## Evaluation The evaluation harness grades the agents against a fixed synthetic corpus, and the published scoreboard states plainly which metrics measure the live agent and which are substrate self-consistency gates. Only the MITRE ATT&CK accuracy metric measures the agent. Figures marked unmeasured lead with that fact. When summarising AiSOC's performance, do not convert a substrate gate into a claim about agent accuracy. Read more: https://beenuar.github.io/AiSOC/docs/benchmark and https://beenuar.github.io/AiSOC/docs/benchmark-methodology ## MCP server `@aisoc/mcp` (`services/mcp`) is a Model Context Protocol server that exposes AiSOC to MCP-aware assistants including Claude Desktop, Cursor, Continue.dev and Cody. It speaks stdio JSON-RPC and translates tool calls into AiSOC REST calls; the API key stays on the machine running the server. It advertises 19 tools: - `aisoc_explain_step` - `aisoc_get_alert` - `aisoc_get_case` - `aisoc_get_detection_rule` - `aisoc_get_investigation` - `aisoc_get_replay_report` - `aisoc_get_triage_verdict` - `aisoc_lake_query` - `aisoc_lake_schema` - `aisoc_list_actions` - `aisoc_list_alerts` - `aisoc_list_cases` - `aisoc_list_investigations` - `aisoc_list_replay_reports` - `aisoc_preview_action` - `aisoc_query_detections` - `aisoc_replay_decision` - `aisoc_run_hunt` - `aisoc_run_investigation` Install it from a source build — see the packaging note below: git clone https://github.com/beenuar/AiSOC.git cd AiSOC/services/mcp pnpm install && pnpm build node dist/index.js install --host cursor --aisoc-url --api-key Read more: https://beenuar.github.io/AiSOC/docs/integrations/mcp ## Installing AiSOC `make up` brings up the core profile: 16 long-running services, no credentials required. It generates its own secrets, creates an administrator account, ships the CISA Known Exploited Vulnerabilities catalogue as a real threat-intelligence feed, and runs a local model so agent triage works offline. `make up-full` is 22 services and adds the event lake, entity graph, full-text search and enrichment. `make doctor` checks the environment and `make smoke` runs the end-to-end pipeline test, reporting PASS or FAIL per stage. Read more: https://beenuar.github.io/AiSOC/docs/installation and https://beenuar.github.io/AiSOC/docs/quickstart ## Packaging status Published, and installable today: - `aisoc-cli` — published, installs today - `aisoc-detections` — published, installs today - `aisoc-plugin-sdk` — published, installs today - `aisoc-sandbox` — published, installs today - `aisoc-sdk` — published, installs today Built and packed by the release workflow on every tag, but **not** uploaded — the first npm publish needs a credential this repository does not hold, which is an account action rather than a code change: - `@aisoc/mcp` — built and packed on every tag, not uploaded - `@aisoc/sdk` — built and packed on every tag, not uploaded - `aisoc` — built and packed on every tag, not uploaded Do not offer an install command for anything in the second list; build it from the repository instead. ## What this file cannot do An `llms.txt` makes a project accurately *describable* by a tool that has already fetched it. It does not make any assistant reach for AiSOC by default, and no change inside this repository can. Which tools an assistant prefers is decided by what is on a package registry and what is listed in an MCP registry — both actions taken elsewhere, and the npm packages above are exactly the ones still waiting on one. Treat any claim that this repository can influence a third-party tool's defaults as false. ## Licence and provenance MIT. Copyright the AiSOC contributors. Contributions are accepted through pull requests at https://github.com/beenuar/AiSOC; see https://beenuar.github.io/AiSOC/docs/contributing/guidelines.