Skip to main content

The knowledge corpus

Vesper's answers are only as good as what it retrieves. This page describes what is in the corpus, how it gets there, and what is deliberately kept out of it.

What is in the corpus​

The corpus is built from resolved community support conversations across the channels where Wazuh support actually happens:

  • the community Slack workspace,
  • the community Discord server,
  • the Google Groups mailing list,
  • GitHub issues.

Conversations are rebuilt into complete threads (the original question plus every reply), so an answer is grounded in the whole exchange, not an out-of-context fragment. Threads that never received an answer are excluded: they carry a question but nothing to ground an answer in.

How threads are processed​

  1. Extraction. Raw parent and comment documents are pulled from the ticket store and grouped into threads by their parent/child identifiers.
  2. Normalization and redaction. Sensitive and identifying fields are stripped before anything is stored (see below).
  3. Embedding. Each normalized thread is embedded into a 1024-dimension vector with Amazon Titan Embed Text v2.
  4. Indexing. Vectors and thread metadata are upserted into the retrieval index that serves your questions.

The same embedding model is used for the corpus and for your questions, which is what makes the similarity scores in the sources panel meaningful.

What never reaches the corpus​

Personal identities are removed at normalization time, before storage. Author names, senders, assignees, per-platform user identifiers and message permalinks are all on the strip list; a thread survives as its technical content (title, question, replies, timestamps, channel) and nothing else. The original raw documents stay in the source system and are never copied into the retrieval corpus.

Freshness​

The corpus is refreshed by re-running the ingestion pipeline against the source ticket store. Ingestion is incremental, with an overlap window so late replies to older threads are not missed, and answered-thread pruning keeps dead weight out of retrieval.