1. Connection model
   1. Forwarding targets
   2. DnsStream Core
   3. DnsStream Assured
2. In-flight event handling
   1. drop
   2. best-effort
   3. durable
   4. Comparison
3. Ring buffer behaviour
   1. Sizing the buffer for Core
   2. Buffer behaviour in Assured
4. Queue persistence

Connection model

Forwarding targets

DnsStream maintains persistent TCP connections to its configured forwarding targets and recovers automatically from network failures without dropping events or requiring manual intervention.

A forwarding target is configured as a host and port in dnsstream.conf. The host may be an IPv4 address, an IPv6 address, or a DNS name. When a DNS name is used, DnsStream resolves it when a connection attempt is made. DnsStream cycles through all resolved addresses until one successfully connects.

forwarder ipv4 siem.corp.local 514
forwarder ipv6 siem-v6.corp.local 6514 tls=on tls-client-certificate=auto

Multiple forwarder directives may be present. Each target is resolved and managed independently.

DnsStream Core

DnsStream resolves configured hostnames and attempts to connect to each resolved address in order, stopping at the first successful connection. That single connection carries all forwarding traffic for as long as it remains healthy.

On connection failure, DnsStream will pause for a short time, then it will re-resolve the hostname and retry from the beginning.

Events accumulate in the ring buffer throughout.

Startup:
  siem.corp.local resolves to 10.0.2.10, 10.0.2.11, 10.0.2.12
  attempt 10.0.2.10 → connected
  10.0.2.11 and 10.0.2.12 not contacted

Connection drops:
  siem.corp.local resolves to 10.0.2.10, 10.0.2.11, 10.0.2.12
  attempt 10.0.2.11 → fails
  attempt 10.0.2.12 → connected
  forwarding resumes to 10.0.2.12

DnsStream Assured

Assured establishes connections to all configured forwarder targets. All connections that succeed are maintained in parallel and all receive forwarded events. When one connection drops, DnsStream continues forwarding to the remaining live connections without interruption. Reconnection to the failed address runs in the background.

Startup:
  siem1.corp.local resolves to 10.0.2.10, 10.0.2.11, 10.0.2.12
  attempt 10.0.2.10 → connected
  10.0.2.11 and 10.0.2.12 not contacted
  siem2.corp.local resolves to 10.0.3.10, 10.0.3.11, 10.0.3.12
  attempt 10.0.3.10 → connected
  10.0.3.11 and 10.0.3.12 not contacted
  all events forwarded to 10.0.2.10 and 10.0.3.10

10.0.2.11 drops:
  siem1.corp.local resolves to 10.0.2.10, 10.0.2.11, 10.0.2.12
  attempt 10.0.2.11 → fails
  attempt 10.0.2.12 → connected
  forwarding resumes to 10.0.2.12
  forwarding continues uninterrupted to 10.0.3.10

In-flight event handling

The forwarder-recovery directive controls what happens to events that were in the process of being sent when a connection failure occurs. Network errors are not always detected immediately - a write may appear to succeed at the TCP layer but the data may not have been received by the collector if the connection dropped between the write and the acknowledgement. The three modes offer a trade-off between event loss and event duplication.

forwarder-recovery durable

drop

Events that were being sent at the moment of connection failure are discarded. DnsStream reconnects and resumes forwarding from new events only.

         Send attempt
              │
    ┌─────────▼──────────┐
    │  Connection drops  │   → events discarded
    └────────────────────┘
    DnsStream reconnects
    Forwarding resumes from next event

Use when your collector deduplicates events and duplicate delivery is more disruptive than occasional loss. Not recommended for security telemetry where completeness matters.

Risk: Events in flight at the time of failure are lost and possibly events sent in the previous batch.

best-effort

On reconnection, DnsStream resends the events that were in flight at the time of the failure. This recovers events that were not received due to the connection dropping mid-send.

         Send attempt
              │
    ┌─────────▼──────────┐
    │  Connection drops  │   → events returned to the queue to be resent
    └────────────────────┘
    DnsStream reconnects
    Retained events are resent
    Forwarding resumes

This handles the common case where the failure is detected promptly and the in-flight events are clearly undelivered.

Risk: If the connection dropped after the collector received the events but before the failure was detected on the DnsStream side, resending produces duplicates. In practice this window is small.

durable

On reconnection, DnsStream resends both the events that were in flight and the events sent in the previous batch. This guards against the case where a network error is not detected until the next send attempt - meaning some apparently-delivered events may not actually have been received by the collector.

         Previous sends          Send attempt
              │                       │
    ┌─────────▼───────────────────────▼──────┐
    │  Connection drops (detected late)      │  → all returned to the queue to be resent
    └────────────────────────────────────────┘
    DnsStream reconnects
    Previous + in-flight events are resent
    Forwarding resumes

This is the default behaviour when forwarder-recovery is not configured. Recommended for deployments where event completeness is the priority and the collector can handle or deduplicate occasional duplicate events.

Risk: Higher probability of duplicate delivery compared to best-effort. The receiving collector should be capable of handling duplicates, either natively or through deduplication rules.

Comparison

Default: durable (when directive is not set).

Ring buffer behaviour

Both the Core and Assured editions buffer events in the in-memory ring buffer whenever no forwarding connection is available. The buffer decouples capture from forwarding - events continue flowing in from the ETW provider regardless of the forwarding state.

Once a connection is re-established, the forwarding thread drains the buffer in order, delivering buffered events before processing new ones. This preserves event ordering across outages.

If the buffer fills completely before a connection is restored, new events are handled according to the queue-size and overflow behaviour configured in dnsstream.conf. The eventsDropped counter in the metrics event will reflect any events lost this way.

queue-size 104857600
queue-recovery on
queue-filename dnsstream.queue

See the Forwarding queue section in the Configuration reference document for full details.

Sizing the buffer for Core

In Core deployments the entire event stream is interrupted during a recovery cycle. The buffer needs to hold all events generated during the expected worst-case outage. As a rough guide:

For Core deployments in excess of 10,000 qps, if free memory is available users can consider increasing queue-size to allow enough time for network recovery.

Buffer behaviour in Assured

In Assured deployments the buffer is only used when all connections have failed simultaneously. During a partial outage where at least one connection remains live, events are forwarded directly without accumulating in the buffer. This means the buffer’s purpose in Assured is to handle complete network partitions rather than routine single-host failures.

Queue persistence

The queue-recovery directive is separate from forwarder-recovery and controls what happens to buffered events across a service stop and restart, rather than across a connection failure within a running service.

queue-recovery on
queue-filename dnsstream.queue

The queue file is only written on a clean shutdown. Events buffered in memory at the time of a crash, forced kill, or power failure are not recoverable regardless of this setting.