Streams
Streams are append-only, immutable logs of data with absolute ordering per partition.
Streams are deployed as independent StatefulSets within a Kubernetes cluster, based purely on a Stream CRD stored in Kubernetes.
Each pod of a Stream's StatefulSet acts as a leader of its own partition. Kubernetes guarantees the identity, stable storage and stable network address of each pod. Replication is based on a deterministic algorithm and does not require consensus due to these powerful identity and stability properties.
apiVersion: hadron.rs/v1beta1
kind: Stream
metadata:
## The name of this Stream.
##
## The generated Kubernetes StatefulSet of this Stream will have the same name.
name: :string
## The Kubernetes namespace of this Stream.
##
## The generated Kubernetes Secret will live in the same namespace.
namespace: :string
spec:
## The number of partitions this Stream should have.
##
## This directly corresponds to the number of replicas of the
## backing StatefulSet of this Stream. Scaling this value up or down will
## cause the backing StatefulSet to be scaled identically.
partitions: :integer
## When true, the Stream processes will use `debug` level logging.
debug: :bool
## The data retention policy for data residing on this Stream.
retentionPolicy:
## The retention strategy to use.
##
## Allowed values:
## - "retain": this strategy preserves the data on the Stream indefinitely.
## - "time": this strategy preserves the data on the Stream for the amount of time
## specified in `.spec.retentionPolicy.retentionSeconds`.
strategy: :string (default "time")
## The number of seconds which data is to be retained on the Stream before it is deleted.
##
## This field is optional and is only evaluated when using the "time" strategy.
retentionSeconds: :integer (default 604800) # Default 7 days.
## The full image name, including tag, to use for the backing StatefulSet pods.
image: :string
## The PVC volume size for each pod of the backing StatefulSet.
pvcVolumeSize: :string
## The PVC storage class to use for each pod of the backing StatefulSet.
pvcStorageClass: :string
## The PVC access modes to use for each pod of the backing StatefulSet.
pvcAccessModes: [:string]