> Configuration

Configuration

Each component of osctrl requires configuration in order to operate properly. As of PR #754, osctrl uses a single YAML configuration file per service, consolidating all settings into one place.

BREAKING CHANGE: The previous JSON-based configuration system with multiple separate files (service.json, db.json, redis.json, jwt.json, saml.json, etc.) has been replaced with a single YAML file per service. If you are upgrading from an older version, you will need to migrate your configuration to the new YAML format.

Single YAML Configuration

All osctrl services (osctrl-tls, osctrl-admin, osctrl-api) now use a unified YAML configuration file that includes all necessary settings in one place. The default filenames are:

tls.yaml for osctrl-tls
admin.yaml for osctrl-admin
api.yaml for osctrl-api

You can specify a different configuration file using the --config-file or -C flag.

Generating and Validating Configuration

Each osctrl binary includes built-in commands to help you create and validate configuration files:

Generate Configuration

To generate a valid YAML configuration file with default values:

./osctrl-tls config-generate -f tls.yaml
./osctrl-admin config-generate -f admin.yaml
./osctrl-api config-generate -f api.yaml

This will create configuration files (tls.yaml, admin.yaml, api.yaml) populated with all required fields and default settings. If not using -f, the file will be created in config/<service>.yml by default.

The generated configuration includes:

All required configuration sections
Default values for each setting
Service-specific options relevant to each binary

Verify Configuration

To validate an existing configuration file before starting the service:

./osctrl-tls config-verify --file tls.yaml
./osctrl-admin config-verify --file admin.yaml
./osctrl-api config-verify --file api.yaml

The verification process checks:

YAML syntax validity
Required fields are present
Configuration values are within valid ranges
Authentication method compatibility
Logger and carver type validity
Database and Redis connection parameters

If the configuration is valid, the command will exit with status code 0. If there are errors, detailed error messages will be displayed indicating what needs to be fixed.

It’s recommended to run config-verify on your configuration files before deploying to production to catch configuration errors early.

Configuration Structure

The YAML configuration file contains the following main sections:

Service Configuration

Basic service settings including network listener, ports, authentication, and logging:

service:
  listener: "0.0.0.0"               # Network interface to bind to
  port: "9000"                      # TCP port for the service
  host: "tls.example.com"           # Public hostname
  auth: "none"                      # Authentication method
  logLevel: "info"                  # Logging level: debug, info, warn, error
  logFormat: "json"                 # Log format: json, console
  auditLog: false                   # Enable audit logging for sensitive actions

Authentication types (auth):

none - No authentication
db - Users stored and retrieved from backend database
saml - SAML-based authentication
headers - Header-based authentication (middleware approach)
jwt - JWT token-based authentication

Database Configuration

Backend database connection settings:

db:
  type: "postgres"                  # Database type: postgres, mysql, sqlite
  host: "127.0.0.1"
  port: "5432"
  name: "osctrl"
  username: "postgres"
  password: "postgres"
  sslmode: "disable"                # SSL mode for database connection
  maxIdleConns: 20                  # Maximum idle connections
  maxOpenConns: 100                 # Maximum open connections
  connMaxLifetime: 30               # Connection max lifetime (minutes)
  connRetry: 5                      # Connection retry timeout (seconds, 0=no retry)
  filePath: "./osctrl.db"           # For SQLite only

Redis Configuration

Cache configuration for session management and performance:

redis:
  host: "127.0.0.1"
  port: "6379"
  password: ""
  db: 0
  connectionString: ""              # Optional: full connection string
  connRetry: 5                      # Connection retry timeout (seconds)

Logger Configuration

Logging destination and settings:

logger:
  type: "db"                        # Logger type: db, splunk, graylog, s3, file, kinesis, kafka, logstash, elastic
  loggerDBSame: true                # Use same DB config as main DB for logging
  alwaysLog: false                  # Always log to DB regardless of other loggers

  # Database logger settings (if type: db and loggerDBSame: false)
  db:
    type: "postgres"
    host: "127.0.0.1"
    # ... (same structure as main db config)

  # S3 logger settings (if type: s3)
  s3:
    bucket: "osctrl-logs"
    region: "us-east-1"
    accessKey: "your-access-key"
    secretAccessKey: "your-secret-key"

  # Splunk logger settings (if type: splunk)
  splunk:
    url: "https://splunk.example.com:8088/services/collector"
    token: "your-hec-token"
    host: "osctrl"
    index: "osquery"

  # Graylog logger settings (if type: graylog)
  graylog:
    url: "http://graylog.example.com:12201/gelf"
    host: "osctrl"
    queries: "osquery_queries"
    status: "osquery_status"
    results: "osquery_results"

  # File logger settings (if type: file)
  local:
    filePath: "/var/log/osctrl/osctrl.log"
    maxSize: 25                     # Max file size in MB before rotation
    maxBackups: 5                   # Number of old log files to retain
    maxAge: 10                      # Max days to retain old log files
    compress: true                  # Compress rotated files with gzip

Logger types:

db - Store logs in the backend database
splunk - Send logs to Splunk HEC
graylog - Send logs to Graylog GELF endpoint
s3 - Upload logs to AWS S3
file - Write logs to local file with rotation
kinesis - Send logs to AWS Kinesis
kafka - Send logs to Kafka topics
logstash - Send logs to Logstash
elastic - Send logs directly to Elasticsearch

Carver Configuration

File carving storage settings:

carver:
  type: "db"                        # Carver type: db, s3, local

  # S3 carver settings (if type: s3)
  s3:
    bucket: "osctrl-carves"
    region: "us-east-1"
    accessKey: "your-access-key"
    secretAccessKey: "your-secret-key"

  # Local carver settings (if type: local)
  local:
    carvesDir: "/var/osctrl/carves" # Directory to store carved files

Carver types:

db - Store carved files in the database
s3 - Upload carved files to AWS S3
local - Store carved files in local directory

TLS Configuration

TLS termination settings:

tls:
  termination: true                 # Enable TLS termination
  certificateFile: "/path/to/cert.pem"
  keyFile: "/path/to/key.pem"

Osquery Configuration

Osquery-specific settings for TLS service:

osquery:
  version: "5.12.1"                 # Osquery version
  tablesFile: "data/5.12.1.json"   # Path to osquery tables JSON
  logger: true                      # Enable remote TLS logger endpoint
  config: true                      # Enable remote TLS config endpoint
  query: true                       # Enable remote TLS query endpoints
  carve: true                       # Enable remote TLS carver endpoints

Metrics Configuration

Prometheus metrics endpoint settings (TLS service only):

metrics:
  enabled: false                    # Enable Prometheus metrics
  listener: "0.0.0.0"
  port: "9090"

Admin UI Configuration

Admin service-specific settings:

admin:
  sessionKey: "your-session-key"    # Secret key for session cookies
  staticDir: "/path/to/static"      # Static files directory
  staticOffline: false              # Use offline static files
  templatesDir: "/path/to/templates"
  backgroundImage: "/path/to/background.svg"
  brandingImage: "/path/to/brand.png"

JWT Configuration

JWT authentication settings:

jwt:
  jwtSecret: "your-jwt-secret"      # Secret for signing JWT tokens
  hoursToExpire: 3                  # Token expiration time in hours

SAML Configuration

SAML authentication settings (Admin service):

saml:
  certPath: "/path/to/saml-cert.pem"
  keyPath: "/path/to/saml-key.pem"
  metadataUrl: "https://idp.example.com/metadata"
  rootUrl: "https://admin.example.com"
  loginUrl: "https://admin.example.com/login"
  logoutUrl: "https://admin.example.com/logout"
  jitProvision: true                # Just-in-time user provisioning
  spInitiated: false                # Service Provider initiated flow

Osctrld Configuration

Settings for osctrld daemon endpoints:

osctrld:
  enabled: false                    # Enable osctrld endpoints

Batch Writer Configuration

Database batch writer settings for TLS service:

batchWriter:
  writerBatchSize: 50               # Events before flushing
  writerTimeout: 60s                # Max wait time before flush
  writerBufferSize: 2000            # Event channel buffer size

Debug Configuration

HTTP request debugging:

debug:
  enableHTTP: false                 # Enable HTTP request debugging
  httpFile: "debug-http.log"        # File to dump HTTP requests
  showBody: false                   # Include request body in dumps

Example Configuration Files

osctrl-tls (tls.yaml)

service:
  listener: "0.0.0.0"
  port: "9000"
  host: "tls.example.com"
  auth: "none"
  logLevel: "info"
  logFormat: "json"

db:
  type: "postgres"
  host: "127.0.0.1"
  port: "5432"
  name: "osctrl"
  username: "postgres"
  password: "postgres"
  sslmode: "disable"
  maxIdleConns: 20
  maxOpenConns: 100
  connMaxLifetime: 30
  connRetry: 5

redis:
  host: "127.0.0.1"
  port: "6379"
  password: ""
  db: 0
  connRetry: 5

logger:
  type: "db"
  loggerDBSame: true
  alwaysLog: false

carver:
  type: "db"

tls:
  termination: false

osquery:
  version: "5.12.1"
  tablesFile: "data/5.12.1.json"
  logger: true
  config: true
  query: true
  carve: true

metrics:
  enabled: true
  listener: "0.0.0.0"
  port: "9090"

osctrld:
  enabled: false

batchWriter:
  writerBatchSize: 50
  writerTimeout: 60s
  writerBufferSize: 2000

debug:
  enableHTTP: false
  httpFile: "debug-http-tls.log"
  showBody: false

osctrl-admin (admin.yaml)

service:
  listener: "0.0.0.0"
  port: "9001"
  host: "admin.example.com"
  auth: "db"
  logLevel: "info"
  logFormat: "json"
  auditLog: true

db:
  type: "postgres"
  host: "127.0.0.1"
  port: "5432"
  name: "osctrl"
  username: "postgres"
  password: "postgres"
  sslmode: "disable"
  maxIdleConns: 20
  maxOpenConns: 100
  connMaxLifetime: 30
  connRetry: 5

redis:
  host: "127.0.0.1"
  port: "6379"
  password: ""
  db: 0
  connRetry: 5

logger:
  type: "db"
  loggerDBSame: true

carver:
  type: "local"
  local:
    carvesDir: "/var/osctrl/carves"

tls:
  termination: false

osquery:
  version: "5.12.1"
  tablesFile: "data/5.12.1.json"

jwt:
  jwtSecret: "change-this-secret"
  hoursToExpire: 3

admin:
  sessionKey: "change-this-session-key"
  staticDir: "static"
  staticOffline: false
  templatesDir: "tmpl_admin"
  backgroundImage: "static/img/circuit.svg"
  brandingImage: "static/img/brand.png"

debug:
  enableHTTP: false
  httpFile: "debug-http-admin.log"
  showBody: false

Command-Line Flags

All configuration values can be overridden using command-line flags. Use --help or -h to see all available flags for each service:

$ ./osctrl-tls --help
$ ./osctrl-admin --help
$ ./osctrl-api --help

Common flags:

--config or -c: Enable configuration from YAML file
--config-file or -C: Path to YAML configuration file
--db-host: Database host
--db-port: Database port
--redis-host: Redis host
--listener or -l: Service listener
--port or -p: Service port

Environment Variables

Configuration values can also be set using environment variables. Each flag has a corresponding environment variable (see --help output for details).

Example:

export SERVICE_PORT="9000"
export DB_HOST="postgres.example.com"
export REDIS_HOST="redis.example.com"

Migration from JSON to YAML

If you’re upgrading from the old JSON-based configuration, you’ll need to consolidate your separate JSON files into a single YAML file:

Old (JSON):

tls.json - Service settings
db.json - Database settings
redis.json - Redis settings
jwt.json - JWT settings
saml.json - SAML settings
logger_tls.json - Logger settings
carver_tls.json - Carver settings

New (YAML):

tls.yaml - All settings in one file

All configuration is now organized into sections within a single YAML file, making it easier to manage and version control your osctrl deployment configuration.