OpenTelemetry

Using OpenTelemetry with HighByte Intelligence Hub

The HighByte Intelligence Hub supports running with the OpenTelemetry Java Agent out-of-the-box. Intelligence Hub metrics, logs, and JVM exporters are currently supported.

OpenTelemetry is enabled by setting the OTEL_COLLECTOR_ENDPOINT environment variable before starting the hub. You can also override or augment agent properties using OTEL_AGENT_PROPERTIES_EXTRA.

Environment Variable Description Example Value
OTEL_COLLECTOR_ENDPOINT Enables the OpenTelemetry Java Agent and sets the collector endpoint http://<collector-host>:4318
OTEL_AGENT_PROPERTIES_EXTRA Optional. Override or augment OTel properties passed to the Java Agent -Dotel.exporter.otlp.protocol=grpc

Once the variable is set, start the hub using the appropriate script for your platform.

Windows

.\start-windows.bat

Linux

./start-linux.sh

macOS

./start-macos.sh

Docker

Pass the environment variables using -e flags on the docker run command. Because there is no local collector running inside the container, a remote endpoint is required.

docker run -p 45245:45245 -p 1885:1885 -p 8885:8885 --name highbyte \
  --mount source=hb_vol,target=/usr/local/highbyte/appData \
  -e OTEL_COLLECTOR_ENDPOINT=http://<collector-host>:4318 \
  highbyte:X.X.X

Application Logs

All INFO, WARN, and ERROR event log messages are exported to OpenTelemetry. Logs are exported with the following attributes:

Attribute Name Description
highbyte.hub.eventlog.source Name of the component generating the event log message.
highbyte.hub.eventlog.type Corresponding to the HighByte message type; INFO, WARN, and ERROR

Connection Metrics

The following metrics are exported for monitoring connections in HighByte Intelligence Hub:

Read Metrics

Metric Name Type Description Attributes
highbyte.hub.connection.stats.read.pending.size Histogram Number of pending reads for a connection across all inputs highbyte.hub.connection.name
highbyte.hub.connection.stats.read.count Counter Total number of reads executed for a connection’s input highbyte.hub.connection.name
highbyte.hub.connection.input.name
highbyte.hub.connection.stats.read.error.count Counter Total number of read errors for a connection’s input highbyte.hub.connection.name
highbyte.hub.connection.input.name
highbyte.hub.connection.stats.read.duration Histogram Read duration for a connection’s input in seconds highbyte.hub.connection.name
highbyte.hub.connection.input.name

Write Metrics

Metric Name Type Description Attributes
highbyte.hub.connection.stats.write.pending.size Histogram Number of pending writes for a connection across all outputs highbyte.hub.connection.name
highbyte.hub.connection.stats.write.dropped.count Counter Number of dropped writes for a connection across all outputs highbyte.hub.connection.name
highbyte.hub.connection.stats.write.count Counter Total number of writes for a connection’s output highbyte.hub.connection.name
highbyte.hub.connection.output.name
highbyte.hub.connection.stats.write.error.count Counter Total number of write errors for a connection’s output highbyte.hub.connection.name
highbyte.hub.connection.output.name
highbyte.hub.connection.stats.write.duration Histogram Write duration for a connection’s output in seconds highbyte.hub.connection.name
highbyte.hub.connection.output.name

Pipeline Metrics

The following metrics are exported for monitoring pipelines in HighByte Intelligence Hub:

Trigger Metrics

Metric Name Type Description Attributes
highbyte.hub.pipeline.stats.trigger.count Counter Total number of triggers for a pipeline’s trigger pipeline.name
pipeline.trigger.name
highbyte.hub.pipeline.stats.trigger.error.count Counter Total number of errors for a pipeline’s trigger pipeline.name
pipeline.trigger.name

Execution Metrics

Metric Name Type Description Attributes
highbyte.hub.pipeline.stats.run.count Counter Total number of runs for a pipeline pipeline.name
highbyte.hub.pipeline.stats.write.pending.size Histogram Number of queued writes for a pipeline pipeline.name
highbyte.hub.pipeline.stats.wait.duration Histogram Wait duration for a pipeline in seconds pipeline.name
highbyte.hub.pipeline.stats.execution.duration Histogram Execution duration for a pipeline in seconds pipeline.name
highbyte.hub.pipeline.stats.error.count Counter Total number of errors for a pipeline’s stage pipeline.name
pipeline.stage.name

See JVM Metrics for more information about JVM metrics generated by the Open Telemetry Java Agent.