Observers¶

The observer classes are monitoring the application being tested and extract useful events. Such events are either required by the Policies in order to evolve the tests, or tracked by the Trackers in order to calculate the test results.

HTTPTimingObserver¶

class performance.driver.classes.observer.HTTPTimingObserver(*args, **kwargs)[source]¶

The HTTP Timing Observer is performing HTTP requests to the given endpoint and is measuring the request and response times.

observers:
  - class: observer.HTTPTimingObserver

    # The URL to send the requests at
    url: http://127.0.0.1:8080/v2/apps

    # [Optional] The interval of the reqeusts (seconds)
    interval: 1

    # [Optional] The body of the HTTP request
    body: |
      {
        "cmd": "sleep 1200",
        "cpus": 0.1,
        "mem": 64,
        "disk": 0,
        "instances": {{instances}},
        "id": "/scale-instances/{{uuid()}}",
        "backoffFactor": 1.0,
        "backoffSeconds": 0
      }

    # [Optional] The HTTP Verb to use (Defaults to 'GET')
    verb: POST

    # [Optional] The HTTP headers to send
    headers:
      Accept: text/plain

This observer is publishing a HTTPTimingResultEvent every time a sample is taken. Refer to the event documentation for more details.

JMXObserver¶

class performance.driver.classes.observer.JMXObserver(*args, **kwargs)[source]¶

The JMX Observer connects to the java management console of a running java application and extracts the given metrics.

observers:
  - class: observer.JMXObserver

    # [Optional] Re-send measured values on ParameterUpdateEvent
    resendOnUpdate: yes

    # Connection information
    connect:

      # [Optional] Specify the host/port where to connect
      host: 127.0.0.1
      port: 9010

      # [Optional] Execute the given shell expression and assume the STDOUT
      # contents is the PID where to attach. If available, {{cmdlinepid}}
      # will contain the PID of the last detected PID from the cmdline
      # channel
      # ------------------------------------------------------------------
      # DANGER!! Evaluated as a shell expression
      # ------------------------------------------------------------------
      pid_from: "pgrep -P $(pgrep -P {{cmdlinepid}})"

    # Which metrics to extract
    metrics:

      # Specify the name of the metric and the source
      - field: tagName

        # The java Management Interface MBean to use (Object Name)
        mbean: "java.lang:type=Threading"

        # The attribute value to extract
        attrib: ThreadCount

        # [Optional] Python evaluation expression for the value
        value: "value"

    # Optional event configuration
    events:

      # [Optional] Wait for this event before activating the observer
      activate: MarathonStartedEvent

      # [Optional] If this event is received the observer is deactivated
      deactivate: ExitEvent

This observer is going to launch a utility process that is going to attach on the specified JVM instance. Upon successful connection it’s going to start extracting all the useful information as JMXMeasurement events in the message bus.

Such events can be passed down to metrics using the JMXTracker tracker.

LogStaxObserver¶

class performance.driver.classes.observer.LogStaxObserver(*args, **kwargs)[source]¶

The Logstax Observer is logstash-like observer for dcos-perf-test-driver that uses some event contents as the line source and a set of rules for creating fields for post-processing.

observers:
  - class: observer.LogStaxObserver

    # An array of filters to apply on every line
    filters:

      # Grok Pattern Matching
      # ---------------------
      - type: grok

        # Match the given field(s) for GROK expressions
        match:
          message: "^%{IP:httpHost} - (?<user>%{WORD}|-).*"

        # [Optional] Overwrite the specified fields with the values
        # extracted from the grok pattern. By default only new fields
        # are appended.
        overwrite: [message, name]

        # [Optional] Add the given fields to the message
        add_field:
          source: grok

        # [Optional] Remove the given fields from the message
        remove_field: [source]

        # [Optional] Add the given tags in the message
        add_tag: [foo, bar]

        # [Optional] Remove the given tags in the message
        remove_tag: [bar, baz]

    # [Optional] Which event(s) to listen for and which fields to extract
    events:

      # By default it's using the `LogLineEvent`
      - name: LogLineEvent
        field: line

    # [Optional] One or more `codecs` to apply on the incoming lines.
    #            These codecs convert one or more log lines into
    codecs:

      # Pass-through every incoming line to a rule matcher
      - type: singleline

      # Group multiple lines into a block and then pass it to the
      # rule matcher as an array of lines. Refer to the `MultilineCodec`
      # for more details.
      - type: multiline
        lines:
          - match: firstLine.*
          - match: secondLine.*

This observer is trying to reproduce a logstash set-up, using the LogLineEvent as the only source. It is first passing the events through a codec that is going to create a processable messages. Each message contains fields and tags.

By default, the singleline codec is populating the message field with the contents of the line. The multiline codec is more elaborate and can be used in order to extract multi-line blocks from the incoming stream.

The messages are then passed to the filters. If a filter matches the incoming message it is going to apply the transformations described.

When the filter process is completed, the observer is going to braodcast a LogStaxMessageEvent that can be processed at a later time by the LogStaxTracker in order to extract useful metrics.

class performance.driver.classes.observer.MarathonEventsObserver(*args, **kwargs)[source]¶

The Marathon Events Observer is extracting high-level events by subscribing to the Server-Side Events endpoint on marathon.

observers:
  - class: observer.MarathonEventsObserver

    # The URL to the marathon SSE endpoint
    url: "{{marathon_url}}/v2/events"

    # [Optional] Use an external curl process for receiving the events
    # instead of the built-in raw SSE client
    curl: no

    # [Optional] Use the timestamp from the event. If set to no, the time
    # the event is arrived to the perf-driver is used
    useEventTimestamp: no

    # [Optional] Additional headers to send
    headers:
      Accept: test/plain

Since this observer requires an active HTTP session to marathon, it also publishes the MarathonStartedEvent when an HTTP connection was successfully established.

The following events are forwarded from the event bus:

MarathonDeploymentStepSuccessEvent

MarathonDeploymentStepFailureEvent

MarathonDeploymentInfoEvent

MarathonDeploymentSuccessEvent

MarathonDeploymentFailedEvent

Note

In order to properly populcate the event’s trace ID, this observer is also listening for http channel requests in order to extract the affected application name(s).

Note

This observer will automatically inject an Authorization header if a dcos_auth_token definition exists, so you don’t have to specify it through the headers configuration.

Note that a dcos_auth_token can be dynamically injected via an authentication task.

MarathonLogsObserver¶

class performance.driver.classes.observer.MarathonLogsObserver(*args, **kwargs)[source]¶

This observer is based on the LogStaxObserver functionality in order to find and filter-out the marathon lines.

observers:
  - class: observer.MarathonLogsObserver

This observer accepts no configuration. It is processing the LogLineEvent messages dispatched by other observers or channels (ex. the CmdlineChannel channel).

Warning

This observer will currently only work if marathon is launched through a CmdlineChannel.

MarathonMetricsObserver¶

class performance.driver.classes.observer.MarathonMetricsObserver(*args, **kwargs)[source]¶

The Marathon Metrics Observer is observing for changes in the marathon /stats endpoint and is emitting events according to it’s configuration

observers:
  - class: observer.MarathonMetricsObserver

    # The URL to the marathon metrics endpoint
    url: "{{marathon_url}}/metrics"

    # [Optional] Additional headers to send
    headers:
      Accept: test/plain

This observer is polling the /metrics endpoint 2 times per second and for every value that is changed, a MetricUpdateEvent event is published.

Note

The name of the parameter is always the flattened name in the JSON response. For example, a parameter change in the following path:

{
  "foo": {
    "bar.baz": {
      "bax": 1
    }
  }
}

Will be broadcasted as a change in the following path:

foo.bar.baz.bax: 1

Note

This observer will automatically inject an Authorization header if a dcos_auth_token definition exists, so you don’t have to specify it through the headers configuration.

Note that a dcos_auth_token can be dynamically injected via an authentication task.

MarathonPollerObserver¶

class performance.driver.classes.observer.MarathonPollerObserver(*args, **kwargs)[source]¶

The Marathon Poller Observer is a polling-based fallback observer that can fully replace the MarathonEventsObserver when the SSE event bus is not available.

observers:
  - class: observer.MarathonPollerObserver

    # The URL to the marathon base
    url: "{{marathon_url}}"

    # [Optional] Additional headers to send
    headers:
      Accept: test/plain

    # [Optional] How long to wait between consecutive polls (seconds)
    interval: 0.5

    # [Optional] How long to wait before considering the deployment "Failed"
    # If set to 0 the deployment will never fail.
    failureTimeout: 0

    # [Optional] How many times to re-try polling the endpoint before
    # considering the connection closed
    retries: 3

    # [Optional] Event binding
    events:

      # [Optional] Which event to wait to start polling
      start: StartEvent

      # [Optional] Which event to wait to stop polling
      stop: TeardownEvent

This observer is polling the /groups endpoint as fast as possible and it calculates deferences from the previously observed state. Any differences are propagated as virtual deployment events as:

MarathonDeploymentSuccessEvent

MarathonDeploymentFailedEvent

If requested, the poller is going to look for MarathonDeploymentStartedEvent events and is going to wait for it to be completed in a given time. If the time is passed, a synthetic failure event will be generated:

MarathonDeploymentFailedEvent

Note

This observer will automatically inject an Authorization header if a dcos_auth_token definition exists, so you don’t have to specify it through the headers configuration.

Note that a dcos_auth_token can be dynamically injected via an authentication task.

Observers¶

HTTPTimingObserver¶

JMXObserver¶

LogStaxObserver¶

MarathonEventsObserver¶

MarathonLogsObserver¶

MarathonMetricsObserver¶

MarathonPollerObserver¶