Introduction

Overload is a distributed, scalable performance testing application. It mainly focuses on ease-of-use.

• Distributed - users don't need to handle complex cluster management - the application creates and manages the cluster by itself once deployed.
• Control - better control over QPS, connection pool, request data

Please check deployment and getting started to begin with.

Deployment

Overload supports two deployment mode -

• Cluster - run on kubernetes
• Standalone - run from CLI or on docker

The repository maintains docker images for both modes with different tags in here.

Deploy on Kubernetes

Cluster mode allows the application run in primary/secondary mode. Currently, only supported on Kubernetes. Application requires at least 4 pods to work in cluster mode.

Running on Kubernetes requires minimum four pods and cluster images. Cluster images are tagged as cluster-{version}.

Addition to that the application requires RBAC authorization to "get", "list" endpoints.

Sample deployment configuration -

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: overload
  name: overload
spec:
  replicas: 4
  selector:
    matchLabels:
      app: overload
  template:
    metadata:
      labels:
        app: overload
      annotations:
        prometheus.io/port: '3030'
        prometheus.io/scrape: 'true'
    spec:
      containers:
        - image: ghcr.io/eipi1/overload:cluster-latest-snapshot
          imagePullPolicy: "Always"
          name: overload
          ports:
            - containerPort: 3030
            - containerPort: 3031
          env:
            - name: DATA_DIR
              value: "/tmp"
            - name: K8S_ENDPOINT_NAME
              value: "overload"
            - name: K8S_NAMESPACE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: RUST_LOG
              value: info
          resources:
            requests:
              memory: "64Mi"
              cpu: "250m"
            limits:
              memory: "1024Mi"
              cpu: "2000m"
---
apiVersion: v1
kind: Service
metadata:
  name: overload
spec:
  #  type: LoadBalancer
  selector:
    app: overload
  ports:
    - protocol: TCP
      port: 3030
      targetPort: 3030
      name: http-endpoint
    - protocol: TCP
      port: 3031
      targetPort: 3031
      name: tcp-remoc #default, pass to env CLUSTER_COM_PORT_NAME if changed

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: overload-endpoints-reader
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: overload-endpoints-reader
subjects:
  - kind: Group
    name: system:serviceaccounts
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: overload-endpoints-reader
  apiGroup: rbac.authorization.k8s.io

Run on Docker

Application can be run on docker in standalone/single instance mode. It requires container images tagged as standalone-{version}.

For example to run the latest snapshot

docker run -p 3030:3030 mdsarowar/overload:latest-standalone-snapshot

Build

To build from source and run locally you'll be needing rust installed on the system.

cargo run

This will start overload server on port 3030

Getting Started

Start Test

The following request will send two GET request per second("countPerSec": 2) to httpbin.org/get for 120 seconds("duration": 120). More example can be found in test cases

curl --location --request POST '{overload_host}:3030/test' \
--header 'Content-Type: application/json' \
--data-raw '<json_request_body>'

Sample JSON request body -

extern crate overload_http;
extern crate serde_json;
use overload_http::Request;
let req = r###"
{
  "duration": 120,
  "name": "demo-test",
  "qps": {
    "ConstantRate": {
      "countPerSec": 2
    }
  },
  "req": {
    "RequestList": {
      "data": [
        {
          "body": null,
          "method": "GET",
          "url": "/get"
        }
      ]
    }
  },
  "target": {
    "host": "httpbin.org",
    "port": 80,
    "protocol": "HTTP"
  },
  "generationMode": {
    "batch": {
      "batchSize": 10
    }
  }
}
"###;
let result = serde_json::from_str::<Request>(req);
assert!(result.is_ok());

It'll respond with a job identifier and status.

{
  "job_id": "demo-test-d2ae5ff0-7bf4-4daf-8784-83b642d7dd6b",
  "status": "Starting"
}

We'll need the job_id if we want to stop the test later.

Get job status

curl --location --request GET '{overload_host}:3030/test/status/'

Stop a job

curl --location --request GET '{overload_host}:3030/test/stop/demo-test-d2ae5ff0-7bf4-4daf-8784-83b642d7dd6b'

Traffic Generation Mode

Overload supports two ways to specify how requests should be generated at any specific moment("second"). Can be set using generationMode field of the test request.

In conjunction with these options along with QPS & concurrent connection configuration, the generator can be configured either to open or closed system

Batch

Sends the requests in batches at certain intervals. The generator keeps track of available time to send all the required batches. If it fails to send a batch or any request in the batch, intervals might be reduced to accommodate remaining requests in the available time.

{
  "generationMode": {
    "batch": {
      "batchSize": 10
    }
  }
}

Example

Consider the generator needs to produce 100 requests in a certain second, and the batch size is 10.

In this case, the second will be sliced into ten 100ms periods. At the beginning of each slice, 10 requests will be sent to the target. If there is enough connection available in the pool, no new connection will be created, otherwise new connections will be attempted. Note that the connection creation is controlled by concurrent connection configuration.

If generator failed to send the second batch due to connection availability, slices will be recalculated. Now 9 batches needs to be sent in ~800ms. the slice duration will be updated to (100/9)=~88ms.

Time mentioned here is simplified and actual slice duration can be shorter. The generator reserves 10%(~100ms) for itself and may consume more than that.

Immediate

The default generation mode.

Tries to send all the requests at the beginning of the second. Crates as much as connection as required as long as it's allowed by concurrent connection configuration. If enough connections are not available, generator waits for next a(or more) connection to be available, and sends the request.

{
  "generationMode": "immediate"
}

Monitoring

The application comes with Prometheus support for monitoring. Metrics are exposed at /metrics endpoint.

Sample Prometheus scraper config

- job_name: overload-k8s
  honor_timestamps: true
  scrape_interval: 5s
  scrape_timeout: 1s
  metrics_path: /metrics
  scheme: http
  kubernetes_sd_configs:
  - api_server: <k8s-api-server>
    role: endpoints
    namespaces:
      names:
      - default
    selectors:
    - role: endpoints
      field: metadata.name=overload

Histogram

Check Prometheus HISTOGRAMS AND SUMMARIES.

By default, the application uses (20, 50, 100, 300, 700, 1100) as buckets to calculate response time quantiles. But each service has difference requirements, so the application provides a way to configure buckets in the test request itself.

Test endpoint accepts an array field histogramBuckets. Users can use this field to configure their own criteria. Currently, the field allows any number of buckets, but it's advisable not to use more than six buckets.

Grafana Dashboard

The application provides sample Grafana dashboard that can be used for monitoring. It has graphs for Request Per Seconds, Response Status count, Average response time and Response time quantiles.

Response Assertion

Overload supports two kind of assertion -

• Lua Assertion
• Simple Assertion (Deprecated)

ResponseAssertion

Lua Assertion

Lua is a powerful, efficient, lightweight scripting language. Overload comes with lua embedded. Users can write complex logic to verify request/response and expose results through metrics. The embedded engine provides functions to parse JSON.

Lua assertion requires following conventions -

• Supported lua version: 5.3
• Lua chunks must be a function
• The function accepts 4 arguments in following order
  • HTTP method
  • URL
  • Request Body (String)
  • Response Body (String)
• The function must return assertion failure as an array of assertion results

Note that, uncaught lua exceptions will be reported in metrics with assertion_id = 2147483647. The details of the error can be found in application info log.

Writing simple assertion

Writing Lua function

The following chunks will verify the request method.

function(method, url, requestBody, responseBody)
    -- initialize assertion result
    result = {}
    -- verify http method
    if method ~= 'POST' then 
        -- create new assertion result
        local err = {}
        -- used for metrics, should avoid duplicate values in the same assertion
        err.id = 1
        err.success = false
        err.error = 'Invalid method'
        -- add to result
        table.insert(result, err)
    end
    return result
end

Assertion Result

Overload transforms the array returned from lua function to an array. Each array element needs to be a table with following keys -

Test request with the lua function

Lua is included in the test request as an array of strings.

extern crate overload_http;
extern crate serde_json;
use overload_http::Request;
let req = r###"
{
  "duration": 10,
  "name": "test-assertion",
  "qps": {
    "ConstantRate": {
      "countPerSec": 3
    }
  },
  "req": {
    "RequestList": {
      "data": [
        {
          "method": "POST",
          "url": "/anything",
          "headers": {
            "Host": "httpbin.org",
            "Connection":"keep-alive"
          },
          "body": "{\"keys\":[\"0001\",\"0002\",\"0004\"]}"
        }
      ]
    }
  },
  "target": {
    "host": "httpbin.org",
    "port": 80,
    "protocol": "HTTP"
  },
  "concurrentConnection": {
    "ConstantRate": {
      "countPerSec": 3
    }
  },
  "responseAssertion": {
    "luaAssertion": [
      "function(method, url, requestBody, responseBody)",
      "    -- initialize assertion result",
      "    result = {}",
      "    -- verify http method",
      "    if method ~= 'POST' then",
      "        -- create new assertion result",
      "        local err = {}",
      "        -- used for metrics, should avoid duplicate values in the same assertion",
      "        err.id = 1",
      "        err.success = false",
      "        err.error = 'Invalid method'",
      "        -- add to result",
      "        table.insert(result, err)",
      "    end",
      "    return result",
      "end"
    ]
  }
}
"###;
let result = serde_json::from_str::<Request>(req);
assert!(result.is_ok());

[Deprecated] Simple Assertion

A set of assertion can be passed to verify response received from API in test. Assertion will fail if response can't be parsed as JSON.

For example, the following request will send request to httpbin.org/get and verify response is a json and the value at $.headers.Host (JsonPath) is httpbin.org.

Failed assertion emits an info log and can also be monitored through metrics.

extern crate overload_http;
extern crate serde_json;
use overload_http::Request;
let req = r###"
{
  "duration": 120,
  "name": "demo-test",
  "qps": {
    "ConstantRate": {
      "countPerSec": 1
    }
  },
  "req": {
    "RequestList": {
      "data": [
        {
          "body": null,
          "method": "GET",
          "url": "/get"
        }
      ]
    }
  },
  "target": {
    "host": "httpbin.org",
    "port": 80,
    "protocol": "HTTP"
  },
  "responseAssertion": {
      "simpleAssertion": [
        {
          "id": 1,
          "expectation": {
            "Constant": "httpbin.org"
          },
          "actual": {
            "FromJsonResponse": {
              "path": "$.headers.Host"
            }
          }
        }
      ]
    }
}
"###;
let result = serde_json::from_str::<Request>(req);
assert!(result.is_ok());

Simple Assertion

Actual

Following actual sources are supported -

Constant

Number, String or JSON value, expect a constant value for all the request.

extern crate response_assert;
extern crate serde_json;
use response_assert::Actual;
let req = r###"
{
  "Constant": {
    "hello": "world"
  }
}
"###;
let result = serde_json::from_str::<Actual>(req);
assert!(result.is_ok());

FromJsonResponse

Derive expectation from JSON response using JsonPath

extern crate response_assert;
extern crate serde_json;
use response_assert::Actual;
let req = r###"
{
  "FromJsonResponse": {
    "path": "$.hello"
  }
}
"###;
let result = serde_json::from_str::<Actual>(req);
assert!(result.is_ok());

Expectation

Specify what do we expect. Following expectations are supported -

Constant

Number, String or Json value, expect a constant value for all the request.

extern crate response_assert;
extern crate serde_json;
use response_assert::Expectation;
let req = r###"
{
  "Constant": {
    "hello": "world"
  }
}
"###;
let result = serde_json::from_str::<Expectation>(req);
assert!(result.is_ok());

RequestPath

Derive from request path using provided segment number

extern crate response_assert;
extern crate serde_json;
use response_assert::Expectation;
let req = r###"
{
   "RequestPath": 1
}
"###;
let result = serde_json::from_str::<Expectation>(req);
assert!(result.is_ok());

RequestQueryParam

Use the value of the request query param as expectation

extern crate response_assert;
extern crate serde_json;
use response_assert::Expectation;
let req = r###"
{
  "RequestQueryParam": "hello"
}
"###;
let result = serde_json::from_str::<Expectation>(req);
assert!(result.is_ok());

Configurations

The application supports following environment variables -

RUST_LOG

Configure logging level, default is info.

You can either configure logging levels for all the components or set different level for different components. For example -

• To enable trace logging - RUST_LOG=trace
• To enable debug log for executor(handles test request), but info for other - RUST_LOG=info,cluster_executor=debug

DATA_DIR

Path to store uploaded CSV files, default is /tmp.

REQUEST_BUNDLE_SIZE

Overload divides total requests to be sent in a second to smaller chunks and process these chunks at certain intervals. This variable can be used to set the max chunk size. Default is 50.

For example, if QPS is 100 at certain point and REQUEST_BUNDLE_SIZE=10, Overload will send 10 requests at 100ms interval; if REQUEST_BUNDLE_SIZE=20, app will send 20 requests at 200ms interval.

Note that, if the value is too less, application may fail to fulfill the QPS, setting too high will increase memory usage. Application should be scaled horizontally in that case.

K8S_ENDPOINT_NAME

Name of the endpoints, default is overload.

K8S_NAMESPACE_NAME

The kubernetes namespace the application is deployed on, default is default.

CLUSTER_UPDATE_INTERVAL

Interval between discovery service calls in seconds, default is 10.

Setting it too low can cause frequent call to k8s API, and if set too high, it may take long time to discover new pods.

CLUSTER_ELECTION_TIMEOUT

Approximate interval in seconds between attempts to elect new leader node. Application will use a randomly chosen value between CLUSTER_ELECTION_TIMEOUT and CLUSTER_ELECTION_TIMEOUT*2.

Default is 30

CLUSTER_MAX_NODE

Maximum number of allowed node in a cluster. Minimum is 4, default 20.

CLUSTER_MIN_NODE

Minimum number of nodes required to create a cluster. Minimum is 4, default 4.

HTTP APIs

The application provides a set of HTTP APIs to manage tests -

• /test - start a test
• /tests - start multiple tests in parallel
• /test/status - get status of running tests
• /test/stop/{test_id} - stop a test by id
• /test/requests-bin - upload request file

POST /test

Start a test with given specification

Request Body

Example

curl --location --request POST '{overload_host}:3030/test' \
--header 'Content-Type: application/json' \
--data-raw '<json_request_body>'

Sample JSON request body -

extern crate overload_http;
extern crate serde_json;
use overload_http::Request;
let req = r###"
{
  "duration": 800,
  "name": "test-overload",
  "histogram_buckets": [
    1,
    2,
    5,
    7,
    10,
    15
  ],
  "qps": {
    "Linear": {
      "a": 1,
      "b": 1,
      "max": 100
    }
  },
  "req": {
    "RequestList": {
      "data": [
        {
          "method": "GET",
          "url": "/serde",
          "headers": {
            "Host": "127.0.0.1:2000",
            "Connection": "keep-alive"
          }
        }
      ]
    }
  },
  "target": {
    "host": "172.17.0.1",
    "port": 2000,
    "protocol": "HTTP"
  },
  "concurrentConnection": {
    "Linear": {
      "a": 1,
      "b": 1,
      "max": 100
    }
  },
  "responseAssertion": {
    "luaAssertion": [
      "function(method, url, requestBody, responseBody)",
      "    -- initialize assertion result",
      "    result = {}",
      "    -- verify valid json response",
      "    jsonResp=json.decode(responseBody);",
      "    if jsonResp == nil then",
      "        -- create new assertion result",
      "        local err = {}",
      "        -- used for metrics, should avoid duplicate values in the same assertion",
      "        err.id = 1",
      "        err.success = false",
      "        err.error = 'Failed to parse to json'",
      "        -- add to result",
      "        table.insert(result, err)",
      "    end",
      "    return result",
      "end"
    ]
  }
}
"###;
let result = serde_json::from_str::<Request>(req);
assert!(result.is_ok());

POST /tests

Start a group of tests with given specifications.

Request Body

Example

curl --location --request POST 'localhost:3030/tests' \
--header 'Content-Type: application/json' \
--data-raw '<json_request_body>'

Sample JSON request body -

extern crate overload_http;
extern crate serde_json;
use overload_http::MultiRequest;
let req = r###"
{
  "name": "multi-req",
  "requests": [
    {
      "duration": 600,
      "name": "demo-test",
      "qps": {
        "Linear": {
          "a": 2,
          "b": 1,
          "max": 400
        }
      },
      "req": {
        "RequestList": {
          "data": [
            {
              "method": "GET",
              "url": "/delay/1",
              "headers": {
                "Host": "127.0.0.1:8080",
                "Connection":"keep-alive"
              }
            }
          ]
        }
      },
      "target": {
        "host": "172.17.0.1",
        "port": 8080,
        "protocol": "HTTP"
      },
      "concurrentConnection": {
        "Linear": {
          "a": 2.5,
          "b": 1,
          "max": 500
        }
      }
    },
    {
      "duration": 500,
      "name": "demo-test",
      "qps": {
        "Linear": {
          "a": 1,
          "b": 1,
          "max": 400
        }
      },
      "req": {
        "RequestList": {
          "data": [
            {
              "method": "GET",
              "url": "/get",
              "headers": {
                "Host": "127.0.0.1:8080",
                "Connection":"keep-alive"
              }
            }
          ]
        }
      },
      "target": {
        "host": "172.17.0.1",
        "port": 8080,
        "protocol": "HTTP"
      },
      "concurrentConnection": {
        "Linear": {
          "a": 0.5,
          "b": 1,
          "max": 70
        }
      }
    }
  ]
}
"###;
let result = serde_json::from_str::<MultiRequest>(req);
assert!(result.is_ok());

This will run two tests with their own QPS/connection rate, one for 600 seconds, another for 500 seconds.

Response

{
  "job_id": "1b826ded-34d5-434c-b983-a1ba46ffff82",
  "status": "Starting"
}

GET /test/status

Returns status of all jobs.

Limitations -

• Keep status only for 10 minutes after test ends, will be cleaned up after that

Query Params

Response

Example

curl --location --request GET '{overload_host}:3030/test/status/'

{
  "60de342e-b18c-4837-80d2-a2c71c1985f8": "Completed"
}

GET /test/stop/{test_id}

Stop a test by id

Params

Example

curl --location --request GET '{overload_host}:3030/test/stop/multi-req-5d1b90e8-a1e0-4d00-b192-e80b8f36d038'

POST /request-file/{sqlite|csv}

Currently, supports CSV & SQLite files. Internally, application converts the uploaded file sqlite database.

Curl sample

curl --location --request POST '{overload_host}:3030/request-file/sqlite' \
--data-binary '@/path/to/requests.sqlite'

CSV format (Deprecated, use sqlite instead)

It's not recommended to use CSV format for large file as Overload have to convert CSV to Sqlite. The conversion process may take time.

The CSV should have header included.

"method","url","body","headers"
"POST","/sample/path/0","{\"sample\":\"json body\",\"host\":\"127.0.0.1\",\"port\":2080,\"protocol\":\"HTTP\"}","{\"Connection\":\"keep-alive\"}"
"POST","/sample/path/1","{\"sample\":\"json body\",\"host\":\"127.0.0.1\",\"port\":2080,\"protocol\":\"HTTP\"}","{\"Connection\":\"keep-alive\"}"
"POST","/sample/path/2","{\"sample\":\"json body\",\"host\":\"127.0.0.1\",\"port\":2080,\"protocol\":\"HTTP\"}","{\"Connection\":\"keep-alive\"}"
"POST","/sample/path/3","{\"sample\":\"json body\",\"host\":\"127.0.0.1\",\"port\":2080,\"protocol\":\"HTTP\"}","{\"Connection\":\"keep-alive\"}"
"POST","/sample/path/4","{\"sample\":\"json body\",\"host\":\"127.0.0.1\",\"port\":2080,\"protocol\":\"HTTP\"}","{\"Connection\":\"keep-alive\"}"
"POST","/sample/path/5","{\"sample\":\"json body\",\"host\":\"127.0.0.1\",\"port\":2080,\"protocol\":\"HTTP\"}","{\"Connection\":\"keep-alive\"}"
"GET","/sample/path/6","","{\"Connection\":\"keep-alive\"}"
"GET","/sample/path/7","","{\"Connection\":\"keep-alive\"}"

Sqlite format

The recommended format for large request file.

One of the many ways to convert CSV to Sqlite is follows -

sqlite3 sample-request.sqlite "VACUUM;"
csvsql -p '\' --db sqlite:///sample-request.sqlite --table http_req --overwrite --insert ./sample-request.csv

Generated sqlite of the sample CSV above -

$ sqlite3 sample-request.sqlite 
SQLite version 3.37.2 2022-01-06 13:25:41
Enter ".help" for usage hints.
sqlite> select * from http_req;
POST|/sample/path/0|{"sample":"json body","host":"127.0.0.1","port":2080,"protocol":"HTTP"}|{"Connection":"keep-alive"}
POST|/sample/path/1|{"sample":"json body","host":"127.0.0.1","port":2080,"protocol":"HTTP"}|{"Connection":"keep-alive"}
POST|/sample/path/2|{"sample":"json body","host":"127.0.0.1","port":2080,"protocol":"HTTP"}|{"Connection":"keep-alive"}
POST|/sample/path/3|{"sample":"json body","host":"127.0.0.1","port":2080,"protocol":"HTTP"}|{"Connection":"keep-alive"}
POST|/sample/path/4|{"sample":"json body","host":"127.0.0.1","port":2080,"protocol":"HTTP"}|{"Connection":"keep-alive"}
POST|/sample/path/5|{"sample":"json body","host":"127.0.0.1","port":2080,"protocol":"HTTP"}|{"Connection":"keep-alive"}
GET|/sample/path/6||{"Connection":"keep-alive"}
GET|/sample/path/7||{"Connection":"keep-alive"}
sqlite> 

To upload using curl -

curl --location '{overload_host}:3030/request-file/sqlite' \
--header 'Content-Type: application/octet-stream' \
--data-binary '@/path/to/test-requests-post.sqlite'

Support for Gzip (version: SNAPSHOT)

To allow larger and faster upload, Overload also supports gzipped sqlite file, but requires "Content-Encoding: gzip" header in the request. The uploaded sqlite file will be unzipped in the server.

To upload using curl

cat /path/to/test-requests-post.sqlite | gzip | \
curl -vi --location '{overload_host}:3030/request-file/sqlite' \
--header "Content-Encoding: gzip" \
--header 'Content-Type: application/octet-stream' \
--data-binary @-

Response

API returns valid count, i.e. count of requests that has been parsed successfully and a file ID. File ID will be required to for testing.

Request Specification

Application supports following request sources -

• RequestList - an array of requests, generally useful for small set of request data
• RandomDataRequest - generate random request based on JSON Schema, useful for large amount of request
• RequestFile - get request from file. Use if RandomDataRequest doesn't satisfy the requirement
• SplitRequestFile - Split the request file among secondary nodes
• JsonTemplate - Json request specification with template support

RequestList

An unordered set of HttpReq

RequestFile

Get requests from a CSV file. File has to be uploaded separately before the test. Upload request returns name/id of the file to be used in the test request.

Note that, application doesn't split file, instead distribute whole file with secondary nodes. The nodes picks up requests randomly from the file.

Example

extern crate overload_http;
extern crate serde_json;
use overload_http::Request;
let req = r###"
{
  "duration": 3,
  "req": {
    "RequestFile": {
      "file_name": "4e1d1b32-0f1e-4b31-92dd-66f51c5acf9a"
    }
  },
  "qps": {
    "ConstantRate": {
      "countPerSec": 8
    }
  },
  "target": {
    "host": "httpbin.org",
    "port": 80,
    "protocol": "HTTP"
  }
}
"###;
let result = serde_json::from_str::<Request>(req);
assert!(result.is_ok());

RandomDataRequest

Generate request with random data based on constraints specified using JSON Schema like syntax.

Supported constraints

Notes

• Pattern
  • Unicode pattern. Careful with character groups, e.g. \d represents digits from english, arabic and other languages.
  • Maximum repeat(*,+) length is 10.
  • For integer, non-numeric pattern (eg. ^a[0-9]{4}z$) will produce 0

Type & Constraint Support

extern crate overload_http;
extern crate serde_json;
use overload_http::Request;
let req = r###"
{
  "duration": 10,
  "req": {
    "RandomDataRequest": {
      "url": "/anything/{param1}/{param2}",
      "method": "POST",
      "bodySchema": {
        "title": "Person",
        "type": "object",
        "properties": {
          "firstName": {
            "type": "string",
            "description": "The person's first name."
          },
          "lastName": {
            "type": "string",
            "description": "The person's last name."
          },
          "age": {
            "description": "Age in years which must be equal to or greater than zero.",
            "type": "integer",
            "minimum": 10,
            "maximum": 20
          },
          "intPattern": {
            "description": "It will produce numbers in range 10000-19999.",
            "type": "integer",
            "pattern": "^1[0-9]{4}$"
          },
          "objArrayKeys": {
            "type": "array",
            "maxLength": 20,
            "minLength": 10,
            "items": {
              "type": "object",
              "properties": {
                "objKey1": {
                  "type": "integer",
                  "minimum": 10,
                  "maximum": 1000
                },
                "objKey2": {
                  "type": "string",
                  "pattern": "^a[0-9]{4}z$"
                }
              }
            }
          },
          "scalarArray": {
            "type": "array",
            "maxLength": 20,
            "minLength": 20,
            "items": {
              "type": "string",
              "pattern": "^a[0-9]{4}z$"
            }
          },
          "constObject": {
            "type": "object",
            "constant": {
              "comment": "everything in this object will be treated as constant",
              "key1": "val1",
              "properties": {
                "comment": "not a schema properties"
              },
              "keyInt": 1234
            }
          }
        }
      },
      "uriParamSchema": {
        "type": "object",
        "properties": {
          "param1": {
            "type": "string",
            "description": "The person's first name.",
            "minLength": 6,
            "maxLength": 15
          },
          "param2": {
            "description": "Age in years which must be equal to or greater than zero.",
            "type": "integer",
            "minimum": 1000000,
            "maximum": 1100000
          }
        }
      }
    }
  },
  "qps": {
    "ConstantRate": {
      "countPerSec": 5
    }
  },
  "target": {
    "host": "httpbin.org",
    "port": 80,
    "protocol": "HTTP"
  }
}
"###;
let result = serde_json::from_str::<Request>(req);
assert!(result.is_ok());

This request will generate POST request to urls like "http://httpbin.org/anything/uejaiwd/1044053", "http://httpbin.org/anything/ryjhwq/1078153" and so on with a randomly generated JSON body.

SplitRequestFile

Get requests from a CSV file, split the file equally between secondary nodes. File has to be uploaded separately before the test. Upload request returns name/id of the file to be used in the test request.

Example

extern crate overload_http;
extern crate serde_json;
use overload_http::Request;
let req = r###"
{
  "duration": 3,
  "req": {
    "SplitRequestFile": {
      "file_name": "4e1d1b32-0f1e-4b31-92dd-66f51c5acf9a"
    }
  },
  "qps": {
    "ConstantRate": {
      "countPerSec": 8
    }
  },
  "target": {
    "host": "httpbin.org",
    "port": 80,
    "protocol": "HTTP"
  }
}
"###;
let result = serde_json::from_str::<Request>(req);
assert!(result.is_ok());

JsonTemplateRequest

Generate request based on a template.

Version

SNAPSHOT

Example

extern crate overload_http;
extern crate serde_json;
use overload_http::{Request, RequestSpecEnum};
let req = r###"
{
  "duration": 10,
  "name": "json-template-post-2.json",
  "qps": {
    "ConstantRate": {
      "countPerSec": 3
    }
  },
  "req": {
    "JsonTemplateRequest": {
      "method": "POST",
      "url": "{{patternStr(\"/anything/[a-z]{4}\")}}",
      "headers": {
        "Host": "httpbin.org",
        "Connection": "keep-alive"
      },
      "body": [
        {
          "id": "{{randomInt(1,10)}}",
          "type": "donut",
          "name": "Cake",
          "ppu": "{{randomFloat(0.2,1.0)}}",
          "available": "{{randomBool()}}",
          "batters": {
            "batter": [
              {
                "id": "{{toStr(randomInt(100,200))}}",
                "type": "Regular"
              },
              {
                "id": "{{toStr(randomInt(100,200))}}",
                "type": "Chocolate"
              }
            ]
          },
          "topping": [
            {
              "id": "5001",
              "type": "None"
            },
            {
              "id": "5002",
              "type": "Glazed"
            },
            {
              "id": "5005",
              "type": "Sugar"
            },
            {
              "id": "5007",
              "type": "Powdered Sugar"
            },
            {
              "id": "5006",
              "type": "Chocolate with Sprinkles"
            },
            {
              "id": "5003",
              "type": "Chocolate"
            },
            {
              "id": "5004",
              "type": "Maple"
            }
          ]
        },
        {
          "id": "0002",
          "type": "donut",
          "name": "{{patternStr(\"[a-z]{5}-[0-9]{3}\")}}",
          "ppu": 0.6,
          "available": "{{randomBool()}}",
          "batters": {
            "batter": [
              {
                "id": "1001",
                "type": "Regular"
              }
            ]
          },
          "topping": [
            {
              "id": "5001",
              "type": "None"
            },
            {
              "id": "5002",
              "type": "Glazed"
            }
          ]
        }
      ]
    }
  },
  "target": {
    "host": "httpbin.org",
    "port": 80,
    "protocol": "HTTP"
  },
  "generationMode": {
    "batch": {
      "batchSize": 3
    }
  },
  "concurrentConnection": {
    "ConstantRate": {
      "countPerSec": 3
    }
  }
}
"###;
let result = serde_json::from_str::<Request>(req);
assert!(result.is_ok());
assert!(matches!(result.unwrap().req, RequestSpecEnum::JsonTemplateRequest(_)));

The above request will generate traffic to httpbin.org/anything/{randomPath} with request body like the following

[
  {
    "id": 6,
    "type": "donut",
    "name": "Cake",
    "ppu": 0.5362728858152506,
    "available": false,
    "batters": {
      "batter": [
        {
          "id": "161",
          "type": "Regular"
        },
        {
          "id": "123",
          "type": "Chocolate"
        }
      ]
    },
    "topping": [
      {
        "id": "5001",
        "type": "None"
      },
      {
        "id": "5002",
        "type": "Glazed"
      },
      {
        "id": "5005",
        "type": "Sugar"
      },
      {
        "id": "5007",
        "type": "Powdered Sugar"
      },
      {
        "id": "5006",
        "type": "Chocolate with Sprinkles"
      },
      {
        "id": "5003",
        "type": "Chocolate"
      },
      {
        "id": "5004",
        "type": "Maple"
      }
    ]
  },
  {
    "id": "0002",
    "type": "donut",
    "name": "mafiq-895",
    "ppu": 0.6,
    "available": false,
    "batters": {
      "batter": [
        {
          "id": "1001",
          "type": "Regular"
        }
      ]
    },
    "topping": [
      {
        "id": "5001",
        "type": "None"
      },
      {
        "id": "5002",
        "type": "Glazed"
      }
    ]
  }
]

Supported Template Functions

Notes

• Template function can be chained - "{{toStr(randomBool())}}" is valid and will generate "true" or "false"
• Request body needs to be valid JSON, so all template including int/boolean fields needs to be string. Datatype will be determined based on the function used.
  • randomInt(..) will produce json integer value, randomBool() will produce boolean
  • Use toStr/toInt to convert for type conversion if necessary
• Pattern
  • Unicode pattern. Careful with character groups, e.g. \d represents digits from english, arabic and other languages.
  • Maximum repeat(*,+) length is 10.
  • For integer, non-numeric pattern (eg. ^a[0-9]{4}z$) will produce 0

QPS Specification

QPS specification determines the number of request per seconds to the target. Currently, supported configurations are -

• ConstantRate - constant rate for whole test
• Linear - linearly increase the QPS
• Steps/Staircase - QPS in steps

ConstantRate

Send request with same rate for the whole test.

Example

extern crate overload_http;
extern crate serde_json;
use overload_http::RateSpecEnum;
let req = r###"
{
    "ConstantRate": {
      "countPerSec": 10
    }
}
"###;
let result = serde_json::from_str::<RateSpecEnum>(req);
assert!(result.is_ok());

If the test run for 5 minutes, it'll generate 10 QPS for the whole 5 minute.

Linear

Increase QPS linearly; rate for any time is calculated using equation qps = ax + b, where x being time(in seconds) passed since the start of the test.

Example

extern crate overload_http;
extern crate serde_json;
use overload_http::RateSpecEnum;
let req = r###"
{
  "Linear": {
    "a": 2,
    "b": 1,
    "max": 12
  }
}
"###;
let result = serde_json::from_str::<RateSpecEnum>(req);
assert!(result.is_ok());

If a test runs for 10 seconds, the generated RPS will be [1, 5, 7, 9, 11, 12, 12, 12, 12, 12]

Step/Staircase QPS

Specify QPS in steps

Step

Example

extern crate overload_http;
extern crate serde_json;
use overload_http::RateSpecEnum;
let req = r###"
{
  "Steps" : {
    "steps": [
      {
        "start": 0,
        "end": 5,
        "rate": 1
      },
      {
        "start": 6,
        "end": 8,
        "rate": 4
      },
      {
        "start": 9,
        "end": 10,
        "rate": 7
      }
    ]
  }
}
"###;
let result = serde_json::from_str::<RateSpecEnum>(req);
assert!(result.is_ok());

If the test run for 15 seconds, from 0 to 5th seconds, generated QPS is 1, from 6th to 8th seconds, generated QPS is 4, from 9 to 10th and 11th to 15th seconds, generated QPS is 7

Connection Pool Specification

concurrentConnection field is used to specify the number of connections to be used during the test.

By default, connection pool is elastic - it create as many connection as needed.

Connections are maintained in a circular queue, i.e. once a connection is used, it goes back to the end of the queue. This way, we can ensure that all connection in the pool are getting used. If a connection is closed by remote target, a new connection will be created.

Currently supports following configurations -

• ConstantRate
• Linear
• Steps/Staircase

ConstantRate

Use the same number of connection for the whole test.

Example

extern crate overload_http;
extern crate serde_json;
use overload_http::RateSpecEnum;
let req = r###"
{
    "ConstantRate": {
      "countPerSec": 10
    }
}
"###;
let result = serde_json::from_str::<RateSpecEnum>(req);
assert!(result.is_ok());

If the test run for 5 minutes, it'll use 10 connections for the whole 5 minute.

Linear

Increase connection pool size linearly; rate for any time is calculated using equation connection_count = ax + b, where x being time(in seconds) passed since the start of the test.

Example

extern crate overload_http;
extern crate serde_json;
use overload_http::RateSpecEnum;
let req = r###"
{
  "Linear": {
    "a": 2,
    "b": 1,
    "max": 12
  }
}
"###;
let result = serde_json::from_str::<RateSpecEnum>(req);
assert!(result.is_ok());

If a test runs for 10 seconds, the number of connections will be [1, 5, 7, 9, 11, 12, 12, 12, 12, 12]

Step/Staircase QPS

Increase or decrease connection pool size in steps.

Step

Example

extern crate overload_http;
extern crate serde_json;
use overload_http::RateSpecEnum;
let req = r###"
{
  "Steps" : {
    "steps": [
      {
        "start": 0,
        "end": 5,
        "rate": 1
      },
      {
        "start": 6,
        "end": 8,
        "rate": 4
      },
      {
        "start": 9,
        "end": 10,
        "rate": 7
      }
    ]
  }
}
"###;
let result = serde_json::from_str::<RateSpecEnum>(req);
assert!(result.is_ok());

If the test run for 15 seconds, from 0 to 5th seconds, 1 connection will be used, from 6th to 8th seconds, it'll be 4, from 9 to 10th and 11th to 15th seconds, 7 connections will be used.

Types

Types describe the data need to passed as the request body.

• Request
• MultiRequest
• HttpReq
• Target
• JobStatus

Request

MultiRequest

HttpReq

Target

Specify target details

JobStatus

Enum stating the current status of the test