Capturing data

Captures are used to search for, and save, values or text from the body of an HTTP response.

There are four different ways to capture data, two focused on parsing JSON data (JSONPath and JMESPath) and two for parsing text (CSS selectors and Regular expressions).

Regardless of which query language you choose, it is wise to quote the expression as it may contain characters that are illegal or otherwise have meaning in YAML (e.g. #, *, &).

JSONPath

JSONPath is similar in concept to XPath, except it is used for parsing JSON rather than XML data. There are many online resources for learning JSONPath.

JMESPath

JMESPath is similar to JSONPath, but with a fully-specified, and therefore standardarized syntax. It lacks some features of JSONPath (e.g. recursion). It is widely known for its use in the AWS and Azure tools.

CSS selectors

CSS selectors are used to locate data by its containing HTML element’s CSS tags or id.

Regular expressions

Regular expressions are the Swiss-army knife of searching. They can parse most anything, but not usually in a robust way. They tend to be the most brittle of the options, as even a minor change in the content can cause the regex to fail to match. You should only resort to regular expressions when parsing unstructured text.

Using variables

Variables are used to store captured data for use in subsequent HTTP requests.

If we have an endpoint /locations that returns the following data:

{
  "locations": [
    {"name": "Seattle", "state": "WA"},
    {"name": "New York", "state": "NY"},
    {"name": "Bellevue", "state": "WA"},
    {"name": "Olympia", "state": "WA"}
  ]
}

this check will capture the cities in Washington as a string and then POST it to a different endpoint in a subsequent request:

session:
- name: capture data
  path: /locations
  method: get
  rules:
  - if:
    - status: [ 200 ]
      capture:
        jmes: 'locations[?state == 'WA'].name | sort(@) | join(', ', @)'
        as: [ cities_string ]

- name: post variables
  path: /post
  method: post
  json: { state: 'WA', cities: $(cities_string) }
  rules:
  - if not:
    - status: [ 200 ]
      event: { policy: critical }

The $(var) syntax is used to interpolate the variable into the string.