Skip to main content
Version: 1.0.0 (Latest)

PDL Quick Reference

This is a reference guide for the Padas Domain Language (PDL).

In order to understand how PADAS works, please review Getting Started.

Introduction​

PDL is a domain-specific language designed for data processing, with features including querying, evaluation, filtering, renaming, and correlation of streaming event data. A PDL expression consists of a combination of zero to many expressions and zero or one correlation statement separated by a pipe '|' character. PDL syntax requires fields to be available in JSON object that it compares against and supports nested JSON objects/fields with dotted notation (e.g. field.subfield.anothersubfield etc.)

Syntax​

PDL can contain one or more expressions and zero or one correlation statement separated by a pipe | character. Output from an expression or correlation statement becomes the input for the expression that comes after the pipe |. Below grammar represents some generic grammar usage.

<expression> | <expression> | ...
<correlation> | <expression> | <expression> | ...
<expression> | ... | <correlation> | <expression> | ...

Field Names and Field Values​

For expressions and correlation statement field names (<fieldName>) represent the JSON field name and field value (<fieldValue>) can be a literal (number or string) or a field name. Literal strings must be enclosed in double quotation marks.

Field names can not have spaces in them and currently following features are provided:

  • Must begin with a letter ([a-zA-Z]) or underscore _
  • Supports dotted notation for nested JSON fields.
  • Does NOT support whitespace in field names.
  • Accepts alphanumeric characters, minus/dash, underscore, and column ([a-zA-Z0-9_-:])

PDL Examples​

JSON Event DataPDL ExpressionExpected Result
{
  "field1":{
    "subfield1":"subvalue1",
    "subfield2":"sub value2"
  },
  "field2":"value2",
  "field3":123
}
field1.subfield2 ?= "value2"
  {
    "field1":{
      "subfield1":"subvalue1",
      "subfield2":"sub value2"
    },
    "field2":"value2",
    "field3":123
  }
{
  "field1":"value1",
  "field2":"value2 text2 value2",
  "field3":123,
  "field4":"value4",
  "field_5":5,
  "field-6":6,
  "field:7":7
}
  field1="va*e1"
  {
    "field1":"value1",
    "field2":"value2 text2 value2",
    "field3":123,
    "field4":"value4",
    "field_5":5,
    "field-6":6,
    "field:7":7
  }
  {
    "field1":{
      "subfield1":"subvalue1",
      "subfield2":"sub value2"
    },
    "field2":"value2",
    "field3":123
  }
(field1.subfield2 = "value2" AND field3=123)
  null
    {
      "field1":{
        "subfield1":"subvalue1",
        "subfield2":"sub value2"
      },
      "field2":"value2",
      "field3":123,
      "field4": [5, 6, 7]
    }

  (field1.subfield2 ?= "value2" AND field3=123) 
  | eval newField=field3+100
  | eval anotherField=if(newField > 300, "above 300", "below 300")
  {
    "field1": {
      "subfield1": "subvalue1",
      "subfield2": "sub value2"
    },
    "field2": "value2",
    "field3": 123,
    "field4": [
      5,
      6,
      7
    ],
    "newField": 223,
    "anotherField": "below 300"
  }
    {
      "field1":{
        "subfield1":"subvalue1",
        "subfield2":"sub value2"
      },
      "field2":"value2",
      "field3":123,
      "field4": [5, 6, 7]
    }

  field2 IN ["value1", "value2"] AND field4 ?= 6
  | eval newField=field3+100
  | fields keep field1, newField
  | rename field1.subfield1 AS myField
  {
    "field1": {
      "subfield2": "sub value2"
    },
    "newField": 223,
    "myField": "subvalue1"
  }
    {
      "field1":{
        "subfield1":"subvalue1",
        "subfield2":"sub value2"
      },
      "field2":"value2",
      "field3":123,
      "field4": [5, 6, 7]
    }

  field2 ?= "value"
  | fields remove field2
  | flatten
  {
    "field1_subfield1": "subvalue1",
    "field1_subfield2": "sub value2",
    "field3": 123,
    "field4_0": 5,
    "field4_1": 6,
    "field4_2": 7
  }