Mutate string processors
You can change the way that a string appears by using a mutate string processesor. For example, you can use the uppercase_string processor to convert a string to uppercase, and you can use the lowercase_string processor to convert a string to lowercase. The following is a list of processors that allow you to mutate a string:
substitute_string
The substitute_string processor matches a key’s value against a regular expression (regex) and replaces all returned matches with a replacement string.
Configuration
You can configure the substitute_string processor with the following options.
| Option | Required | Description |
|---|---|---|
entries | Yes | A list of entries to add to an event. |
source | Yes | The key to be modified. |
from | Yes | The regex string to be replaced. Special regex characters such as [ and ] must be escaped using \\ when using double quotes and \ when using single quotes. For more information, see Class Pattern in the Java documentation. |
to | Yes | The string that replaces each match of from. |
Usage
To get started, create the following pipeline.yaml file:
pipeline:
source:
file:
path: "/full/path/to/logs_json.log"
record_type: "event"
format: "json"
processor:
- substitute_string:
entries:
- source: "message"
from: ":"
to: "-"
sink:
- stdout:
Next, create a log file named logs_json.log. After that, replace the path of the file source in your pipeline.yaml file with your file path. For more detailed information, see Configuring Data Prepper.
Before you run Data Prepper, the source appears in the following format:
{"message": "ab:cd:ab:cd"}
After you run Data Prepper, the source is converted to the following format:
{"message": "ab-cd-ab-cd"}
from defines which string is replaced, and to defines the string that replaces the from string. In the preceding example, string ab:cd:ab:cd becomes ab-cd-ab-cd. If the from regex string does not return a match, the key is returned without any changes.
split_string
The split_string processor splits a field into an array using a delimiter character.
Configuration
You can configure the split_string processor with the following options.
| Option | Required | Description |
|---|---|---|
entries | Yes | A list of entries to add to an event. |
source | Yes | The key to be split. |
delimiter | No | The separator character responsible for the split. Cannot be defined at the same time as delimiter_regex. At least delimiter or delimiter_regex must be defined. |
delimiter_regex | No | A regex string responsible for the split. Cannot be defined at the same time as delimiter. Either delimiter or delimiter_regex must be defined. |
Usage
To get started, create the following pipeline.yaml file:
pipeline:
source:
file:
path: "/full/path/to/logs_json.log"
record_type: "event"
format: "json"
processor:
- split_string:
entries:
- source: "message"
delimiter: ","
sink:
- stdout:
Next, create a log file named logs_json.log. After that, replace the path in the file source of your pipeline.yaml file with your file path. For more detailed information, see Configuring Data Prepper.
Before you run Data Prepper, the source appears in the following format:
{"message": "hello,world"}
After you run Data Prepper, the source is converted to the following format:
{"message":["hello","world"]}
uppercase_string
The uppercase_string processor converts the value (a string) of a key from its current case to uppercase.
Configuration
You can configure the uppercase_string processor with the following options.
| Option | Required | Description |
|---|---|---|
with_keys | Yes | A list of keys to convert to uppercase. |
Usage
To get started, create the following pipeline.yaml file:
pipeline:
source:
file:
path: "/full/path/to/logs_json.log"
record_type: "event"
format: "json"
processor:
- uppercase_string:
with_keys:
- "uppercaseField"
sink:
- stdout:
Next, create a log file named logs_json.log. After that, replace the path in the file source of your pipeline.yaml file with the correct file path. For more detailed information, see Configuring Data Prepper.
Before you run Data Prepper, the source appears in the following format:
{"uppercaseField": "hello"}
After you run Data Prepper, the source is converted to the following format:
{"uppercaseField": "HELLO"}
lowercase_string
The lowercase string processor converts a string to lowercase.
Configuration
You can configure the lowercase string processor with the following options.
| Option | Required | Description |
|---|---|---|
with_keys | Yes | A list of keys to convert to lowercase. |
Usage
To get started, create the following pipeline.yaml file:
pipeline:
source:
file:
path: "/full/path/to/logs_json.log"
record_type: "event"
format: "json"
processor:
- lowercase_string:
with_keys:
- "lowercaseField"
sink:
- stdout:
Next, create a log file named logs_json.log. After that, replace the path in the file source of your pipeline.yaml file with the correct file path. For more detailed information, see Configuring Data Prepper.
Before you run Data Prepper, the source appears in the following format:
{"lowercaseField": "TESTmeSSage"}
After you run Data Prepper, the source is converted to the following format:
{"lowercaseField": "testmessage"}
trim_string
The trim_string processor removes whitespace from the beginning and end of a key.
Configuration
You can configure the trim_string processor with the following options.
| Option | Required | Description |
|---|---|---|
with_keys | Yes | A list of keys from which to trim the whitespace. |
Usage
To get started, create the following pipeline.yaml file:
pipeline:
source:
file:
path: "/full/path/to/logs_json.log"
record_type: "event"
format: "json"
processor:
- trim_string:
with_keys:
- "trimField"
sink:
- stdout:
Next, create a log file named logs_json.log. After that, replace the path in the file source of your pipeline.yaml file with the correct file path. For more detailed information, see Configuring Data Prepper.
Before you run Data Prepper, the source appears in the following format:
{"trimField": " Space Ship "}
After you run Data Prepper, the source is converted to the following format:
{"trimField": "Space Ship"}