Customizing stream execution with configuration patches
Each stream is composed by a DAG of processors though which the data is processed. Each processor has a set of properties that can be configured to customize its behavior. Those properties are defined inside a stream configuration template, and it can look similar to this:
The system follows a convention over configuration approach, where the default values are defined in the processor implementation. Generally, the default values are good enough for most of the cases, but sometimes you may need to customize the behavior of a processor.
For example, you may want to change the tone of the summary generated by the LLM processor from Conversational
to Formal
.
This can be done by creating a configuration patch that overrides the default value of the tone
property.
A configuration patch is a JSON Patch that respects the RFC 6902 standard, with some imposed limitations.
The limitations are:
add
, replace
, and remove
properties
object of a processorThe accepted primitive values are:
For example, the configuration patch that changes the tone of the summary generated by the LLM processor from Conversational
to Formal
looks like this:
The json paths can be also expressed using .
.
For example, properties.0.properties.tone
, properties/0/properties/tone
and /properties/0/properties/tone
are all equivalent paths.
The JSON Patch format proposed in the RFC 6902 standard is not always suitable for specifying configuration patches.
For this reason, the system allows specifying configuration patches as query parameters
in the URL (all gateways) or using a configuration message (WS Gateway
only).
The query parameters are specified in the URL of the request, and they are used to specify the configuration patches that should be applied to the stream configuration template.
Every patch query parameter can be explicitly marked as a patch by using the patches.
prefix. If the prefix is not specified, all undocumented query parameters are considered as patches.
For example, the equivalent of this JSON Patch:
is the following query parameter:
or even shorter:
By default, the operation is replace
.
Add/Replace
To use the add
operation, prefix the patch value with +
. For example, to add the language
property with the value en
, use the following query parameter:
The replace
operation will behave as the add
operation if the property does not exist.
So, the equivalent of the previous query parameter is:
Remove
To use the remove
operation, set the patch value to -
. For example, to remove the tone
property, use the following query parameter:
Escape
To escape the +
and -
characters, use the \
character. For example, to set the tone
property to +Formal
, use the following query parameter:
In most of the cases, the replace
operation is enough, and you don’t need to specify it explicitly.
The configuration message is a WebSocket message sent by the client as an initial message to the WS Gateway
, with the following structure:
The rules of the configuration patches are the same as for the query parameters.
For example, the configuration message that changes the tone of the summary generated by the LLM processor from Conversational
to Formal
looks like this:
or using the .
notation:
Each patch path must contain the index of the processor to which the patch should be applied. This may be problematic if the processors order is changed, and the indexes are not updated accordingly. This can be avoided by using the processor ID instead of the index.
For example, the following patch:
can be rewritten as:
Each processor can specify a path alias that can be used to simplify the patch path. Given the following stream configuration template:
The following patch:
can be rewritten as:
To better understand, we’ll take the following example, and we’ll rewrite it from the most complex to the simplest form.
Given the following stream configuration template:
And the following patch:
The most complex form of the patch is:
then we’ll infer the redundant patches.
prefix
then we’ll substitute the processor index with the processor ID
then we’ll replace the add
operation for language
with replace
then we’ll use the path aliases for tone
and language
Replace
Given:
To enable the summary, use the following query parameter:
Add
Given:
To add the summary language, use the following query parameter:
Set array
Given:
To set the allowed languages to ["en", "fr"]
, use the following query parameter:
Replace with configuration message
Given:
To enable the summary and set the tone to Formal
, use the following configuration message:
Customizing stream execution with configuration patches
Each stream is composed by a DAG of processors though which the data is processed. Each processor has a set of properties that can be configured to customize its behavior. Those properties are defined inside a stream configuration template, and it can look similar to this:
The system follows a convention over configuration approach, where the default values are defined in the processor implementation. Generally, the default values are good enough for most of the cases, but sometimes you may need to customize the behavior of a processor.
For example, you may want to change the tone of the summary generated by the LLM processor from Conversational
to Formal
.
This can be done by creating a configuration patch that overrides the default value of the tone
property.
A configuration patch is a JSON Patch that respects the RFC 6902 standard, with some imposed limitations.
The limitations are:
add
, replace
, and remove
properties
object of a processorThe accepted primitive values are:
For example, the configuration patch that changes the tone of the summary generated by the LLM processor from Conversational
to Formal
looks like this:
The json paths can be also expressed using .
.
For example, properties.0.properties.tone
, properties/0/properties/tone
and /properties/0/properties/tone
are all equivalent paths.
The JSON Patch format proposed in the RFC 6902 standard is not always suitable for specifying configuration patches.
For this reason, the system allows specifying configuration patches as query parameters
in the URL (all gateways) or using a configuration message (WS Gateway
only).
The query parameters are specified in the URL of the request, and they are used to specify the configuration patches that should be applied to the stream configuration template.
Every patch query parameter can be explicitly marked as a patch by using the patches.
prefix. If the prefix is not specified, all undocumented query parameters are considered as patches.
For example, the equivalent of this JSON Patch:
is the following query parameter:
or even shorter:
By default, the operation is replace
.
Add/Replace
To use the add
operation, prefix the patch value with +
. For example, to add the language
property with the value en
, use the following query parameter:
The replace
operation will behave as the add
operation if the property does not exist.
So, the equivalent of the previous query parameter is:
Remove
To use the remove
operation, set the patch value to -
. For example, to remove the tone
property, use the following query parameter:
Escape
To escape the +
and -
characters, use the \
character. For example, to set the tone
property to +Formal
, use the following query parameter:
In most of the cases, the replace
operation is enough, and you don’t need to specify it explicitly.
The configuration message is a WebSocket message sent by the client as an initial message to the WS Gateway
, with the following structure:
The rules of the configuration patches are the same as for the query parameters.
For example, the configuration message that changes the tone of the summary generated by the LLM processor from Conversational
to Formal
looks like this:
or using the .
notation:
Each patch path must contain the index of the processor to which the patch should be applied. This may be problematic if the processors order is changed, and the indexes are not updated accordingly. This can be avoided by using the processor ID instead of the index.
For example, the following patch:
can be rewritten as:
Each processor can specify a path alias that can be used to simplify the patch path. Given the following stream configuration template:
The following patch:
can be rewritten as:
To better understand, we’ll take the following example, and we’ll rewrite it from the most complex to the simplest form.
Given the following stream configuration template:
And the following patch:
The most complex form of the patch is:
then we’ll infer the redundant patches.
prefix
then we’ll substitute the processor index with the processor ID
then we’ll replace the add
operation for language
with replace
then we’ll use the path aliases for tone
and language
Replace
Given:
To enable the summary, use the following query parameter:
Add
Given:
To add the summary language, use the following query parameter:
Set array
Given:
To set the allowed languages to ["en", "fr"]
, use the following query parameter:
Replace with configuration message
Given:
To enable the summary and set the tone to Formal
, use the following configuration message: