Configuration patches

Overview

Each stream is composed by a DAG of processors though which the data is processed. Each processor has a set of properties that can be configured to customize its behavior. Those properties are defined inside a stream configuration template, and it can look similar to this:

{
  "_id": "674c1f1ea3fc806ecb01df55",
  "processors": [
    {
      "_id": "llm",
      "propertiesSchema": "tech.vatis.schema.stream.processor.properties.llm.LlmPropertiesDto",
      "properties": {
        "summary": {
          "enabled": true,
          "flushInterval": "300",
          "properties": {
            "tone": "Conversational",
            "length": "Brief",
            "structure": "Paragraphs"
          }
        }
      },
      ...
    }
  ]
}

The system follows a convention over configuration approach, where the default values are defined in the processor implementation. Generally, the default values are good enough for most of the cases, but sometimes you may need to customize the behavior of a processor.

For example, you may want to change the tone of the summary generated by the LLM processor from Conversational to Formal. This can be done by creating a configuration patch that overrides the default value of the tone property.

A configuration patch is a JSON Patch that respects the RFC 6902 standard, with some imposed limitations.

The limitations are:

the only supported operations are add, replace, and remove
it can only apply to the properties object of a processor
the added/replaced values can be either a primitive value or an array of primitive values

The accepted primitive values are:

string
number
boolean
null

For example, the configuration patch that changes the tone of the summary generated by the LLM processor from Conversational to Formal looks like this:

[
  {
    "op": "replace",
    "path": "/processors/0/properties/summary/properties/tone",
    "value": "Formal"
  }
]

The json paths can be also expressed using ..

For example, properties.0.properties.tone, properties/0/properties/tone and /properties/0/properties/tone are all equivalent paths.

Specifying patches

The JSON Patch format proposed in the RFC 6902 standard is not always suitable for specifying configuration patches. For this reason, the system allows specifying configuration patches as query parameters in the URL (all gateways) or using a configuration message (WS Gateway only).

Query parameters

The query parameters are specified in the URL of the request, and they are used to specify the configuration patches that should be applied to the stream configuration template.

Every patch query parameter can be explicitly marked as a patch by using the patches. prefix. If the prefix is not specified, all undocumented query parameters are considered as patches.

For example, the equivalent of this JSON Patch:

[
  {
    "op": "replace",
    "path": "/processors/0/properties/summary/properties/tone",
    "value": "Formal"
  }
]

is the following query parameter:

https://gateway.vatis.tech/url?patches.processors.0.properties.summary.properties.tone=Formal

or even shorter:

https://gateway.vatis.tech/url?processors.0.properties.summary.properties.tone=Formal

Operation

By default, the operation is replace.

To use the add operation, prefix the patch value with +. For example, to add the language property with the value en, use the following query parameter:

https://gateway.vatis.tech/url?processors.0.properties.summary.properties.language=+en

The replace operation will behave as the add operation if the property does not exist.

So, the equivalent of the previous query parameter is:

https://gateway.vatis.tech/url?processors.0.properties.summary.properties.language=en

To use the remove operation, set the patch value to -. For example, to remove the tone property, use the following query parameter:

https://gateway.vatis.tech/url?processors.0.properties.summary.properties.tone=-

To escape the + and - characters, use the \ character. For example, to set the tone property to +Formal, use the following query parameter:

https://gateway.vatis.tech/url?processors.0.properties.summary.properties.tone=\+Formal

In most of the cases, the replace operation is enough, and you don’t need to specify it explicitly.

Configuration message

The configuration message is a WebSocket message sent by the client as an initial message to the WS Gateway, with the following structure:

{
  "type": "CONFIGURATION",
  "patches": {
    "<path>": "<value>"
  }
}

The rules of the configuration patches are the same as for the query parameters.

For example, the configuration message that changes the tone of the summary generated by the LLM processor from Conversational to Formal looks like this:

{
  "type": "CONFIGURATION",
  "patches": {
    "/processors/0/properties/summary/properties/tone": "Formal"
  }
}

or using the . notation:

{
  "type": "CONFIGURATION",
  "patches": {
    "processors.0.properties.summary.properties.tone": "Formal"
  }
}

A configuration message can contain as many patches as needed, and without any (reasonable) length limitation.

The configuration message can be used alongside the query parameters, and the patches from the configuration message will overwrite the patches from the query parameters.

Use the configuration message only when you need to specify a longer value that would not fit in the URL.

Simplifying paths

Processor index

Each patch path must contain the index of the processor to which the patch should be applied. This may be problematic if the processors order is changed, and the indexes are not updated accordingly. This can be avoided by using the processor ID instead of the index.

For example, the following patch:

https://gateway.vatis.tech/url?processors.0.properties.summary.properties.language=en

can be rewritten as:

https://gateway.vatis.tech/url?processors.llm.properties.summary.properties.language=en

Path alias

Each processor can specify a path alias that can be used to simplify the patch path. Given the following stream configuration template:

{
  "_id": "674c1f1ea3fc806ecb01df55",
  "processors": [
    {
      "_id": "llm",
      "propertiesSchema": "tech.vatis.schema.stream.processor.properties.llm.LlmPropertiesDto",
      "properties": {
        "summary": {
          "enabled": true,
          "flushInterval": "300",
          "properties": {
            "tone": "Conversational",
            "length": "Brief",
            "structure": "Paragraphs"
          }
        }
      },
      "patchAliases": {
        "summary": "properties/summary/enabled",
        "summary_tone": "properties/summary/properties/tone",
        "summary_length": "properties/summary/properties/length",
        "summary_structure": "properties/summary/properties/structure"
      },
      ...
    }
  ]
}

The following patch:

https://gateway.vatis.tech/url?processors.0.properties.summary.properties.tone=Formal

can be rewritten as:

https://gateway.vatis.tech/url?summary_tone=Formal

The equivalence chain

To better understand, we’ll take the following example, and we’ll rewrite it from the most complex to the simplest form.

Given the following stream configuration template:

{
  "_id": "674c1f1ea3fc806ecb01df55",
  "processors": [
    {
      "_id": "llm",
      "propertiesSchema": "tech.vatis.schema.stream.processor.properties.llm.LlmPropertiesDto",
      "properties": {
        "summary": {
          "enabled": true,
          "flushInterval": "300",
          "properties": {
            "tone": "Conversational",
            "length": "Brief",
            "structure": "Paragraphs"
          }
        }
      },
      "patchAliases": {
        "summary_tone": "properties/summary/properties/tone",
        "summary_language": "properties/summary/properties/language"
      },
      ...
    }
  ]
}

And the following patch:

[
    {
        "op": "replace",
        "path": "/processors/0/properties/summary/properties/tone",
        "value": "Formal"
    },
    {
        "op": "add",
        "path": "/processors/0/properties/summary/properties/language",
        "value": "en"
    }
]

The most complex form of the patch is:

https://gateway.vatis.tech/url?patches.processors.0.properties.summary.properties.tone=Formal&patches.processors.0.properties.summary.properties.language=+en

then we’ll infer the redundant patches. prefix

https://gateway.vatis.tech/url?processors.0.properties.summary.properties.tone=Formal&processors.0.properties.summary.properties.language=+en

then we’ll substitute the processor index with the processor ID

https://gateway.vatis.tech/url?processors.llm.properties.summary.properties.tone=Formal&processors.llm.properties.summary.properties.language=+en

then we’ll replace the add operation for language with replace

https://gateway.vatis.tech/url?processors.llm.properties.summary.properties.tone=Formal&processors.llm.properties.summary.properties.language=en

then we’ll use the path aliases for tone and language

https://gateway.vatis.tech/url?summary_tone=Formal&summary_language=en

Examples

Given:

{
  "_id": "674c1f1ea3fc806ecb01df55",
  "processors": [
    {
      "_id": "llm",
      "propertiesSchema": "tech.vatis.schema.stream.processor.properties.llm.LlmPropertiesDto",
      "properties": {
        "summary": {
          "enabled": false,
          "flushInterval": "300",
          "properties": {
            "tone": "Conversational",
            "length": "Brief",
            "structure": "Paragraphs"
          }
        }
      },
      "patchAliases": {
        "summary": "properties/summary/enabled"
      },
      ...
    }
  ]
}

To enable the summary, use the following query parameter:

https://gateway.vatis.tech/url?summary=true

Given:

{
  "_id": "674c1f1ea3fc806ecb01df55",
  "processors": [
    {
      "_id": "llm",
      "propertiesSchema": "tech.vatis.schema.stream.processor.properties.llm.LlmPropertiesDto",
      "properties": {
        "summary": {
          "enabled": false,
          "flushInterval": "300",
          "properties": {
            "tone": "Conversational",
            "length": "Brief",
            "structure": "Paragraphs"
          }
        }
      },
      "patchAliases": {
        "summary_language": "properties/summary/properties/language"
      },
      ...
    }
  ]
}

To add the summary language, use the following query parameter:

https://gateway.vatis.tech/url?summary_language=en

Given:

{
  "_id": "674c1f1ea3fc806ecb01df55",
  "processors": [
    {
      "_id": "llm",
      "propertiesSchema": "tech.vatis.schema.stream.processor.properties.whispergpu.WhisperGpuProcessorPropertiesDto",
      "properties": {
        "allowedLanguages": null
      },
      "patchAliases": {
        "allowed_languages": "properties/allowedLanguages"
      },
      ...
    }
  ]
}

To set the allowed languages to ["en", "fr"], use the following query parameter:

https://gateway.vatis.tech/url?allowed_languages=en&allowed_languages=fr

Given:

{
  "_id": "674c1f1ea3fc806ecb01df55",
  "processors": [
    {
      "_id": "llm",
      "propertiesSchema": "tech.vatis.schema.stream.processor.properties.llm.LlmPropertiesDto",
      "properties": {
        "summary": {
          "enabled": false,
          "flushInterval": "300",
          "properties": {
            "tone": "Conversational",
            "length": "Brief",
            "structure": "Paragraphs"
          }
        }
      },
      "patchAliases": {
        "summary": "properties/summary/enabled",
        "summary_tone": "properties/summary/properties/tone"
      },
      ...
    }
  ]
}

To enable the summary and set the tone to Formal, use the following configuration message:

{
  "type": "CONFIGURATION",
  "patches": {
    "summary": "true",
    "summary_tone": "Formal"
  }
}

Vatis Docs

Speech to Text

Audio Intelligence

Infrastructure

Integration

Configuration patches

Overview