Figure 1 Example of glue logic functionality
¶ Purpose of the glue component
- Forward messages from proprietary topics to standard COFY onthology topics
- Reformat payloads from proprietary format to standard COFY onthology format
- JSONify payloads
- Add or remove attributes
- Modify payloads
- Add, substract, rescale payload value with fixed values or payloads of other topics
- Combine topics and their payloads into one
The glue component is implemented as a Python service which, at startup, reads in a JSON configuration file.
This configuration file contains:
- a list of ingredients: the MQTT topics to subscribe to and, optional, the payload attributes to extract,
- a list of recipes: the operations to be performed with the ingredients.
¶ Example configuration file
{
"version": "202111120",
"ingredients": {
"total_consumption": {
"friendly_name": "Total energy consumed",
"topic_in": "shellies/+/emeter/1/total",
"json_template": null
},
"total_injection": {
"friendly_name": "Total energy injected",
"topic_in": "shellies/+/emeter/1/total_returned",
"json_template": null
},
"total_power": {
"friendly_name": "Total energy injected",
"topic_in": "shellies/+/emeter/1/power",
"json_template": null
},
"voltage": {
"friendly_name": "Grid voltage",
"topic_in": "shellies/+/emeter/1/voltage",
"json_template": "{{foo}}"
}
},
"recipes": {
"total_consumed": {
"friendly_name": "Total consumption",
"topic_out": "data/devices/utility_meter/total_energy_consumed",
"recipe": "{{total_consumption}}",
"entity": "utility_meter",
"channel": "total_energy_consumed",
"unit": "Wh",
"metricKind": "cumulative",
"metric": "electricityImport",
"ignore_after": 300,
"ignore": false
},
"total_injected": {
"friendly_name": "Total injection",
"topic_out": "data/devices/utility_meter/total_energy_injected",
"recipe": "{{total_injection}}",
"entity": "utility_meter",
"channel": "total_energy_injected",
"unit": "Wh",
"metricKind": "cumulative",
"metric": "electricityImport",
"ignore_after": 300,
"ignore": false
},
"total_power": {
"friendly_name": "Total active power",
"topic_out": "data/devices/utility_meter/total_active_power",
"recipe": "{{total_power}}",
"entity": "utility_meter",
"channel": "total_active_power",
"unit": "W",
"metricKind": "cumulative",
"metric": "electricityImport",
"ignore_after": 300,
"ignore": false
},
"voltage_phase_l1": {
"friendly_name": "Voltage phase 1",
"topic_out": "data/devices/utility_meter/voltage_phase_l1",
"recipe": "{{voltage}}",
"entity": "utility_meter",
"channel": "voltage_phase_l1",
"unit": "V",
"metricKind": "gauge",
"metric": "electricityImport",
"ignore_after": 300,
"ignore": false
}
}
}
¶ Documentation of configuration file
| Ingredients | |
|---|---|
friendly_name |
Human readable name of the ingredient (optional) |
topic_in |
Topic to subscribe to |
json_template |
Extract the value of this template from the topic_in payload (optional) |
| Recipes | |
|---|---|
friendly_name |
Human readable name of the recipe (optional) |
topic_out |
Topic to publish result of recipe to |
recipe |
The actual recipe: a Jinja2 template describing the operations to be performed on one or more of the ingredients. The ingredients can be referenced by name here. |
entity |
Name of virtual entity (see COFY onthology) |
channel |
Name of data channel (see COFY onthology) |
unit |
Unit of recipe result (see COFY onthology) |
ignore_after |
Discard ingredients if they are older than this number of seconds (optional) (default: 300) |
ignore |
Discard entire recipe if one of the ingredients went stale (true), or go on without the stale ingredients (false) (optional) (default: true) |
metricKind |
Description of the value type: cumulative - The value constantly increases over time. The value can periodically restart from zero when the maximum possible value of the counter is exceeded. delta - The value measures the change since it was last recorded. gauge - The value measures a specific instant in time. For example, metrics measuring temperature. actuator - The value indicates a state. Can be 0 or 1. |
metric |
The type of measurement, see list here: https://docs.cofybox.io/cofycloud/hub-listener#data-format |
¶ Operation of the glue component
Because MQTT messages arrive asynchronously and not necessarily with a retain flag, the glue component contains its own internal data cache which stores the most recently received message and, if configured for this ingredient, the extracted template value for all subscribed topics.
If a message is received on one the subscribed topics_in, the payload is stored in the cache, if required the ingredient is extracted through its template, and the execution of the recipes requiring this ingredient is triggered.
The recipes are stored in a list of dictionaries. Upon triggering (receiving one or more of the ‘fresh’ ingredients), the specific recipes incorporating the fresh ingredient are evaluated.
Specifically, the ignore_after and ignore conditions of the recipe are checked. These check, when the recipe requires more than one ingredient, if the time difference between the ingredients (based on the timestamp of each ingredient in the dataframe) is not too big (‘the ingredient went stale’) and what to do if this is the case (drop execution of the recipe or continue with only the fresh ingredients). If all the ingredients are fresh, the recipe is executed.
¶ Evaluating the Ingredients
Ingredients can either be taken as the raw input of the specified MQTT topic, or in the case of JSON format payloads, can be processed using Jinja templates.
¶ Example: Parsing a JSON payload from MQTT
Take the following Ingredient configured in the Glue Code YAML:
temperature_living_room:
friendly_name: 'Living room temperature'
topic_in: 'stat/COFY24E8/ds18b20/282e85350c0000d0'
json_template: '{{value}}'
Say the following JSON is posted to the MQTT topic specified in topic_in:
{
"timestamp": "2021-08-04T13:53:25Z",
"value": 20.3,
"units": "celsius"
}
By defining a json_template field in our ingredients configuration, we have told the glue component to parse the JSON payload as a dictionary and to extract the value element.
¶ Example: A (slightly) more complex template
Say you want a new ingredient, which must specify the same temperature in fahrenheit. We can perform mathematical processing before handing the result to recipes:
temperature_living_room_fahrenheit:
friendly_name: 'Living room temperature (fahrenheit)'
topic_in: 'stat/COFY24E8/ds18b20/282e85350c0000d0'
json_template: '{{value * 1.8000 + 32.00}}'
Jinja templates are very versatile, you can read more about in-built functions in their documentation.
¶ Evaluating the recipe
The recipe is a Jinja2 template which describes how ingredients are combined. The ingredients are referenced by
Example: we have the following two ingredients
temperature_living_room, in °C
temperature_outside, in Kelvin
We want to calculate the output temperature_difference in °C. The recipe would then look like something like this:
[...]
"diff_temperature": {
"friendly_name": "Difference in temperature between inside/outside in °C.",
"topic_out": "data/devices/tempsensor/temp",
"recipe": "{{temperature_living_room-(temperature_outside-273)}}",
"entity": "tempsensor",
"channel": "temp",
"unit": "C",
"metricKind": "gauge",
"metric": "indoorTemperature",
}
[...]
¶ Getting particular devices working in glue
We have a page with information about integrating devices that we use with glue, see here devices
This process inevitably involves publishing data on new MQTT topics. We are trying use a naming convention for these:
cb/<Device Class>/<Quantity> or cb/<Device Class>/<Sub-topic>/<Quantity>
A naming convention allows us to use universal glue keys regardless of the type of device a user has, and this particular convention differentiates the source of the data from the streams coming from glue.
"cb" (CoFyBox) to indicate that this MQTT topic is meant for use entirely within the CoFyBox. Device Class indicates the type of device that the data comes from, e.g. "battery", "ev" etc. Quantity gives the type of data, e.g. "power_output", "supply_voltage" etc. Sub-topic may be useful in cases where the device produces a lot of interesting data and it can be further categorised.