Supported device types and versions 
About the MQTT Sparkplug protocol
Communication line configuration
Communication line parameters
Communication station configuration
I/O tag configuration
Literature
Document revisions

Supported device types and versions


The protocol is an implementation of the MQTT 3.1.1 standard (October 2014). MQTT protocol is a client/server protocol of a subscribe/publish type. It is simple, has little overhead, and is easy to implement. It is used for M2M communication (Machine to Machine) and in the IoT context (Internet of Things). The MQTT server is also called the MQTT broker.
D2000 KOM implements the client part of the protocol. The protocol is implemented on a TCP/IP line. MQTTS (Secure MQTT, MQTT over TLS) is also supported - either directly in the D2000 KOM process (using TLS settings on TCP/IP-TCP line) or via the tunnel utility.
For the transfer of LoRaWAN data encapsulated within the MQTT protocol, see LoRaWan protocol description.

The Sparkplug 3.0.0 standard (Sparkplug B MQTT Payload) is also partially supported. Support includes:

The communication was tested/deployed with:

Note: communication with the cloud liveobjects.orange-business.com via WebSockets (wss://liveobjects.orange-business.com:443/mqtt) was also tested. The program https://github.com/jimparis/unwebsockify.git was used as a WSS wrapper. This program started with the following parameters:
./unwebsockify.py --port 1883 --listen 172.16.0.1 wss://liveobjects.orange-business.com:443/mqtt
The D2000 KOM process was connected to address 172.16.0.1 on port 1883. The WSS wrapper was connected to the defined URL and wrapped the MQTT communication data in a WebSocket envelope.


Each PUBLISH message contains a topic (Topic), data (Payload), and level of confirmation (QoS). PUBLISH messages can be sent both by the client and the server. The clients at the beginning of the communication will use the SUBSCRIBE message to indicate what topics (parameter of Topic Filter protocol) they are interested in. 
The protocol defines the following levels of confirmation of PUBLISH messages - QoS (Quality of Service):

 

The level of confirmation of the messages sent by the D2000 KOM process is defined by the protocol parameter Publish QoS. The D2000 KOM process considers the writing of the output tag to be successfully finished depending on the QoS:

 

The MQTT communication starts with the CONNECT message sent by the client (D2000 KOM). The message contains User Name, Passwordand other parameters, which can be modified:  Clean Session FlagClient ID, Will Flag, Will QoSWill RetainKeep Alive, and Will Topic. The server replies with a CONNACK message with a return code that contains information about the success of the connect operation.

Then the client sends a SUBSCRIBE message with a filter of topics (Topic Filter parameter), specifying which topics it is interested in, and with the required maximum level of confirmation (parameter Subscribe QoS).

The server responds with a return code that contains information about the success and maximum QoS that was assigned to the requested topics.

Then follows a phase of communication, during which both the client and the server send PUBLISH messages (the client with any topic, the server with topics relating to the filter of topics of the received SUBSCRIBE message) and confirm them according to the value of the QoS parameter of the received PUBLISH messages.

If the server does not send a message for longer than Ping Interval seconds, the client sends the PING request message, to which the server must respond with the PING response message (within the time specified by the Reply Timeout parameter).
If parameters change on the line, the connection is closed and re-created.

About the MQTT Sparkplug protocol

Sparkplug's MQTT protocol defines 3 groups of applications:

The D2000 can operate in Host Application mode or Edge Node mode (plus one or more Device/Sensor devices) according to the Payload Type line parameter. In Edge Node mode, exactly one station with an Edge Node address and optionally additional stations with slave Device/Sensor addresses must be configured on the line.

MQTT Sparkplug defines the following message types:

MQTT Sparkplug defines a Topic syntax for all message types, which has the form namespace/group_id/message_type/edge_node_id/[device_id], where:

Examples:

Note: STATE messages have a Topic of the form spBv1.0/STATE/sparkplug_host_id, where sparkplug_host_id is the unique name of the host application  (Sparkplug Host ID parameter).

Examples:

All message types except STATE have a binary Sparkplug payload. The STATE messages have a JSON payload - they contain only the items "online" (True/False value) and "timestamp" (time stamp in milliseconds).

The Sparkplug payload contains metrics - each required metric can be configured into a single I/O tag. The metric contains a text identifier (specified in the I/O tag configuration as sparkplug_address), value type, value (simple/array/structured), optionally a timestamp, and other attributes. D2000 supports working with simple values ​​and array values ​​- the first field item is stored in the I/O tag; it is possible to configure the Destination Column for the I/O tag, in which all array items are stored (or the first N, if the structure does not have enough rows). D2000 can read items of template (UDT) and Dataset metrics.
Note: If a metric arrives with the is_null flag set, the I/O tag will have the Invalid value.


Communication line configuration


Note: The default port for the MQTT protocol is 1883 or 8883 for the SSL/TLS version. D2000 KOM contains an implementation of the SSL/TLS protocol variant, alternately t is possible to configure it by using the stunnel utility http://www.stunnel.org working in a client mode (client = yes). Stunnel running on the same computer as the D2000 KOM should listen to the 1883 local port and after connecting of D2000 KOM process to the port should encrypt the communication using SLL/TLS and send it to the target MQTT server (typically on port 8883).

Note: when using an SSL/TLS connection, you must pay attention to the expiration of the certificates used. The expiration date can be found on Linux with the command

openssl x509 -text -in file.crt | grep "After"

Forced disconnection: If all stations on the line are in the simulation mode or the communication is stopped for them, the line will be disconnected (the communication socket will be closed). If the simulation is disabled for at least one station and the communication is not stopped for it (the Parameters tab of the Station type object), the line will be connected again.

Communication line parameters


Dialog link configuration - Protocol parameters tab.
They affect some optional protocol parameters. The following protocol line parameters can be entered:

Table 1

ParameterDescriptionUnit / sizeDefault value

Full Debug

Activates detailed debug information about sending and receiving values.YES/NONO
User Name
User name used in a CONNECT message to connect to the MQTT server.-
Password
Password used in a CONNECT message to connect to the MQTT server.-
Topic Filter
The name of one topic or a multiple-topic filter sent within the SUBSCRIBE message. Using the filter the MQTT client specifies topics, within which it wants to receive messages.
Note: topics are hierarchically sorted, a slash (/) is used as the separator, a plus (+) is used as a one-level mask, a hash (#) character is used as a mask for multiple levels.
Examples of filter: a/b , level1/+ , # , +/+/+/up
Note: the change of the Topic Filter parameter will be reflected after restarting the communication - e.g. due to the breakdown of the TCP connection, as long as all stations on the line are switched off (StOff) and switched on again, or after a restart of the KOM process. In the first two cases, the message UNSUBSCRIBE is sent to the original Topic Filter and then SUBSCRIBE to the new Topic Filter (this can be important in so-called persistent sessions when the Client ID parameter is specified and the MQTT server remembers the state of the client even after the TCP connection is broken).
Note: for Payload Type = Sparkplug Host, the filter spBv1.0/# is sufficient to receive all Sparkplug messages.
Note: for Payload Type = Sparkplug Edge Node, a SUBSCRIBE message is automatically sent to all NCMD/DCMD topics derived from the addresses of Edge Node/Device stations configured on the link.

Example:

  • for an Edge Node station with the address myGroup/myEdgeNode, a SUBSCRIBE message is sent to the topic spBv1.0/myGroup/NCMD/myEdgeNode
  • for a Device station with the address myGroup/myEdgeNode/myDevice1, a SUBSCRIBE message is sent to the topic spBv1.0/myGroup/DCMD/myEdgeNode/myDevice1

Note: Payload Type = Sparkplug Edge Node, a SUBSCRIBE message is automatically sent to the topic spBv1.0/STATE/<primaryHostApplication> if the Sparkplug Host ID parameter is specified (<primaryHostApplication> is the value of this parameter), so that the D2000 KOM process is informed about the connection/disconnection of the Primary Host Application. If a STATE message about the Primary Host Application disconnection is received (STATE message with online=false) and multiple IP addresses are specified in the line configuration, the D2000 KOM process disconnects from the current MQTT server and tries other MQTT servers until it finds one to which the Primary Host Application is connected (STATE message with online=true).

-#

Subscribe QoS

The desired maximum level of validation (QoS) sent within the SUBSCRIBE message.
The MQTT server can then send PUBLISH messages with such or lower levels of confirmation (but not higher). PUBLISH messages sent by the MQTT server will be confirmed by the D2000 KOM process according to the level of confirmation specified in them. The higher the level of confirmation, the more messages between the client and the server are exchanged (1 at QoS_0, 2 at QoS_1, and 4 at QoS_2).
QoS_0
QoS_1
QoS_2
QoS_1
Client ID

Unique client identifier (Client Identifier) sent within the CONNECT message.
Note: it is possible to enter a blank string - in which case the server can assign a unique name to the client (if it supports such functionality) or return an error. However, if the Client ID is not specified, the Clean Session Flag parameter settings will be ignored (as the server will assign a unique name each time).

The tested MQTT server (thethings.network) returned an error if the Client ID was blank and Clean Session Flag=NO.

Note: Some MQTT brokers (PIXII.COM, Eclipse Mosquitto) identified clients only by Client ID. In practice, this meant that two different D2000 systems that connected to the same broker were considered as one client, and the broker closed an existing connection that it considered old when a new connection was established, or it did not allow a new connection to be created and returned the error Connection Refused, identifier rejected (2).
After setting the Client ID to a unique value, the communications started to work without connection breakdowns.

-D2000kom
Clean Session Flag

Parameter Clean Session Flag of the CONNECT message. The NO value means that the server uses the current session state (connection) - e. g. after the collapse and recovery of the TCP connection. This means that all unconfirmed PUBLISH messages with QoS_1 and QoS_2 are resent (optionally also QoS_0, depending on the implementation).
The YES value means that the session is re-created, and unconfirmed PUBLISH messages are not repeated.
Note: If Payload Type = Sparkplug Host and the Sparkplug Host ID parameter is not empty, this parameter is ignored, and Clean Session Flag is set to YES (see the description of the Sparkplug Host ID parameter).
Note: If Payload Type = Sparkplug Edge Node, this parameter is ignored, and Clean Session Flag is set to YES (see the description of the Sparkplug Host ID parameter).

YES/NONO

Publish QoS

Level of confirmation (QoS) used to send PUBLISH messages through the D2000 KOM process.
Sending the PUBLISH message is the outcome of writing into the output tag with the OUT_VALUE address. The higher the confirmation level, the more messages between the client and server are exchanged (1 for QoS_0, 2 for QoS_1, and 4 for QoS_2).
QoS_0
QoS_1
QoS_2
QoS_0

Publish Retain

Setting the Retain flag used when sending PUBLISH messages by the D2000 KOM process. Activating the Retain flag causes the last message sent by the D2000 KOM process to be available on the MQTT server to other clients immediately after they are connected, as well as after the D2000 KOM process is disconnected.YES/NONO

Keep Alive

Parameter Keep Alive sent as part of a CONNECT message.
If it is non-zero, it indicates the interval in seconds in which the client must send a message (if it does not send it, the MQTT broker can disconnect the client). If it has no data to send, it will at least send a PING request. A zero value means that the MQTT broker does not require periodic messaging.

The recommended Keep Alive value is several minutes.

The D2000 KOM process sends PING requests according to the settings of the Keep Alive and Ping Interval parameters (whichever interval expires first).

0-65535 sec0

Ping Interval

If the MQTT server did not send any message during the specified time interval, the D2000 KOM process sends a PING request and waits for a PING response (until time Reply Timeout).

A value of 0 turns off sending the PING request messages. The parameter allows detection of TCP connection failure.

0-3600 sec60

Payload Type

The setting of message parsing:

  • Text only - the message is not parsed, it is assigned to the I/O tag with the address IN_TOPIC
  • JSON - the message is parsed as JSON data. If there is an I/O tag with the address IN_TOPIC, the whole message will be assigned to it.
    If there are I/O tags with addresses JA=json_address, they will be populated with the appropriate data from the JSON message. If no such addresses exist in the message, the I/O tags will be invalidated.
  • Sparkplug Host - the message is parsed as Sparkplug B payload (binary coded), The D2000 KOM process behaves as a Host Application (data consumer).
  • Sparkplug Edge Node- the message is parsed as Sparkplug B payload (binary coded), The D2000 KOM process behaves as an Edge Node (data producer).

Text only

JSON

Sparkplug Host

Sparkplug Edge Node

Text only

Time Field Name

If Payload Type=JSON, the name of the field with a timestamp. If the field name is not specified or the field is not found, the current time is assigned to the values.
For more information on the field name format, see  I/O tags with addresses JA=json_address.
--
Time Mask
Mask for parsing a value in the field with a timestamp. Special masks are:
  • UNIX - the numeric value represents the number of seconds from epoch 00:00:00 01.01.1970 UTC.
  • UNIXMS - the numeric value represents the number of milliseconds from epoch 00:00:00.000 01.01.1970 UTC.

Note: Whether the time is interpreted as local or UTC with a configured offset depends on the time station parameters settings.

-yyyy-mm-dd hh:mi:ss.mss

Ignore Missing Time

Ignoring a missing timestamp - if it is not present in the JSON payload, no warning will be issued.YES/NONO

Will Flag

Parameter Will Flag of a CONNECT message. A value of Yes means that the server will send a Last Will message to interested parties if the connection to the D2000 KOM process is lost.
Note: If Payload Type = Sparkplug Host and the Sparkplug Host ID parameter is not empty, this parameter is ignored and Last Will will be sent (see the description of the Sparkplug Host ID parameter).
Note: If Payload Type = Sparkplug Edge Node, this parameter is ignored and Last Will will be sent (see the description of the Sparkplug Host ID parameter).
YES/NONO

Will QoS

The acknowledgment level (QoS) used when sending a Last Will message in the event of a loss of connection to the D2000 KOM process.
Note: If Payload Type = Sparkplug Host and the Sparkplug Host ID parameter is not empty, this parameter is ignored, and QoS_1 level will be set (see the description of the Sparkplug Host ID parameter).
Note: If Payload Type = Sparkplug Edge Node, this parameter is ignored, and QoS_1 level will be set (see the description of the Sparkplug Host ID parameter).
QoS_0
QoS_1
QoS_2
QoS_0

Will Retain

The setting of the Retain flag used when sending a Last Will message if the connection to the D2000 KOM process is lost.
Note: If Payload Type = Sparkplug Host and the Sparkplug Host ID parameter is not empty, this parameter is ignored and Retain is set to YES (see the description of the Sparkplug Host ID parameter).
Note: If Payload Type = Sparkplug Edge Node, this parameter is ignored and Retain is set to NO (see the description of the Sparkplug Host ID parameter).
YES/NONO

Will Topic

The topic used to send the Last Will message if the connection to the D2000 KOM process is lost.
Note: If Payload Type = Sparkplug Host and the Sparkplug Host ID parameter is not empty, this parameter is ignored (see the description of the Sparkplug Host ID parameter).
Note: If Payload Type = Sparkplug Edge Node, this parameter is ignored, and Topic is set according to the Sparkplug standard (see the description of the Sparkplug Host ID parameter).
-

Will Message

Contents of the Last Will report if the connection to the D2000 KOM process is lost.
Note: If Payload Type = Sparkplug Host and the Sparkplug Host ID parameter is not empty, this parameter is ignored (see the description of the Sparkplug Host ID parameter).
Note: If Payload Type = Sparkplug Edge Node, this parameter is ignored, and Last Will is set according to the Sparkplug standard (see the description of the Sparkplug Host ID parameter).
-
Reply Timeout

If the MQTT server does not respond to the SUBSCRIBE, UNSUBSCRIBE, and PING requests within the required time or the D2000 KOM process fails to read a complete message (and only part of it is read), the D2000 KOM process declares an error, closes the connection, and opens it again. Value 0 turns off the timeout.
The parameter enables the handling of problematic behavior of the MQTT server.

sec20
Wait Timeout

A timeout of a single reading from a TCP connection. D2000 KOM repeats reading of spontaneous data Max. Wait Retry times and if no data is read, the reading is timeouted and finished (and may be followed by a further reading or writing). By lowering Wait Timeout and Max. Wait Retry parameters, it is possible to achieve a faster writing response of the D2000 KOM process at the expense of a higher CPU load when the MQTT server has no data.
Note: if a lot of messages come from the MQTT server and the D2000 KOM also needs to write values, we recommend setting a lower parameter value (e.g. 0.005 sec) so that writing is not blocked by reading (in any case, after 10 received messages, there is an interruption during which the accumulated writes can be performed).

sec0.100
Max. Wait Retry
The number of repetitions of reading from the TCP connection. See the description of the Wait Timeout parameter.-3

Payload Encoding

Payload field encoding. The MQTT protocol does not specify the content of the Payload field; the ISO-8859-1 standard encoding is suitable for both text and binary content, and UTF-8 is suitable if UTF-8 encoded texts are transmitted. Currently supported encodings are:

  • UTF-8

  • ISO-8859-1

  • Windows-1250

-

ISO-8859-1

Sparkplug parameters

Report Unknown Metrics

The parameter activates listings of unknown metrics and topics that do not have their own stations, but end up at a station with the address ".*" (if there is any). The listings will be in the line log as error messages even if the debug on the line is turned off (to facilitate the addition of I/O tags).

YES/NONO

Nonstandard Topic

Activation of parsing of non-standard SparkPlug topics (if Payload Type = Sparkplug Host).
The standard topic has the form 'namespace/group_id/message_type/edge_node_id/[device_id]' e.g. 'spBv1.0/Owner/DDATA/devicebase1/local_inputs'.
A non-standard topic has several levels instead of group_id, e.g., 'spBv1.0/Basel/Packaging/DDATA/Falcon11/plannedProductionQuantity'.
Note: writing to I/O tags on stations with a non-standard SparkPlug topic is not supported.

YES/NONO

Ignore Unknown Quality

Within the metric, it is possible to define a property called Quality of type Int32. According to the Sparkplug standard, it must be one of the values ​​0=BAD, 192=GOOD, 500=STALE. Any other D2000 Kom process reports as an error. The Ignore Unknown Quality parameter can be used to suppress this error message.

YES/NONO

Browse Dataset Mode

The parameter specifies which addresses of the Dataset metric are displayed when browsing:

  • Columns only: one address is displayed for each dataset column (e.g. Dataset[*]^ColA)
  • Items only: the address of each item is displayed (e.g. Dataset[1]^ColA, Dataset[2]^ColA, ...)
  • Columns + Items: addresses for both columns and items are displayed
-
Columns only

Item Separator

Separator of individual levels in Templates used when entering the address of the I/O tag in Sparkplug mode.
The default value is "->", so the address of the I/O tag can be e.g. SA=Template1->SubTemplate2->Item. If the template name contains the pair "->", it is possible to replace this separator (e.g. "=>").


->

Sparkplug Host ID

If Payload Type = Sparkplug Host:

Identifier of Host Application (D2000 is a Host Application). 
If specified, the D2000 KOM process will send a STATE message (with online=true) according to the MQTT Sparkplug standard after connecting to the MQTT server. This message announces that it is alive (equivalent to the NBIRTH and DBIRTH messages sent by Edge Nodes and Devices). At the same time, it sets the Will Topic/Will Message in the CONNECT message according to the Sparkplug standard (STATE message with online=false), with Will QoS=QoS_1, Will Retain=YES, Clean Session Flag=YES.
If the identifier is not specified, the D2000 KOM does not send the STATE message (and the Will parameters are configurable).
According to the MQTT Sparkplug standard, the identifier is mandatory for the Sparkplug Host mode (without specifying it, the D2000 KOM process pretends to be an "anonymous" client and does not send a STATE message)..

If Payload Type = Sparkplug Edge Node:

Identifier of Primary Host Application (Identifier of the Host Application that is defined as primary - the most important - for this Edge Node).
If the identifier is specified, the D2000 KOM process waits for a STATE message after connecting to the MQTT server. When it is received and contains online=true, it sends an NBIRTH message with definitions and values ​​of all output I/O tags of the station corresponding to the Edge Node and optionally DBIRTH messages for stations corresponding to Device/Sensor devices (if such stations are configured). If the STATE message contains online=false (i.e., the Primary Host Application is not connected to the MQTT server), the D2000 KOM disconnects from the MQTT server and tries another MQTT server according to the communication link configuration, provided that multiple MQTT servers are configured. If a single MQTT server is configured, the D2000 KOM waits to receive a STATE message with online=true from the Primary Host Application and only then starts sending NBIRTH/DBIRTH/NDATA/DDATA messages.
If the identifier is not specified, the D2000 KOM process sends NBIRTH and possibly DBIRTH messages immediately after connecting to the MQTT server (and subsequently sends NDATA/DDATA messages with value changes).
According to the MQTT Sparkplug standard, the specification of the Primary Host Application is optional for the Sparkplug Edge Node mode.
Note: Regardless of the value of the Sparkplug Host ID parameter, the D2000 KOM sets the Will Topic/Will Message in the CONNECT message according to the Sparkplug standard, with Will QoS=QoS_1, Will Retain=NO, Clean Session Flag=YES.

--

Convert Datatype/Timestamp to Text

Adding a textual representation of the value type (e.g. Int32) and a timestamp to the text I/O tag with IN_SP2JS address used to convert the Sparkplug payload to JSON for optional parsing in an ESL script.

YES/NONO

Communication station configuration


For a SparkPlug MQTT server, Topic has the form 'namespace/group_id/message_type/edge_node_id/[device_id]', where message_type indicates the message type (e.g., DDATA, DBIRTH, DDEATH).

If Payload Type = Sparkplug Host: A regular expression (e.g., spBv1.0/Sparkplug Devices/.*/MyDevice/Sensor2) can be used instead of message_type to cover all message types. It is possible to omit the namespace and message_type parts and write the Topic in the abbreviated form 'group_id/edge_node_id/[device_id]' (e.g., Sparkplug Devices/MyDevice/Sensor2).

If Payload Type = Sparkplug Edge Node: One station on the line should correspond to the Edge Node; its address must be in the form group/node (e.g., myGroup/myNode). There may be other stations corresponding to the Device/Sensor on the line, their addresses must be in the form group/node/device (e.g., myGroup/myNode/myDevice1), where group/node is the same as the address of the station corresponding to the Edge Node. According to the Sparkplug standard, these Device/Sensor stations belong to the corresponding Edge Node station (are its substations) and represent devices with which the Edge Node station communicates (e.g., via Modbus protocol).


Note:
If Payload Type = Sparkplug Host and the station address is in abbreviated form, the station doesn't process commands (DCMD, NCMD). If it is in the form of a regular expression ('namespace/group_id/.*/edge_node_id/[device_id]'), the station also processes commands (including the command sent by the D2000 KOM process if Send Node Control/Rebirth=YES). Therefore, we recommend specifying the station address in abbreviated form. If it is also necessary to process commands (from other Host Applications), then create another station with an address in the form of a regular expression (e.g. spBv1.0/Sparkplug Devices/DCMD/MyDevice).
If Payload Type = Sparkplug Edge Node, the station processes commands (DCMD, NCMD) and ignores data messages (NDATA, DDATA, NBIRTH, DBIRTH).


The Browse button opens a browse dialog for the station address. As long as the communication is functional, a dialogue containing the Topics received so far will be displayed. The list of received topics can be cleared with the Refresh button.
Double-clicking on a particular line will insert the value from the Address column into the configuration of the station from which the browse dialog was opened.
Note: The Station column shows the station to which the Topic has been assigned (based on the abovementioned processing priorities). For Sparkplug addresses, the abbreviated form of the address is displayed.

Station protocol parameters

Communication station - configuration dialog box - "Protocol parameters" tab.
They influence some of the optional parameters of the protocol.

Table 2

KeywordFull name

Meaning

UnitDefault value

SSA

Subscribe Station Address

If this parameter has the value YES, the Topic related to the station address is also added to the SUBSCRIBE message:

  • If Payload Type = Sparkplug Host, an NDATA/DDATA topic is created from the station address (e.g. spBv1.0/myGroup/NDATA/myEdgeNode or spBv1.0/myGroup/DDATA/myEdgeNode/myDevice1)
  • If Payload Type = Sparkplug Edge Node, the parameter is ignored - NCMD/DCMD topic is always created (e.g. . spBv1.0/myGroup/NCMD/myEdgeNode or spBv1.0/myGroup/DCMD/myEdgeNode/myDevice1)
  • If Payload Type = Text only or JSON, the topic is the same as the station address (i.e. it makes no sense to set this parameter on stations that have addresses in the form of a regular expression, e.g. status/batt.*)
YES/NONO

SWT

Station Will Topic

Will topic of the device. If this parameter is set and a message with the same topic is received, the station will go into a communication error (StHardErr) and the values of the I/O tags will be invalidated. In this way, it is possible to emulate the standard behavior that occurs when there is a communication error with the device (even if the communication between the D2000 Kom process and the MQTT broker is functional).
If Payload Type = Sparkplug Host, it is not necessary to enter this parameter - if an NDEATH/DDEATH message arrives with a Topic that matches the station address, the station will go into a communication error.



SWP

Station Will Payload

Content of the Will message. If this parameter is set and a message with the same topic as defined by the Station Will Topic parameter is received, the Payload must also be the same. If this parameter is an empty string, matching the topic with the Station Will Topic parameter is sufficient.
Note: This parameter was implemented because MQTT brokers send messages with the same Topic when connecting/disconnecting the device, the difference being only in the payload.



FPT

Payload Type

The setting of message parsing (overriding the line parameter Payload Type):

  • Default - the line parameter Payload Type is respected
  • Text only - the message is not parsed, it is assigned to the I/O tag with the address IN_TOPIC
  • JSON - the message is parsed as JSON data. If there is an I/O tag with the address IN_TOPIC, the whole message will be assigned to it.
    If there are I/O tags with addresses JA=json_address, they will be populated with the appropriate data from the JSON message. If no such addresses exist in the message, the I/O tags will be invalidated.
  • Sparkplug - the message is parsed as Sparkplug B payload (binary coded). Based on the line parameter Payload Type, this is Sparkplug Host/Edge Node mode, Sparkplug Host by default.
Default
Text only
JSON
Sparkplug
Default

FTF

Time Field Name

If Payload Type = JSON, the name of the field with a timestamp - overriding the line parameter Time Field Name--

FTM

Time Mask

Mask for parsing a value in the field with a timestamp - overriding the line parameter Time Mask.

Note: Whether the time is interpreted as local or UTC with a configured offset depends on the time station parameters settings.

--

Sparkplug parameters



SR

Send Node Control/Rebirth

If Payload Type = Sparkplug Host, when connecting to the MQTT server, a command (NCMD or DCMD) with the metric 'Node Control/Rebirth' is sent to the SparkPlug station. The response should be a message (NBIRTH/DBIRTH) with all current metrics.

YES/NOYES


I/O tag configuration


Possible value types of I/O tags: Ci, Co, TxtI, TxtO, Qi, Ci, Co, Ai, Ao, Di, Do, TiR, ToR, TiA, ToA.

The MQTT implementation supports three work modes:


Type of  I/O tagAddressDescription
I/O tags for reading data sent by the MQTT server through a PUBLISH message.
Note: values of I/O tags are set by the D2000 KOM process in the order IN_TOPIC, IN_DATA, and IN_ID. It is not necessary for the configuration to contain all three I/O tags.
TxtI
IN_TOPIC
Topic (Topic) of received PUBLISH message.
TxtI
IN_DATA
Data (Payload) of received PUBLISH message.
Ci
IN_ID
Identifier of a packet (Packet Identifier) of PUBLISH message that depends on the level of validation (QoS).
For messages sent with QoS_0, the identifier is zero, for QoS_1 and QoS_2, it is a positive 16-bit number.
Note: if the MQTT server sends also messages with the QoS_0 level of validation and the ACK_ID I/O tag is configured, then we recommend activating the option New value when changing time in the Filter tab, so that repeated writing of the value 0 will cause a new value that differs only in a timestamp to be generated.
I/O tag to confirm the received data to the MQTT server.
Co
ACK_ID
If an output I/O tag with ACK_ID address is defined, the D2000 KOM expects confirmation of the processing of each message by writing a copy of the value of the IN_ID tag. Only after, it sets values from the next received PUBLISH message (if it was received in the meantime) into the IN_TOPIC, IN_DATA, and IN_ID I/O tags (in this order). 
In the case of the QoS_0 level of confirmation, it is, therefore, necessary to repeatedly set the value of the I/O tag ACK_ID to 0. 
If the I/O tag ACK_ID does not exist, the values are written into the IN_TOPIC, IN_DATA, and IN_ID I/O tags immediately after the PUBLISH message is received and processed.
Note: for the messages received with the QoS_0 level of validation, no confirmation is sent to the MQTT server, only the values of the received PUBLISH message will be published. 
I/O tags for sending values to the MQTT server through a PUBLISH message.
Note: in order for the D2000 KOM process to send the PUBLISH messages to the MQTT server, both I/O tags must be defined within one station.
TxtO
OUT_TOPIC
The topic of the PUBLISH message being sent.
TxtO

OUT_VALUE

Data (Payload) of the PUBLISH message being sent.
Note: sending the message is performed out as a result of writing to the OUT_VALUE I/O tag (i.e. if the Topic does not change then it is sufficient to set the OUT_TOPIC point once - e.g. by using default value).
I/O tags for parsing JSON messages

TxtI, TxtO, Qi,
Ci, Co,
Ai, Ao,
Di, Do,
TiR, ToR

JA=json_address

If Payload Type=JSON, the message is parsed as JSON data. The json_address value specifies the name of the JSON field whose value is to be assigned to the I/O tag
For JSON messages that can be structured, the syntax level1.level2.level3 ... is supported, e.g. rx.current, and if they contain fields (indexed from 1) also syntax level1[index1].level2[index2].level3 ... is possible, e.g. rx.gwrx[1].time.
Since the JSON message itself can be an array, the address can also start with an index, e.g. JA=[1].batt_cell_v_avg

For other examples, see the description of the LoRaWAN protocol's Envelope type I/O tags.

I/O tags for parsing Sparkplug messages

TxtI, TxtO, Qi,
Ci, Co,
Ai, Ao,
Di, Do,
TiR, ToR, TiA, ToA

Input I/O tags:
SA=sparkplug_address

Output I/O tags:
ST=type;SA=sparkplug_address

ST=type;SAL=alias;SA=sparkplug_address

If Payload Type = Sparkplug Host/Edge Node, the message is parsed as Sparkplug data (a binary format built on Google Protocol Buffers). Sparkplug data contains metrics that have text identifiers (sparkplug_address).

Reading template items  is possible by specifying sparkplug_address in the format <TemplateName1><Separator><TemplateName2><Separator> ... <Separator><ItemName> where:

  • <TemplateNameX> is the name of the template/sub-template
  • <Separator> is the separator for individual levels (standard characters "->", which can be changed with the Item Separator parameter if this sequence occurs in template/item names)
  • <ItemName> is the item name of the deepest nested template

Examples of template item addresses:
SA=Template1->SubTemplate2->Item
SA=secUDT->sec

Reading dataset items (equivalent to structured variables in D2000) is possible by specifying sparkplug_address in the format <DatasetName>[<Row>]^<ColumnName> where:

  • <DatasetName> is the name of the dataset (it can also be part of a structure, e.g. Template1->SubTemplate2->Dataset3)
  • <Row> is the column number (1..N) or the "*" character. In the latter case, it is possible to configure the Destination column to which all rows are written (values ​​from the first row of the corresponding column are written to the I/O tag)
  • <ColumnName> is the name of the dataset column

Examples of dataset item addresses:
SA=Performance[3]^ActivePower
SA=Machine2->Parameters[1]^ActivePower
SA=DHS/Formation Data->Reservoir Parameter[*]^Gas density


For output I/O tags, the value type must be specified (ST=type). Simple types are supported (not template items/dataset items):

  • Int8
  • Int16
  • Int32
  • Int64
  • UInt8
  • UInt16
  • UInt32
  • UInt64
  • Float
  • Double
  • Boolean
  • String
  • DateTime
  • Text

Note: There is no difference between the String and Text types.

For output I/O tags, it is also possible to specify a numerical alias (SAL=alias) as a natural number (0, 1, 2, ...). The alias of the I/O tag within the station must be unique. Aliasing allows you to reduce the size of the transmitted data: in the NBIRTH/DBIRTH message, both the text identifier (sparkplug_address) and the alias are specified for each I/O tag, in the NDATA/DDATA messages, only aliases that are shorter than the text addresses are specified.


The PUBLISH message created during writing contains a Topic derived from the station address. The message type depends on the station address:

  • for Payload Type = Sparkplug Host, depending on whether it is an Edge Node (NCMD) or Device/Sensor (DCMD)
  • for Payload Type = Edge Node, depending on whether it is an Edge Node (NDATA) or Device/Sensor (DDATA)

The Payload of the message contains a timestamp, a value type (type), a written value (encoded according to the specified value type), and a metric name (sparkplug_address).

TxtI

IN_SP2JS

The I/O tag is used to convert the Sparkplug payload into a JSON representation, which can then be processed, e.g. in ESL script. Depending on the Convert Datatype/Timestamp to Text parameter, a textual representation of the value type and timestamp is also added. 
An example of value:

{"metrics":[{"datatype":3,"int_value":7338992,"name":"Corrected Vol Acc Stn","timestamp":1729664005479}],"seq":32,"timestamp":1729664005479}

After formatting into readable form:

{
    "metrics": [
        {
            "datatype": 3,
            "int_value": 7338992,
            "name": "Corrected Vol Acc Stn",
            "timestamp": 1729664005479
        }
    ],
    "seq": 32,
    "timestamp": 1729664005479
}

An example of a more complex value containing properties and a dataset and also displaying a textual representation of the data type (datatype_txt) and timestamp (timestamp_txt) as a result of the set parameter Convert Datatype/Timestamp to Text:

{
    "metrics": [
        {
            "datatype": 12,
            "datatype_txt": "String",
            "name": "Node Properties/Configuration",
            "string_value": "{}",
            "timestamp": 1730305529539,
            "timestamp_txt": "30-10-2024 17:25:29.539"
        },
        {
            "alias": 30064771073,
            "datatype": 5,
            "datatype_txt": "Uint8",
            "int_value": 0,
            "name": "Node Properties/Missing Param",
            "properties": {
                "keys": [
                    "usage"
                ],
                "values": [
                    {
                        "string_value": "technical information",
                        "type": 12,
                        "type_txt": "String"
                    }
                ]
            },
            "timestamp": 1730305529537,
            "timestamp_txt": "30-10-2024 17:25:29.537"
        },
        {
            "alias": 0,
            "dataset_value": {
                "columns": [
                    "topic_name",
                    "offset",
                    "length",
                    "crc"
                ],
                "num_of_columns": 4,
                "rows": [
                    {
                        "row": [
                            "N/A",
                            0,
                            0,
                            0
                        ]
                    }
                ],
                "types": [
                    12,
                    7,
                    7,
                    7
                ],
                "types_txt": [
                    "String",
                    "UInt32",
                    "UInt32",
                    "UInt32"
                ]
            },
            "datatype": 16,
            "datatype_txt": "DataSet",
            "name": "Node Control/FW Update",
            "timestamp": 1730305529537,
            "timestamp_txt": "30-10-2024 17:25:29.537"
        }
    ],
    "seq": 0,
    "timestamp": 1730305529536,
    "timestamp_txt": "30-10-2024 17:25:29.536"
}


Note: it is also possible to monitor the status of other Sparkplug Host Applications connected to the MQTT server. If the Identifier of Host Application is e.g. "ACME", so it is necessary to create a station with the address "spBv1.0/STATE/ACME" (or in abbreviated form "ACME") and on it an I/O tag of type Di with the address "JA=online" (since the Host Application sends a STATE message with a JSON payload).


Browse

In the Sparkplug Host mode, for the I/O tags, it is possible to discover the list of metrics, as long as the KOM process is running and communication with an MQTT server is established.
(For I/O tags in Sparkplug Edge Node, it is only possible to find the list of metrics to which writes were made using NCMD/DCMD Topics).
Clicking the Browse button opens the MQTT Item Browser window and displays a list of received metrics. The object list is created dynamically as a result of received messages.

The list of objects is dynamic, i.e. when a new value arrives in the KOM process, it is updated. Filtering in individual columns (with the exception of Time column) is also supported, asterisks can be used in the mask (e.g. *Short*).

Double-clicking on a particular line will insert the value from the Address column into the configuration of the I/O tag from which the MQTT Item Browser window was opened. At the same time, it is also inserted into the clipboard.

The Refresh button clears the list of values in both the CNF and the KOM process and optionally sends the Rebirth command (In the Sparkplug Host mode, if Send Node Control/Rebirth=YES).

The Value column contains the received value.

Literature


Links
Official website of MQTT protocol http://mqtt.org

Official website of MQTT Sparkplug protocol https://sparkplug.eclipse.org


Specifications and Standards
MQTT 3.1.1 specification http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/mqtt-v3.1.1.html
ISO/IEC 20922:2016 http://www.iso.org/standard/69499.html

Descriptions of Data Formats and API
www.loriot.io - Application API Data Format https://www.loriot.io/home/documentation.html#docu/app-data-format
www.thethingsnetwork.org - API Reference https://www.thethingsnetwork.org/docs/applications/mqtt/api.html


The attached ZIP contains the configuration of two lines and two stations with the MQTT protocol and the definition of the S.MqttTest scheme. Data written over one line is received over the other line. The functionality was verified against the Mosquitto broker installed locally on Windows, listening on TCP port 1883.


The attached ZIP contains a configuration of two lines and four stations with the MQTT Sparkplug B protocol. Both lines connect (alternatively) to the servers test.mosquitto.org and localhost.

The line L.MQTT_MOSQUITTO_EN (parent MQTT_MOSQUITTO_EN.KOM) is in Sparkplug Edge Node (data producer) mode, has two stations B.MQTT_MOSQUITTO_EN (Edge Node with address myGroup/myEdgeNode) and B.MQTT_MOSQUITTO_EN.Device1 (Device with address myGroup/myEdgeNode/myDevice1). These stations publish the values ​​of the objects Hour, Min, Sec.
The line parameter Sparkplug Host ID is set to "D2000komHA" (The Host application with this name is considered primary). The line L.MQTT_MOSQUITTO_EN connects alternately to the servers test.mosquitto.org and localhost until it finds a connected Primary Host Application.

The L.MQTT_MOSQUITTO_HA line (parent MQTT_MOSQUITTO_HA.KOM) is in Sparkplug Host mode (data consumer), has two stations B.MQTT_MOSQUITTO_HA (receiving data with the topic myGroup/myEdgeNode) and B.MQTT_MOSQUITTO_HA.Device1 (receiving data with the address myGroup/myEdgeNode/myDevice1). These stations receive object values ​​published by I/O tags on Sparkplug Edge Node stations.
The line parameter Sparkplug Host ID is set to "D2000komHA" (this is the Host application identification).

Using the output I/O tags on Sparkplug Host stations (M.MQTT_MOSQUITTO_HA.Output1_Int32, M.MQTT_MOSQUITTO_HA.Output2_Float, and M.MQTT_MOSQUITTO_HA.Device1.Output1_Bool), it is possible to send DCMD and NCMD messages and write to the input I/O tags on Sparkplug Edge Node stations (M.MQTT_MOSQUITTO_EN1.Input1, M.MQTT_MOSQUITTO_EN1.Input2, and M.MQTT_MOSQUITTO_EN.Device1.Input1_Bool).



You can read a blog about the MQTT protocol


Document revisions



Communication Protocols