pdserv  3.1
Process data server
MSR language syntax

Table of Contents

Introduction

The MSR data server is one means for a client to interact and communicate with a process encapsulated by the pdserv library, possibly running in real time.

The client attaches to the pdserv server via a TCP network socket, by default on port 2345. Communication is bi-directional.

The communication syntax between client and server follows an XML-like format.

Upon connection, the server sends an initial greeting and waits for commands from the client.

Command stream from client to server

Commands are sent by the client to the server in a single XML element. The XML-like command format in BNF is:

command := '<' tag attribute* '/>'
tag := name
attribute := ' ' name '="' value '"'
name := any character except the obvious (space, angled brackets, qoutes)

Example of setting a parameter value:

<write_parameter index="55" value="123457"/>
Note
The command parser is very relaxed (attributes can have single ('), double (") or even no quotes if the value does not contain spaces, the closing can be '>' or '/>', boolean attributes can use the attribute name stand alone without '="1"'), but do not rely on this feature! This is only useful when a human interacts on the network socket directly. Programs must always use the strict definition as shown above.

Unrecognized commands return a warning:

<silly/>
<warn num="1000" text="unknown command" command="silly"/>

Unrecognized attributes are silently ignored.

There are state changing and query commands, and they may or may not generate a reply. Commands are processed in order of reception. It is not required to synchronize commands to replies, although the client will do that for its own purpose.

To determine whether a command was processed by the server, the id attribute is used within the command element. The reply (if one is generated) will have an id attribute with the same value. Additionally an <ack> element with the same id attribute value is sent immediately after command is processed.

For example, here are two commands sent in one line:

<rk index="99" id="rk99"/><rk index="3" id="rk3"/>
<ack id="rk99"/>
<channel index="3" name="/osc/amplitude/0/0" datasize="8" typ="TDBL"
bufsize="1000" task="0" HZ="100" time="1565346298.661905"
value="1.358077306222715e-312" id="rk3"/>
<ack id="rk3"/>

The first line contains two <rk> (query a signal) commands with an id attribute. The first is unsuccessful and produces no reply, however the <ack id="rk99"/> is replied before successfully processing the second query. Note the id attribute in <channel>.

This mechanism can be used to synchronize communication between client and server, especially when the commands do not produce a reply.

Reply stream from server to client

The reply stream is well formed XML that can be processed with a streaming XML parser like SAX or expat. A message is a single XML element that may contain child elements.

Note
Although the server sends a greeting in the form of a <connected> element, it is a complete element with a closing and is thus a finished document! Normal parsers will stop there and need to be reset for parsing to continue. Thus parsers must first be primed with any well formed opening element (like <msr>) which is never closed. Content generated by the server is ASCII which is valid UTF-8, although variable path names are encoded in native program format. To avoid problems, use UTF-8 throughout.

The communication from server to client is interleaved with solicited and unsolicited messages:

The client must carefully separate the return stream to distinguish between these message types.

The problem with unsolicited messages is that they occur at any point in time without any cause from the client. Thus the client has to monitor this stream continuously or run the risk of its input stream overflowing and thus being cut off by the server. For most clients (GUI's, etc) this is no problem, but for clients that want to excercise synchronous single threaded communication could choke. To alleviate this problem, unsolicited messages can be disabled using the <remote_host/> command.

Data types

When querying the server for its variables (signals or parameters), its data type and binary size (sizeof operator) is returned in the typ and datasize attributes respectively.

An example of quering a parameter:

<rp index="4"/>
<parameter index="4" name="/osc/amplitude/Limit" datasize="8" typ="TDBL"
flags="7" mtime="0.000000" value="20"/>

The following table maps these types to the corresponding C data type.

typ datasize c-type
TUCHAR 1 uint8_t
TCHAR 1 int8_t
TUSHORT 2 uint16_t
TSHORT 2 int16_t
TUINT 4 uint32_t
TINT 4 int32_t
TULINT 8 uint64_t
TLINT 8 int64_t
TFLT 4 float
TDBL 8 double
COMPOUND n struct {}

Dimension

The dimensionality of a variable is reflected in attributes when querying the variables.

Generally a variable can be scalar or non-scalar. Scalar variables contain only one of the data types as shown above (a compound can also be a scalar). If a variable contains more than one element, it is said to be non-scalar.

For non-scalar variables, the reply has the following additional attributes:

The data values are ordered in a C-like fasion. Adjacent values are the last dimension of a multi-dimensional array: for the

double array[4][5];

with 4 rows and 5 columns, the 5's will be adjacent (row major).

An example of a scalar:

<channel index="32" name="/SawTooth" datasize="1" typ="TUCHAR"
bufsize="100" task="0" HZ="10"/>

An example of a vector:

<parameter index="5" name="/Event/State" datasize="4" typ="TUINT_LIST"
anz="5" cnum="5" rnum="1" orientation="VECTOR" dir="1" flags="7"
mtime="0.000000" value="0,0,0,0,0"/>

An example of a matrix:

<channel index="2" name="/osc/amplitude" datasize="8" typ="TDBL_MATRIX"
anz="25" cnum="5" rnum="5" orientation="MATRIX_ROW_MAJOR"
dir="1" bufsize="1000" task="0" HZ="100"/>

Incedentally, the dir flag in the last two examples show that there are more variables listed under its name (see the list command).

Binary value representation

Binary values are transmitted in native machine format. Check the endian flag in the greeting.

MSR uses two methods of binary encoding:

hexdec-format

This format simply walks though the value's entire memory space byte by byte and prints it as a 2-digit hex value.

Example: pi = 182D4454FB210940 on a little endian platform.

This representation is used between client and server interaction.

To query the value of a variable, set the hex boolean attribute in the command and the value is returned in the hexvalue reply attribute.

To set the value of a parameter, use the hexvalue instead of the value** attribute in the command.

Base64 encoding

The value is encoded as base64 string which is more efficient than hexdec (four base64 vs six hexdec characters per three bytes binary data).

Example: pi = GC1EVPshCUA= on a little endian platform.

This representation is used for streaming subscribed signals when the coding attribute is set to Base64

Atomized variables

To support old clients that can only handle scalar variables, the server can atomize non-scalar variables in a path beneath it. This behaviour is set in the configuration file.

Atomized variables can cause an exponential explosion (think of a 10x10 matrix yielding 100 separate variables)

When requesting a directory list, use the attribute noderived to suppress listing of directories containing atomized variables only.

Commands

ping

Ping the server. The server will reply with its current time. This function is used to detect network presence of the server.

Command attributes: none

Reply: <ping/> with attributes:

Example:

<ping/>
<ping time="1565257713.113707"/>

write_parameter

Alias: wp

Set the value of a parameter. The connection must be authorized, see access in remote_host

Command attributes:

*) One of index (dominant) or name is mandatory, otherwise nothing happens.

**) One of hexvalue (dominant) or value is mandatory, otherwise nothing happens.

Values are written until either the parameter's end is reached or input value is exhaused.

Reply: none (possibly a parameter update notification follows)

See also
pdserv_parameter()

Example:

<wp index="6" hexvalue="DEADBEEF"/>

read_parameter

Alias: rp

Read a parameter's value value and attributes.

Command attributes:

*) If neither name (dominant) or index is specified, a complete parameter listing of all parameters is performed. The listing is enclosed in a <parameters></parameters> element with individual child <parameter/> elements.

Reply: <parameter/> with attributes:

*) These attributes are not transmitted when short attribute is set.

**) hexvalue is returned when hex attribute is set.

See also
pdserv_parameter()

Example:

<rp index="5"/>
<parameter index="5" name="/Event/State" datasize="4" typ="TUINT_LIST"
anz="5" cnum="5" rnum="1" orientation="VECTOR" dir="1" flags="7"
mtime="1565276030.213719" value="56746,1,0,0,0"/>

xsad

Subscribe a signals to the data stream sent by the server.

Command attributes:

Note
When channels is not specified, only sync takes affect. If a channel exists in a group, it is overwritten with the new attribute values.

Reply: none (but see the unsolicited <data> message)

Example:

<xsad channels="2,4" group="3" reduction="5" blocksize="10"/>

xsod

Removes signal subscription from a data transmission group.

Command attributes:

Reply: none

Example:

<xsod channels="4" group="3"/>

xsap

Version: after v3.0

Monitor parameters, transmitting the value on change.

Command attributes:

Reply: none (but see the unsolicited <data> message)

Example:

<xsap parameters="5,6"/>

xsop

Version: after v3.0

Cancel monitoring of parameter changes.

Command attributes:

Example:

<xsop parameters="6"/>

echo

Unused

read_kanaele

Alias: rk rc

Read the value and all attributes of a signal.

Command attributes:

*) If neither name (dominant) or index is specified, a complete signal listing of all signals is performed. The listing is enclosed in a <channels></channels> element with individual child <channel/> elements. Polling signal values is expensive, thus the value is not transmitted when a complete listing is performed,

Reply: <channel/> with attributes:

*) These attributes are not transmitted when short attribute is set.

**) hexvalue is returned when hex attribute is set.

Example:

<rk index="29"/>
<channel index="29" name="/osc/derivative" comment="Derivative of [cos,sin]"
datasize="8" typ="TDBL_LIST" anz="2" cnum="2" rnum="1" orientation="VECTOR"
text="Derivative of [cos,sin]" dir="1" bufsize="1000" task="0" HZ="100"
time="1565280630.398467" value="11.04316657549318,-5.031003686345138"/>

read_param_values

Alias: rpv

Return a list of all parameter values, sequencially sorted by index.

Reply: <param_values/> with attributes:

See also
pdserv_signal()

Example:

<read_param_values/>
<param_values value="1.2;10;1;0;20;56746,1,0,0,0"/>

auth

Version: since v3.0

Feature test: login

Perform SASL authentication dialog.

Command attributes:

Reply: <saslauth/> with attributes:

1) The first step is to obtain a list of supported mechanisms.

<auth/>
<saslauth mechlist="DIGEST-MD5 CRAM-MD5 NTLM PLAIN LOGIN ANONYMOUS"/>
Note
When login is mandatory, the <saslauth mechlist="..."/> element is sent automatically upon connection without prior request.

The list of mechanisms does not change. This first step is required only once upon initial connection by the client.

This step is not required if the mechanism is known apriori.

2) The client chooses a mechanism. Depending on the mechanism, the client library will start the conversation with some data or not. Supplying a mech** attribute initiates the server side of the conversation (client-send-first). Depending on the mechanism, the server may require more data or even signal success or failure.

<auth mech="PLAIN" clientdata="lkjsdlfjkl"/>
<saslauth serverdata="lkjasdflkj" success="1"/>

The mech attribute may only be sent on initiation. Certain mechanisms may require more than one interaction. Subsequent commands will only have clientdata, until the server replies with a success attribute.

list

Version: since v2.0

Feature test: list

Return a list of parameters, signals and directories.

Command attributes:

Reply: <listing></listing> element with children:

Example:

<list path="/"/>
<listing>
<dir path="/Event"/>
<channel index="32" name="/SawTooth" datasize="1" typ="TUCHAR"
bufsize="100" task="0" HZ="10"/>
<dir path="/Taskinfo"/>
<dir path="/osc"/>
</listing>

starttls

Version: since v3.0

Feature test: tls

Start tls data transmission.

Command attributes: none

Reply: <tls/>\r\n

Upon reception of this command, the server switches over to TLS transmission. However, the client must continue reception in plain text until the reply is received, including \r\n!

The client may switch its command channel to TLS after sending this command, but reception must continue in unencrypted until the reply (including \r\n!) is received before switching reception to TLS as well.

Example:

<starttls/>
<tls>\r\n

remote_host

Version: since v2.0

Feature test: polite

Command to set miscellaneous states.

Command attributes:

Example:

<remote_host name="myhostname" applicationname="GUI" access="1"/>

read_statistics

Alias: read_statics (no, this is not a spelling mistake) rs

Get information about other MSR clients connected to the server.

Reply: <clients></clients> element with <client/> child elements with attributes:

Example:

<read_statistics/>
<clients>
<client name="lansim (192.168.22.56:52276)"
apname="Persistent Manager, Version: 0.3.1"
countin="19908501" countout="27337577"
connectedtime="1282151176.659208"/>
<client .../>
</clients>

message_history

Feature test: history

Arguments:

Request the server to send past messages. Without attributes, the latest message including all active messages are sent enclosed in a <message_history></message_history> element in increasing order of sequence numbers.

With seq argument, the specific message with that sequence number is returned if it exists.

Reply:

Notes:

Typically this command is used to discover all past messages. To achieve this, first send

<message_history/>

and process all messages returned, taking note of the sequence numbers in the replies.

Then use

<message_history seq="XX" id="nn"/>

to retrieve individual messages, counting down from the highest sequence number. If a message does not exist, nothing is returned. Therefor the id tag should be used in this context to detect an empty reply.

Note
The sequence number is a uint32_t data type and can wrap (though very unlikely, but possible). Counting down from 0 will yield sequence number 4294967295 (2^32-1).
See also
pdserv_event()

Example of retrieving active messages:

<message_history/>
<message_history>
<crit_error name="/Limit" index="3" seq="150" prio="0"
time="1565386296.209379" text="Event message 3"/>
<crit_error name="/Limit" index="1" seq="4196" prio="0"
time="1565389793.155808" text="Event message 2"/>
<reset name="/Limit" index="0" seq="4220" time="1565389796.195809"/>
</message_history>

The last message has sequence number 4220 (/Limit[0]) but it is inactive. Messages /Limit[1] and /Limit[3] still persist.

To regenerate the list, retrieve messages with sequence numbers counting down from the highest sequence number (4220): 4219, 4218... (possibly, but not necessarily skipping 4196):

<message_history seq="4219" id="mh"/>
<crit_error name="/Limit" index="0" seq="4219" prio="0"
time="1565389795.655814" text="Event message 1" id="mh"/>
<ack id="mh"/>

Example of retrieving a specific message with sequence number 3220:

<message_history seq="3220" id="mh"/>
<ack id="mh"/>

Message 3220 does not exist any more becuase it was purged by the server (in this example it keeps 1000 messages - see configuration file). The list is now complete as far as possible.

However, even though the history has been exhausted, message /Limit[3] with sequence number 150 is still known to be active.

Unsolicited stream

ack

This element is sent just after the reply to any command with an id attribute is processed. This feature can be used to synchronize communication with the server.

Note
If id is specified in [<starttls/>] command, the <ack> is already encrypted!

Element attributes:

Example:

<echo id="echo id"/>
<ack id="echo id" time="1565618806.761301"/>

connected

This element is sent as a greeting upon connection to the server. It is used to greet the client with a information about the server, running process and a list of capabilities.

Element attributes:

Example:

<connected name="MSR" host="lappi" app="PdServ Test" appversion="1.234"
version="393226"
features="pushparameters,binparameters,eventchannels,statistics,pmtime,
aic,messages,polite,list,login,history,xsap,tls"
endian="little" recievebufsize="8192"/>

data

A <data></data> element is a container enclosing child elements of values for signals that were subscribed using xsad.

Element attributes:

Child elements:

Example of event-based subscription:

<xsad channels="29" event="1" group="9" id="1234"/>
<ack id="1234" time="1565613895.117706"/>
<data group="9" level="0" time="1565613895.122557">
<time d="XWcUZ5ItuhU="/>
<E c="29" d="-9.844291201868986,-5.137457097023935"/>
</data>

pu

Notification to all clients of a parameter change. Clients interested in its value must then request it.

See also: xsap

Element attributes:

Example:

<pu index="5"/>
<pu index="7"/>

parameter

A client can subscribe to monitor parameter value changes using xsap. Upon change, the value is sent automatically by the server using a <parameter/> element upon value change.

Example:

<xsap parameters="5" id="pmon"/>
<ack id="pmon" time="1565627421.396449"/>
<parameter index="5" name="/Event/State" datasize="4" typ="TUINT_LIST"
anz="5" cnum="5" rnum="1" orientation="VECTOR" flags="7"
mtime="1565627429.852331"
hexvalue="0100000000000000000000000000000001000000" pm="1"/>
<parameter index="5" name="/Event/State" mtime="1565627438.417570"
hexvalue="0000000000000000000000000000000001000000" pm="1"/>
Note
This this is very similar to read_parameter reply, except for:
  • the pm attribute, indicating that this is a monitored parameter
  • no atomized versions of the parameter is sent
  • only a hexdec value is sent
  • parameters that are known to the client (it was sent to the client in a long version already) produce a short reply (see sort attribute in read_parameter.

Event: crit_error, error, warn, info, reset

A message is generated by the process when some event occurs. They have the following categories as tag name:

<reset/> is a special event element signalling that the event has been cancelled.

Attributes are:

See also
pdserv_event()

Example:

<warn name="/Limit" index="1" seq="7" prio="4" time="1565689720.991359"
text="Event message 2"/>
<reset name="/Limit" index="1" seq="8" time="1565689724.251307"/>