Implementing protocols is the main work when one wants to support new IOBoards models that involves technologies not already supported : either new OSs or new boards famillies.

A first simple example

Project description

When one wants to support new Protocols for instance for a non supported OS or board familly, it first has to create a new package that will provide a library containing protocols definitions. Let’s take as example the hardio-linux package.

It’s root CMakeLists.txt looks like:

cmake_minimum_required(VERSION 3.19.8)
...
project(hardio-linux)

PID_Package(
    ...
)

PID_Dependency(hardio-core VERSION 2.0)
...

PID_Publishing(PROJECT https://gite.lirmm.fr/hardio/boards/hardio-linux
    FRAMEWORK hardio
    CATEGORIES protocol
    ...
)

build_PID_Package()

It uses the hardiocore package as dependency for getting possible protocols definitions. Also we declare it as a member of the hardio framework, contributing to the protocol category.

We declare the library linux in src/CMakeLists.txt:

PID_Component(linux
  EXPORT hardio/core
  DEPEND posix
  C_STANDARD 11
  CXX_STANDARD 17
)

And we define the root public header include/linux/hardio/linux.h whose main role is to contains the include directive for each protocol implementation headers of the package and also to the hardio/core.h the library exports:

#pragma once
#include <hardio/core.h>
#include <hardio/impl/posix_serial_protocol.h>
#include <hardio/impl/posix_ip_protocol.h>
#include <hardio/impl/posix_udp_protocol.h>
#include <hardio/impl/posix_tcp_protocol.h>
#include <hardio/impl/linux_network_interfaces.h>
...

By convention all protocol implementation headers are put into the hardio/impl folder. In a first time we take as example the posix_serial_protocol.h file that contains an implementation of serial protocol using posix API.

Declaration

Here is the example code for serial protocol implementation:

#pragma once
#include <hardio/core.h>

namespace hardio {

class PosixSerialProtocol : public SerialProtocol {
public:
    PosixSerialProtocol() = delete;
    PosixSerialProtocol(const PosixSerialProtocol&) = delete;

    PosixSerialProtocol(size_t baudrate);

    virtual ~PosixSerialProtocol();

    void flush() final;

    bool write(size_t length, const uint8_t* data) final;

    bool read(size_t length, uint8_t* data, size_t& read_data) final;

    bool can_bind_to(const std::shared_ptr<Interface>& interf,
                     std::string& reason) const final;

    bool custom_initialization(std::string& reason,
                               bool managed_thread = true) final;

private:
    int uart_;
};
}

The PosixSerialProtocol inherits from the abstract class SerialProtocol defined in hardiocore, that is why the #include <hardio/core.h> in necessary. This simply tells it is an implementation of serial protocol. It defines a uart_ attribute that is a posix file descriptor representing the serial connection.

It overrides (using final keyword) flush/read/write pure virtual methods defined in SerialProtocol. These methods are those that will really be used through SerialInterface to put in place communciation with a device.

In addition it overrides can_bind_to and custom_initialization methods. Those two methods are automatically used by the hardio subsystem (respectively during description and initialization phases), in order to check validity of the description and configure communication using the protocol.

Implementation

We now can see details on these methods implementation.

The constructor is really simple, it simply forwards arguments to the SerialProtocol construtor:

PosixSerialProtocol::PosixSerialProtocol(size_t baudrate)
    : SerialProtocol(baudrate) {
}

The can_bind_to method is used to check that the interface with which the protocol is bound is of adequate type:

bool PosixSerialProtocol::can_bind_to(const std::shared_ptr<Interface>& other,
                                      std::string& reason) const {
    if (other->type_id() == pid::type_id<SerialBus>() or
        other->type_id() == pid::type_id<USBProtocol>()) {
        // linux implementation of serial protocol is identical for real
        // RS232 or USB connections
        return true;
    }
    reason = "serial protocol can only bind to SerialBus or USB Interface";
    return false;
}

other represents the interface we are trying to bind with PosixSerialProtocol, and reason is the error message returned if something went wrong. The check is quite simple: the underlying interface must be a USBInterface or directly a SerialBus IO, otherwise PosixSerialProtocol cannot be used.

Implementation of flush/read/write methods is presented:

#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <termios.h>
#include <unistd.h>

...

void PosixSerialProtocol::flush() {
    tcflush(uart_, TCIFLUSH);
}

bool PosixSerialProtocol::write(size_t length, const uint8_t* data) {
    if (uart_ < 0) {
        return false;
    }
    flush();
    ssize_t len = ::write(uart_, (uint8_t*)(data), length);
    return (len != -1 and size_t(len) == length);
}

bool PosixSerialProtocol::read(size_t length, uint8_t* data,
                               size_t& read_data) {
    if (uart_ < 0) {
        return false;
    }
    auto len = ::read(uart_, (uint8_t*)(data), length);

    if (len > -1) {
        read_data = len;
        return true;
    } else {
        read_data = 0;
        return false;
    }
}

This is just wrapper code around the standard posix API. One thing to remind is that read()/write() functions must return false if an error occurred.

Finally the big part of serial communication implemnetation is performed in the custom_initialize() function. This function is automatically called when the initialize() function of the main board is called.

Its code is:

bool PosixSerialProtocol::custom_initialization(
    std::string& reason, [[maybe_unused]] bool managed_thread) {
    struct termios paramport;
    // used is the interface bound to the serial protocol (either basically a
    // SerialBus OR a virtual device like a USB FTDI interface)
    std::string id;
    if (used_interface()->type_id() == pid::type_id<SerialBus>()) {
        id = std::static_pointer_cast<SerialBus>(used_interface())->id();
    } else { // USB Interface -> use the device ID provided by the USB interface
        id = std::static_pointer_cast<USBInterface>(used_interface())
                 ->device_id();
    }

    uart_ = open(id.c_str(), O_RDWR | O_NOCTTY | O_NDELAY);
    if (uart_ < 0) {
        reason = "cannot open device " + id;
        return false;
    }

    // get config port
    int result = tcgetattr(uart_, &paramport);
    if (result < 0) {
        reason = "cannot get port configuration tcgetattr error=" +
                 std::string(strerror(errno));
        return false;
    }
    // set options
    speed_t comm_rate = B0;

    switch (this->baudrate()) {
    case 300:
        comm_rate = B300;
        break;
    case 1200:
        comm_rate = B1200;
        break;
    case 2400:
        comm_rate = B2400;
        break;
    ...
    default:
        comm_rate = B9600;
        break;
    }

    if ((result = cfsetispeed(&paramport, comm_rate)) < 0) {
        reason = "Erroneous IN speed =" + std::string(strerror(errno));
        return false;
    }
    if ((result = cfsetospeed(&paramport, comm_rate)) < 0) {
        reason = "Erroneous OUT speed =" + std::string(strerror(errno));
        return false;
    }

    paramport.c_cflag &= ~CSIZE;
    paramport.c_cflag &= ~PARENB; // no parity
    paramport.c_cflag &= ~CSTOPB; // 1 stop bit
    paramport.c_cflag |= CS8;     // 8 bits

    paramport.c_cflag &= ~CRTSCTS; // no control

    // raw mode , no echo , no signal
    paramport.c_lflag &= ~(ICANON | ECHO | ISIG);

    // input mode : no xon/xoff, no conversion CR/LF
    paramport.c_iflag &= ~(IXOFF | IXON | IGNCR | ICRNL);

    // outpu mode : no transformation or delay
    paramport.c_oflag = 0;

    // characters reception policy
    // no waiting , immdiat return with read parameters number
    paramport.c_cc[VMIN] = 0;
    paramport.c_cc[VTIME] = 0;

    paramport.c_cc[VKILL] = _POSIX_VDISABLE;

    // write config
    result = tcsetattr(uart_, TCSAFLUSH, &paramport);
    if (result < 0) {
        reason = "port configuration failed = " + std::string(strerror(errno));
        return false;
    }
    return true;
}

reason argument is filled whenever an error occurrred. managed_thread is not useful here because the implementation does not require thread creation.

First part consists in getting the device ID used to identify the serial connection, that is different whether the serial protocol is defined from a SerialBus or from a USBInterface:

if (used_interface()->type_id() == pid::type_id<SerialBus>()) {
    id = std::static_pointer_cast<SerialBus>(used_interface())->id();
} else { // USB Interface -> use the device ID provided by the USB interface
    id = std::static_pointer_cast<USBInterface>(used_interface())
                ->device_id();
}

Then:

uart_ = open(id.c_str(), O_RDWR | O_NOCTTY | O_NDELAY);

Create the serial communication.

Finally last part configures the baudrate for the connection:

tcsetattr(uart_, TCSAFLUSH, &paramport);

Once this custom_intialization is done you can use the serial interface.

Protocol providing multiple interfaces

A lot of protocols are more complex than serial protocol, basically because they define an additional addressing process that allows to provide multiple interfaces. This is the case for UDP protocol for instance.

Declaration

Here is the example code for UDP protocol implementation found in include/hardio/impl/posix_udp_protocol.h:

#pragma once
#include <hardio/protocols/udp.h>

namespace hardio {

class PosixUDPEndpoint : public UDPEndpoint {
public:
    PosixUDPEndpoint(const PosixUDPEndpoint&) = delete;

    PosixUDPEndpoint(size_t port = -1);
    virtual ~PosixUDPEndpoint();
    
    std::vector<EndpointSpec> connected_endpoints() const final;

    bool memorize(std::string_view endpoint = "",
                  ssize_t remote_port = -1) final;

    bool write_to(std::string_view remote_ip, ssize_t remote_port,
                  size_t length, const uint8_t* data) final;

    bool read_from(std::string_view remote_ip, ssize_t remote_port,
                   size_t length, uint8_t* data, size_t& data_read) final;

    bool read_from_if(std::string_view remote_ip, ssize_t remote_port,
                      size_t length, uint8_t* data, size_t& data_read) final;

private:
    struct server_info {
        std::string ip_;
        struct sockaddr_in end_pt_addr_;
        int sock_;
    };
    std::map<uint64_t, std::map<ssize_t, server_info>> known_endpoints_;

    bool check_endpoint(std::string_view endpoint_ip, ssize_t endpoint_port,
                        const struct sockaddr_in& to_check,
                        const struct sockaddr_in& known) const;

    server_info* find_endpoint(std::string_view endpoint, ssize_t remote_port);
};
}

Logic is the same as previously: PosixUDPEndpoint inherits UDPEndpoint base class and specializes its pure virtual methods. In this case we do not need to override can_bind_to because the base class already define it with adequate constraints checking. We do not either need to implement custom_initialization method because each OS level configuration will be made for each UDP interface and nothing needs to be done globally for the whole protocol management. Indeed anytime an interface is added to the protocol a new socket is created so socket creation (which is the the OS configruation action) is made at interface creation time.

The constructor PosixUDPEndpoint takes as paramater a port number with default value set to -1 which means “no local port specified”.

memorize() method is automatically called any time an UDPInterface object is provided from this protocol. We explain this in implementation section.

write_to, read_from and read_from_if are overrides of the pure virtual methods defined by the UDPEndpoint protocol class and those methods implements communication model of UDP.

Considering attributes we define known_endpoints_ that simply memorize information (related to posix sockets) of all UDP endpoints known by the protocol. Private methods are just utilities.

Compared to the previous example an important point is so to manage configuration information related to each interface created from this protocol. A simplified implementation is explained in next subsection.

Implementation

Constructor and destructor look like:

PosixUDPEndpoint::PosixUDPEndpoint(size_t port)
    : UDPEndpoint(port), known_endpoints_{} {
}

PosixUDPEndpoint::~PosixUDPEndpoint() {
    for (auto& end_pt : known_endpoints_) {
        for (auto& end_point_with_port : end_pt.second) {
            ::close(end_point_with_port.second.sock_);
        }
    }
}

The constructor simply forwards its argument to the UDPEndpoint base class while memorized endpoints set is let empty. The destructor simply ensures that each UDP endpoint socket is correctly close. Nothing really surprising for anyone who already did socket programming.

The implementation of communication methods is straightforward:

bool PosixUDPEndpoint::write_to(std::string_view endpoint_ip, ssize_t endpoint_port,
                                size_t length, const uint8_t* data) {
    auto info = find_endpoint(endpoint_ip, endpoint_port);
    if (info == nullptr) {
        return false;
    }
    return (::sendto(info->sock_, data, length, 0,
                     (struct sockaddr*)&info->end_pt_addr_,
                     sizeof(info->end_pt_addr_)) > -1);
}

bool PosixUDPEndpoint::read_from(std::string_view endpoint_ip,
                                 ssize_t endpoint_port, size_t length,
                                 uint8_t* data, size_t& data_read) {
    auto info = find_endpoint(endpoint_ip, endpoint_port);
    if (info == nullptr) {
        return false;
    }
    struct sockaddr_in serv;
    auto srvaddrlen = sizeof(serv);
    ssize_t result =
        ::recvfrom(info->sock_, data, length, 0, (struct sockaddr*)&serv,
                   (socklen_t*)(&srvaddrlen));
    if (result > -1 and
        check_endpoint(endpoint_ip, endpoint_port, serv, info->end_pt_addr_)) {
        data_read = result;
        return true;
    }
    data_read = 0;
    return false;
}

bool PosixUDPEndpoint::read_from_if(std::string_view endpoint_ip,
                                    ssize_t endpoint_port, size_t length,
                                    uint8_t* data, size_t& data_read) {
    auto info = find_endpoint(endpoint_ip, endpoint_port);
    if (info == nullptr) {
        return false;
    }
    struct sockaddr_in serv;
    auto srvaddrlen = sizeof(serv);
    ssize_t result =
        ::recvfrom(info->sock_, data, length,
                   MSG_DONTWAIT, // non blocking read
                   (struct sockaddr*)&serv, (socklen_t*)(&srvaddrlen));

    data_read = 0;
    if (result > -1) {
        if (check_endpoint(endpoint_ip, endpoint_port, serv,
                           info->end_pt_addr_)) {
            data_read = result;
        }
        return true;
    } else {
        return (errno == EAGAIN or errno == EWOULDBLOCK);
    }
}

Simply suppose that the find_endpoint function get the correct endpoint memorized in known_endpoints_ depending on the given remote_ip and remote_port. Once done we have the socket information so we just need to use the correct posix functions to read from (either in blocking or non blocking mode) or write to the socket. Please notice that the read_from_if should return true even if no message has been received, simply because it is a normal behavior and not an error.

To understand how this works it is simpler to have a look at the UDPInterface class:

UDPInterface::UDPInterface(std::string_view remote_ip, ssize_t remote_port)
    : ProtocolInterface<UDPEndpoint>(),
      remote_ip_{remote_ip},
      remote_port_{remote_port} {
}

bool UDPInterface::write(size_t length, const uint8_t* data) {
    return protocol()->write_to(remote_endpoint_ip(), remote_port(), length,
                                data);
}

bool UDPInterface::read(size_t length, uint8_t* data, size_t& data_read) {
    return protocol()->read_from(remote_endpoint_ip(), remote_port(), length,
                                 data, data_read);
}

bool UDPInterface::read_if(size_t length, uint8_t* data, size_t& data_read) {
    return protocol()->read_from_if(remote_endpoint_ip(), remote_port(), length,
                                    data, data_read);
}

Anytime an interface is created for any implementation of the UDPEndpoint protocol, an instance of the UDPInterface class is created and memorizes the remote_ip and the remote_port declared for this interface. The write, read, read_if functions simply forward the call to the corresponding UDPEndpoint protocol implementation with correct addressing information (IP and port for the interface). This way end users can use any type of the (correct) protocol implementation (here PosixUDPEndpoint) with the exact same API.

So in the end most of the complexity of the code lies in the memorize() function that is in charge of creating sockets for each interface. First of all let’s have a look at the source code of UDPEndpoint class:

bool UDPEndpoint::manage_add_interface(
    const std::shared_ptr<ProtocolInterface<UDPEndpoint>>& dev,
    std::string& reason) {
    auto specialized_obj = std::static_pointer_cast<UDPInterface>(dev);
    if (this->port() != -1) { // if a local port is specified
        if (not this->provided_interfaces().empty()) {
            // only one provided interface is possible because no more than 1
            // interface can listen on the given port
            reason =
                "UDP layer on top of IP interface is defined with a "
                "listening port (" +
                std::to_string(this->port()) +
                ") and another UDP endpoint is already using this IP interface";
            return false;
        }
    }

    // check that same connection is not already established
    auto epts = connected_endpoints();
    for (const auto& ip : epts) {
        if (this->port() == -1 and specialized_obj->remote_port() == -1) {
            reason = "UDP endpoint does not define a target port";
            // a port must be specified (either local, remote or both)
            return false;
        }

        if (ip.first == specialized_obj->remote_endpoint_ip() and
            specialized_obj->remote_port() == ip.second) {
            reason = "UDP endpoint target same IP (" +
                     specialized_obj->remote_endpoint_ip() + ") and port (" +
                     std::to_string(specialized_obj->remote_port()) +
                     ") as another endpoint";

            return false; // ip already used
        }
    }
    // now checking that the device can connect
    if (not memorize(specialized_obj->remote_endpoint_ip(),
                     specialized_obj->remote_port())) {
        reason = "failed to create socket for IP " +
                 specialized_obj->remote_endpoint_ip() + " and port " +
                 std::to_string(specialized_obj->remote_port());
        return false;
    }
    return true;
}

The principal function of this class is manage_add_interface().It is in charge of performing checks and required configuration anytime a new interface is added to the protocol (i.e. when add_interface() is called).

dev argument represents the new UDPInterface object to be added to the protocol (but not yet added). reason simply contains error description if any error occurred (e.g. check failed).

First part of the function performs checks related to new interface information, we do not enter into details on that point but the code is mostly self explanatory. The second part calls memorize() function of the protocol implementation (using dynamic polymorphism), that is the one of PosixUDPEndpoint in the current tutorial context. A way to generally explain the manage_add_interface is that it describes generic rules any UDPEndpoint protocol implementation has to follow and delegate the specific rules for specific implementations to memorize() function.

So basically in the end memorize() has to do the real configuration for trully putting in place the UDP communication, that is, in this tutorial context, creating the posix sockets and memorizing them for each interface. This is what is done in the following code:

bool PosixUDPEndpoint::memorize(std::string_view remote_ip,
                                ssize_t remote_port) {
    // IP interface in use must define an IP !!!
    if (this->ip() == "") {
        return false;
    }
    /* gethostbyname: get the server's DNS entry */
    std::string end_pt{remote_ip};
    if (end_pt == "") {
        struct hostent* server_info = ::gethostbyname(end_pt.c_str());
        if (server_info == nullptr) {
            return false;
        }
    }

    /* socket: create the socket */
    auto sock = ::socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
    if (sock < 0) {
        return false;
    }
    if (port() != -1) {
        // if a local port is defined we need to bind it !!
        // NOTE: we force use of this port with the given IP address
        struct sockaddr_in local_addr;
        local_addr.sin_family = AF_INET; // IPv4
        local_addr.sin_addr.s_addr = inet_addr(this->ip().c_str());
        local_addr.sin_port = htons(this->port());

        if (::bind(sock, (struct sockaddr*)&local_addr, sizeof(local_addr)) ==
            -1) {
            ::close(sock);
            return false;
        }
    } else {
        if (remote_ip == "" or remote_port == -1) {
            return false;
        }
    }
    auto id = pid::hashed_string(remote_ip);
    known_endpoints_[id][remote_port] = {end_pt, sockaddr_in{}, sock};
    /* build the server's Internet address */
    auto& end_pt_addr = known_endpoints_[id][remote_port].end_pt_addr_;
    ::memset(&end_pt_addr, 0, sizeof(end_pt_addr));
    end_pt_addr.sin_family = AF_INET;
    if (end_pt != "") {
        end_pt_addr.sin_addr.s_addr = inet_addr(end_pt.c_str());
    }
    if (remote_port != -1) {
        end_pt_addr.sin_port = htons(remote_port);
    }
    return true;
}

The code basically follows the classic posix sockets creation process. Here we want to create a UDP socket so we call:

auto sock = ::socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);

Then the socket is bound to a port ONLY if a local port has been defined on the protocol.

Finally the socket is memorized in the known_endpoints_ attributes and the remote endpoint IP and port are translated into a data compatible with the sockaddr_in format. Socket and remote information will then be used by the read_from()/read_from_if()/write_to() functions whenever communication with the endpoint is needed.

Conclusion

You now know the basics of providing new implementation for existing protocols. Of course such a tutorial cannot conver all the subtle difference among all protocols, so the best way is to check hardiocore source code and see what is needed for each protocol. To do that have a look at the adequate protocol definition source file locate in src/core/protocols and src/core/interfaces folders.

Also this tutorial does not cover the possibility of creating completely new protocols. This is the purpose of this tutorial.