muRPC: JSON-RPC Protocol Implementation in C++

Introduction

There is a JSON-RPC protocol specification JSON-RPC Specification. During my search, I couldn't quickly find an implementation of this protocol for C/C++ languages, and since there wasn't one, I had to write it myself. This article describes the muRPC library for creating a server and client for the JSON-RPC protocol. The implementation includes examples and unit tests to verify functionality and performance. The typical mode of operation assumes that one of the JSON-RPC clients provides some methods and notifies the server about them. Then other JSON-RPC clients can call these methods and receive responses. The server provides routing and validation of messages between clients. As an example of operation, the examples contain code for a client that performs computations on CUDA cores and returns the result to whoever requested it. The application area is any systems where messaging is required, conveniently connected with calling software functions. The server turned out to be high-performance and scalable across CPU cores almost linearly.

Project code is available at: https://github.com/skbuff-ru/muRPC

About the JSON-RPC Protocol

JSON-RPC is a Remote Procedure Call (RPC) protocol that uses JSON (JavaScript Object Notation) as the data exchange format. The protocol allows a client to call methods on a server as if they were executing locally, but uses JSON for data serialization and, for example, TCP as the transport protocol.

Protocol Standard

JSON-RPC 2.0 is the official protocol standard published in 2010. It defines the message format, structure of requests, responses, and notifications. The main elements of the protocol are:

jsonrpc: Protocol version (should be "2.0")
method: Name of the called method
params: Call parameters (array or object)
id: Request identifier (for matching responses with requests)
result: Result of successful call
error: Error description in case of failed call

Example method call:

{
  "jsonrpc": "2.0",
  "method": "subtract",
  "params": [42, 23],
  "id": 1
}

Example response:

{
  "jsonrpc": "2.0",
  "result": 19,
  "id": 1
}

The complete protocol specification is provided at https://www.jsonrpc.org/specification

Mapping JSON-RPC Calls to C-like Functions

JSON-RPC calls are mapped to C-like functions as follows: parameters passed in the JSON object are converted to arguments of the target function, and the function execution result is wrapped in a JSON response. This scheme allows easy integration of existing C/C++ functions into the RPC interface.

Call from the example above:

{"jsonrpc": "2.0", "method": "subtract", "params": [42, 23], "id": 1}

Calls the function named subtract with parameters 42 and 23. The result returned is:

int subtract(int a, int b)
{
    return a - b;
}

muRPC Architecture

muRPC is a lightweight RPC library in C++ that provides JSON-RPC functionality. It provides developers with a simple way to create client-server applications using method calls over the network using the JSON format.

System Components

Server: Main component responsible for routing messages between clients
Client: Provides functionality for performing RPC calls to the server, method registration, listening and sending notifications.

Server Side

muRPC is a library that allows creating a server or client. The muRPC server is implemented to support a large number of simultaneous connections. The server in muRPC acts as an intermediary, allowing clients to register their methods and call methods of other clients. It's important to note that the server typically routes messages between clients rather than providing its own methods. Although muRPC also supports creating methods on the server. This allows building decentralized systems where each client can provide its services to other clients through the server.

Key server components:

Connection manager
Registry of clients and their registered methods
System for routing requests between clients
Notification system (publish/subscribe)
Session and timeout management mechanisms
Maintaining statistics on connections and processed messages
Tracking client status and their registered methods
Logging all operations for debugging and monitoring
Providing centralized control over the system
Parsing and validation of JSON messages

Client Side

The client side of muRPC provides an API for performing and providing remote calls. It supports both synchronous and asynchronous calls, registration of own methods, notifications and subscription to events from other clients.

Call Modes

muRPC supports two modes of performing calls:

Asynchronous calls - main operating mode:
- Uses the call() method in C++ and Python
- Call does not block program execution
- Result is handled through a callback function
- Suitable for high-performance applications
Synchronous calls - auxiliary mode:
- Uses the call_sync() method in C++ and Python
- Call blocks execution until result is received
- Suitable for simple scenarios and initialization

The client can:

Perform calls to methods of other clients (in both modes)
Provide its own methods for calling by other clients
Subscribe and unsubscribe from events (notifications)
Send notifications to other clients
Register its own methods for calling by other clients

Using cepoll in muRPC

I wrote the first version using boost::asio::read_until. Overall, it worked well, but it was necessary to support delimiters between JSON messages consisting of multiple characters, and few people understood how boost::asio works internally due to the extensive source code. There was an attempt to minimize boost::asio, but all the C++ header chain used took about 5 MB. And honestly, I wanted complete control over the low-level system. So it was decided to rewrite boost::asio::read_until in pure C and use it. Since the main language is C, the name cepoll seemed appropriate to me.

cepoll is a library that is part of this project and provides high-speed multithreaded I/O. This library implements the epoll system call approximately the same way as nginx does. At the same time, EPOLLEXCLUSIVE distribution with binding connections to handler threads (or workers in English) is used.

The cepoll library provides:

Multithreaded event processing
Support for message delimiters
Connection and timeout management
CPU affinity for performance optimization
Nginx-like architecture with master-workers
Simple implementation for better understanding and debugging

Nginx-like cepoll Architecture:

The cepoll library implements an architecture similar to Nginx, using master-workers:

Master process: Accepts new connections and distributes them among workers
Worker processes: Handle I/O operations using epoll
Load distribution: New clients are distributed among workers using round-robin algorithm
EPOLLEXCLUSIVE: Used for more even distribution of events among workers

Distribution of Information Across Threads:

When a new connection arrives:

Master process accepts the connection
Connection is assigned to one of the workers (by round-robin algorithm)
File descriptor is added to the epoll instance of the assigned worker
All subsequent operations with this connection are handled only by the assigned worker

Each worker has its own epoll instance and handles only its connections. This allows efficient distribution of load between multiple threads and utilization of all available CPU cores.

Support for Message Delimiters:

The cepoll library implements the ce_read_until function, which allows reading data until a specific delimiter, which is especially useful for protocols based on text messages, such as JSON-RPC. This function efficiently handles buffered data and searches for the specified delimiter, allowing correct separation of messages even when data is partially received.

Buffer Management and Message Processing:

The cepoll library manages buffers for each connection, allowing processing of large messages and multiple messages in one packet. When receiving data:

Data is placed in the connection buffer
Search for delimiter is performed in the buffer
When delimiter is found, the corresponding handler is called
Remaining data stays in the buffer for subsequent processing

Error Handling and Timeouts:

cepoll includes mechanisms for error and timeout handling:

Detection of connection breaks
Management of timeouts for inactive connections
Protection against denial-of-service attacks through message size limitations
Proper resource cleanup when closing connections

Examples of Using muRPC

Let's consider examples of using muRPC to create a system with multiple clients communicating through a server:

// Server example
#include <muRPC/server.hpp>

int main() {
    // Creating server with configuration
    std::string config = R"({
        "port": 8080,
        "thread_count": 4,
        "keepalive_timeout_ms": 5000,
        "max_message_size": 4096
    })";

    muRPC::server srv(config);

    // Server performs routing, not method provision
    // Methods are registered by clients

    return 0;
}

// Example client 1 - registers method
#include <muRPC/client.hpp>

int main() {
    std::string config = R"({
        "host": "localhost",
        "port": 8080
    })";

    muRPC::Client cli(config);

    // Registering client
    auto reg_result = cli.register_client("client1");
    if (!reg_result.contains("result")) {
        std::cout << "Failed to register client: " << muRPC::serialize_json(reg_result["error"]) << std::endl;
        return 1;
    }

    // Registering method on client
    cli.register_method("add", [](const muRPC::Json& params) {
        int a = static_cast<int>(params[0].get_number());
        int b = static_cast<int>(params[1].get_number());
        return a + b;
    });

    return 0;
}

// Example client 2 - calls method from another client
#include <muRPC/client.hpp>

int main() {
    std::string config = R"({
        "host": "localhost",
        "port": 8080
    })";

    muRPC::Client cli(config);

    // Registering client
    auto reg_result = cli.register_client("client2");
    if (!reg_result.contains("result")) {
        std::cout << "Failed to register client: " << muRPC::serialize_json(reg_result["error"]) << std::endl;
        return 1;
    }

    // Preparing parameters for call
    muRPC::Json params = muRPC::Json::array();
    params.set(0, static_cast<int64_t>(5));
    params.set(1, static_cast<int64_t>(3));

    // Calling method registered by another client
    auto result = cli.call_sync("add", params);
    std::cout << "Result: " << result.as<int>() << std::endl;

    return 0;
}

TCP Transport in muRPC

The current version of muRPC uses the TCP protocol as the transport layer. A single TCP connection is opened between the client and server for all communication.

The client sends requests to the server in the form of messages packaged in JSON format:

{"jsonrpc": "2.0", "method": "keepAlive" }\n{"jsonrpc": "2.0", "method": "subtract", "params": [42, 23], "id": 1}\n

And receives responses from the server also in JSON format:

{ "jsonrpc": "2.0", "result": 19, "id": 1 }

The TCP protocol provides reliable and ordered data delivery: bytes written to a socket on one side will be received on the remote side in the same order, without losses and duplication. However, TCP works with a stream of bytes, not individual "messages", so applications must separate logical messages themselves (for example, using length or delimiters). Provided that each message is written completely and sequentially (for example, from a single thread), its content will not mix with other messages. Due to TCP's guarantee of delivery order, the application does not need to handle packet reordering on its own. To match the response with the request, the id field in JSON format is used.

Interestingly, when sending messages smaller than the standard packet length (1500 bytes message + TCP headers), the message will be transmitted as a single network packet. Therefore, there's little point in compressing or using a binary protocol. If a large amount of information needs to be transmitted, it's worth compressing the payload specifically. On the plus side, this creates a convenient debugging tool - tcpdump can see the protocol requests and responses.

Timeouts in muRPC

muRPC provides flexible timeout configuration options for various usage scenarios. This allows adapting the system's behavior to specific performance and reliability requirements.

Types of Timeouts

muRPC implements the following types of timeouts:

Call timeout: Defines the maximum time to wait for a response to an RPC call. If no response is received within the specified time, the call is considered failed and a timeout error is generated.
Keepalive timeout: Defines the maximum idle time of a connection before it is automatically closed. This helps free resources occupied by inactive connections.
Registration timeout: Defines the maximum time within which a client must register on the server after establishing a connection. If the client does not register within this time, the connection may be closed.

Timeout Configuration

Timeouts can be configured on both the server and client sides:

Setting Timeouts on the Server

On the server, timeouts can be set in the configuration file or when creating a server instance:

std::string config = R"({
    "port": 8080,
    "thread_count": 4,
    "keepalive_timeout_ms": 5000,      // 5 seconds timeout for idle connections
    "registration_timeout_ms": 10000,  // 10 seconds for client registration
    "max_message_size": 4096
})";

muRPC::server srv(config);

Setting Timeouts on the Client

On the client, timeouts can be set in the configuration:

std::string config = R"({
    "host": "localhost",
    "port": 8080,
    "client_id": "client1",
    "call_timeout_ms": 30000           // 30 seconds timeout for calls
})";

muRPC::client cli(config);

Python Interface

In the muRPC project repository, there is a client Python interface. The Python interface is slower than C++, but implements all capabilities of the C++ version. The Python client is implemented in the murpc_client.py module and includes:

Support for synchronous and asynchronous calls
Connection management through a connection pool
Handling of timeouts and connection losses
Support for method registration, notifications and subscriptions

The Python client code was generated from C++ code, it is fully functional, but for real use it may require cleaning.

Example of using the Python client:

from murpc_client import MuRPCClient

# Client configuration
config = {
    "host": "127.0.0.1",
    "port": 8080,
    "client_id": "python_example_client",
    "call_timeout_ms": 10000
}

# Creating client
client = MuRPCClient(config)

# Registering method
def hello_handler(params):
    name = params.get("name", "World")
    return {"message": f"Hello, {name}!"}

client.register_method("hello", hello_handler)

# Preparing parameters for call
params = {"param": "value"}

# Asynchronous call
def callback(response):
    print(f"Response: {response}")

client.call("some_method", params, callback)

# Preparing parameters for call
params = {}

# Synchronous call
response = client.call_sync("get_server_stats", params)
print(f"Server stats: {response}")

muRPC Performance

To test muRPC performance, a utility perf_test was developed that measures key metrics: PS (requests per second) and channel throughput (message exchange speed). Here are the values when running on my laptop 13th Gen Intel(R) Core(TM) i9-13900H on 12 thread_count in the server config without CPU binding:

Maximum Performance

During testing with 10 connections and small payload (50 bytes), maximum performance was achieved:

RPS: 24,000 requests per second
Throughput: 33 Mbps (outgoing traffic)

When working with large payloads (102400 bytes), the system demonstrates high throughput:

RPS: 1,700 requests per second (with 50 connections)
Throughput: 1,400 Mbps (outgoing traffic)

The load across workers on the server scales almost horizontally and workers do not block each other.

The main server load is actually the RapidJSON library as seen on the perf trace:

  27,94%  libmuRPC.so          [.] void rapidjson::GenericReader<rapidjson::UTF8<char>, rapidjson::UTF8<char>,
                                   rapidjson::CrtAllocator>::ParseStringToStream<0u, rapidjson::UTF8<char>, rapidjson::UTF
  18,67%  libmuRPC.so          [.] rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<char>,
                                   rapidjson::CrtAllocator>, rapidjson::UTF8<char>, rapidjson::UTF8<char>, rapidjson::CrtAlloc
  12,53%  libmuRPC.so          [.] ce_read_until
   9,94%  libc.so.6            [.] __memmove_avx_unaligned_erms
   2,25%  [kernel]             [k] sync_regs
   1,71%  [kernel]             [k] _copy_to_iter
   1,41%  [kernel]             [k] native_irq_return_iret
   1,13%  [kernel]             [k] clear_page_erms
   0,71%  [kernel]             [k] __mod_memcg_lruvec_state
   0,63%  [kernel]             [k] __pte_offset_map_lock
   0,59%  [kernel]             [k] __count_memcg_events
   0,55%  libc.so.6            [.] __strlen_avx2
   0,49%  libc.so.6            [.] _int_malloc
   0,46%  [kernel]             [k] lru_gen_add_folio
   0,46%  [kernel]             [k] _copy_from_iter

Why RapidJSON?

The first experiment was with libjsoncpp. However, its parsing speed didn't seem high enough to me. Alternatives were nlohmann - since it's essentially a single include file or something else. On the nlohmann github page, I found an interesting link - "Speed. There are certainly faster JSON libraries out there." (Speed comparison of JSON libraries) for comparing the speed of different JSON libraries. To measure them on real tests, I decided to support different variants. The repository currently contains RapidJSON (json_adapter_rapidjson.cpp) and Nlohmann (json_adapter_nlohmann.cpp). The fastest turned out to be the combination of RapidJson and tcmalloc.

License

This muRPC and cepoll project is distributed under the MIT license, which allows free use, copying, modification and distribution of the software, provided that copyright notices and disclaimers of warranty are preserved.

Dependencies Licenses

nlohmann/json: Library for working with JSON, MIT license
RapidJSON: Alternative library for working with JSON, MIT license

There is also a commercial version of muRPC with extended capabilities. For additional information, contact info@skbuff.ru.

Conclusion

muRPC represents a powerful and flexible library for implementing RPC communication using JSON-RPC. The architectural feature is that the server routes messages between clients rather than providing its own methods. This allows building decentralized systems where each client can be both a consumer and a provider of services.

Combining asynchronous processing with cepoll makes muRPC particularly suitable for high-load applications with a large number of clients.

Possible Improvements and Future Development Directions

Future development prospects for muRPC include several key directions, each aimed at expanding functionality, increasing security and system performance.

One important direction is expanding support for transport protocols. Currently, muRPC uses TCP as the main transport, but support for other protocols such as HTTP and UDP is planned to be added. This will allow using the library in a wider range of applications, including those where low latency (UDP) or compatibility with web standards (HTTP) is important. Such flexibility in transport selection will make muRPC an even more universal solution for various usage scenarios.

Security remains a priority in the project's development plans. Currently, possibilities for implementing authentication and encryption mechanisms are being considered. There is already a test patch for supporting TLS connections using certificates, which will allow secure data transmission between clients and the server. This is especially important for applications processing sensitive information or operating in unreliable network environments.

Another important direction is expanding monitoring and diagnostic capabilities. Plans include adding more detailed performance metrics, profiling and debugging tools, and improving logging. This will help developers more easily identify and eliminate problems, as well as better understand system behavior in production.

Regarding performance, future plans include researching possibilities for further optimization. One promising area is using SIMD (Single Instruction, Multiple Data) to accelerate JSON parsing. This could significantly increase performance when processing large amounts of data, especially on modern processors supporting the relevant instructions.

All these improvements are aimed at making muRPC an even more reliable, secure and high-performance library for building distributed systems. The project continues to evolve as open source, and the community can contribute to its development. We invite all interested developers to participate in the project and work together to make it better.

Author: Maxim Uvarov