GTNet

Work in Progress

The implementation of GTNet is not yet complete. This documentation describes the planned and partially implemented functionality.

GTNet (Grafioschtrader Network) is a decentralized peer-to-peer system that enables multiple Grafioschtrader instances to exchange financial data. Each installation can simultaneously act as a data provider and data consumer in the network.

Concept and Functionality

GTNet connects different Grafioschtrader instances with each other and enables the mutual exchange of price data. Each instance can decide for itself whether it provides data, receives data, or does both simultaneously. The exchange is automated via secure machine-to-machine communication (M2M).

Types of Data Exchange

GTNet supports three types of data for exchange:

  • Intraday Prices: Current day prices for securities and currency pairs with OHLCV data (Open, High, Low, Close, Volume).
  • Historical Prices: Daily closing prices for the past.
  • Security Metadata: Master data of securities such as ISIN, name, asset class, exchange information and connector settings. This enables the import of security configurations from other GTNet instances.

Connection Setup and Authentication

The connection between two GTNet instances is established via a handshake procedure. During the first contact, authentication tokens are exchanged, which are used for further secure communication. Each instance can decide whether it automatically accepts unknown servers or only allows predefined servers.

The tokens can be renewed later. Because the two sides do not adopt the new token at the same moment, the token being replaced stays valid for a limited time. If the answer to a renewal is lost on the way, the connection therefore does not break; the next attempt puts it right again on its own.

Approving the Data Exchange

A completed handshake only means that two instances know each other and are allowed to talk. It does not yet entitle either of them to price data. For intraday prices and historical prices a data request must also have been made for that kind of data and approved by the other side. Without that approval the instance being asked refuses the request, even when it offers the kind of data in principle.

Security metadata is exempt from this. It is queried individually when needed rather than agreed as an ongoing exchange; here your own setting of whether such requests are accepted is enough.

If a data request is refused, or an existing approval is withdrawn, the exchange ends in the same state on both sides. Both instances then show the kind of data as no longer agreed, and taking it up again requires a new data request.

Existing connections

Peers with which price data has already been exchanged keep their approval automatically when the version changes. Only those who so far had merely a handshake but never an exchange have to make a data request.

Messages and Status Notifications

GTNet uses a message-based communication protocol for various purposes:

  • Handshake Messages: For establishing and confirming connections between instances.
  • Status Messages: For communicating changes such as online/offline status, maintenance mode, or capacity utilization. When a peer signs off while shutting down, it is recorded as offline straight away; previously it stayed entered as online until the next send failed.
  • Data Requests: For requesting and approving the exchange of specific data types.

Who May Do What

GTNet concerns the whole instance rather than an individual tenant. Only a user with administrator rights may therefore change anything about it: creating, editing or deleting peers, sending and answering messages, deleting messages, and maintaining the rules for automatic answers. Every other logged-in user may read the GTNet views, that is the list of peers with their messages, the announced maintenance windows and the exchange log.

What a user is not allowed to do is not offered to them either: the corresponding menu entries and dialogs do not appear at all rather than being refused on save. The exchange options of an individual instrument under Exchange price data are the one exception. They belong to the instrument and follow its ordinary editing rights: an administrator and a «Privileged user» may change every instrument, every other user only the instruments they created themselves.

Two areas stay reserved for administrators even for reading. The Automatic Message reveals the terms on which this instance admits a peer, and the export of the GTNet data hands out the entire holdings at once. Admin messages marked «Admin only» likewise remain visible to administrators in every view, including where the messages of an individual peer are expanded.

A conversation marked «Admin only» stays that way for every message in it. A reply cannot widen it — neither one written here nor one that arrives from the peer, which has no way of knowing how the thread it answers is classified. The classification is decided by the message that started the conversation.

A message that is still waiting for an answer cannot be deleted, and neither can the peer it belongs to, until the answer has arrived or the request has been rejected. This holds for every kind of request, including the ones the instances exchange among themselves such as the token refresh and the synchronisation of the exchange settings.

Query Limits and Load Management

To protect servers from overload, GTNet knows two limits. The Daily query limit determines how many requests a single peer may make per UTC day; it works in both directions, because your own instance stops asking of its own accord once the allowance a peer has published is used up. The Query limit restricts, per exchange kind, how many instruments a single request may cover. Both are described under Query Limits. Additionally, an instance can be marked as “busy”, whereby only status messages are communicated.


Getting Started with GTNet

This section provides a step-by-step guide to set up your GT instance for participation in the GTNet network.

Prerequisites

To participate in GTNet, your own Grafioschtrader server must be accessible from outside via HTTPS. Ensure that your server is properly configured for machine-to-machine (M2M) communication. If you encounter issues with external connectivity, consult the Problem Solving Guide in the wiki.

Step 1: Enable GTNet in Global Settings

First, activate GTNet functionality via the Global Settings GTNet:

  • Set g.gnet.use to a value greater than 0 to enable GTNet
  • Optionally enable g.gnet.use.log to track data exchanges

Step 2: Register Your Own Instance

Before communicating with other participants, you must register your own server in the GTNet Setup view:

  1. Open the GTNet Setup view
  2. Create an entry for your own server with your domain URL
  3. After saving, the system stores your instance ID in the global setting g.gnet.my.entry.id
Tip

If you are migrating an existing database to a new server, you should export the GTNet data on the old server before migration and then import it on the new server. This preserves authentication tokens, peer connections, and message history. See Export and Import of GTNet Data for details.

Step 3: Add Remote Instances

Add one or more remote GTNet instances to establish communication:

  1. In the GTNet Setup view, add entries for remote servers
  2. Recommendation: Prefer PUSH_OPEN servers, as they maintain an active data pool and enable bidirectional exchange without requiring local instruments
  3. Configure the daily request limit and other settings as needed

Step 4: Establish Initial Connection

Request data exchange directly, which automatically handles the handshake process:

  1. Select the remote server entry
  2. Use the context menu to send a Request for Data Exchange message
  3. The system automatically performs the handshake and token exchange if this is the first contact
  4. Wait for the response (acceptance or rejection)
  5. Once accepted, the data exchange is activated and authentication tokens are stored for future communication

To avoid manually responding to every incoming request, configure automatic response rules in the Automatic Answer view:

  1. Define rules for different message types (e.g., first contact, data exchange requests)
  2. Set conditions using variables like time of day, daily request count, or domain patterns
  3. Specify whether to accept or reject requests automatically
  4. Configure waiting times after rejections to prevent repeated requests
Tip

Without automatic answer rules, every incoming request requires manual approval by the administrator. Setting up appropriate rules significantly reduces administrative overhead.

Step 6: Configure Securities for Exchange

Define which instruments should participate in data exchange:

  1. Open the Exchange Price Data view
  2. For each security or currency pair, configure the four exchange options:
    • Receive intraday prices
    • Receive historical prices
    • Send intraday prices
    • Send historical prices
  3. Save your changes

Where GTNet Is Used

GTNet integrates into various parts of the application, both automatically by the system and through user actions.

System-Triggered Usage

graph LR;
    A[Application Start] -->|Broadcast| B[Server Online Message]
    C[Application Shutdown] -->|Broadcast| D[Server Offline Message]
    E[Settings Changed] -->|Broadcast| F[Settings Updated Message]
  • Application Startup: When the GT server starts with GTNet enabled, an “online” status is automatically broadcast to all connected peers
  • Application Shutdown: During graceful shutdown, an “offline” status is broadcast to inform peers
  • Settings Changes: Configuration changes are automatically synchronized to connected instances

User-Triggered Usage

  • Watchlist Update: When updating a watchlist, instruments configured for GTNet exchange can receive intraday prices from connected peers. See Last Price Exchange for details.
  • Historical Data Loading: When loading historical price data, GTNet can fill gaps from peer instances. See Historical Price Exchange for details.
  • Security Creation: When creating new securities, the GTNet Security Search allows importing security metadata and connector settings from other GTNet instances.

Monitoring GTNet

GTNet provides several ways to monitor the status and activity of data exchanges.

Static Overview: Exchange Price Data

The Exchange Price Data view provides a static overview of which instruments are configured for exchange and which OPEN instances can supply data:

  • View all securities and currency pairs with their exchange settings
  • Expand rows to see supplier details (which peers can provide data for each instrument)
  • Useful for understanding the current configuration state

Dynamic View: Exchange Log

The Exchange Log view offers a dynamic, time-based view of actual exchange activity:

  • See statistics for both intraday and historical price exchanges
  • Track supplier and consumer statistics per connected instance
  • View aggregated data by day, week, month, and year
  • Requires logging to be enabled via g.gnet.use.log in Global Settings

Background Tasks

Several background tasks handle GTNet operations automatically. These can be monitored in the Task Data Change Monitor:

Task IDNameDescription
20Track online statusChecks the online/busy status of all configured peers once shortly after server start; Online requires a successful GTNet protocol response, and peers without a completed outbound handshake are set to “Unknown”. Peers that are out of service or inside an announced maintenance window are skipped. A single instance can be re-checked at any time via the “Check status now” context menu entry in the GTNet server overview.
22Aggregate exchange logsSummarizes exchange log entries into daily, weekly, monthly, and yearly periods
23Synchronize configurationsSyncs exchange configurations with connected peers and updates supplier details
24Broadcast settings changesSends settings updates to all connected peers when local settings change
25Deliver future messagesHandles scheduled messages like maintenance announcements and sets instances whose announced discontinuation date has been reached to “Out of service”

For detailed descriptions of each task, see Background Tasks.


Setup Pages

The detailed setup is done via the following subpages: