For Erlay to work efficiently, some degree of transaction fanout is necessary. This ensures that a sufficient number of peers receive the transaction quickly. The key reason for this is that fanout is more bandwidth-efficient if the receiving node has not already seen the transaction. However, if the node already knows the transaction, fanout becomes wasteful. Additionally, Erlay introduces some latency to the transaction exchange process.
The ideal approach would be to fanout until a transaction has reached a sufficiently large portion of the network, then switch to reconciliation. However, this is impractical because a node does not know how widely a transaction has propagated at the time of reception.
A more feasible strategy is to set a minimum fanout rate and adhere to it, allowing reconciliation to handle the rest. This ensures that a targeted percentage of the network receives the transaction through fanout, while reconciliation takes care of the rest.
A viable approach is to target a minimum fanout rate and stick to it while leaving the rest to reconciliation. This way a target percentage of the network will receive the transaction via fanout, and the rest will use reconciliation for it 1️⃣.
To determine reasonable Erlay configurations, we can begin by simulating how a transaction propagates across the network using the current protocol. This will establish a baseline in terms of both data usage and propagation time. Once we have this baseline, we can run the same simulation with different Erlay configurations, each targeting different fanout levels. By analyzing the relationship between bandwidth saving and latency we can fine-tune our parameters to achieve an acceptable configuration.
We will start by defining our baseline. For this, as in previous simulations, we are using a random network of 110000 nodes, 10000 reachable, 100000 unreachable. Given Erlay has the additional goal of allowing nodes to increase their peer count without incurring significant bandwidth costs, we will be simulating this for 8, 10, and 12 outbound connections per node (to do this, we can simply specify the --outbound option in the simulator).
cargo run --release -- -n 100 --erlay --seed 8482337119851113330 --outbound 8
cargo run --release -- -n 100 --erlay --seed 8482337119851113330 --outbound 10
cargo run --release -- -n 100 --erlay --seed 8482337119851113330 --outbound 12
We will compare the baseline with a wide variety of Erlay configurations for OUTBOUND_FANOUT_DESTINATIONS and INBOUND_FANOUT_DESTINATIONS_FRACTION. For outbounds, we will select values on the [0-8] range, whereas for inbounds, we will increase the count percentage-wise (in 10% increments) starting from 0.
Given the scope of this experiment, running all the necessary simulations would take considerable time. To streamline the process, we have added a multi-run script to the simulator. This script automates a series of simulations, testing different ranges for OUTBOUND_FANOUT_DESTINATIONS and INBOUND_FANOUT_DESTINATIONS_FRACTION. The results are then stored in a CSV file for further analysis.
We will be running instances of the script simultaneously (one for each --outbound configuration) and interleave the results afterward:
hyperion/multi-run.sh --outbounds=8 --seed=8482337119851113330 --out=result_8.csv
hyperion/multi-run.sh --outbounds=10 --seed=8482337119851113330 --out=result_10.csv
hyperion/multi-run.sh --outbounds=12 --seed=8482337119851113330 --out=result_12.csv
Since this is a wide-range experiment, the results are extensive and only meaningful when compared. Documenting every configuration in detail would make this report tedious to read. Instead, we will focus on three specific results to analyze how the charts change based on the selected number of peers. An interested reader can refer to https://docs.google.com/spreadsheets/d/1YGG3Lm6__xmKsd9HlmNuQOaw_Lg7rLwsBNcWYJsUsIw/edit?usp=sharing to view the complete set of results. The interleaved data is located in columns A-Z, while the corresponding charts can be found further to the right in the document.