Sharding network topology

[Mikerah]

Last year, @jannikluhn posted a proposal for a Torus-shaped sharding network. This was before the deprecation of EIP1011. Since then, we have, for the time being, decided on using gossipsub for the pubsub routing algorithm. The underlying network topology for gossipsub is a strongly connected mesh (see this issue.). However, there has been little talk about what the sharding network topology will be or if it has been decided.

I want to bring this up because I think it'll be important for several analyses that will need to be done in the future with regards to how we can make gossipsub more efficient, validator privacy, peer discovery, etc.

[mhchia]

It's truly a critical topic to be discussed! From my understanding, gossipsub is based on the existing network topology, and its behavior does not affect the underlying topology. Therefore, I think we also need to take DHT(or any other peer discovery mechanism) into consideration. It affects the topology when performing discovery(e.g. bootstrapping when joining, or resolving a peer ID if some discovery scheme is used when only the peer ID is available).

Besides, how signatures are aggregated is probably a factor affecting the topology.

[Mikerah]

Would it make sense to start compiling a list of viable network topologies and then start simulations of each? Since the testnet is approaching soon, we definitely need to figure out these details sooner, rather than later.

[jannikluhn]

Unfortunately, we can't simply pick a network topology that we like. It emerges indirectly from the discovery mechanism, the peer selection rules, node lifetimes, potentially latencies and bandwidths, etc. So all we can do is design our protocols in such a way that should lead to a good topology.

In Eth1.0, peers are essentially selected randomly, with the exception of nodes that are "full" already and don't accept new peers. Intuitively, I'd expect a fully homogeneous topology to be perfect for gossipsub, so this seems already close to optimal.

[Mikerah]

@jannikluhn, So are you saying that the sharding topology should be a mesh as in gossipsub, as the fully homogeneous topology (I don't think a star topology would work. It has too many problems for our needs) ? This is very different from the Torus shaped topology that you suggested last year.

[fjl]

I think random connectivity per shard is sufficient for pubsub and syncing. It has the massive advantage of being unstructured and is a lot harder to attack than any structured topology.

[fjl]

To clarify: when I say 'random connectivity, I mean we should establish a system like eth 1.0 has, but per-shard. Any node interested in a certain shard should select random peers from the set of all nodes participating in that shard.