What Problems Do Blockchain Sharding Need to Solve?

Challenges of Public Chain

Horizontal Expansion and Vertical Expansion

In the early days, many public chains use vertical expansion as an attempt to overcome the challenge. For example, adjusting block intervals or its block size, all of which is to increase the functionality of a single chain by adjusting different parameters. This is similar to expansion for centralized system whereby one would need to use the fastest CPU or a bigger hard disk. Vertical expansion has one Achilles heel however, which is limited increase in functionality because there are only that much wiggle room within which one can play with the parameters. One instance for illustration is the limitation in the speed of global P2P web broadcasting. For blockchain, increasing or decreasing the interval between blocks or adjusting the block size will heighten the possibility in producing orphan blocks and thus wasting hash power. Another similar method is to use PoS or DPoS, such as the DPoS used by EOS, which allows only 21 nodes to produce blocks and hurt the level of decentralization of the network.

Types of Shards

Sharding is one of the solutions allows expansion on the chain. Different kinds of blockchain sharding can be ranked according to level of functionalities. Here is the rank starting with the weakest to the strongest: network sharding, transaction sharding, computational sharding, and state sharding.

Network Sharding

Network sharding is the most basic kind of sharding, which is to divide the entire blockchain network into several subnetworks of which each constitutes one shard. All shards within the network process different transactions in parallel. Developers need to invent a mechanism to confirm which node can be kept on which shard securely, so to avoid attacks from those who are in the procession of a majority of shards.

Transaction Sharding

Transaction sharding is based upon network sharding and take one step further to classify all transactions and route to different shards for authentication and packaging. The problems ighag arise are for example what kind of characters and which kind of transactions should be mapped to which shards.

Computational Sharding

Only a subnetwork within a network executes the computation which can complete the task with high performance.

State Sharding

State sharding takes the entire ledger information to store in different shards, which means to divide up the state of the blockchain into multiple states, each of which can process rather independently and also to interact with other shards seamlessly.

Comparison of different types of shardings

Network sharding is the foundation that transaction, computational, and state sharding build upon. Network sharding and transaction sharding can solve the bottleneck for computational power while state sharding can solve the bottleneck for bandwidth and storage capacity. Additional, state sharding only lowers the number of nodes required for authentication, therefore state sharding the most ideal format for sharding.

Challenges Encountered By Blockchain Sharding

The most pressing problem at hand is to maintain the atomicity while running a cross-shard operation. For example, different processes on different shards need to be run according to a certain timeline like to confirm the time sequence between operations, which is indeed a daunting task.

Comparisons of projects related to blockchain sharding

At present, there are a number of projects that implement sharding technology. We have made a comparison table to differentiate sharding/cross-chain public chain platforms.

Boson Consensus

Consensus mechanism is a basic component of blockchain. After introducing sharding to blockchain, consensus mechanism is adapting to such design correspondingly. Here we will introduce an important sharding consensus mechanism, the Boson Consensus. The Boson Consensus was first proposed by QuarkChain as a framework of consensus mechanism specifically catering for sharding. It can be applied to QuarkChain, Polkadot, and even ETH2.0 (still in the process of design). The Boson Consensus has been operating smoothly in the QuarkChain mainnet production environment.

  1. High compatibility, supporting customization of shard chains
  2. Support cross-chain, cross-shard transactions
  3. High level of security for all shards. When a shard is added to the network, all shards are protected by the root chain, which allows each shard to enjoy high level of security immediately
  4. Dynamic addition of shards

The Advantages Of Developing Blockchain Using Go

Blockchain requires high-performance operations for computing hashing and encryption algorithms, processing large amount of transaction data, broadcasting, and storing data. Blockchain has high level of coding complexity and requires network programming and multi-platform support. Blockchain also demands a high level of maintainability and guarantee for its security. Using Go for development satisfies most of the technical requirements that we just mentioned above.

Implementation of blockchain sharding using Go

There are several sharding projects that use Go for development. Here we will use the Go version of Quarkchain as an example to illustrate how QuarkChain implements sharding with this language. Here is a short bio for QuarkChain:

Structure of QuarkChain

QuarkChain borrows the concept of cluster, which is often used in solutions for centralized large-scale distributed systems. Each node is one cluster. Each cluster has a master and one or multiple slave. Master includes root chain and provides Json-PRC interface and P2P interface for inter-node communications. Slave includes one or more shards. Between master and slave(s), and between slaves, communications are transferred using the GRPC interface.

Inter-Shard Communication

Different nodes in blockchain communicate using P2P protocol to reach consensus. Within QuarkChain, shards from different nodes would not establish P2P connection directly because as the connection of existing P2P protocol has not reached bottleneck yet, doing so would introduce a higher level of complexity to the code. Currently, the communication between shards are routed through root chain P2P layer. Here is the structure of root chain P2P message:

Support For Heterogeneous Shards

For blockchain, the most important objects are transaction, header, block, and block chain. For heterogeneous shards, this means that the definition of these objects and its processing logic would be different for each shard. At the same time, blockchain has standard processing flow and thus we have created corresponding interface for each of these objects.

Cross-shard Transactions

QuarkChain has designed for each user to have an address on each shard. Therefore the flow for cross-shard transactions is divided into two simple steps.

  1. Extracting deposit: this function extracts tokens from User A’s deposit enough for transferring amount and transaction fee expected (calculated by multiplying gas price with the price of start gas. This process takes place within shard X.
  2. Execution and return funds: For extracting deposit, the process will first increase the deposit in user B. Or for visiting the smart contract signed and execute the code for smart contract C. After finishing transferring of token or deployment of a smart contract, the system will calculate the actual transaction cost and return any leftover tokens to user A’s address on shard Y. Such process will take place on shard Y.

Introduction of the Competition:

During last year’s competition, the winning team reached throughput that surpassed 55000 TPS. We have recently open-sourced the Go version of QuarkChain which has even better functionalities. We would like to test the functionality of the Go version through hosting this TPS competition with 3 BTC reward. It is easy to join the competition. You will only need 5 minutes to initialize the node. During the competition period, you can submit as many submissions as you would like and we will rank according to your best submission. First place for the TPS Peak Challenge will receive 0.5 BTC and 1 million QKC. The first to reach 100000+ TPS will receive 0.1BTC + 0.2 million QKC. There are other awards including participation bonus and referral bonus and we welcome all of you to join!




Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store