simplify block production

[anshalshukla]

🗒️ Description

Simplies block production. Uses a more efficient and simpler approach to build blocks.

• Earlier we were calculating attestations from the aggregated_payloads, these attestations were applied on the state until no more attestations can be applied
• With the change in signature and payload maps we don't need to fetch attestations and can directly look at the attestation_data based on the current_justified checkpoint, select proofs greedily and apply them as long as the current_justified checkpoint continues to update

Backwards compatible as it doesn't change anything in state or stf

🔗 Related Issues or PRs

✅ Checklist

• Ran tox checks to avoid unnecessary CI fails:

  uvx tox

• Considered adding appropriate tests for the changes.
• Considered updating the online docs in the ./docs/ directory.

[tcoratger]

Here it seems super complex to follow no? With 4 levels of nesting in total, couldn't we simplify? I'm afraid that this is cause a lot of trouble to implement by clients?

  while True:                                  # fixed-point
      for att_data, proofs in sorted(...):     # iterate attestation data
          while True:                          # greedy selection
              for proof in proofs:             # scan candidates

[anshalshukla]

I'll look into this as well later today

[anshalshukla]

This issue was also addressed through _extend_proofs_greedily. Strictly speaking, this still makes the process four loops, since _extend_proofs_greedily itself contains two loops. Previously, the flow had three loops: fixed point, selection, and scanning.

Greedy selection introduces one additional loop, although we can remove it if client teams find it too burdensome to implement.

Looking ahead, once recursive aggregation is available, I plan to aggregate all proofs for a given message into a single proof during block production, even when the validator is not an aggregator. However, if this is done without care, proposers may miss the production interval. To prevent that, we would need an upper bound on the number of signatures/proofs aggregated within a production interval.

Under such a bound, greedy selection appears to be the best strategy for minimizing the number of proofs. My current understanding of LeanVM suggests that recursively aggregating 3 signatures may perform significantly better than aggregating 5, though this remains a hypothesis. It is possible that these gains would be offset by the cost of the extra loop. For that reason, we may choose not to require greedy selection in the spec, and instead allow client teams to implement whichever approach they prefer, anyway the spec should prioritize readability over optimization.