Skip to content

Conversation

@jofrev
Copy link
Contributor

@jofrev jofrev commented Sep 23, 2025

Description

Making use of the read/write multiple block commands of the SD/MMC has the potential of speeding up SD card access significantly.
On ESP32 and ESP32-S3 and an SD card in SDC mode speed increased from 200kB/s to >2MB/s with 8kiB of additional buffer.
In general, it should speed up access significantly for all ESP devices without a PSRAM capable SDC peripheral when reading/writing data to/from PSRAM.

This PR actually uses these multi-block read/write commands by calling the sdmmc_read/write_sectors_dma functions from their respective sdmmc_read/write_sectors counterpart with a block_count different from 1. (Before sdmmc_read/write_sectors would call sdmmc_read/write_sectors_dma repeatedly with a block_count of 1 regardless of the amount of data to be read/written.)

To limit the temporary buffer size for large transfers, the data is transferred in chunks whose maximum size in number of blocks is defined by the MAX_NUM_BLOCKS_PER_MULTI_BLOCK_RW macro.
Setting this macro to 1 should result in the same behavior as before, while a value of 16 for example results in a size of 16 * 512B = 8192B for the temporary buffer improving the speed by one order of magnitude.

It may be nice to allow users to change the value of this macro possibly leaving the default at 1 to not change the behavior except if explicitly configured by the user since temporary buffer size and transmission speed are a trade off "game".
I'm open for suggestions how to enable them to do that. I would have added a Kconfig option, but there is no other Kconfig option present for this component as of v5.5.1 only on master.

Related

Testing

Tested on two custom boards with a ESP32-WROVER-E/ESP32-S3 and ESP-IDF v5.5.1 and an SD card in SDC mode.

Name:  SC32G
Type:  SDHC
Speed: 20.00 MHz (limit: 20.00 MHz)
Size:  30436MB
CSD:   ver=2, sector_size=512, capacity=62333952 read_bl_len=9
SSR:   bus_width=4
SSR:   alloc_unit_kb=4096

Checklist

Before submitting a Pull Request, please ensure the following:

  • 🚨 This PR does not introduce breaking changes.
  • All CI checks (GH Actions) pass.
  • Documentation is updated as needed.
  • Tests are updated or added as necessary.
  • Code is well-commented, especially in complex areas.
  • Git history is clean — commits are squashed to the minimum necessary.

Note

Adds multi-block SDMMC read/write with chunking controlled by a new Kconfig option, using a DMA-aligned buffer when available and improving error/status handling.

  • SDMMC Driver:
    • Multi-block transfers: sdmmc_read_sectors/sdmmc_write_sectors now chunk unaligned transfers and call *_sectors_dma with block_count > 1.
    • Buffering: Prefer host.dma_aligned_buffer; otherwise allocate DMA-capable temp buffer sized by actual allocation and sector size; validate and free as needed.
    • Error/status handling: Enhanced logs; on multi-block write failure, wait for idle and query number of successfully written blocks; consistent busy/status checks.
  • Config:
    • Add Kconfig option CONFIG_SD_UNALIGNED_MULTI_BLOCK_RW_MAX_CHUNK_SIZE to bound per-chunk blocks (controls single vs. multi-block ops).
  • API/Docs:
    • Document sdmmc_host_t.dma_aligned_buffer use cases (multi-block RW and SDIO IO) and sizing guidance.
  • Misc:
    • Include <sys/param.h> for MIN/MAX.

Written by Cursor Bugbot for commit 21df91a. This will update automatically on new commits. Configure here.

@github-actions
Copy link

github-actions bot commented Sep 23, 2025

Messages
📖 🎉 Good Job! All checks are passing!

👋 Hello jofrev, we appreciate your contribution to this project!


📘 Please review the project's Contributions Guide for key guidelines on code, documentation, testing, and more.

🖊️ Please also make sure you have read and signed the Contributor License Agreement for this project.

Click to see more instructions ...


This automated output is generated by the PR linter DangerJS, which checks if your Pull Request meets the project's requirements and helps you fix potential issues.

DangerJS is triggered with each push event to a Pull Request and modify the contents of this comment.

Please consider the following:
- Danger mainly focuses on the PR structure and formatting and can't understand the meaning behind your code or changes.
- Danger is not a substitute for human code reviews; it's still important to request a code review from your colleagues.
- To manually retry these Danger checks, please navigate to the Actions tab and re-run last Danger workflow.

Review and merge process you can expect ...


We do welcome contributions in the form of bug reports, feature requests and pull requests via this public GitHub repository.

This GitHub project is public mirror of our internal git repository

1. An internal issue has been created for the PR, we assign it to the relevant engineer.
2. They review the PR and either approve it or ask you for changes or clarifications.
3. Once the GitHub PR is approved, we synchronize it into our internal git repository.
4. In the internal git repository we do the final review, collect approvals from core owners and make sure all the automated tests are passing.
- At this point we may do some adjustments to the proposed change, or extend it by adding tests or documentation.
5. If the change is approved and passes the tests it is merged into the default branch.
5. On next sync from the internal git repository merged change will appear in this public GitHub repository.

Generated by 🚫 dangerJS against 21df91a

@jofrev jofrev force-pushed the feat/sdmmc-multi-block-rw branch from aa3b708 to dddcc57 Compare September 23, 2025 15:55
@CLAassistant
Copy link

CLAassistant commented Sep 23, 2025

CLA assistant check
All committers have signed the CLA.

@github-actions github-actions bot changed the title feat(sdmmc): support multi-block read/writes feat(sdmmc): support multi-block read/writes (IDFGH-16505) Sep 23, 2025
@espressif-bot espressif-bot added the Status: Opened Issue is new label Sep 23, 2025
@jofrev jofrev force-pushed the feat/sdmmc-multi-block-rw branch from dddcc57 to 8a6bb5b Compare September 24, 2025 15:38
Copy link
Member

@igrr igrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jofrev Regarding the question on setting the buffer size:

Perhaps the best option would be to allow reusing the existing sdmmc_host_t::dma_aligned_buffer which is currently only used for SDIO. That would have two advantages:

  • Callers can configure the size of the scratch buffer by simply allocating it with the size they need.
  • The buffer is reused between reads or writes.

However that would likely be a much larger change than your PR currently is. I think adding a Kconfig option would be sufficient to accept this PR.

#include "esp_private/sdmmc_common.h"

// the maximum size in blocks of the chunks a SDMMC write/read will be split into
#define MAX_NUM_BLOCKS_PER_MULTI_BLOCK_RW (16u)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's introduce a Kconfig option to set this value, with the default being 1 to match the current heap usage.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the option in 3b3d530.

if (err != ESP_OK) {
ESP_LOGD(TAG, "%s: error 0x%x writing block %d+%d",
__func__, err, start_block, i);
ESP_LOGD(TAG, "%s: error 0x%x writing blocks %zu+[%zu..%zu]",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately we can't rely on %z format specifier, since it's not supported by newlib-nano (i.e. if CONFIG_NEWLIB_NANO_FORMAT is enabled.)

Same note for sdmmc_read_sectors below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted back to %d in c6581d0.

@jofrev jofrev force-pushed the feat/sdmmc-multi-block-rw branch from 6588e0f to 98cbd0a Compare October 6, 2025 05:00
@jofrev
Copy link
Contributor Author

jofrev commented Oct 6, 2025

I had a look at the dma_aligned_buffer and liked the idea.
98cbd0a is a first version that favors this buffer (if allocated) over the temporary buffer.
What do you think @igrr?

@jofrev jofrev force-pushed the feat/sdmmc-multi-block-rw branch from 98cbd0a to 758c1b8 Compare October 24, 2025 14:55
@jofrev jofrev requested a review from igrr October 24, 2025 15:47
@jofrev jofrev force-pushed the feat/sdmmc-multi-block-rw branch from 758c1b8 to 2e33490 Compare November 24, 2025 07:46
@jofrev jofrev marked this pull request as ready for review November 24, 2025 07:47
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is being reviewed by Cursor Bugbot

Details

Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

@jofrev jofrev force-pushed the feat/sdmmc-multi-block-rw branch from 2e33490 to 16fd56c Compare November 27, 2025 15:24
This has the potential of speeding up SD card access significantly.

Reusing the DMA aligned buffer gives the additional option of having to
allocate the transaction buffer only once instead of for every
transaction, while still keeping the improved transaction time.

To give users the maximum amount of control, the Kconfig option for the
transaction buffer size applies only to the temporary buffer, which is
only allocated if the DMA aligned buffer has not been pre-allocated.
@jofrev jofrev force-pushed the feat/sdmmc-multi-block-rw branch from 16fd56c to e1e8d53 Compare November 27, 2025 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status: Opened Issue is new

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants