feat(js-client): create multi-queued rate limits

[alexjoverm]

Note

Implements a multi-queue throttle manager with dynamic rate limits (CDN/Management), parses server rate limit headers, integrates across client methods, and updates docs/tests.

• Core rate limiting:
  • Introduces ThrottleQueueManager to run separate throttle queues per rate (e.g., 1000, 50, 15, 10, 6 req/s).
  • Adds rateLimit.ts with determineRateLimit, parseRateLimitHeaders, createRateLimitConfig, and MANAGEMENT_API_DEFAULT_RATE_LIMIT.
  • New constants: DEFAULT_PER_PAGE, PER_PAGE_THRESHOLDS for tiered uncached limits.
• Client integration (src/index.ts):
  • Replaces single throttle with throttleManager and per-request rate selection via determineRateLimit.
  • Applies rate limiting to get (via cacheResponse) and Management API methods (post, put, delete).
  • Parses X-RateLimit(-Policy) response headers to adjust future limits.
  • For published requests, uses in-memory cache to bypass rate limiting when cached.
• Documentation (README):
  • Adds detailed Rate Limiting section; updates rateLimit config option description and behavior notes.
• Tests:
  • Adds comprehensive tests for rate limit tiers, header parsing, and multi-queue behavior (rateLimit.test.ts, throttleQueueManager.test.ts).
  • Updates existing tests to use throttleManager.execute; adds timing-based throttling scenarios in index.test.ts.
  • Minor test tweak for optional abort in throttlePromise.test.ts.

^{Written by Cursor Bugbot for commit 227642a. This will update automatically on new commits. Configure here.}

[cursor]

This PR is being reviewed by Cursor Bugbot

Details

Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

[cursor]

Bug: Trailing slash incorrectly identified as single story

The isSingleStoryRequest function incorrectly identifies listing endpoints with trailing slashes (like /cdn/stories/) as single story requests. When the URL is /cdn/stories/, the function returns true because url.split('/').length is 4 and !url.endsWith('/cdn/stories') is true (it ends with /cdn/stories/ not /cdn/stories). This causes listings to use the 50 req/s single-story rate limit instead of the appropriate tier based on per_page, potentially causing rate limit issues for users who include trailing slashes in their API calls.

 

[maoberlehner]

Some comments are technically wrong and some other minor comments. I let you decide if you want to fix them or not.

[maoberlehner]

question (non-blocking): do we have this magic number 25 in some constant already that we could use here?

[maoberlehner]

nitpick (non-blocking): I think it should be 26-50, 51-75, and 76-100

[maoberlehner]

And it would be nice to have the values as constants here, too, instead of hardcoded below. Comments can get old, constants not.

[maoberlehner]

suggestion (non-blocking): instead of inspecting the throttleManager itself, we could use vi.advanceTimersByTime(N) and check how often get was actually called for the different kinds of requests. That way we remove the coupling of the test to the concrete implementation.

it.only('should independently throttle different request types to their specific limits', async () => {
  vi.useFakeTimers();
  const limits = {
    published: 1000,
    small: 50,
    medium: 15,
    large: 10,
    veryLarge: 6,
  } as const;
  const attempts = {
    published: limits.published + 10,
    small: limits.small + 10,
    medium: limits.medium + 10,
    large: limits.large + 10,
    veryLarge: limits.veryLarge + 10,
  } as const;

  const promises = [];
  for (let i = 0; i < attempts.veryLarge; i++) {
    promises.push(
      client.get('cdn/stories', { version: 'draft', per_page: 76 }).catch(() => {}),
    );
  }
  for (let i = 0; i < attempts.large; i++) {
    promises.push(
      client.get('cdn/stories', { version: 'draft', per_page: 51 }).catch(() => {}),
    );
  }
  for (let i = 0; i < attempts.medium; i++) {
    promises.push(
      client.get('cdn/stories', { version: 'draft', per_page: 26 }).catch(() => {}),
    );
  }
  for (let i = 0; i < attempts.small; i++) {
    promises.push(
      client.get('cdn/stories', { version: 'draft', per_page: 25 }).catch(() => {}),
    );
  }
  for (let i = 0; i < attempts.published; i++) {
    promises.push(
      client.get('cdn/stories', { version: 'published' }).catch(() => {}),
    );
  }

  await vi.advanceTimersByTimeAsync(999);

  const countCallsContaining = (param: string, value: string | number) => {
    return client.client.get.mock.calls.filter(call => call[1][param] === value).length;
  };

  const veryLargeCalls = countCallsContaining('per_page', 76);
  expect(veryLargeCalls).toBe(limits.veryLarge);

  const largeCalls = countCallsContaining('per_page', 51);
  expect(largeCalls).toBe(limits.large);

  const mediumCalls = countCallsContaining('per_page', 26);
  expect(mediumCalls).toBe(limits.medium);

  const smallCalls = countCallsContaining('per_page', 25);
  expect(smallCalls).toBe(limits.small);

  const medium = countCallsContaining('version', 'published');
  expect(medium).toBe(limits.published);

  vi.useRealTimers();
});

[alexjoverm]

Totally agree. Def better when they're more input-output based. Thanks for the suggestion

[maoberlehner]

nitpick (non-blocking): I think the ranges here are also not quite correct (26-50 and so on).

[pkg-pr-new]

Open in StackBlitz

@storyblok/astro

npm i https://pkg.pr.new/@storyblok/astro@380

storyblok

npm i https://pkg.pr.new/storyblok@380

@storyblok/eslint-config

npm i https://pkg.pr.new/@storyblok/eslint-config@380

@storyblok/js

npm i https://pkg.pr.new/@storyblok/js@380

storyblok-js-client

npm i https://pkg.pr.new/storyblok-js-client@380

@storyblok/management-api-client

npm i https://pkg.pr.new/@storyblok/management-api-client@380

@storyblok/nuxt

npm i https://pkg.pr.new/@storyblok/nuxt@380

@storyblok/react

npm i https://pkg.pr.new/@storyblok/react@380

@storyblok/region-helper

npm i https://pkg.pr.new/@storyblok/region-helper@380

@storyblok/richtext

npm i https://pkg.pr.new/@storyblok/richtext@380

@storyblok/svelte

npm i https://pkg.pr.new/@storyblok/svelte@380

@storyblok/vue

npm i https://pkg.pr.new/@storyblok/vue@380

commit: 227642a