Calling Supadata from Node.js without hitting limit-exceeded

Disclosure: we pay for Supadata and use it in production. Supadata gives us API credits for publishing this tutorial, and the Supadata sign-up links in this post are our referral link: supadata.ai/r/3YW2NR6K. The code and opinions are our own.

One of the backends we run at Bowen Chang Pty Ltd turns short cooking videos into recipes. Someone shares a TikTok or Instagram link, and we need two things before a language model can write the recipe: the caption the creator typed and the words they spoke. Supadata gives us both. Its metadata endpoint returns the caption, author and cover image, and its transcript endpoint returns the speech.

We moved every TikTok and Instagram call to Supadata for a practical reason. Supadata fetches the post from its own infrastructure, so platforms that rate-limit our server’s IP address never see us. When TikTok’s public oEmbed endpoint stopped returning captions in October 2026, our TikTok imports kept working because we were already reading captions from Supadata.

The first version was a single SDK call. Then real traffic hit it, and two problems showed up:

1. Bursts get rejected. Supadata limits requests per second by plan (the Free plan allows one per second). Two people importing at the same moment produced a limit-exceeded error.
2. Slow calls hold a worker. The @supadata/js SDK makes a plain fetch() with no timeout, so one slow response could hold a request open for minutes. This tutorial builds the fix as one file, supadata.ts. It compiles against @supadata/js 1.4 with TypeScript in strict mode, and each section below is a piece of that file.

Setup

You need Node.js 18 or later. Create a Supadata account (our referral link) to get an API key, then set up a project:

npm init -y && npm pkg set type=module
npm install @supadata/js
npm install -D tsx @types/node
export SUPADATA_API_KEY=your-api-key

1. Give every call a deadline

Promise.race against a timer is enough. The finally clears the timer so it can’t keep the process alive.

import { createHash } from 'node:crypto';
import { Supadata, type Transcript, type TranscriptOrJobId } from '@supadata/js';

export function withTimeout<T>(promise: Promise<T>, ms: number, label: string): Promise<T> {
  let timer: NodeJS.Timeout | undefined;
  const deadline = new Promise<never>((_, reject) => {
    timer = setTimeout(() => reject(new Error(`${label} timed out after ${ms}ms`)), ms);
  });
  return Promise.race([promise, deadline]).finally(() => clearTimeout(timer));
}

2. Queue calls so they never burst

The queue starts calls at least minIntervalMs apart and caps how many run at once. It also limits how long a caller will wait for its turn, because a user staring at a spinner would rather get a quick failure than wait 30 seconds. backoff() delays every call that hasn’t started yet, which is what you want when the upstream says you are going too fast.

export class QueueWaitTimeoutError extends Error {}

type Waiter = { resolve: () => void; reject: (err: Error) => void; timer?: NodeJS.Timeout };

export class RequestQueue {
  private active = 0;
  private nextStartAt = 0;
  private waiters: Waiter[] = [];
  private pumpTimer: NodeJS.Timeout | null = null;

  constructor(private minIntervalMs: number, private maxConcurrent: number) {}

  async run<T>(fn: () => Promise<T>, maxWaitMs: number): Promise<T> {
    await this.acquire(maxWaitMs);
    try {
      return await fn();
    } finally {
      this.active--;
      this.pump();
    }
  }

  /** Hold every call that hasn't started yet for at least `ms`. */
  backoff(ms: number): void {
    this.nextStartAt = Math.max(this.nextStartAt, Date.now() + ms);
    if (this.pumpTimer) clearTimeout(this.pumpTimer);
    this.pumpTimer = null;
    this.pump();
  }

  private acquire(maxWaitMs: number): Promise<void> {
    return new Promise((resolve, reject) => {
      const waiter: Waiter = { resolve, reject };
      waiter.timer = setTimeout(() => {
        const i = this.waiters.indexOf(waiter);
        if (i === -1) return; // already started
        this.waiters.splice(i, 1);
        reject(new QueueWaitTimeoutError(`no queue slot within ${maxWaitMs}ms`));
      }, Math.max(0, maxWaitMs));
      this.waiters.push(waiter);
      this.pump();
    });
  }

  private pump(): void {
    if (this.pumpTimer) return;
    while (this.waiters.length > 0 && this.active < this.maxConcurrent) {
      const wait = this.nextStartAt - Date.now();
      if (wait > 0) {
        this.pumpTimer = setTimeout(() => {
          this.pumpTimer = null;
          this.pump();
        }, wait);
        return;
      }
      const waiter = this.waiters.shift()!;
      clearTimeout(waiter.timer);
      this.active++;
      this.nextStartAt = Date.now() + this.minIntervalMs;
      waiter.resolve();
    }
  }
}

The queue lives in one process. If you run several instances, each one spaces its own calls, so the occasional collision between instances still happens. That is why the caller in step 3 retries too.

3. Route every call through one function

createSupadataCaller puts the pieces together. Each call waits its turn in the queue and runs with a deadline. If Supadata still answers limit-exceeded (another instance may have used the same second), the whole queue backs off, first for one second and then for two, and the call goes back in line as long as its total wait budget allows. Any other error, such as not-found, fails immediately.

export function isRateLimited(err: unknown): boolean {
  const e = err as { error?: unknown; message?: unknown } | null;
  return e?.error === 'limit-exceeded' || /limit exceeded/i.test(String(e?.message ?? ''));
}

export interface CallerOptions {
  minIntervalMs?: number; // 1100 suits the Free plan (1 request/second)
  maxConcurrent?: number;
  maxWaitMs?: number; // total time one call may spend queued or backing off
  timeoutMs?: number;
  retries?: number;
  backoffMs?: number;
  client?: Supadata; // pass a fake in tests
}

export type SupadataCall = <T>(label: string, fn: (client: Supadata) => Promise<T>) => Promise<T>;

export function createSupadataCaller(apiKey: string, opts: CallerOptions = {}): SupadataCall {
  const {
    minIntervalMs = 1100,
    maxConcurrent = 4,
    maxWaitMs = 5000,
    timeoutMs = 10_000,
    retries = 2,
    backoffMs = 1000,
  } = opts;
  const queue = new RequestQueue(minIntervalMs, maxConcurrent);
  const client = opts.client ?? new Supadata({ apiKey });

  return async function call<T>(label: string, fn: (client: Supadata) => Promise<T>): Promise<T> {
    const waitDeadline = Date.now() + maxWaitMs;
    for (let retry = 0; ; retry++) {
      try {
        return await queue.run(() => withTimeout(fn(client), timeoutMs, `Supadata ${label}`), waitDeadline - Date.now());
      } catch (err) {
        if (retry >= retries || !isRateLimited(err)) throw err;
        const delay = backoffMs * 2 ** retry;
        if (Date.now() + delay >= waitDeadline) throw err;
        queue.backoff(delay); // the whole queue slows down, not just this call
      }
    }
  };
}

The client option exists for tests. You can pass a fake client that throws new SupadataError({ error: 'limit-exceeded' }) and check that the queue backs off and retries without spending real credits.

4. Fetch the caption and transcript, with a cache

A transcript for a given video never changes, so we cache it. The same video gets shared with different tracking parameters (?_r=1&_t=…), so we strip those before building the cache key. Otherwise every share would cost a fresh call. The transcript content can be either a string or an array of timed chunks. Long videos may return a jobId for asynchronous processing instead. Recipe clips are short, so we treat that case as "no transcript" and fall back to the caption.

/** Share links differ only in tracking params; one video should be one cache entry. */
export function canonicalVideoUrl(url: string): string {
  const u = new URL(url);
  if (!/(^|\.)(tiktok|instagram)\.com$/i.test(u.hostname)) return url;
  return `https://${u.hostname.toLowerCase()}${u.pathname.replace(/\/+$/, '')}`;
}

export function transcriptText(result: TranscriptOrJobId): string {
  if (!('content' in result)) return ''; // a jobId: long video, processed asynchronously
  const { content } = result as Transcript;
  if (typeof content === 'string') return content.trim();
  return content.map((chunk) => chunk.text).filter(Boolean).join(' ').trim();
}

const transcriptCache = new Map<string, string>();

export async function getVideoText(rawUrl: string, call: SupadataCall) {
  const url = canonicalVideoUrl(rawUrl);
  const cacheKey = createHash('sha256').update(url).digest('hex');

  const [meta, transcript] = await Promise.allSettled([
    call('metadata', (client) => client.metadata({ url })),
    transcriptCache.has(cacheKey)
      ? Promise.resolve(transcriptCache.get(cacheKey)!)
      : call('transcript', (client) => client.transcript({ url, mode: 'auto' })).then(transcriptText),
  ]);

  if (transcript.status === 'fulfilled' && transcript.value) transcriptCache.set(cacheKey, transcript.value);
  return {
    caption: meta.status === 'fulfilled' ? (meta.value.description ?? '').trim() : '',
    transcript: transcript.status === 'fulfilled' ? transcript.value : '',
  };
}

Promise.allSettled means one failed call doesn’t sink the other. A caption with no transcript is often still enough to write a recipe.

Run it

Save the four sections above as supadata.ts, then add demo.ts:

import { createSupadataCaller, getVideoText } from './supadata.js';

const apiKey = process.env.SUPADATA_API_KEY;
if (!apiKey) throw new Error('Set SUPADATA_API_KEY first');
// A batch script can afford to wait; a request a user is waiting on can't.
const call = createSupadataCaller(apiKey, { maxWaitMs: 60_000 });

const urls = process.argv.slice(2);
const results = await Promise.all(urls.map((url) => getVideoText(url, call)));
results.forEach((result, i) => {
  console.log(`\n${urls[i]}`);
  console.log(`caption:    ${result.caption.slice(0, 120) || '(none)'}`);
  console.log(`transcript: ${result.transcript.slice(0, 120) || '(none)'}`);
});

npx tsx demo.ts "https://www.tiktok.com/@creator/video/1234567890" "https://www.instagram.com/reel/ABC123/"

Each URL makes two calls, metadata and transcript, and the queue starts them a little over a second apart. Ten URLs take about 22 seconds and none of them hits limit-exceeded. The demo raises maxWaitMs for that reason. With the 5-second default, any call still queued after 5 seconds fails with QueueWaitTimeoutError. That is the behaviour you want behind a user-facing request, where a quick failure beats a long spinner, but not in a batch job.

What we run in production

The biggest difference in production is the cache. It is an S3 bucket rather than an in-memory Map, so it survives deploys and is shared by every instance.

The defaults above (1.1-second spacing, a 10-second deadline, a 5-second total wait and two retries) are the values we run on the Free tier. On a paid plan with a higher per-second allowance, lower minIntervalMs to match: 110 ms suits 10 requests a second.

If you are building something similar, you can try Supadata through our referral link. If you would rather have someone build it with you, tell us about your project.