Rate limits, quotas and caching to scale a BCE/KBO application: reference architecture

In brief

The Company Belgium BCE/KBO API exposes rate-limit headers your code can read directly. To scale without exhausting your quota, combine a 4-layer cache (CDN, memory, shared cache, DB snapshot) with webhooks for continuous tracking. This architecture absorbs 95 % of traffic before the API and keeps latency under 50 ms.

The problem: going from 100 to 100,000 requests/day

You start with a few calls a day. The product grows. Six months later, your app does 100,000 lookups daily. If you call the API on every page view by every user, you:

• Saturate your quota

• Receive 429 Too Many Requests

• Pay the volume bill

• Increase user-side latency

The architecture in this article prevents that. It's production-tested.

Understanding rate limits

Anatomy of a window

An API typically exposes a quota per sliding window:

• Free plan: 10 req/min, 100 req/day

• Starter plan: 60 req/min, 5,000 req/day

• Pro plan: 600 req/min, 100,000 req/day

• Enterprise plan: per contract

Company Belgium headers:

1
2
3
4

X-RateLimit-Limit: 600
X-RateLimit-Remaining: 432
X-RateLimit-Reset: 1746541234
Retry-After: 30   (only on 429)

The 3 limit types

A good app must handle all three, not just the occasional 429.

Reference architecture in 4 layers

1
2
3
4
5
6
7
8
9
10
11

[ User Request ]
       ↓
[ Browser cache + CDN ]            ← Layer 1 (free, outside quota)
       ↓
[ App memory cache (LRU TTL) ]     ← Layer 2 (10-100 ms)
       ↓
[ Shared cache (memcached) ]       ← Layer 3 (1-10 ms network)
       ↓
[ Database snapshot ]              ← Layer 4 (10-50 ms, freshness ~24h)
       ↓
[ BCE/KBO API ]                    ← Quota & cost

Goal: less than 5 % of user requests actually hit the API in layer 5.

Layer 1 — CDN / Browser

For public endpoints without sensitive data, expose your proxy routes with Cache-Control: public, max-age=300, s-maxage=600. CloudFront, Cloudflare, Vercel Edge Network do the rest.

Typical effect: 50 to 70 % of requests absorbed before your app.

Layer 2 — App memory cache

LRU with TTL (see the Python tutorial and Node.js tutorial):

• Capacity: 500 to 5,000 records based on RAM

• TTL: 5 min for records, 30 min for NACE codes

• Eviction: LRU

Typical effect: +20 % absorption.

Layer 3 — Shared cross-process cache

If you have multiple app instances (Docker, Kubernetes, Vercel functions), a local cache is not shared. Use:

• Memcached: simple, fast, shared-nothing

• In-cluster cache like Caffeine (Java), Tinybird (analytics)

• Postgres with a cache table + key index (enough for typical SaaS volume)

Effect: a MISS on instance A can be a HIT on layer 3, populated by instance B 10 seconds earlier.

Layer 4 — DB snapshot

For apps that do many lookups on their own clients (CRM, ERP, accounting), maintain a local copy of BCE records for tracked companies. Synchronized via webhooks.

Pros:

• Local queries: 10-50 ms

• No network dependency on reads

• SQL filter/group available

Cons:

• Data possibly hours old

• One more table to maintain

The right tradeoff for 80 % of B2B SaaS cases.

Layer 5 — Direct API call

When all previous layers miss, call the API. Wrapped with:

• Exponential retry on 429/5xx honoring Retry-After

• Circuit breaker: if > 50 % errors over 1 minute, stop calls for 30 seconds

• Hard timeout (10 s)

• Local limit: semaphore of max N concurrent requests

Strategies per use case

A. Supplier verification (infrequent, must be fresh)

• Memory cache 1 minute

• No CDN (client-specific)

• Direct call on miss

Typical API volume: 1 req per file.

B. Public directory display (frequent, can be slightly stale)

• CDN 30 minutes

• Memory cache 5 min

• DB snapshot refreshed nightly

• Direct call only on full miss

Typical API volume: 1 req per 100 views.

C. Full-text search (frequent, must be fresh)

• No CDN (unique query URLs)

• Memory cache 1 min with key = hash(query+filters)

• Direct call on miss

Typical API volume: 1 req per 5-10 searches.

D. Monitoring/alerts (continuous, event-driven)

• No polling — webhooks

• DB snapshot kept current via webhooks

• Memory cache optional

Typical API volume: 0 req per day (all via webhooks).

Webhooks vs polling: the math

You track 1,000 clients and want to detect changes within 15 minutes.

Polling:

• 1,000 companies × (60/15) lookups/h × 24h = 96,000 req/day

• Pro plan required (100,000/day)

• Cost: ~€300/month

Webhooks:

• ~20 changes/day (observed rate)

• 20 webhooks received = 20 optional lookups = 20 req/day

• Free plan enough

• Cost: €0/month

The gap is 4,800×. Webhooks are almost always the right answer for monitoring.

Handling 429s properly

Pseudo-code for smart retry:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

function callApi(req):
  attempt = 0
  while attempt < 4:
    res = http.send(req)
    if res.status == 429:
      delay = res.header("Retry-After") || base * 2^attempt
      sleep(delay + jitter())
      attempt++
      continue
    if res.status in [500, 502, 503, 504]:
      sleep(base * 2^attempt + jitter())
      attempt++
      continue
    return res
  throw RateLimitExhausted()

Jitter = random ±25 % delay to avoid the thundering herd (all instances retrying at once).

Observability

Metrics to expose (Prometheus / Datadog / OpenTelemetry):

Alert when X-RateLimit-Remaining < 10 % → fail over to DB snapshot.

Capacity table

Common pitfalls

How Company Belgium helps you scale

• Transparent rate-limit headers: your code adapts

• Webhooks for 9 event types (company.updated, company.ubo.changed, company.peppol.registered, etc.) — see endpoints reference

• Plan tiers aligned with real architectures (Free, Starter, Pro, Enterprise)

• Sandbox to test without consuming production quota

• Bulk endpoint on pro plans (100 records per call)

• Functional comparison with other APIs: 2026 comparison and official vs third-party APIs

Bottom line

Scaling a BCE/KBO app is not just upgrading to a bigger plan. The right architecture absorbs 95 % of traffic before the API, via 4 cache layers and webhooks. The quota becomes a formality, latency stays under 50 ms, and the bill follows value, not noise.

Three levers to activate in order: memory cache + retry, webhooks instead of polling, DB snapshot for your own clients. With that, you serve 100 K user req/day consuming only 5 K API req/day.