> ## Documentation Index
> Fetch the complete documentation index at: https://docs.asobeast.com/llms.txt
> Use this file to discover all available pages before exploring further.
# How much can one instance collect?
> Measured orchestration cost and projected daily wall clock at three scales, the configuration where a run stops finishing inside a day, and what the per market multiplier really costs.
Collection is bounded by rate limits, not by hardware. asobeast asks the stores for one thing at a time on purpose, so the honest question is not how fast your server is, it is how many store requests a day you have room for.
Every number here was measured on a deterministic fixture and can be reproduced with `pnpm --filter api run bench:target`.
## What the pipeline costs to orchestrate
This is the code's own overhead: building the target list, estimating the budget, and enqueueing the whole fan-out with a provider that returns instantly.
| Scale | Apps | Tracked keywords | Ranking rows | Budget query | Fan-out | Jobs enqueued | Redis growth |
| -------- | ---- | ---------------- | ------------ | ------------ | -------- | ------------- | ------------ |
| `small` | 3 | 75 | 525 | 12 ms | 11 ms | 81 | 0.4 MB |
| `target` | 20 | 8,000 | 720,000 | 91 ms | 468 ms | 8,040 | 9.5 MB |
| `stress` | 50 | 75,000 | 6,750,000 | 547 ms | 4,029 ms | 75,100 | 77 MB |
Orchestration is not the constraint and never becomes one. Enqueueing 75,100 jobs takes four seconds, against a run that will take days. Budget Redis at roughly 1 KB per queued job.
## What the pipeline costs in wall clock
This is arithmetic, not measurement, and it is what you actually wait for. Job count divided by the configured rate limit.
| Scale | App Store requests | At 15 rpm | Google Play job starts | At 10 rpm |
| -------- | ------------------ | ---------- | ---------------------- | ---------- |
| `small` | 81 | 0.1 hours | 0 | 0 hours |
| `target` | 4,020 | 4.5 hours | 4,020 | 6.7 hours |
| `stress` | 37,550 | 41.7 hours | 37,550 | 62.6 hours |
The `target` row is the stated benchmark: 20 apps, 200 keywords each, two storefronts, both stores. It finishes inside the night. The `stress` row does not finish inside a day at all.
The Google Play column counts **job starts**, which is what `SCRAPE_GPLAY_RPM` spaces. A Play scoring job fans out to roughly 15 to 18 sequential requests and a depth 200 Play rank job costs about eight, against one for the Apple equivalent. The real Play request count is therefore several times the number in that column, and it is the request count that risks Google throttling you. Treat the Play projection as a floor.
## Where the day runs out
A run stops finishing inside 24 hours at:
| Store | Ceiling per day | What that is |
| ----------- | --------------------- | --------------------------------------------- |
| App Store | **21,600 requests** | `SCRAPE_ITUNES_RPM` of 15, times 60, times 24 |
| Google Play | **14,400 job starts** | `SCRAPE_GPLAY_RPM` of 10, times 60, times 24 |
Keywords dominate the count, and a keyword is counted once per market. So the Apple ceiling is roughly 21,600 keyword and market pairs at default limits. Some shapes that reach it:
* 20 apps, 200 keywords each, 5 markets is 20,000 searches, about 22 hours. A sixth market does not fit.
* 50 apps, 200 keywords each, 2 markets is 20,000 searches, the same wall from a different direction.
* 10 apps, 400 keywords each, 5 markets is 20,000 searches again.
The shape does not matter. The product does.
## The per market multiplier
Adding a storefront for a keyword set you already track **doubles** the searches for that set. The same phrase in two markets is two keyword rows, checked by two searches, because rankings differ per storefront and one search cannot answer for both.
At the `target` scale, dropping from two markets to one halves the App Store projection from 4.5 hours to about 2.3. Adding a third takes it to about 6.7.
This is the single most expensive decision you make, and it is invisible until the run stops finishing. Check it before you add a market, not after.
## Check your own numbers
`GET /jobs/budget` estimates the fan-out from your real data, broken down per store, and the settings page renders the same figures as a budget card. The estimate is verified against the actual fan-out in CI on every pull request, so the number the card shows is the number of jobs the pipeline will enqueue.
```bash theme={null}
curl --silent http://127.0.0.1:3000/api/backend/jobs/budget | jq '{total, utilization, stores}'
```
`utilization` is the fraction of a day the run will occupy at your configured limits. Above 1.0 the run cannot finish before the next one starts.
## Should you raise the limits?
Usually not.
`SCRAPE_ITUNES_RPM` defaults to 15 against an informal ceiling of roughly 20 requests per minute per IP address. There is headroom, and there is not much of it. Going past what the store tolerates does not produce an error you can plan around: requests start failing, jobs land in the failed set, and in the worst case the store stops answering that IP address for a while. That cost falls on you, not on the project, and the recovery is waiting.
Prefer, in this order:
1. Remove keywords you do not act on. Most tracked sets have a long tail nobody reads.
2. Remove markets you do not sell in. This is the multiplier.
3. Only then consider raising `SCRAPE_ITUNES_RPM`, in small steps, watching the failed job count on the health endpoint after each change.
Both workers run at concurrency 1 behind their limiter deliberately. That is not a performance oversight, it is the thing keeping your address in good standing. See [The daily pipeline and rate limits](/concepts/pipeline).
## The read path holds up
Query plans were captured at the `target` scale, with 720,000 ranking rows loaded, after `ANALYZE`:
| Query | Rows examined | Time |
| --------------------------------- | ------------- | ------- |
| 90 days of rankings for one app | 36,000 | 4.1 ms |
| Latest position per keyword | 36,000 | 46.5 ms |
| Visibility aggregate over 90 days | 36,000 | 7.5 ms |
All three use the existing `("appId", date)` index rather than scanning the table, so no index was added. The dashboard stays responsive at the benchmark scale; the collection window is the thing that runs out first.
## Related
Why concurrency is 1 and what the limiter does.
Why a market multiplies rather than adds.