What runs and when?
Every schedule is a cron expression evaluated in UTC. The defaults below come fromapps/api/.env.example.
Batched alerts do not have their own schedule. They send once the daily pipeline reports every child job settled, so the delivery time depends on workload and retries rather than on a fixed clock.
What are the two workers?
The difference matters. The Apple limiter spaces requests, while the Google Play limiter spaces job starts, because one Google Play job fans out into many sequential requests of its own.
Work a person starts to explore rather than to collect, today the lookups of a deep search, is queued with a priority. A worker always takes every waiting job without a priority first, the daily run’s jobs among them, so a deep search waits behind them. Such a lookup also checks, before it reaches the store, that its workspace still has a plan in force and is not suspended, and does nothing otherwise.
Reading screenshot captions runs in a third queue,
screenshots, with its own worker at concurrency 1. While reading is on, every snapshot that records App Store screenshots queues one read, a market listing refresh included. With SCREENSHOT_OCR=false the screenshots are recorded as skipped and nothing is queued. It spends no store request budget and no OpenAI call. See Read screenshot captions.
Downloading store console reports runs in a fourth queue, store-connect, with its own worker at concurrency 1. It talks to App Store Connect with the key an owner connected, not to the public store, so it spends no store request budget and no proxy. A sync job per workspace is queued by CRON_STORE_REPORTS and by the connect, verify and sync buttons, and one already waiting is not queued twice. See Read impressions and conversion from your console.
Why are the limits conservative?
The iTunes endpoints informally tolerate roughly 20 requests per minute per IP address. Google Play is more sensitive still. Neither store publishes a limit or an error you can negotiate with, so asobeast stays well underneath by default. RaisingSCRAPE_ITUNES_RPM or SCRAPE_GPLAY_RPM puts the host IP address at risk of being throttled or blocked by the store, which stops collection for every app on that host. Do not point several instances at the stores from one IP address.
How do you estimate your daily budget?
Daily request volume is roughly tracked keywords multiplied by tracked markets, plus per app work for snapshots, reviews and category ranks, and one listing refresh per app, competitor and extra market.GET /jobs/budget returns an estimate broken down by kind, and the settings page renders it as a budget card with a utilization meter. The dashboard warns when the daily pipeline would exceed 85 percent of capacity. Both are shown only to the platform operator.
Category charts are the smallest slice of the daily budget, but they count, because the daily run captures every chart kind of each category an app sits in and of the overall chart, and every followed category adds its three charts. A chart is fetched once a day however many apps and workspaces sit in it. See Read top charts.
Can you call the store outside the queue?
No, with two exceptions. All collection goes through the queue so the limiters apply. The exceptions are a small, user initiated suggestion lookup and a store lookup when you search a store by name or ask for similar apps, both bounded by a person clicking rather than by a schedule, and by their hourly allowances.What is retained?
The retention job prunes each table on its own window.0 means keep forever. Every variable is described in the configuration reference.
Three exceptions are worth remembering. The newest snapshot per app is always kept regardless of age, open or snoozed action items are never pruned by age, and AI calls from the current month are never pruned because the monthly AI allowance still counts them. The same job also gives back AI calls that were reserved more than an hour ago and never settled.
What happens when a job fails?
A parse failure fails the job. BullMQ retries it with backoff, and request handling is never affected, so a broken parser degrades collection rather than taking the instance down. An App Store request that stops answering is abandoned after 15 seconds and retried like any other failure, so one slow connection cannot hold theappstore worker. Google Play requests have their own 30 second bound per attempt.
An empty answer is not trusted when the same question had an answer within the last seven days: a search for a keyword that ranked apps, or a review feed for an app that was receiving reviews. On the App Store the job asks again three times, four seconds apart, and then fails its attempt so the queue retries it, because Apple’s origin sometimes answers empty for a moment. App Store searches, lookups, charts and review feeds always reach Apple’s origin rather than a copy cached at Apple’s edge, so each retry sees a fresh answer. A review feed that is still empty on the last attempt ends the sync without a failed job: nothing is written, the reviews already stored stay as they are, and the next daily sync reads the feed again. A keyword search that is still empty on the last attempt ends its check the same way: no position is recorded for that day, the keyword keeps its last check, and the next check asks the store again. On Google Play the same empty answer is a soft block, and the job fails at once without retrying.
Top charts follow the same rule, with the job’s attempts as the second look. A chart that listed at least ten apps within the last seven days and answers empty fails every attempt, because that is how a blocked request looks. A chart that listed fewer, such as the paid chart of a small category, can honestly empty overnight, so it is treated like any other chart. Any other chart that answers empty is asked again on each attempt and stored as a capture with no rows only after the last one fails to find an app, so one blocked request on the first day of a new chart is never stored as an empty chart. The next day the chart is already known to be empty, is stored on the first attempt, and costs one request a day and no failed job. See What does an empty chart mean?
Failed jobs are visible on the queue dashboard at /admin/queues, which answers only to the platform operator. A failed chart job left by an earlier version stays in the failed count, and the header badge keeps reading degraded, until the operator cleans it on the dashboard. See Troubleshooting.
How do you follow today’s run for one app?
Every app page shows a pill beside Run daily with where today’s run of that app stands: how many of its listings, position checks and chart reads are in, out of how many the day needs. A chart read is a rank row of the app: the chart of its price class and the grossing chart, each in its category and overall, so an app counts four, not the six charts the daily run captures for everyone. A market listing counts only where the store sells the app: when a refresh finds that a storefront does not list the app or one of its competitors, the pill expects no listing of it there until a later refresh of that storefront finds it, so an app tracked in a market it is not sold in still reaches Run complete. Open it for one row per stage and the time the run started. It counts what landed since the daily trigger, or since midnight UTC once the trigger has passed, says Run complete when every count is in and Run delayed when the run started over twenty hours ago with counts still missing. A workspace the daily run skips, because an operator suspended it or its plan is no longer in force, shows no pill and is not polled, andGET /apps/{id}/run-status answers paused: true next to the unchanged counts. A run that finished before the pause still reads Run complete. See Rate limits for suspension and Billing for plans. While the run is collecting it asks again every ten seconds and refreshes the tables as positions land, so Run daily is followed by visible progress. The same answer is available as GET /apps/{id}/run-status and as the app_run_status MCP tool, computed from stored rows and never from the queue. Reviews are not counted, because a sync that finds nothing new leaves no trace.
What happens when the store stops listing an app?
A store can stop listing an app you or a competitor tracks: the developer removed it, the store took it down or it moved to a new store id. A listing refresh in the app’s home storefront then gets an answer that the app does not exist, the iTunes lookup with no result or a Google Play not found, and records the time of that first answer asdelistedAt on the app. The refresh job finishes instead of failing and being retried, and an on demand Refresh still answers the store’s 404.
While delistedAt is set:
- Every page of the app opens with a No longer listed notice naming the store, the store id, the home storefront and the UTC day, with the next step: open the store page, import the app again from its new link if it moved, or delete it, or remove a competitor, if it is gone for good.
- The app’s dashboard card and table row, and the competitor card on the competitors page, carry a Delisted badge.
- The guided setup stops waiting for a listing the store can no longer answer.
GET /apps/{id},GET /apps,GET /apps/{id}/competitorsandGET /portfolioanswerdelistedAtas a timestamp, andnullfor an app the store lists.- The run pill expects no listing of that app, in its home storefront or in any market, so today’s run can reach Run complete on its position checks and chart reads.
delistedAt, so the notice and the badges go away and the run pill counts the listing again. A market that does not list the app leaves delistedAt alone, because many apps are sold in some storefronts only; the run pill stops expecting that one listing instead, as How do you follow today’s run for one app? describes.