Cron jobs

yf runs the periodic work of a site as cron jobs: _class('cron') (yf_cron, plugin sys) keeps the list, runs a job under its lock and writes its runs; the admin page is Tools / Cron jobs (?object=manage_cron); the runner is the console command cron:run.

Jobs

  • File jobs: a file cron_jobs/<name>.cron.php of the framework, the app, the project or a plugin (the old share/cron_jobs/ folders too) that returns a function. Its return value (and anything it prints) is the output of the run; an exception or false is a failure. It runs in a child php (cron:run <job> --child), as the old runner ran it; $PROJECT_CONF['cron']['INCLUDE_PHP_CHILD'] = false (the default from 4.0) runs it in the runner's process, where an exit() ends the whole pass and what the job defines stays for the jobs after it. Exec types sh and php_script run the file as a command instead. Only files of the cron_jobs folders run.
  • Class jobs: declared by the framework or by the site in $PROJECT_CONF['cron']['JOBS']:
    $PROJECT_CONF['cron']['JOBS']['shop_reindex'] = [
        'class' => 'shop_search',      // _class() name
        'method' => 'cron_reindex',     // gets the run context: budget, deadline, trigger, admin_id, job
        'frequency' => '1 hours',       // N minutes | hours | days | weeks
        'timeout' => 900,               // seconds
        'title' => 'Shop search index',
        'module' => 'manage_shop',      // admin module it belongs to
        'step' => false,                // true: does part of the work per call and returns ['done' => false] until finished
    ];
    The method returns its output text, or ['ok' => bool, 'output' => text, 'done' => bool].

The Cron jobs page and cron:run register every job they find as a row of cron_tasks, turned off: an admin turns it on. The schedule, time limit and comment of a row are edited on the page (Edit).

Running them: the system crontab

* * * * * cd /path/to/site && php vendor/yfix/yf/.dev/console/yf.php cron:run --due > /dev/null 2>&1

Every minute the runner runs the jobs that are on and due (never run, or started one interval ago), one after the other, each under its lock: a file lock in storage/cron/locks/ and a claim on its cron_tasks row, so a job never runs twice at once, also across servers. A claim left by a process that was killed is taken over at once on its own server (or, for a stepped job, once its step is idle past STEP_IDLE), otherwise after the job's time limit plus 5 minutes. It writes the heartbeat (storage/cron/heartbeat.json) that the page and the doctor read. Run it as the web server user. A long job can get its own line (cron:run <job>). The old runner .dev/scripts/cron_runner.php still works; it writes no status and no heartbeat.

Without a system cron: the pseudo-cron

$PROJECT_CONF['cron']['PSEUDO_CRON'] = true (off by default): an admin page view runs the due jobs after the page is sent, at most once every PSEUDO_CRON_INTERVAL seconds (60) and for PSEUDO_CRON_SECONDS of work (20). Risks: jobs run only while admins use the panel; a long job ties up a PHP worker (and the FPM or proxy time limit can cut it); without fastcgi_finish_request() (Apache mod_php) only stepped jobs run, since the admin would wait. A job that is not stepped runs only when its time limit fits the pass's seconds. Prefer the crontab.

The page

  • Jobs: cards (jobs on, the runner's last pass, failing, overdue, running; the refresh control) and every job with its module, schedule, last run, status, duration, next due time and an On / Off switch (a POST with a confirm, audited).
  • Run now (super-admins, manage_cron RUN_NOW): runs the job in the request, under its lock (a running job is refused), with the time limit of the job; a stepped job does WEB_SECONDS of work per request and the page asks for the next step while it stays open. Each run goes to the admin wall. cron WEB_EXEC off keeps the shell jobs to the console.
  • Job page: what it runs, the last output, its runs (status, trigger, admin, duration, output) and the Live log (the log stream source cron). The output of runs is shown to super-admins only; other admins see the status.
  • Runs: the newest runs of every job. Setup: the crontab line, the heartbeat and the pseudo-cron.

Runs are kept in cron_logs with their status, trigger and admin (columns added on the first run where the database user may ALTER); cron HISTORY_KEEP limits the runs kept per job (0, the default, keeps all).

Console

php .dev/console/yf.php cron:run              # list the jobs and the last runner pass
php .dev/console/yf.php cron:run <job>        # run one now, on or off, without a time limit
php .dev/console/yf.php cron:run --due        # the crontab runner
php .dev/console/yf.php cron:run --json

Bot IP ranges

The class job bot_ip_ranges (off until turned on) downloads, once a day, the JSON IP lists that bot operators publish: OpenAI (GPTBot, ChatGPT-User, OAI-SearchBot), Anthropic, DuckDuckGo (DuckDuckBot, DuckAssistBot), Google (Googlebot and the special crawlers), Bing, Apple and Perplexity, over https, at most 2 MB each, into storage/bot_ip_ranges/. A list that fails keeps its last good copy. The visitor popup of User / Online users then marks such a bot verified when its IP is in its operator's lists (yf_bot_registry::verify(), bot_registry USE_IP_RANGES), from the files only: no network on a page view. $PROJECT_CONF['bot_ip_ranges']['SOURCES'] adds a list (id => ['url' => 'https://...', 'operator' => ..., 'bots' => [keys of plugins/user/share/bots.php]]) or removes one (id => false).

The job geonames_update (where the geo plugin is installed) updates the GeoNames tables of Tools / Geo, in steps.

Doctor

The site doctor's Cron group reports active jobs that never ran or are late (overdue), the runner stopped (its heartbeat older than cron HEARTBEAT_STALE, 15 minutes, once it ran on this server) and active jobs whose last run failed.