Cron jobs
yf runs the periodic work of a site as cron jobs: _class('cron') (yf_cron, plugin sys) keeps
the list, runs a job under its lock and writes its runs; the admin page is Tools / Cron jobs (?object=manage_cron);
the runner is the console command cron:run.
Jobs
- File jobs: a file
cron_jobs/<name>.cron.phpof the framework, the app, the project or a plugin (the oldshare/cron_jobs/folders too) that returns a function. Its return value (and anything it prints) is the output of the run; an exception orfalseis a failure. It runs in a child php (cron:run <job> --child), as the old runner ran it;$PROJECT_CONF['cron']['INCLUDE_PHP_CHILD'] = false(the default from 4.0) runs it in the runner's process, where anexit()ends the whole pass and what the job defines stays for the jobs after it. Exec typesshandphp_scriptrun the file as a command instead. Only files of the cron_jobs folders run. - Class jobs: declared by the framework or by the site in
$PROJECT_CONF['cron']['JOBS']:
The method returns its output text, or$PROJECT_CONF['cron']['JOBS']['shop_reindex'] = [ 'class' => 'shop_search', // _class() name 'method' => 'cron_reindex', // gets the run context: budget, deadline, trigger, admin_id, job 'frequency' => '1 hours', // N minutes | hours | days | weeks 'timeout' => 900, // seconds 'title' => 'Shop search index', 'module' => 'manage_shop', // admin module it belongs to 'step' => false, // true: does part of the work per call and returns ['done' => false] until finished ];['ok' => bool, 'output' => text, 'done' => bool].
The Cron jobs page and cron:run register every job they find as a row of cron_tasks, turned off: an
admin turns it on. The schedule, time limit and comment of a row are edited on the page (Edit).
Running them: the system crontab
* * * * * cd /path/to/site && php vendor/yfix/yf/.dev/console/yf.php cron:run --due > /dev/null 2>&1
Every minute the runner runs the jobs that are on and due (never run, or started one interval ago), one after the other, each
under its lock: a file lock in storage/cron/locks/ and a claim on its cron_tasks row, so a job never runs twice at
once, also across servers. A claim left by a process that was killed is taken over at once on its own server (or, for a stepped
job, once its step is idle past STEP_IDLE), otherwise after the job's time limit plus 5 minutes. It writes the heartbeat (storage/cron/heartbeat.json) that the page and the doctor read. Run it as
the web server user. A long job can get its own line (cron:run <job>). The old runner
.dev/scripts/cron_runner.php still works; it writes no status and no heartbeat.
Without a system cron: the pseudo-cron
$PROJECT_CONF['cron']['PSEUDO_CRON'] = true (off by default): an admin page view runs the due jobs after the page is
sent, at most once every PSEUDO_CRON_INTERVAL seconds (60) and for PSEUDO_CRON_SECONDS of work (20). Risks: jobs
run only while admins use the panel; a long job ties up a PHP worker (and the FPM or proxy time limit can cut it); without
fastcgi_finish_request() (Apache mod_php) only stepped jobs run, since the admin would wait. A job that is not stepped runs only
when its time limit fits the pass's seconds. Prefer the crontab.
The page
- Jobs: cards (jobs on, the runner's last pass, failing, overdue, running; the refresh control) and every job with its module, schedule, last run, status, duration, next due time and an On / Off switch (a POST with a confirm, audited).
- Run now (super-admins,
manage_cron RUN_NOW): runs the job in the request, under its lock (a running job is refused), with the time limit of the job; a stepped job doesWEB_SECONDSof work per request and the page asks for the next step while it stays open. Each run goes to the admin wall.cron WEB_EXECoff keeps the shell jobs to the console. - Job page: what it runs, the last output, its runs (status, trigger, admin, duration, output) and the Live log (the log
stream source
cron). The output of runs is shown to super-admins only; other admins see the status. - Runs: the newest runs of every job. Setup: the crontab line, the heartbeat and the pseudo-cron.
Runs are kept in cron_logs with their status, trigger and admin (columns added on the first run where the database user may
ALTER); cron HISTORY_KEEP limits the runs kept per job (0, the default, keeps all).
Console
php .dev/console/yf.php cron:run # list the jobs and the last runner pass
php .dev/console/yf.php cron:run <job> # run one now, on or off, without a time limit
php .dev/console/yf.php cron:run --due # the crontab runner
php .dev/console/yf.php cron:run --json
Bot IP ranges
The class job bot_ip_ranges (off until turned on) downloads, once a day, the JSON IP lists that bot operators publish:
OpenAI (GPTBot, ChatGPT-User, OAI-SearchBot), Anthropic, DuckDuckGo (DuckDuckBot, DuckAssistBot), Google (Googlebot and the special
crawlers), Bing, Apple and Perplexity, over https, at most 2 MB each, into storage/bot_ip_ranges/. A list that fails keeps
its last good copy. The visitor popup of User / Online users then marks such a bot verified when its IP is in its operator's
lists (yf_bot_registry::verify(), bot_registry USE_IP_RANGES), from the files only: no network on a page view.
$PROJECT_CONF['bot_ip_ranges']['SOURCES'] adds a list (id => ['url' => 'https://...', 'operator' => ..., 'bots'
=> [keys of plugins/user/share/bots.php]]) or removes one (id => false).
The job geonames_update (where the geo plugin is installed) updates the GeoNames tables of Tools / Geo, in steps.
Doctor
The site doctor's Cron group reports active jobs that never ran or are late (overdue), the runner stopped (its heartbeat older than
cron HEARTBEAT_STALE, 15 minutes, once it ran on this server) and active jobs whose last run failed.