Online users
yf records who is online (users, guests, admins and bots) in Redis or Valkey only. No database tables are used. A sorted set per
visitor type holds each visitor's last request time, and a small hash per visitor holds the details. Both expire on their own. The
admin page User / Online users (?object=watch_online_users) shows them live. The engine is
_class('online_users') (yf_online_users, plugin user).
Turning it on
// a Redis or Valkey server (environment, $CONF or a constant)
define('REDIS_HOST', '127.0.0.1'); // or VALKEY_HOST; REDIS_PORT, REDIS_PASSWORD, REDIS_PREFIX as for the cache
define('YF_SECRET_KEY', '...64 random hex characters...'); // keys the visitor ids (see Privacy)
$PROJECT_CONF['main']['TRACK_ONLINE_STATUS'] = true; // record visitors
$PROJECT_CONF['main']['TRACK_ONLINE_DETAILS'] = true; // also IP, user agent and country (personal data)
$PROJECT_CONF['online_users']['MASK_IP'] = true; // store 203.0.113.0 instead of 203.0.113.77
$PROJECT_CONF['online_users']['COUNTRY_HEADERS'] = ['HTTP_CF_IPCOUNTRY']; // only the header your proxy sets
$PROJECT_CONF['client_utils']['TRUSTED_PROXIES'] = ['10.0.0.0/8']; // behind a proxy: the real visitor IP
The same switches are on the Settings page, tab Online users. Without a Redis server tracking does nothing. The tracking call
only reads the config, opens no connection, and costs nothing measurable. The admin page and the site doctor then say Redis required.
auth_user STORE_ONLINE_USERS records visitors through the same engine.
Redis databases: sessions, cache, online users
Keep each kind of data in a database of its own on the Redis server, so that clearing one never clears another:
| Data | Database | Set by |
|---|---|---|
| caches and everything else | 0 | REDIS_DATABASE (default 0) |
| online users | 2 | $PROJECT_CONF['online_users']['REDIS_DB'] (default 2; null: the connection's own database) |
| PHP sessions | 10 (for example) | php.ini only: session.save_handler = redis, session.save_path = "tcp://redis:6379?database=10&timeout=300". yf never configures it; it only reads it |
The site doctor (group Cache, "Redis databases of sessions, cache and online users") reads session.save_path, the cache
connection and the online users connection, and reports any database two of them share. Sharing the cache database with the sessions
is critical: a cache flush there logs everyone out.
Cache flushes are safe. "Purge all caches" (with manage_redis ALLOW_FLUSHDB) and cache()->flush()
(the Settings cache purge; the Redis cache driver runs FLUSHDB unless FLUSH_BY_PREFIX is on) never run
FLUSHDB on a database that also holds the PHP sessions or the online users. They delete the cache's own keys by its
prefix instead (REDIS_CACHE_PREFIX or cache CACHE_NS), or refuse and say why when the cache has no prefix, or
when its prefix would also match the session or online keys (the online keys start with REDIS_PREFIX +
online:, so a cache prefix equal to REDIS_PREFIX covers them). When the layout cannot be read, they refuse too.
The Redis manager's own "Flush DB" warns in red when the selected database holds sessions or online users. The online users purge
deletes only the online keys of the site (a SCAN of its prefix), never the database.
Moving the online users to database 2 needs no migration: the entries in the old database expire within the window (5 minutes by
default). Redis Cluster and some managed servers have database 0 only and refuse SELECT 2: the online users then record
nothing and the doctor says so; set $PROJECT_CONF['online_users']['REDIS_DB'] = null there (the online keys keep their own
prefix; give the cache a prefix too). Servers are compared by host name and port: two names of one server (a docker service name and its
IP) are not recognised as the same.
What is stored, and for how long
Key (below REDIS_PREFIX) | Holds | Lifetime |
|---|---|---|
online:<SITE_ID>:z:<type> | sorted set of visitor ids by last seen; type user, guest, admin, bot; at most MAX_PER_TYPE (10000) | trimmed on every write, the key WINDOW + 60 s |
online:<SITE_ID>:v:<type>:<id> | type, user id, first and last seen, section, page path, module/action, device, bot name; with details also IP, user agent (200 characters), country | WINDOW; deleted with its entry when it leaves the window or the cap |
online:<SITE_ID>:t:<type>:<id> | throttle gate of a visitor without a session | THROTTLE |
online:<SITE_ID>:a:<type>:<minute> | HyperLogLog of the visitors of that minute (the sparklines) | SPARK_MINUTES (60, at most 360) + 2 min |
online:<SITE_ID>:salt | random salt for the visitor ids when the site has no YF_SECRET_KEY | kept |
Memory budget. A visitor costs roughly 0.1 KB in its sorted set and 0.3-0.7 KB of details (the user agent is most of it), so the
cap of 4 × MAX_PER_TYPE visitors stays under about 30 MB. The sparklines add up to 4 × SPARK_MINUTES
HyperLogLogs: a few hundred bytes each on a small site, 12 KB each at most (60 minutes: about 3 MB at most; 360 minutes: about 17 MB).
Users and admins are identified by their id. A guest is identified by the user_id_tmp cookie when the browser has it.
The cookie holds a random number and is HttpOnly, SameSite=Lax, and Secure on https. A guest without the cookie, and every bot, gets a
keyed hash of the IP and user agent.
Privacy
- The data is pseudonymised, not anonymised. A page path, an IP or a user agent can still point at one person. The page is the path
only, never the query string. A path segment that looks like an identifier is stored as
:token: one that contains@, a UUID, 12 or more digits, or a run of 16 or more letters, digits,_and-with a digit or a capital. Lowercase word slugs without digits are kept. - The visitor hash is keyed with a key derived from
YF_SECRET_KEY. Without it the key comes from the salt in Redis: whoever can read Redis can then test IPs against the ids. The doctor shows a notice in that case. - The IP is
get_ip(). Behind a load balancer or proxy, list it inclient_utils.TRUSTED_PROXIES. OtherwiseX-Forwarded-Foris taken from anyone and can be forged. - The country is read only from the headers named in
COUNTRY_HEADERS(none by default), because a client can send any header.COUNTRY = 'geoip'also asks the GeoIP file whenmain()->USE_GEO_IPis on. - IP, user agent and country are stored only with
TRACK_ONLINE_DETAILS.MASK_IP(off) zeroes the last IPv4 octet or the last 80 IPv6 bits before storing.
Cost per request
One visitor is written at most once every THROTTLE seconds (30), or after PAGE_THROTTLE seconds (5) on
another page. The time of the last write is kept in the session, so a browser's requests in between send nothing to Redis. A visitor
without a session (bots, API clients) first passes a gate key (SET ... NX EX THROTTLE): one round trip, and the write only
when the gate opens. A write is one pipelined round trip of O(log n) commands; entries older than the window are trimmed by score in it
(their details expire with them). Above the cap, the oldest entries and their details are removed under WATCH, so a
visitor written meanwhile is never removed. Ajax and htmx requests refresh the last seen time but keep the page.
The tracker uses a connection of its own with short timeouts (CONNECT_TIMEOUT and READ_TIMEOUT, 0.2 s; the
connect handshake waits at most 1 s) unless the site sets REDIS_TIMEOUT / REDIS_READ_TIMEOUT itself; the admin reads and purge use
ADMIN_READ_TIMEOUT (2 s) on it. When a visitor's write for DOWN_TTL seconds (5): a marker in APCu, or a file in STORAGE_PATH (else a 0700 folder of the process user
in the system temp folder; a marker that is a link is ignored). So
a hung or stopped server costs one request at most a second, and the others nothing.
API
$o = _class('online_users');
$o->_is_online($user_id); // true / false; [id => bool] for an array; 'admin_id' or 'user_id_tmp' as the 2nd argument
$o->counts(); // ['user' => 12, 'guest' => 40, 'admin' => 1, 'bot' => 7], null when Redis does not answer
$o->online_ids('user_id'); // [5, 9, ...] newest first
$o->visitors('user', 100); // rows with path, device, country, ip, ...
$o->track('user_id', $id); // record a visitor yourself (throttled)
$o->forget('user', $id); // remove one (logout does it)
The admin page
The page has cards per type with a sparkline of the last hour, type / device / country filters, a search over page, IP, user agent and
name, and a compact sortable table (table2 in htmx mode) with one line per visitor: a type icon and the name (bots with the icon of
their purpose), flag, country, device and IP, the page, and how long ago the visitor was seen and has been online. From 100 visitors on
(watch_online_users.PAGE_FROM) the table has pages of 50 (PER_PAGE) that load in place and keep the filters.
The i button opens the visitor in a dialog on the same page (htmx; without JavaScript it is a page of its own): every stored field,
the bot's operator, purpose and documentation, and links to the admin pages about that visitor: the account, logins, pages visited
(admins), logins and failed logins from the IP, "Send notification" and Duplicate accounts (?object=check_multi_accounts,
also in the menu under User: the accounts online from the same IP and the matches the user login records; which IP holds which accounts
is shown to super-admins, other admins see the counts). A link shows only when that admin module exists and the
admin may open it. Super-admins can ban or unban the IP there (the banned_ips list, a POST with the session token,
written to the admin wall). A banned IP cannot log in while auth_user / auth_admin BLOCK_BANNED_IPS
is on; the dialog says when it is off. The list takes IPv4 addresses. A ban is refused, with the reason, when IP masking is on (the
stored IP is not the visitor's), for the admin's own IP, for a proxy listed in client_utils.TRUSTED_PROXIES (everybody
behind it would be locked out) and for private or reserved addresses. A row already on the list can always be unbanned. At login a row
matches the IP itself, a wildcard over the whole address (1.* matches 1.2.3.4, not 11.2.3.4;
a bare * row matches nobody, and the site doctor warns about it) or a CIDR mask (10.0.0.0/8).
The page updates itself while the tab is visible at the interval of its refresh control next to "Live update": 5, 10, 15, 30 or 60
seconds or off, 10 seconds by default, kept per admin. It does not update while the visitor dialog is open. Admins the block rules let open watch_online_users
see it (online_users.VIEW_GROUPS narrows that). Purge stored data deletes every online key of the site at once,
including the details of visitors who left earlier (a bounded SCAN of the site's keys). When Redis fails half way, the page
says how many keys were deleted, so you can run it again. It is for super-admins: a POST with
the session token, written to the admin wall. The doctor (group Cache) reports a missing or unreachable Redis server, a missing
YF_SECRET_KEY, settings that do not fit and the old database tables. The admin home has an Online users card.
The ban list in nginx
Optionally the ban list is also written as an nginx include file of deny lines, so that nginx refuses a banned IP
before PHP runs (403 on every page, not only at login). Off by default: Settings, Online users (super-admins), "IP ban list as an nginx
file" (banned_ips_nginx.ENABLED) and "nginx ban file: path" (banned_ips_nginx.FILE). The path defaults to
STORAGE_PATHnginx/banned_ips.conf of the app, for example /var/www/myapp/storage/nginx/banned_ips.conf:
outside the web root (www/), on the app mount, and its directory is created (0755). Another path must be absolute, end
.conf, lie outside this app's web root (and outside the root directories of nginx.conf when PHP can
read it) and in a directory PHP may write to. A file that already exists there must start with the header yf writes
(# Generated by yf from the IP ban list): a file of nginx or of another app is never overwritten. The path is checked when the
setting is saved and again at every write; a saved path that is refused later (a directory made read-only) stays saved, is shown in red on
the Settings page and is not written to until it is fixed. The file is written again after each ban or unban of this page, by Regenerate now in its "nginx ban file" panel
(state, last written, entries, skipped rows; a POST with the session token, written to the admin wall) and by the console:
php .dev/console/yf security:nginx-bans; --check writes nothing and exits 1 when the file is missing, stale or its
path is refused (--json for monitoring). The doctor (group Security) reports a refused path, a missing or stale file and, when
it can read nginx.conf, a file no include line names.
# storage/nginx/banned_ips.conf as written (sorted, one line per address or range)
# Generated by yf from the IP ban list (sys_banned_ips): do not edit, the next change overwrites it.
# 2026-11-15T12:00:00Z, 2 deny, 0 allow, 1 skipped.
# Include it at http level; a server or location with allow / deny rules of its own does not inherit these.
deny 101.34.223.139;
deny 101.34.250.0/24;
What gets written: an IP or a CIDR range (IPv4 and IPv6, host bits cleared; an IPv4-mapped form such as ::ffff:1.2.3.4 as
its IPv4 address), and a wildcard over whole IPv4 octets as its range (1.2.* is 1.2.0.0/16). Skipped, with a log
line and a note in the panel: other wildcards (1.2*, a bare *), anything that is not an address (other
characters are refused, never stripped), a range wider than /8 (IPv4) or /16 (IPv6, MIN_PREFIX_V4 /
MIN_PREFIX_V6; so never 0.0.0.0/0 or ::/0), a range over the IPv4-mapped block, a rule that covers
your own IP, a client_utils.TRUSTED_PROXIES entry or an entry of "nginx ban file: never denied" (ALLOW; with
INCLUDE_ALLOW also written as allow lines first), and private or reserved addresses and ranges (unless
ALLOW_PRIVATE). A rule left out for the writer's own IP is named in the header, so the freshness check gives the same answer
to the console and to every admin. The file is written to a temp file in the same directory and renamed over the old one, never through
a symlink, mode 0644, under a lock (.banned_ips.conf.lock next to it; a write waits at most 3 s for another one), and not at
all when the rules did not change. The login check (BLOCK_BANNED_IPS) still reads the table, so both apply the same list.
yf only writes the file; nginx reads it at its next reload. With PHP and nginx in separate containers that both bind-mount the app
(./www:/var/www), nginx includes the file from that mount at http level, after the realip settings when it is
behind a proxy or CDN (set_real_ip_from / real_ip_header; otherwise nginx sees the proxy, which is never denied).
The access rules of http are inherited only by a server or location without allow /
deny of its own.
http {
include geoip.conf;
include vars.conf;
include gzip.conf;
include /var/www/myapp/storage/nginx/banned_ips.conf;
include limits.conf;
...
}
services:
php:
volumes: [ "./www:/var/www" ] # writes /var/www/myapp/storage/nginx/banned_ips.conf
nginx:
image: yfix/nginx # or any image that reloads nginx when its config changes
volumes: [ "./www:/var/www", "./nginx:/etc/nginx" ]
Use an nginx image that reloads on a config change (the yfix/nginx images do), or reload nginx yourself after a change
(nginx -t && nginx -s reload). Until then nginx keeps
the list it loaded; the first include needs the file to exist, so turn the feature on and press Regenerate now before adding the line.
A note on trust: nginx's master process reads this file, and PHP writes it. Whoever can write the file can add any nginx directive to
it, not only allow / deny. The generator itself only writes allow / deny lines and comments.
Keep the directory outside the web root (the default is), writable by the PHP user only, and use an image that runs nginx -t
before it reloads, so that a broken file is refused rather than taking nginx down.
Bots
Bots are recognised by their user agent from one list, plugins/user/share/bots.php: search engines (Googlebot and its
variants, Bingbot, YandexBot, Baiduspider, DuckDuckBot, Applebot, Sogou, Seznam, Naver Yeti, PetalBot, Qwantbot...), AI crawlers and
assistants (ClaudeBot, Claude-User, Claude-SearchBot, GPTBot, ChatGPT-User, OAI-SearchBot, PerplexityBot, Perplexity-User,
meta-externalagent, Bytespider, Amazonbot, CCBot, DuckAssistBot, MistralAI-User and more), SEO crawlers (AhrefsBot, SemrushBot, MJ12bot,
DotBot...), monitoring (UptimeRobot, Pingdom...), link previews and tools (curl, python-requests). Each entry has the bot's name, its
operator, its purpose (search, AI training, AI search, AI assistant, SEO, monitoring, link preview, ads, feed, archive, tool), the
operator's documentation and the host names of its crawlers. A site adds or overrides entries with
$PROJECT_CONF['bot_registry']['EXTRA'] (same format, checked first). A user agent the list does not know falls back to the
bot keywords of yf_spider_detection. Matching is plain string search on the request: no file read, no network.
Verified bots (opt-in): with online_users.VERIFY_BOTS on, the visitor dialog checks that a bot really comes from its
operator: the IP's reverse DNS name must end with one of the operator's domains (for example .googlebot.com) and resolve back
to the IP. The DNS queries run only when a super-admin opens the dialog, never on a visitor's request, at most 5 per request and 2 seconds
in all; other admins see the cached answer. A definite answer is cached for a day; no reverse name or no forward answer for 2 minutes.
Operators that publish IP lists instead of host names (OpenAI, Anthropic, DuckDuckGo) show "not verifiable by reverse DNS" for now. It
needs the full IP (details on, IP masking off).
Country and GeoIP
The country comes from the request header your proxy sets (COUNTRY_HEADERS), or with COUNTRY = 'geoip' from
a GeoIP lookup. _class('geoip_status') is the one place that knows the GeoIP sources of the server: the web server variable
(GEOIP_COUNTRY_CODE, MM_COUNTRY_CODE of the nginx / Apache geoip2 modules), a country header of a proxy listed in
client_utils.TRUSTED_PROXIES (CF-IPCountry, CloudFront, X-Geo-Country), the PECL geoip extension,
a MaxMind .mmdb file (GeoLite2 / GeoIP2 Country or City in /usr/share/GeoIP/, read by the maxminddb extension or
the geoip2 library) and the legacy .dat file. status() says which work, what is missing and how to fix it;
country($ip) asks them in that order. The site doctor shows it (group Environment, "GeoIP: country of a visitor").
Upgrading from the database storage
Older versions wrote to the users_online, users_online_details and sys_online tables. These
tables are no longer created, read or written. Existing tables still hold personal data of past visitors: session ids, full URLs with
query strings, IPs and user agents. Dropping them is a privacy step. Run make check-site SITE=/path (pattern
online_db) first to find site code that still reads them. main()->_online_users is no longer filled. The
settings _ONLINE_TTL, _CACHE_UPDATE_TTL, ONLINE_AUTO_CLEAN and ONLINE_MAX_IDS have no
effect. Use WINDOW and THROTTLE instead.