SEO
The admin page Tools / SEO (?object=manage_seo) looks at the site the way a search engine does: it requests the site's own
pages over HTTP, reads what they say to search engines and lists what to improve, each finding with a severity (critical, warning, notice), the
pages it concerns and how to fix it. Safe fixes can be applied from the page, one tick each, with a ledger and an undo. Nothing changes on a
site until a super-admin acts: the page, the doctor checks and the admin home card only read the result of the last scan.
Who. Every admin the block rules let open manage_seo sees the last scan, its pages, the performance results, the fixes on
offer and the fix ledger (yf_seo::$VIEW_GROUPS narrows it to some admin groups). Scans, fixes, undo, Lighthouse and PageSpeed
Insights are for super-admins (group 1) only.
Scanning
- What is requested. The home page, the URLs of
/sitemap.xml, then the links found, breadth first, up to the chosen number of pages (10 to 500, defaultyf_seo::$MAX_PAGES50) and withinTIME_BUDGET_MS(20 s); a scan stopped by a limit says so. A gzip sitemap may unpack to 20 MB at most (GUNZIP_MAX); a sitemap with a NUL byte, a UTF-16 or UTF-32 byte order mark (a UTF-8 one is fine) or a DOCTYPE is refused. robots.txt, the sitemap, an unknown URL (soft 404),http://of an https site, the www / non-www twin and one page with and without its trailing slash are asked too. - Only this site. The site URL is
$PROJECT_CONF['seo']['BASE_URL'], else the trusted base URL of the doctor's network checks (theirBASE_URL, orWEB_PATHwhen the site sets it or its host is inDOMAINS; never the Host header of a request alone). Links to other hosts are counted, never requested; more host names of the site go inyf_seo::$HOSTS. - Like a visitor. No cookies, no session, only GET, the User-Agent
yf-seo. Links whose path or query matchyf_seo::$PATH_DENY(logout, delete, vote, cart, ...), the admin path, files (images, PDF, archives) andrel="nofollow"links are not followed; neither is a redirect to a denied path, a sitemap URL on a denied path or an asset on a denied path (a pattern that fails counts as denied). - The one exception to "this host only" is the www / non-www twin of the base host, asked once without following, and only when the
host name is public (not an IP address, not
localhost,.local,.test,.internal...) and the twin resolves to public addresses only (no private, loopback or link-local address); the request goes to the address that was checked. - Bounded. Every request has a hard time limit (
NET_TIMEOUT_MS) and a byte cap (MAX_BYTES); redirects are followed to this site only (MAX_REDIRECTS); one scan runs at a time. - Cached. The result (findings and a compact record of each page, no page bodies) is kept in
STORAGE_PATH/seo/with the scan before it, so the page shows what is new and what is gone. The folder gets a deny-all.htaccess, which only Apache reads: on nginx (or any other server)STORAGE_PATHmust lie outside the web root or be denied in the server config, as for the doctor's reports.
Indexing and robots
A production page with noindex (robots meta tag or X-Robots-Tag) is critical; a development or staging copy that
search engines may index is a warning (production is the doctor's environment setting). robots.txt: missing, answered with an HTML page,
Disallow: / on production, no Sitemap: line, sitemap URLs that robots.txt blocks. The framework's robots.txt is the
robotstxt module: it allows indexing only on the hosts of main()->PRODUCTION_DOMAIN; the setting
robotstxt.SITEMAP_LINE (Settings, tab SEO, off by default) adds the Sitemap: line there.
Sitemap
Missing, not XML, empty, URLs of another host, URLs that do not answer 200. The framework's sitemap is the site_map module
(site_map.SITE_MAP_ENABLED, Settings tab SEO, off by default): modules add their pages with _hook_sitemap().
Titles, descriptions and headings
Missing, several, too short or too long, the same on several pages (pages whose canonical names another URL are left out). A module sets the
title with _hook_title() and the meta tags with _hook_meta() (keys description, canonical,
og_title ...); static pages and news have their own fields. One <h1> per page.
Canonical URLs and languages
Canonical: missing (notice), several, to another host, to a page that does not answer 200. A multi-language site (pages in several
languages, ?lang= links or any hreflang) is checked for hreflang links: missing, invalid codes, no x-default, no link
to itself, a target that does not link back.
Links, redirects and 404
Internal links to pages that answer an error, links to redirects, redirect chains and loops, pages that answer 5xx, and the soft 404: an
unknown URL that answers 200 (turn on core_errors.ERROR_PAGES or let the site answer 404).
A public site gets requests from scanners for paths it never had (/wp-admin/..., /.env). Each one answers 404
and, by default, also writes the warning yf_core_blocks: Task not found to the error log, where it hides real errors. Set
$PROJECT_CONF['graphics']['NOT_FOUND_RAISE_WARNING'] = false; in the site config to answer the 404 without the log row.
Images, social cards and structured data
Images without an alt attribute (alt="" is right for a decorative image; the page view suggests text from the
file names), without width and height, all loaded at once. OpenGraph (og:title, og:description,
og:image) and twitter:card. JSON-LD blocks must be JSON with @context and @type; the home
page should have one (Organization or WebSite).
HTTPS and hosts
The site URL on http, http not redirecting to https, mixed content (http scripts and styles on an https page are critical), both www and non-www answering, a page answering with and without the trailing slash.
Language and content
No <html lang>, no viewport meta tag, pages with the same text under several URLs, pages with fewer than 50 words.
Performance
Lite audit (every scan, no browser): the home page's HTML size and compression, its requests (assets of other hosts are counted, not
fetched), the weight with the assets of this site, render-blocking scripts and stylesheets in <head>, images over 200 KB,
text answers without gzip / brotli, static assets without cache headers, slow answers, inline code; per page the HTML size and the blocking
assets.
Lighthouse (Apache-2.0, Google): the paint metrics and scores of a real browser. It needs Node, Chrome or Chromium and the
lighthouse program on the web server (npm install -g lighthouse); the framework never installs it. Chrome loads the page
like a visitor's browser: it runs the page's JavaScript, third-party scripts included, with network access from the web server. Turn it on
only for a site whose pages you trust. The Performance tab
says what it finds (a file lookup; nothing is run for that). A super-admin may run it on the site URL or a scanned page when
seo.LIGHTHOUSE_ALLOW is on (Settings, tab SEO): it runs as an argument list without a shell, with a timeout
(LIGHTHOUSE_TIMEOUT) after which the whole process group (Chrome too) is killed (with setsid and the posix extension;
without them GNU timeout signals the group, and without either only the program itself is stopped), one run at a time, and keeps the scores and main metrics of the last runs. LIGHTHOUSE_BIN,
CHROME_PATH and CHROME_FLAGS set the programs and the browser flags (no --no-sandbox unless the server
needs it).
PageSpeed Insights (Google API, opt-in): Google runs Lighthouse on its own servers and adds the speed real visitors had (Chrome UX
Report). The URL of the page is sent to Google. It is off by default (seo.PAGESPEED_ALLOW, Settings tab SEO, super-admin,
typed confirmation) and needs an API key in the site config, never saved by a page and shown masked:
$PROJECT_CONF['seo']['PAGESPEED_KEY'] = 'AIza...'; // Google Cloud console: PageSpeed Insights API
Only a super-admin's button sends a request, only for a production site, and only a URL of the site's own public host name (the base URL
or a scanned page); the key never goes into the results, messages or the audit. PAGESPEED_STRATEGY picks mobile or desktop.
Fixes
The Fixes tab lists what can be fixed now, each with its own tick; nothing is applied without it.
- Settings the scan says are missing on a production site (the robots.txt Sitemap line, sitemap generation): saved through the Settings page's save (its checks and backup), as the doctor's quick fixes are; settings for super-admins only or with a typed confirmation stay on their own pages.
- robots.txt file: when robots.txt is a static file in the web root, the
Sitemap:line is appended to it (not a symbolic link, at most 64 KB) under a lock, so two requests never mix; the previous text is kept for the undo. The file is written as a new file and renamed over the old one: its mode stays, but it then belongs to the PHP user, and ACLs or extended attributes of the old file are not kept. - Empty meta fields of static pages (meta description, page title) and news (meta description): a suggestion from the row's own text
(tags and braces taken out, cut at a word near 155 characters), which can be edited before applying. A field is written only while it is
still empty (the condition is part of the
UPDATE): a value never gets overwritten. Fields with nothing to suggest from take typed text.
Every fix goes to the fix ledger (who, when, what before, what after) and the admin wall. Undo puts the value back only while it is still what the fix wrote: a field or file edited since is kept.
Safety
- Viewing for the admins the block rules let in; every scan, fix, undo, Lighthouse and PageSpeed Insights run is for super-admins (group 1), a POST with the session token.
- Crawled text (titles, descriptions, URLs) is shown escaped, template braces included.
- Content writes use whitelisted tables and columns (
yf_seo::$META_TARGETS), integer ids and escaped values. - XML is parsed without network access and refused with a DOCTYPE; HTML is parsed with DOMDocument, which loads nothing.
Elsewhere
The site doctor has an SEO group: a production site with ROBOTS_NO_INDEX on (critical), the problems of the last scan, and the
two settings above as quick fixes. The admin home has an SEO card with the score of the last scan.