Manticore Search
Manticore Search is a full-text search server; the search plugin talks
to it with the manticore driver of _class('wrapper_search'). The admin page Tools / Manticore
(?object=manage_manticore) shows how the server is doing and holds the everyday management: the tables and their size,
optimize, flush, truncate, drop, creating the tables the site needs, a query console, the threads with kill, the variables with the
risky ones marked, and the replication clusters. The engine is _class('admin_manticore') (yf_admin_manticore,
plugin search). No client library is used: the page speaks SQL to Manticore over the MySQL protocol (port 9306) with the
PHP mysqli extension.
Which server
The page uses the search plugin's server, first match wins:
$PROJECT_CONF['admin_manticore']['HOST'](for the page only),$PROJECT_CONF['search_driver_manticore']['HOST'],- the
SPHINX_HOST/SPHINX_PORTconstants (Manticore is a drop-in for Sphinx), 127.0.0.1:9306.
$PROJECT_CONF['wrapper_search']['driver'] = 'manticore';
$PROJECT_CONF['search_driver_manticore']['HOST'] = '127.0.0.1:9306';
The menu item shows while the site names a Manticore server (one of the settings above, or the manticore search driver).
The page first checks that the port answers, so a stopped server costs one connect timeout (2 s) and no PHP warning. Error messages
never show passwords or user:password@ parts.
The page
- Overview: uptime and version, queries (per second, average time, searches and writes), connections and the refused ones
(
maxed_out), the thread pool (busy workers, clients, queue, load), the query cache and the tables by type. The ⟳ button refreshes the cards every 5 to 60 seconds; it is off until you choose an interval. Below: the components (SHOW VERSION), the checks, the site's index definitions that have no table, and clusters that are not healthy. - Tables: every table (
SHOW TABLES) with its type (rt, plain, percolate, distributed), documents, size (RAM and disk), RAM chunk and disk chunks, read fromSHOW TABLE ... STATUS; sort by any column, filter by name, 50 a page. A warning sign marks a table with many disk chunks or many deleted documents (optimize it). A table's page shows its status, its settings (SHOW TABLE ... SETTINGS), its columns (DESCRIBE) andSHOW CREATE TABLE. - Console: one SQL statement at a time, results in a table with the time it took and
SHOW METAafter aSELECT. - Threads:
SHOW THREADS(the idle ones too on request) with Kill on each connection. - Variables:
SHOW VARIABLESandSHOW SETTINGS(the configuration searchd runs with), the risky values highlighted. - Cluster (only with replication): each cluster from
SHOW STATUS LIKE 'cluster%': status, size, node state, its tables, nodes and queues; a node that is not in the primary component or not synced is reported.
Table actions
| Action | Statement | Tables | Confirm |
|---|---|---|---|
| Optimize | OPTIMIZE TABLE t: merges the disk chunks in the background and frees deleted documents | rt | a click and a confirm |
| Flush RAM chunk | FLUSH RAMCHUNK t: writes the RAM chunk to a new disk chunk | rt | a click and a confirm |
| Truncate | TRUNCATE TABLE t (cluster:t for a replicated table): deletes every document, keeps the table | rt, percolate | the table name typed again |
| Drop | DROP TABLE t: off unless ALLOW_DROP; a replicated table must leave its cluster first | all | the table name typed again |
| Create | CREATE TABLE from the site's definition, through the search driver's create_index() | missing ones | a click and a confirm |
| Kill | KILL id of a connection (the ConnID of SHOW THREADS) | - | a click and a confirm |
Table names must match [A-Za-z_][A-Za-z0-9_]* (at most 64 characters) before they reach a statement, and are sent in
backticks. Every change is a POST with the session token; a plain link (GET) only shows a confirm form, a script (ajax) gets 403.
The table and the connection id are read from the POST body only, never from the link. Each change writes a row to the admin wall
(Logs / Admin / Activity) naming the table and the server, before it runs; a failed one writes a second row with the error.
The site's index definitions
List the indexes the site needs in wrapper_search, with the schema of create_index(). Search does not read
it; the Tables tab and the overview offer Create for each one the server lacks, and the doctor reports them.
$PROJECT_CONF['wrapper_search']['INDEXES'] = [
'products' => ['fields' => ['title' => 'text', 'body' => 'text', 'cat_id' => 'int', 'price' => 'float', 'tag' => 'keyword'], 'suggest' => true],
];
Query console
Reads run at once: SELECT, SHOW, DESCRIBE / DESC, EXPLAIN,
CALL KEYWORDS, CALL SUGGEST, CALL QSUGGEST, CALL SNIPPETS and CALL PQ.
Anything else is a write. Writes are on (CONSOLE_WRITE, the default) while the page is not read-only: the page shows the
statement and runs it only after you confirm, and writes it to the admin wall first. DROP also needs ALLOW_DROP.
With CONSOLE_WRITE off the console runs reads only.
- One statement a run: comments are taken out, one trailing
;is fine, a;between two statements is refused. Strings are single-quoted; a double quote outside a string is refused (write a phrase inside the string:MATCH('"red boots"')). - Server files: a statement that mentions
load_files(CALL SNIPPETSwithload_filesorload_files_scatteredreads any file the Manticore server can read) is refused unlessALLOW_LOAD_FILESis on; then it is a write (confirmed and audited). - Rows: at most
CONSOLE_MAX_ROWS(1000) are shown; a read with a largerLIMIT, a subquery's too, is refused. Without aLIMITManticore returns 20 rows. - Time: a
SELECTwithoutmax_query_timegetsOPTION max_query_time=10000(CONSOLE_TIMEOUT_MS), except withFACET; amax_query_timeof 0 or above it is lowered to it. - Every value is shown as text; long values are cut to 1000 characters.
Checks
| Check | Why it matters |
|---|---|
searchd.listen on a public address (warning) or on every interface (notice) | Manticore has no passwords: whoever reaches the port can read, change and drop every table. Listen on 127.0.0.1 or a private address, or firewall the ports. |
binlog_path empty | Without the binary log, RT documents written since the last flush are lost on a crash. |
auto_optimize=0 | Disk chunks pile up and searches get slower. |
log_level debug | A lot of log output. |
secondary_indexes=0 | Attribute filters scan every document. |
maxed_out above 0 | Connections were refused because every worker was busy. |
Disk chunks above twice optimize_cutoff, 20 % or more deleted documents (Tables tab) | Optimize the table. |
The site doctor (group Database) runs these checks too while a Manticore server is configured: the server answers (critical when the site search uses Manticore), its version (Manticore 6 or newer; a Sphinx server is reported), the tables of the index definitions, and the risky settings. The admin home has a Manticore Search card for super-admins.
Access and settings
Super-admins (group 1) only; a block rule does not open the page to other groups. Settings page, tab Search:
$PROJECT_CONF['manage_manticore'] = [
'READ_ONLY' => false, // true: no change at all (the read-only console still works)
'ALLOW_DROP' => false, // offer DROP TABLE (name typed again) and DROP in the console write mode
'CONSOLE_WRITE' => true, // the console may run writes, each confirmed first; false: reads only
'ALLOW_LOAD_FILES' => false, // allow load_files (CALL SNIPPETS reading server files) as a console write
'CONSOLE_MAX_ROWS' => 1000,
'CONSOLE_TIMEOUT_MS' => 10000, // max_query_time added to a SELECT, and its cap; 0 = none
'PER_PAGE' => 50, // tables a page
'REFRESH_DEFAULT' => 0, // seconds between card refreshes until an admin chooses (0 = off)
'MENU_ITEM_NEEDS_CONFIG' => true, // false: the menu item always shows
];
$PROJECT_CONF['admin_manticore'] = [
'HOST' => '', // host:port for this page only; '' = the search plugin's
'CONNECT_TIMEOUT' => 2,
'TABLES_STATUS_MAX' => 300, // tables whose status is read for the list
];
The page works with Sphinx 3 in part (the status, the console, the threads); it says so when the server is not Manticore. The HTTP JSON API (port 9308) is not used.