Project | Requirement |
TencentDB for PostgreSQL version | 15.x or later |
Extension version | tencentdb_wal_stat v1.0 |
Execution permission | Members of the pg_tencentdb_superuser role |
Execution environment | Executable only on the Primary (Primary). |
CREATE EXTENSION IF NOT EXISTS tencentdb_wal_stat;
SELECT extname, extversion FROM pg_extensionWHERE extname = 'tencentdb_wal_stat';
extname | extversion |
tencentdb_wal_stat | 1.0 |
tencentdb_wal_stat(wal_num integer)RETURNS SETOF record
Parameter | Type | Scope | Description |
wal_num | integer | 1 ~ 500 | The number of WAL files to analyze. A larger value indicates a broader analysis scope. |
Field | Type | Description |
database_oid | oid | Database OID (0 for shared objects and system-level WAL). |
database_name | text | Database name. |
schema_oid | oid | Schema OID |
schema_name | text | Schema name (resolvable only within the current database and shared catalog). |
relation_oid | oid | Relation (table/index) OID. |
relation_name | text | Relation name (displayed as <filenode:N> for deleted objects). |
relation_kind | text | Relation type: table, index, toast, sequence, matview, shared_table, shared_index, system, unknown. |
rmgr_name | text | WAL resource manager name (such as Heap, Btree, Transaction, and so on). |
wal_records | bigint | Number of WAL records. |
wal_bytes | bigint | WAL bytes (excluding FPI). |
wal_fpi | bigint | Number of Full Page Image (FPI) records. |
wal_fpi_bytes | bigint | FPI bytes. |
-- Summarize WAL generation volume by database (analyzing the latest 50 WAL files)SELECT database_name,SUM(wal_records) AS total_records,pg_size_pretty(SUM(wal_bytes + wal_fpi_bytes)::bigint) AS total_size,ROUND(SUM(wal_bytes + wal_fpi_bytes) * 100.0 /SUM(SUM(wal_bytes + wal_fpi_bytes)) OVER (), 1) AS pctFROM tencentdb_wal_stat(50)GROUP BY database_nameORDER BY SUM(wal_bytes + wal_fpi_bytes) DESC;
database_name | total_records | total_size | pct |
func_veri | 447033 | 49 MB | 79.6 |
postgres | 53164 | 7812 KB | 12.3 |
business_analysis | 8 | 49 KB | 0.1 |
monitor | 7 | 44 KB | 0.1 |
instance | 7 | 44 KB | 0.1 |
pg_instance | 14 | 34 KB | 0.1 |
-- Summarize by schema dimensionSELECT database_name, schema_name,SUM(wal_records) AS total_records,pg_size_pretty(SUM(wal_bytes + wal_fpi_bytes)::bigint) AS total_sizeFROM tencentdb_wal_stat(50)WHERE database_name = 'func_veri' -- Replace with the target database nameAND schema_name IS NOT NULL AND schema_name != ''GROUP BY database_name, schema_nameORDER BY SUM(wal_bytes + wal_fpi_bytes) DESC;
database_name | schema_name | total_records | total_size |
func_veri | public | 185846 | 19 MB |
func_veri | pg_catalog | 9110 | 2230 KB |
func_veri | test_schema | 20521 | 1443 KB |
func_veri | pg_toast | 190 | 179 KB |
-- View the top 10 tables that generate the most WALSELECT database_name, schema_name, relation_name, relation_kind,wal_records,pg_size_pretty(wal_bytes::bigint) AS wal_size,pg_size_pretty(wal_fpi_bytes::bigint) AS fpi_size,pg_size_pretty((wal_bytes + wal_fpi_bytes)::bigint) AS total_sizeFROM tencentdb_wal_stat(50)WHERE relation_kind IN ('table', 'index')ORDER BY (wal_bytes + wal_fpi_bytes) DESCLIMIT 10;
database_name | schema_name | relation_name | relation_kind | wal_records | wal_size | fpi_size | total_size |
func_veri | public | idx_test_orders | table | 50000 | 4932 KB | 0 bytes | 4932 KB |
func_veri | public | idx_test_products | table | 30000 | 3757 KB | 0 bytes | 3757 KB |
func_veri | public | idx_test_orders_pkey | index | 50411 | 3226 KB | 0 bytes | 3226 KB |
func_veri | public | idx_test_orders_customer_id_status_id_product_name_amount_o_idx | index | 444 | 21 KB | 3166 kB | 3187 KB |
func_veri | public | idx_test_products_pkey | index | 30246 | 1935 KB | 0 bytes | 1935 KB |
postgres | pg_catalog | pg_attribute | table | 2475 | 1023 KB | 142 kB | 1165 KB |
func_veri | public | idx_test_customers | table | 10000 | 1034 KB | 0 bytes | 1034 KB |
func_veri | test_schema | orders | table | 10000 | 732 KB | 0 bytes | 732 KB |
func_veri | test_schema | orders_pkey | index | 10084 | 645 KB | 0 bytes | 645 KB |
func_veri | public | idx_test_customers_pkey | index | 10084 | 645 KB | 0 bytes | 645 KB |
-- View WAL distribution by RMGR typeSELECT rmgr_name,SUM(wal_records) AS total_records,pg_size_pretty(SUM(wal_bytes)::bigint) AS wal_size,ROUND(SUM(wal_records) * 100.0 / SUM(SUM(wal_records)) OVER (), 1) AS pctFROM tencentdb_wal_stat(50)GROUP BY rmgr_nameORDER BY SUM(wal_records) DESC;
rmgr_name | total_records | wal_size | pct |
Btree | 246781 | 16 MB | 44.9 |
Heap | 229351 | 25 MB | 41.7 |
Heap2 | 46408 | 3726 KB | 8.4 |
Generic | 11119 | 4349 KB | 2.0 |
Transaction | 7120 | 2015 KB | 1.3 |
Sequence | 6288 | 608 KB | 1.1 |
Standby | 1737 | 73 KB | 0.3 |
XLOG | 1163 | 49 KB | 0.2 |
Storage | 116 | 4912 bytes | 0.0 |
CLOG | 1 | 30 bytes | 0.0 |
-- FPI proportion analysisSELECTpg_size_pretty(SUM(wal_bytes)::bigint) AS wal_data_size,pg_size_pretty(SUM(wal_fpi_bytes)::bigint) AS fpi_size,pg_size_pretty(SUM(wal_bytes + wal_fpi_bytes)::bigint) AS total_size,ROUND(SUM(wal_fpi_bytes) * 100.0 /NULLIF(SUM(wal_bytes + wal_fpi_bytes), 0), 1) AS fpi_pctFROM tencentdb_wal_stat(50);
wal_data_size | fpi_size | total_size | fpi_pct |
51MB | 11MB | 62MB | 17.5 |
Step 1: View the overall WAL generation overview.└── tencentdb_wal_stat(50) aggregated by database│Step 2: Locate the target database.└── Summarize by schema to locate abnormal schemas.│Step 3: Drill down to the table level.└── View the top 10 tables by relation.│Step 4: Analyze the cause.├── A high proportion of Heap type → A large volume of INSERT/UPDATE/DELETE operations├── A high proportion of B-tree type → High index maintenance overhead└── A high FPI proportion → Excessive checkpoint frequency│Step 5: Take measures.├── Limit the write rate for abnormal tenants.├── Suspend non-core batch tasks.└── Optimize the index policy.
-- Record the WAL generation volume for each database hourly (can be written to a monitoring table)INSERT INTO wal_monitor_log (check_time, database_name, wal_records, wal_bytes)SELECT now(), database_name,SUM(wal_records), SUM(wal_bytes + wal_fpi_bytes)FROM tencentdb_wal_stat(10)GROUP BY database_name;
Scenario | Recommended wal_num | Description |
Emergency troubleshooting | 5 ~ 10 | Return quickly to view the most recent WAL sources. |
Daily Inspection | 20 ~ 50 | Covers a longer time range, making the data more representative. |
Deep Analysis | 100 ~ 200 | Use when a comprehensive understanding of WAL distribution is required. |
Term | Description |
WAL | Write-Ahead Logging (WAL). PostgreSQL ensures transaction durability and crash recovery by writing logs before data files. |
LSN | log sequence number (LSN). Uniquely identifies a location within the WAL. |
RMGR | Resource Manager. WAL is categorized by operation type, such as Heap (table data), Btree (B-tree index), and Transaction. |
FPI | Full Page Image (FPI). When a data page is modified for the first time after a checkpoint, its full content is written to the WAL to avoid partial writes during crash recovery. |
Replication Slot | Replication Slot. Retains WAL logs until downstream subscribers finish consumption. If downstream consumption is slow, WAL cannot be purged. |
<oid:N> or <filenode:N>. This is normal because catalogs cannot be resolved across databases.フィードバック