tencent cloud

TencentDB for PostgreSQL

WAL Log Source Analysis (tencentdb_wal_stat)

ダウンロード
フォーカスモード
フォントサイズ
最終更新日: 2026-07-02 17:00:34
TencentDB for PostgreSQL provides the WAL log source analysis extension. This document introduces the description and practices for the WAL log source analysis (tencentdb_wal_stat) extension.

Overview

tencentdb_wal_stat is a WAL log source analysis extension provided by TencentDB for PostgreSQL. It supports online statistics of WAL log generation volume across three dimensions: database, schema, and table. This helps you quickly identify objects that generate a large amount of WAL, enabling timely intervention to prevent disk space from reaching critical levels.
Core Value:
Precise Location: Drill down layer by layer from database to schema to table to quickly identify the major WAL generators.
Online Analysis: It is invoked directly via SQL, eliminating the need for offline parsing of WAL files.
Multi-dimensional Aggregation: It simultaneously displays detailed information such as RMGR types (Heap/Btree/Transaction) and FPI statistics.
Production Safety: It performs read-only analysis, which does not affect business operations.
Typical Scenarios:
In a SaaS multi-tenant environment, a sudden surge of writes from a specific tenant can cause WAL accumulation.
When a backlog occurs at the logical replication consumer end, you need to identify the table with the highest WAL generation volume for throttling.
Monitor WAL generation trends during routine Ops to prevent disk space issues.

Environment Requirements

Project
Requirement
TencentDB for PostgreSQL version
15.x or later
Extension version
tencentdb_wal_stat v1.0
Execution permission
Members of the pg_tencentdb_superuser role
Execution environment
Executable only on the Primary (Primary).

Installation and Verification

Installing an Extension

CREATE EXTENSION IF NOT EXISTS tencentdb_wal_stat;

Verifying the Installation

Input:
SELECT extname, extversion FROM pg_extension
WHERE extname = 'tencentdb_wal_stat';
Output:
extname
extversion
tencentdb_wal_stat
1.0

Function Description

Functions

tencentdb_wal_stat(wal_num integer)
RETURNS SETOF record
Parameters:
Parameter
Type
Scope
Description
wal_num
integer
1 ~ 500
The number of WAL files to analyze. A larger value indicates a broader analysis scope.
Note:
A larger wal_num value results in a longer analysis time. It is recommended to perform large-scale analysis during off-peak business hours.

Output Fields

Field
Type
Description
database_oid
oid
Database OID (0 for shared objects and system-level WAL).
database_name
text
Database name.
schema_oid
oid
Schema OID
schema_name
text
Schema name (resolvable only within the current database and shared catalog).
relation_oid
oid
Relation (table/index) OID.
relation_name
text
Relation name (displayed as <filenode:N> for deleted objects).
relation_kind
text
Relation type: table, index, toast, sequence, matview, shared_table, shared_index, system, unknown.
rmgr_name
text
WAL resource manager name (such as Heap, Btree, Transaction, and so on).
wal_records
bigint
Number of WAL records.
wal_bytes
bigint
WAL bytes (excluding FPI).
wal_fpi
bigint
Number of Full Page Image (FPI) records.
wal_fpi_bytes
bigint
FPI bytes.
Note:
The total WAL size = wal_bytes + wal_fpi_bytes. FPI is a full page replica written when a page is modified for the first time after a checkpoint, typically accounting for 20% to 40% of the total WAL volume.

Common Query Scenarios

Scenario 1: Disk Alarm - Quickly Locating the Database Generating the Most WAL

Input:
-- Summarize WAL generation volume by database (analyzing the latest 50 WAL files)
SELECT database_name,
SUM(wal_records) AS total_records,
pg_size_pretty(SUM(wal_bytes + wal_fpi_bytes)::bigint) AS total_size,
ROUND(SUM(wal_bytes + wal_fpi_bytes) * 100.0 /
SUM(SUM(wal_bytes + wal_fpi_bytes)) OVER (), 1) AS pct
FROM tencentdb_wal_stat(50)
GROUP BY database_name
ORDER BY SUM(wal_bytes + wal_fpi_bytes) DESC;
Output:
database_name
total_records
total_size
pct
func_veri
447033
49 MB
79.6
postgres
53164
7812 KB
12.3
business_analysis
8
49 KB
0.1
monitor
7
44 KB
0.1
instance
7
44 KB
0.1
pg_instance
14
34 KB
0.1

Scenario 2: After Locating the Database, Further Identifying the Schema Generating the Most WAL

Input:
-- Summarize by schema dimension
SELECT database_name, schema_name,
SUM(wal_records) AS total_records,
pg_size_pretty(SUM(wal_bytes + wal_fpi_bytes)::bigint) AS total_size
FROM tencentdb_wal_stat(50)
WHERE database_name = 'func_veri' -- Replace with the target database name
AND schema_name IS NOT NULL AND schema_name != ''
GROUP BY database_name, schema_name
ORDER BY SUM(wal_bytes + wal_fpi_bytes) DESC;
Output:
database_name
schema_name
total_records
total_size
func_veri
public
185846
19 MB
func_veri
pg_catalog
9110
2230 KB
func_veri
test_schema
20521
1443 KB
func_veri
pg_toast
190
179 KB

Scenario 3: Identifying the Top 10 Tables by WAL Generation at the Table Level

Input:
-- View the top 10 tables that generate the most WAL
SELECT database_name, schema_name, relation_name, relation_kind,
wal_records,
pg_size_pretty(wal_bytes::bigint) AS wal_size,
pg_size_pretty(wal_fpi_bytes::bigint) AS fpi_size,
pg_size_pretty((wal_bytes + wal_fpi_bytes)::bigint) AS total_size
FROM tencentdb_wal_stat(50)
WHERE relation_kind IN ('table', 'index')
ORDER BY (wal_bytes + wal_fpi_bytes) DESC
LIMIT 10;
Output:
database_name
schema_name
relation_name
relation_kind
wal_records
wal_size
fpi_size
total_size
func_veri
public
idx_test_orders
table
50000
4932 KB
0 bytes
4932 KB
func_veri
public
idx_test_products
table
30000
3757 KB
0 bytes
3757 KB
func_veri
public
idx_test_orders_pkey
index
50411
3226 KB
0 bytes
3226 KB
func_veri
public
idx_test_orders_customer_id_status_id_product_name_amount_o_idx
index
444
21 KB
3166 kB
3187 KB
func_veri
public
idx_test_products_pkey
index
30246
1935 KB
0 bytes
1935 KB
postgres
pg_catalog
pg_attribute
table
2475
1023 KB
142 kB
1165 KB
func_veri
public
idx_test_customers
table
10000
1034 KB
0 bytes
1034 KB
func_veri
test_schema
orders
table
10000
732 KB
0 bytes
732 KB
func_veri
test_schema
orders_pkey
index
10084
645 KB
0 bytes
645 KB
func_veri
public
idx_test_customers_pkey
index
10084
645 KB
0 bytes
645 KB

Scenario 4: Analyzing WAL Type Distribution (Heap Writes vs. Index Writes)

Input:
-- View WAL distribution by RMGR type
SELECT rmgr_name,
SUM(wal_records) AS total_records,
pg_size_pretty(SUM(wal_bytes)::bigint) AS wal_size,
ROUND(SUM(wal_records) * 100.0 / SUM(SUM(wal_records)) OVER (), 1) AS pct
FROM tencentdb_wal_stat(50)
GROUP BY rmgr_name
ORDER BY SUM(wal_records) DESC;
Output:
rmgr_name
total_records
wal_size
pct
Btree
246781
16 MB
44.9
Heap
229351
25 MB
41.7
Heap2
46408
3726 KB
8.4
Generic
11119
4349 KB
2.0
Transaction
7120
2015 KB
1.3
Sequence
6288
608 KB
1.1
Standby
1737
73 KB
0.3
XLOG
1163
49 KB
0.2
Storage
116
4912 bytes
0.0
CLOG
1
30 bytes
0.0
Note:
A high proportion of Btree and Heap types indicates that table writes (INSERT/UPDATE/DELETE) and their associated index maintenance are the primary source of WAL. If the Btree proportion is abnormally high, consider reducing unnecessary indexes.

Scenario 5: Viewing the FPI (Full Page Image) Ratio

Input:
-- FPI proportion analysis
SELECT
pg_size_pretty(SUM(wal_bytes)::bigint) AS wal_data_size,
pg_size_pretty(SUM(wal_fpi_bytes)::bigint) AS fpi_size,
pg_size_pretty(SUM(wal_bytes + wal_fpi_bytes)::bigint) AS total_size,
ROUND(SUM(wal_fpi_bytes) * 100.0 /
NULLIF(SUM(wal_bytes + wal_fpi_bytes), 0), 1) AS fpi_pct
FROM tencentdb_wal_stat(50);
Output:
wal_data_size
fpi_size
total_size
fpi_pct
51MB
11MB
62MB
17.5
Note:
If the FPI proportion exceeds 40%, you can consider increasing the max_wal_size to reduce the checkpoint frequency, thereby decreasing FPI generation.

Ops Practical Guide

WAL Accumulation Emergency Handling Process

When you receive a disk space alarm or detect WAL accumulation, follow the steps below to troubleshoot.
Step 1: View the overall WAL generation overview.
└── tencentdb_wal_stat(50) aggregated by database
Step 2: Locate the target database.
└── Summarize by schema to locate abnormal schemas.
Step 3: Drill down to the table level.
└── View the top 10 tables by relation.
Step 4: Analyze the cause.
├── A high proportion of Heap type → A large volume of INSERT/UPDATE/DELETE operations
├── A high proportion of B-tree type → High index maintenance overhead
└── A high FPI proportion → Excessive checkpoint frequency
Step 5: Take measures.
├── Limit the write rate for abnormal tenants.
├── Suspend non-core batch tasks.
└── Optimize the index policy.

Daily Monitoring Recommendations

It is recommended to periodically run the following query to record the WAL generation trend:
-- Record the WAL generation volume for each database hourly (can be written to a monitoring table)
INSERT INTO wal_monitor_log (check_time, database_name, wal_records, wal_bytes)
SELECT now(), database_name,
SUM(wal_records), SUM(wal_bytes + wal_fpi_bytes)
FROM tencentdb_wal_stat(10)
GROUP BY database_name;

Recommendations for Selecting the wal_num Parameter

Scenario
Recommended wal_num
Description
Emergency troubleshooting
5 ~ 10
Return quickly to view the most recent WAL sources.
Daily Inspection
20 ~ 50
Covers a longer time range, making the data more representative.
Deep Analysis
100 ~ 200
Use when a comprehensive understanding of WAL distribution is required.
Note:
A larger wal_num value results in a longer analysis time. It is recommended to perform large-scale analysis during off-peak business hours.

Definitions

Term
Description
WAL
Write-Ahead Logging (WAL). PostgreSQL ensures transaction durability and crash recovery by writing logs before data files.
LSN
log sequence number (LSN). Uniquely identifies a location within the WAL.
RMGR
Resource Manager. WAL is categorized by operation type, such as Heap (table data), Btree (B-tree index), and Transaction.
FPI
Full Page Image (FPI). When a data page is modified for the first time after a checkpoint, its full content is written to the WAL to avoid partial writes during crash recovery.
Replication Slot
Replication Slot. Retains WAL logs until downstream subscribers finish consumption. If downstream consumption is slow, WAL cannot be purged.

Must-Knows

Available only on the primary database: An error will be reported if the command is executed on a standby database because the standby database lacks independent WAL writes.
Permission Requirement: Membership in the pg_tencentdb_superuser role is required.
Parameter Range: The wal_num value must be between 1 and 500. An error will be reported if the value is out of this range.
Name Resolution: Object names from other databases may appear as <oid:N> or <filenode:N>. This is normal because catalogs cannot be resolved across databases.
Performance Impact: This function reads WAL files for statistics. When wal_num is large, the execution time will increase. It is recommended to avoid using a large wal_num during peak hours.
WAL Availability: Only WAL files that have not been cleaned up can be analyzed. Historical WAL files that have been reclaimed cannot be analyzed.

ヘルプとサポート

この記事はお役に立ちましたか?

フィードバック