64 lines
4.3 KiB
Markdown
64 lines
4.3 KiB
Markdown
# Zabbix PostgreSQL Partitioning Architecture
|
|
|
|
This document provides a brief technical overview of the components, logic, and dynamic querying mechanisms that power the PostgreSQL partitioning solution for Zabbix.
|
|
|
|
## Schema-Agnostic Design
|
|
|
|
A core architectural principle of this solution is its **schema-agnostic design**. It does not assume that your Zabbix database is installed in the default `public` schema.
|
|
|
|
When the procedures need to create, drop, or manipulate a partitioned table (e.g., `history`), they do not hardcode the schema. Instead, they dynamically query PostgreSQL's internal system catalogs (`pg_class` and `pg_namespace`) to locate exactly which schema the target table belongs to:
|
|
|
|
```sql
|
|
SELECT n.nspname INTO v_schema
|
|
FROM pg_class c
|
|
JOIN pg_namespace n ON n.oid = c.relnamespace
|
|
WHERE c.relname = v_table;
|
|
```
|
|
|
|
This ensures that the partitioning scripts will work flawlessly, even in custom Zabbix deployments where tables are housed in alternative schemas.
|
|
|
|
## File Structure & Queries
|
|
|
|
The solution is divided into a series of SQL scripts that must be executed sequentially to set up the environment.
|
|
|
|
### 1. `00_partitions_init.sql`
|
|
* **Purpose:** Initializes the foundation for the partitioning system.
|
|
* **Actions:**
|
|
* Creates the isolated `partitions` schema to keep everything separate from Zabbix's own structure.
|
|
* Creates the `partitions.config` table (which stores retention policies).
|
|
* Creates the `partitions.version` table for tracking the installed version.
|
|
|
|
### 2. `01_auditlog_prep.sql`
|
|
* **Purpose:** Prepares the Zabbix `auditlog` table for partitioning.
|
|
* **Actions:**
|
|
* PostgreSQL range partitioning requires the partition key (in this case, `clock`) to be part of the Primary Key.
|
|
* This script dynamically locates the existing Primary Key (usually just `auditid`) and alters it to a composite key `(auditid, clock)`.
|
|
|
|
### 3. `01_maintenance.sql`
|
|
* **Purpose:** Contains the core PL/pgSQL procedural logic that manages the lifecycle of the partitions.
|
|
* **Key Functions/Procedures:**
|
|
* `partition_exists()`: Queries `pg_class` to verify if a specific child partition partition exists.
|
|
* `create_partition()`: Executes the DDL `CREATE TABLE ... PARTITION OF ... FOR VALUES FROM (x) TO (y)` to generate a new time-bound chunk.
|
|
* `drop_old_partitions()`: Iterates over existing child partitions (using `pg_inherits`) and calculates their age based on their suffix. Drops those older than the defined `keep_history` policy.
|
|
* `maintain_table()`: The orchestrator for a single table. It calculates the necessary UTC timestamps, calls `create_partition()` to build the future buffer, calls `create_partition()` recursively backward to cover the retention period, and finally calls `drop_old_partitions()`.
|
|
* `run_maintenance()`: The global loop that iterates through `partitions.config` and triggers `maintain_table()` for every configured Zabbix table.
|
|
|
|
### 4. `02_enable_partitioning.sql`
|
|
* **Purpose:** The migration script that actually executes the partition conversion on the live database.
|
|
* **Actions:**
|
|
* It takes the original Zabbix table (e.g., `history`) and renames it to `history_old` (`ALTER TABLE ... RENAME TO ...`).
|
|
* It immediately creates a new partitioned table with the original name, inheriting the exact structure of the old table (`CREATE TABLE ... (LIKE ... INCLUDING ALL) PARTITION BY RANGE (clock)`).
|
|
* It triggers the first maintenance run so new incoming data has immediate partitions to land in.
|
|
|
|
### 5. `03_monitoring_view.sql`
|
|
* **Purpose:** Provides an easy-to-read observability layer.
|
|
* **Actions:**
|
|
* Creates the `partitions.monitoring` view by joining `pg_class`, `pg_inherits`, `pg_tablespace`, and `pg_size_pretty`.
|
|
* This view aggregates the total size of each partitioned family and calculates how many "future partitions" exist as a safety buffer.
|
|
|
|
## Automated Scheduling (`pg_cron`)
|
|
|
|
While `systemd` timers or standard `cron` can be used to trigger the maintenance, the recommended approach (especially for AWS RDS/Aurora deployments) is using the `pg_cron` database extension.
|
|
|
|
`pg_cron` allows you to schedule the `CALL partitions.run_maintenance();` procedure directly within PostgreSQL, ensuring the database autonomously manages its own housekeeping without requiring external OS-level access or triggers.
|