diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..86c0167 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,63 @@ +# Zabbix PostgreSQL Partitioning Architecture + +This document provides a brief technical overview of the components, logic, and dynamic querying mechanisms that power the PostgreSQL partitioning solution for Zabbix. + +## Schema-Agnostic Design + +A core architectural principle of this solution is its **schema-agnostic design**. It does not assume that your Zabbix database is installed in the default `public` schema. + +When the procedures need to create, drop, or manipulate a partitioned table (e.g., `history`), they do not hardcode the schema. Instead, they dynamically query PostgreSQL's internal system catalogs (`pg_class` and `pg_namespace`) to locate exactly which schema the target table belongs to: + +```sql +SELECT n.nspname INTO v_schema +FROM pg_class c +JOIN pg_namespace n ON n.oid = c.relnamespace +WHERE c.relname = v_table; +``` + +This ensures that the partitioning scripts will work flawlessly, even in custom Zabbix deployments where tables are housed in alternative schemas. + +## File Structure & Queries + +The solution is divided into a series of SQL scripts that must be executed sequentially to set up the environment. + +### 1. `00_schema_create.sql` +* **Purpose:** Initializes the foundation for the partitioning system. +* **Actions:** + * Creates the isolated `partitions` schema to keep everything separate from Zabbix's own structure. + * Creates the `partitions.config` table (which stores retention policies). + * Creates the `partitions.version` table for tracking the installed version. + +### 2. `01_auditlog_prep.sql` +* **Purpose:** Prepares the Zabbix `auditlog` table for partitioning. +* **Actions:** + * PostgreSQL range partitioning requires the partition key (in this case, `clock`) to be part of the Primary Key. + * This script dynamically locates the existing Primary Key (usually just `auditid`) and alters it to a composite key `(auditid, clock)`. + +### 3. `01_maintenance.sql` +* **Purpose:** Contains the core PL/pgSQL procedural logic that manages the lifecycle of the partitions. +* **Key Functions/Procedures:** + * `partition_exists()`: Queries `pg_class` to verify if a specific child partition partition exists. + * `create_partition()`: Executes the DDL `CREATE TABLE ... PARTITION OF ... FOR VALUES FROM (x) TO (y)` to generate a new time-bound chunk. + * `drop_old_partitions()`: Iterates over existing child partitions (using `pg_inherits`) and calculates their age based on their suffix. Drops those older than the defined `keep_history` policy. + * `maintain_table()`: The orchestrator for a single table. It calculates the necessary UTC timestamps, calls `create_partition()` to build the future buffer, calls `create_partition()` recursively backward to cover the retention period, and finally calls `drop_old_partitions()`. + * `run_maintenance()`: The global loop that iterates through `partitions.config` and triggers `maintain_table()` for every configured Zabbix table. + +### 4. `02_enable_partitioning.sql` +* **Purpose:** The migration script that actually executes the partition conversion on the live database. +* **Actions:** + * It takes the original Zabbix table (e.g., `history`) and renames it to `history_old` (`ALTER TABLE ... RENAME TO ...`). + * It immediately creates a new partitioned table with the original name, inheriting the exact structure of the old table (`CREATE TABLE ... (LIKE ... INCLUDING ALL) PARTITION BY RANGE (clock)`). + * It triggers the first maintenance run so new incoming data has immediate partitions to land in. + +### 5. `03_monitoring_view.sql` +* **Purpose:** Provides an easy-to-read observability layer. +* **Actions:** + * Creates the `partitions.monitoring` view by joining `pg_class`, `pg_inherits`, `pg_tablespace`, and `pg_size_pretty`. + * This view aggregates the total size of each partitioned family and calculates how many "future partitions" exist as a safety buffer. + +## Automated Scheduling (`pg_cron`) + +While `systemd` timers or standard `cron` can be used to trigger the maintenance, the recommended approach (especially for AWS RDS/Aurora deployments) is using the `pg_cron` database extension. + +`pg_cron` allows you to schedule the `CALL partitions.run_maintenance();` procedure directly within PostgreSQL, ensuring the database autonomously manages its own housekeeping without requiring external OS-level access or triggers. diff --git a/postgresql/procedures/00_partitions_init.sql b/postgresql/procedures/00_schema_create.sql similarity index 100% rename from postgresql/procedures/00_partitions_init.sql rename to postgresql/procedures/00_schema_create.sql diff --git a/postgresql/procedures/03_monitoring_view.sql b/postgresql/procedures/03_monitoring_view.sql index 7b123bb..15c06a0 100644 --- a/postgresql/procedures/03_monitoring_view.sql +++ b/postgresql/procedures/03_monitoring_view.sql @@ -2,7 +2,8 @@ -- Creates a view to monitor partition status and sizes. -- ============================================================================ -CREATE OR REPLACE VIEW partitions.monitoring AS +DROP VIEW IF EXISTS partitions.monitoring; +CREATE VIEW partitions.monitoring AS SELECT parent.relname AS parent_table, c.table_name, diff --git a/postgresql/procedures/README.md b/postgresql/procedures/README.md index 00f9932..da4724e 100644 --- a/postgresql/procedures/README.md +++ b/postgresql/procedures/README.md @@ -55,7 +55,7 @@ If you are deploying Zabbix on a fresh database instance (like AWS RDS) rather t ## Installation The installation is performed by executing the SQL procedures in the following order: -1. Initialize schema (`00_partitions_init.sql`). +1. Initialize schema (`00_schema_create.sql`). 2. Install maintenance procedures (`01_maintenance.sql`). 3. Enable partitioning on tables (`02_enable_partitioning.sql`). 4. Install monitoring views (`03_monitoring_view.sql`). @@ -70,7 +70,7 @@ DB_HOST="localhost" # Or your RDS endpoint DB_NAME="zabbix" DB_USER="zabbix" -for script in 00_partitions_init.sql 01_maintenance.sql 02_enable_partitioning.sql 03_monitoring_view.sql; do +for script in 00_schema_create.sql 01_maintenance.sql 02_enable_partitioning.sql 03_monitoring_view.sql; do echo "Applying $script..." psql -h $DB_HOST -U $DB_USER -d $DB_NAME -f "$script" done @@ -247,6 +247,9 @@ GRANT USAGE ON SCHEMA partitions TO zbx_monitor; GRANT SELECT ON partitions.monitoring TO zbx_monitor; ``` +> [!NOTE] +> If you ever apply updates to `03_monitoring_view.sql`, you should run the script as the `zabbix` database user (the original creator of the view). The script drops and recreates the view, so running it as `zabbix` ensures the view retains its ownership and preserves existing `GRANT` permissions for read-only users. + ## Implementation Details ### `auditlog` Table