From d2de7f8b0284e4cf0dcdd943b7967c5b904a38a8 Mon Sep 17 00:00:00 2001 From: Maksym Buz Date: Tue, 16 Dec 2025 23:05:44 +0100 Subject: [PATCH] Change: Adjusted README to the new changes. Added runtime parameters. --- partitioning/README.md | 81 ++++++++++++++++++++++++++++++------------ 1 file changed, 59 insertions(+), 22 deletions(-) diff --git a/partitioning/README.md b/partitioning/README.md index 1e0b8ba..befbbe4 100644 --- a/partitioning/README.md +++ b/partitioning/README.md @@ -47,8 +47,16 @@ The script manages MySQL table partitions based on time (Range Partitioning on t --- ## 3. Configuration -Edit `/etc/zabbix/zabbix_partitioning.conf`: +The preferred way to create the configuration file is using the **Interactive Wizard**: +```bash +/opt/zabbix_partitioning/zabbix_partitioning.py --wizard +``` + +Follow the on-screen prompts to configure your database connection, retention periods, and other settings. The wizard will generate the YAML configuration file for you. + +### Automatic Config Structure (Reference) +If you prefer to edit manually `/etc/zabbix/zabbix_partitioning.conf`: ```yaml database: host: localhost @@ -56,6 +64,7 @@ database: passwd: YOUR_PASSWORD db: zabbix # port: 3306 # Optional, default is 3306 + # socket: /var/run/mysqld/mysqld.sock # Overrides host if present partitions: daily: @@ -74,9 +83,7 @@ partitions: - **`yearly`**: Partitions are created for each year. - Retention Format: `14d` (days), `12w` (weeks), `12m` (months), `1y` (years). -- **`initial_partitioning_start`**: Controls how the very FIRST partition is determined during initialization (`--init` mode). - - `db_min`: (Default) Queries the table for the oldest record (`MIN(clock)`). Accurate but **slow** on large tables. - - `retention`: (Recommended for large DBs) Skips the query. Calculates the start date as `Now - Retention Period`. Creates a single `p_archive` partition for all data older than that date. +- **`replicate_sql`**: Controls MySQL Binary Logging for partitioning commands. - **`premake`**: Number of future partitions to create in advance. - Default: `10`. Ensures you have a buffer if the script fails to run for a few days. @@ -90,7 +97,25 @@ partitions: --- -## 4. Zabbix Preparation (CRITICAL) +--- + +## 4. Runtime Parameters + +| Argument | Description | +|---|---| +| `-c`, `--config FILE` | Path to configuration file (Default: `/etc/zabbix/zabbix_partitioning.conf`) | +| `-i`, `--init` | Initialize partitions (converts tables). | +| `--fast-init` | Skip slow table scan during initialization. Starts from retention period. | +| `--wizard` | Launch interactive configuration wizard. | +| `-r`, `--dry-run` | Simulate queries without executing. Logs expected actions (Safe mode). | +| `-v`, `--verbose` | Enable debug logging (DEBUG level). | +| `--stats TABLE` | Output JSON statistics (Size, Count, Days Left) for a specific table. | +| `--discovery` | Output Zabbix Low-Level Discovery (LLD) JSON. | +| `-V`, `--version` | Show script version. | + +--- + +## 5. Zabbix Preparation (CRITICAL) Before partitioning, you **must disable** Zabbix's internal housekeeping for the tables you intend to partition. If you don't, Zabbix will try to delete individual rows while the script tries to drop partitions, causing conflicts. 1. Log in to Zabbix Web Interface. @@ -102,24 +127,35 @@ Before partitioning, you **must disable** Zabbix's internal housekeeping for the --- -## 5. Initialization +## 6. Initialization This step converts existing standard tables into partitioned tables. 1. **Dry Run** (Verify what will happen): ```bash /opt/zabbix_partitioning/zabbix_partitioning.py --init --dry-run ``` - *Check the output for any errors.* 2. **Execute Initialization**: + There are two strategies for initialization: + + **A. Standard Initialization (Default)**: + Scans the database to find the oldest record (`MIN(clock)`) and creates partitions from that point forward. + *Best for smaller databases or when you need to retain ALL existing data.* ```bash /opt/zabbix_partitioning/zabbix_partitioning.py --init ``` - *This may take time depending on table size.* + + **B. Fast Initialization (`--fast-init`)**: + Skips the slow table scan. It calculates the start date based on your configured **Retention Period** (e.g., `Now - 365d`). + It creates a single catch-all `p_archive` partition for all data older than the retention start date, then creates granular partitions forward. + *Recommended for large databases to avoid long table locks/scans.* + ```bash + /opt/zabbix_partitioning/zabbix_partitioning.py --init --fast-init + ``` --- -## 6. Automation (Cron Job) +## 7. Automation (Cron Job) Set up a daily cron job to create new partitions and remove old ones. 1. Open crontab: @@ -133,7 +169,7 @@ Set up a daily cron job to create new partitions and remove old ones. --- -## 7. Automation (Systemd Timer) — Recommended +## 8. Automation (Systemd Timer) — Recommended Alternatively, use systemd timers for more robust scheduling and logging. 1. **Create Service Unit** (`/etc/systemd/system/zabbix-partitioning.service`): @@ -176,12 +212,12 @@ Alternatively, use systemd timers for more robust scheduling and logging. --- -## 8. Troubleshooting +## 9. Troubleshooting - **Connection Refused**: Check `host`, `port` in config. Ensure MySQL is running. - **Access Denied (1227)**: The DB user needs `SUPER` privileges to disable binary logging (`replicate_sql: False`). Either grant the privilege or set `replicate_sql: True` (if replication load is acceptable). - **Primary Key Error**: "Primary Key does not include 'clock'". The table cannot be partitioned by range on `clock` without schema changes. Remove it from config. -## 9. Docker Usage +## 10. Docker Usage You can run the partitioning script as a stateless Docker container. This is ideal for Kubernetes CronJobs or environments where you don't want to manage Python dependencies on the host. @@ -267,8 +303,8 @@ docker run --rm \ | `RETENTION_TRENDS` | 365d | Retention for `trends*` tables | | `RETENTION_AUDIT` | 365d | Retention for `auditlog` (if enabled) | | `ENABLE_AUDITLOG_PARTITIONING` | false | Set to `true` to partition `auditlog` | -| `RUN_MODE` | maintenance | `init`, `maintenance`, `dry-run`, `discovery`, `check` | -| `CHECK_TARGET` | - | Required if `RUN_MODE=check`. Table name to check (e.g. `history`). | +| `RUN_MODE` | maintenance | `init`, `maintenance`, `dry-run`, `discovery`, `stats` | +| `CHECK_TARGET` | - | Required if `RUN_MODE=stats`. Table name to check (e.g. `history`). | | `PARTITION_DAILY_[TABLE]` | - | Custom daily retention (e.g., `PARTITION_DAILY_mytable=30d`) | | `PARTITION_WEEKLY_[TABLE]` | - | Custom weekly retention | | `PARTITION_MONTHLY_[TABLE]` | - | Custom monthly retention | @@ -282,36 +318,37 @@ docker run --rm \ zabbix-partitioning ``` -#### Scenario G: Monitoring (Health Check) -Check days remaining for a specific table (e.g., `history`). Returns integer days. +#### Scenario G: Monitoring (Stats) +Get detailed JSON statistics for a specific table. ```bash docker run --rm \ -e DB_HOST=zabbix-db \ - -e RUN_MODE=check \ + -e RUN_MODE=stats \ -e CHECK_TARGET=history \ zabbix-partitioning ``` --- -## 10. Monitoring +## 11. Monitoring The script includes built-in features for monitoring the health of your partitions via Zabbix. ### 10.1 CLI Usage - **Discovery (LLD)**: + Output the list of configured tables for Zabbix Discovery rules. ```bash ./zabbix_partitioning.py --discovery # Output: [{"{#TABLE}": "history", "{#PERIOD}": "daily"}, ...] ``` -- **Check Days**: +- **Statistics (JSON)**: + Get detailed stats (Size, Partition Count, Days Remaining). ```bash - ./zabbix_partitioning.py --check-days history - # Output: 30 (integer days remaining) + ./zabbix_partitioning.py --stats history + # Output: {"table": "history", "size_bytes": 102400, "partition_count": 5, "days_left": 30} ``` - **Version**: ```bash ./zabbix_partitioning.py --version - # Output: zabbix_partitioning.py 0.3.1-test ``` ### 10.2 Zabbix Template