Compare commits
4 Commits
08ba77cdf0
...
test
| Author | SHA1 | Date | |
|---|---|---|---|
| 094debc46c | |||
| d2de7f8b02 | |||
| 3b2bebf6b9 | |||
| 059aa81c5c |
@@ -5,6 +5,32 @@ All notable changes to this project will be documented in this file.
|
|||||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
## [0.6.0] - 2025-12-20
|
||||||
|
### Added
|
||||||
|
- **SSL**: Added support for simplified SSL configuration (`ssl: required`) for Managed/Cloud databases and updated Wizard/Config.
|
||||||
|
- **Safety**: Added automatic cleanup of `housekeeper` table during init (targeted) and conflict detection during maintenance.
|
||||||
|
- **CLI**: Added short flags for all arguments (`-vv`, `-f`, `-d`, `-s`, `-w`).
|
||||||
|
- **Validation**: Enforced strict unit validation for retention periods and strict table existence checks.
|
||||||
|
- **Error Handling**: Improved error reporting used to identify specific tables causing configuration errors.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- **CLI**: Refactored `--init` and `--fast-init` into mutually exclusive standalone commands (Breaking Change).
|
||||||
|
- **CLI**: Reorganized argument parsing logic for better maintainability.
|
||||||
|
- **Safety**: Updated initialization warnings to explicitly mention duration ("SEVERAL HOURS") and disk space requirements.
|
||||||
|
- **Logic**: Refined housekeeper conflict check to only warn about tables actively configured for partitioning.
|
||||||
|
|
||||||
|
## [0.5.0] - 2025-12-16
|
||||||
|
### Added
|
||||||
|
- **Wizard**: Added interactive configuration wizard (`--wizard`).
|
||||||
|
- **CLI**: Added `--fast-init` flag to skip table scanning during initialization, instead of config option.
|
||||||
|
- **Monitoring**: Added `--stats` argument to output JSON statistics (Size, Count, Days Left).
|
||||||
|
- **Template**: Updated `zabbix_mysql_partitioning_template.yaml` to use Master/Dependent items for efficient stats collection.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- **Config**: Removed `initial_partitioning_start` from configuration file (moved to `--fast-init`).
|
||||||
|
- **Template**: Replaced legacy check item with comprehensive stats master item.
|
||||||
|
- **CLI**: Removed legacy `--check-days` argument (replaced by `--stats`).
|
||||||
|
|
||||||
## [0.4.1] - 2025-12-16
|
## [0.4.1] - 2025-12-16
|
||||||
### Added
|
### Added
|
||||||
- **CLI**: Added `--verbose` / `-v` flag to switch between INFO (default) and DEBUG logging levels.
|
- **CLI**: Added `--verbose` / `-v` flag to switch between INFO (default) and DEBUG logging levels.
|
||||||
|
|||||||
@@ -47,8 +47,16 @@ The script manages MySQL table partitions based on time (Range Partitioning on t
|
|||||||
---
|
---
|
||||||
|
|
||||||
## 3. Configuration
|
## 3. Configuration
|
||||||
Edit `/etc/zabbix/zabbix_partitioning.conf`:
|
The preferred way to create the configuration file is using the **Interactive Wizard**:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
/opt/zabbix_partitioning/zabbix_partitioning.py --wizard
|
||||||
|
```
|
||||||
|
|
||||||
|
Follow the on-screen prompts to configure your database connection, retention periods, and other settings. The wizard will generate the YAML configuration file for you.
|
||||||
|
|
||||||
|
### Automatic Config Structure (Reference)
|
||||||
|
If you prefer to edit manually `/etc/zabbix/zabbix_partitioning.conf`:
|
||||||
```yaml
|
```yaml
|
||||||
database:
|
database:
|
||||||
host: localhost
|
host: localhost
|
||||||
@@ -56,6 +64,7 @@ database:
|
|||||||
passwd: YOUR_PASSWORD
|
passwd: YOUR_PASSWORD
|
||||||
db: zabbix
|
db: zabbix
|
||||||
# port: 3306 # Optional, default is 3306
|
# port: 3306 # Optional, default is 3306
|
||||||
|
# socket: /var/run/mysqld/mysqld.sock # Overrides host if present
|
||||||
|
|
||||||
partitions:
|
partitions:
|
||||||
daily:
|
daily:
|
||||||
@@ -74,9 +83,7 @@ partitions:
|
|||||||
- **`yearly`**: Partitions are created for each year.
|
- **`yearly`**: Partitions are created for each year.
|
||||||
- Retention Format: `14d` (days), `12w` (weeks), `12m` (months), `1y` (years).
|
- Retention Format: `14d` (days), `12w` (weeks), `12m` (months), `1y` (years).
|
||||||
|
|
||||||
- **`initial_partitioning_start`**: Controls how the very FIRST partition is determined during initialization (`--init` mode).
|
- **`replicate_sql`**: Controls MySQL Binary Logging for partitioning commands.
|
||||||
- `db_min`: (Default) Queries the table for the oldest record (`MIN(clock)`). Accurate but **slow** on large tables.
|
|
||||||
- `retention`: (Recommended for large DBs) Skips the query. Calculates the start date as `Now - Retention Period`. Creates a single `p_archive` partition for all data older than that date.
|
|
||||||
|
|
||||||
- **`premake`**: Number of future partitions to create in advance.
|
- **`premake`**: Number of future partitions to create in advance.
|
||||||
- Default: `10`. Ensures you have a buffer if the script fails to run for a few days.
|
- Default: `10`. Ensures you have a buffer if the script fails to run for a few days.
|
||||||
@@ -90,7 +97,25 @@ partitions:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 4. Zabbix Preparation (CRITICAL)
|
---
|
||||||
|
|
||||||
|
## 4. Runtime Parameters
|
||||||
|
|
||||||
|
| Argument | Description |
|
||||||
|
|---|---|
|
||||||
|
| `-c`, `--config FILE` | Path to configuration file (Default: `/etc/zabbix/zabbix_partitioning.conf`) |
|
||||||
|
| `-i`, `--init` | Initialize partitions (Standard Mode: Scan Table). |
|
||||||
|
| `-f`, `--fast-init` | Initialize partitions (Fast Mode: Skip Scan, use Retention). |
|
||||||
|
| `--wizard` | Launch interactive configuration wizard. |
|
||||||
|
| `-r`, `--dry-run` | Simulate queries without executing. Logs expected actions (Safe mode). |
|
||||||
|
| `-v`, `--verbose` | Enable debug logging (DEBUG level). |
|
||||||
|
| `--stats TABLE` | Output JSON statistics (Size, Count, Days Left) for a specific table. |
|
||||||
|
| `--discovery` | Output Zabbix Low-Level Discovery (LLD) JSON. |
|
||||||
|
| `-V`, `--version` | Show script version. |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Zabbix Preparation (CRITICAL)
|
||||||
Before partitioning, you **must disable** Zabbix's internal housekeeping for the tables you intend to partition. If you don't, Zabbix will try to delete individual rows while the script tries to drop partitions, causing conflicts.
|
Before partitioning, you **must disable** Zabbix's internal housekeeping for the tables you intend to partition. If you don't, Zabbix will try to delete individual rows while the script tries to drop partitions, causing conflicts.
|
||||||
|
|
||||||
1. Log in to Zabbix Web Interface.
|
1. Log in to Zabbix Web Interface.
|
||||||
@@ -102,38 +127,52 @@ Before partitioning, you **must disable** Zabbix's internal housekeeping for the
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 5. Initialization
|
## 6. Initialization
|
||||||
This step converts existing standard tables into partitioned tables.
|
This step converts existing standard tables into partitioned tables.
|
||||||
|
|
||||||
1. **Dry Run** (Verify what will happen):
|
1. **Dry Run** (Verify what will happen):
|
||||||
```bash
|
```bash
|
||||||
/opt/zabbix_partitioning/zabbix_partitioning.py --init --dry-run
|
/opt/zabbix_partitioning/zabbix_partitioning.py --init --dry-run
|
||||||
```
|
```
|
||||||
*Check the output for any errors.*
|
|
||||||
|
|
||||||
2. **Execute Initialization**:
|
2. **Execute Initialization**:
|
||||||
|
> [!WARNING]
|
||||||
|
> For large databases, standard initialization may take **SEVERAL HOURS** and require significant disk space (approx 2x table size).
|
||||||
|
|
||||||
|
There are two strategies for initialization:
|
||||||
|
|
||||||
|
**A. Standard Initialization (`--init`)**:
|
||||||
|
Scans the database to find the oldest record (`MIN(clock)`) and creates partitions from that point forward.
|
||||||
|
*Best for smaller databases or when you need to retain ALL existing data.*
|
||||||
```bash
|
```bash
|
||||||
/opt/zabbix_partitioning/zabbix_partitioning.py --init
|
/opt/zabbix_partitioning/zabbix_partitioning.py --init
|
||||||
```
|
```
|
||||||
*This may take time depending on table size.*
|
|
||||||
|
**B. Fast Initialization (`--fast-init`)**:
|
||||||
|
Skips the slow table scan. It calculates the start date based on your configured **Retention Period** (e.g., `Now - 365d`).
|
||||||
|
It creates a single catch-all `p_archive` partition for all data older than the retention start date, then creates granular partitions forward.
|
||||||
|
*Recommended for large databases to avoid long table locks/scans.*
|
||||||
|
```bash
|
||||||
|
/opt/zabbix_partitioning/zabbix_partitioning.py --fast-init
|
||||||
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 6. Automation (Cron Job)
|
## 7. Automation (Cron Job)
|
||||||
Set up a daily cron job to create new partitions and remove old ones.
|
Set up a cron job to create new partitions and remove old ones.
|
||||||
|
|
||||||
1. Open crontab:
|
1. Open crontab:
|
||||||
```bash
|
```bash
|
||||||
crontab -e
|
crontab -e
|
||||||
```
|
```
|
||||||
2. Add the line (run daily at 00:30):
|
2. Add the line (run twice a day at 00:10 and 04:10):
|
||||||
```cron
|
```cron
|
||||||
30 0 * * * /usr/bin/python3 /opt/zabbix_partitioning/zabbix_partitioning.py -c /etc/zabbix/zabbix_partitioning.conf >> /var/log/zabbix_partitioning.log 2>&1
|
10 0,4 * * * /usr/bin/python3 /opt/zabbix_partitioning/zabbix_partitioning.py -c /etc/zabbix/zabbix_partitioning.conf >> /var/log/zabbix_partitioning.log 2>&1
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 7. Automation (Systemd Timer) — Recommended
|
## 8. Automation (Systemd Timer) — Recommended
|
||||||
Alternatively, use systemd timers for more robust scheduling and logging.
|
Alternatively, use systemd timers for more robust scheduling and logging.
|
||||||
|
|
||||||
1. **Create Service Unit** (`/etc/systemd/system/zabbix-partitioning.service`):
|
1. **Create Service Unit** (`/etc/systemd/system/zabbix-partitioning.service`):
|
||||||
@@ -151,10 +190,10 @@ Alternatively, use systemd timers for more robust scheduling and logging.
|
|||||||
2. **Create Timer Unit** (`/etc/systemd/system/zabbix-partitioning.timer`):
|
2. **Create Timer Unit** (`/etc/systemd/system/zabbix-partitioning.timer`):
|
||||||
```ini
|
```ini
|
||||||
[Unit]
|
[Unit]
|
||||||
Description=Run Zabbix Partitioning Daily
|
Description=Run Zabbix Partitioning twice a day
|
||||||
|
|
||||||
[Timer]
|
[Timer]
|
||||||
OnCalendar=*-*-* 00:30:00
|
OnCalendar=*-*-* 00:10:00 *-*-* 04:10:00
|
||||||
Persistent=true
|
Persistent=true
|
||||||
|
|
||||||
[Install]
|
[Install]
|
||||||
@@ -176,12 +215,12 @@ Alternatively, use systemd timers for more robust scheduling and logging.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 8. Troubleshooting
|
## 9. Troubleshooting
|
||||||
- **Connection Refused**: Check `host`, `port` in config. Ensure MySQL is running.
|
- **Connection Refused**: Check `host`, `port` in config. Ensure MySQL is running.
|
||||||
- **Access Denied (1227)**: The DB user needs `SUPER` privileges to disable binary logging (`replicate_sql: False`). Either grant the privilege or set `replicate_sql: True` (if replication load is acceptable).
|
- **Access Denied (1227)**: The DB user needs `SUPER` privileges to disable binary logging (`replicate_sql: False`). Either grant the privilege or set `replicate_sql: True` (if replication load is acceptable).
|
||||||
- **Primary Key Error**: "Primary Key does not include 'clock'". The table cannot be partitioned by range on `clock` without schema changes. Remove it from config.
|
- **Primary Key Error**: "Primary Key does not include 'clock'". The table cannot be partitioned by range on `clock` without schema changes. Remove it from config.
|
||||||
|
|
||||||
## 9. Docker Usage
|
## 10. Docker Usage
|
||||||
|
|
||||||
You can run the partitioning script as a stateless Docker container. This is ideal for Kubernetes CronJobs or environments where you don't want to manage Python dependencies on the host.
|
You can run the partitioning script as a stateless Docker container. This is ideal for Kubernetes CronJobs or environments where you don't want to manage Python dependencies on the host.
|
||||||
|
|
||||||
@@ -267,8 +306,8 @@ docker run --rm \
|
|||||||
| `RETENTION_TRENDS` | 365d | Retention for `trends*` tables |
|
| `RETENTION_TRENDS` | 365d | Retention for `trends*` tables |
|
||||||
| `RETENTION_AUDIT` | 365d | Retention for `auditlog` (if enabled) |
|
| `RETENTION_AUDIT` | 365d | Retention for `auditlog` (if enabled) |
|
||||||
| `ENABLE_AUDITLOG_PARTITIONING` | false | Set to `true` to partition `auditlog` |
|
| `ENABLE_AUDITLOG_PARTITIONING` | false | Set to `true` to partition `auditlog` |
|
||||||
| `RUN_MODE` | maintenance | `init`, `maintenance`, `dry-run`, `discovery`, `check` |
|
| `RUN_MODE` | maintenance | `init`, `maintenance`, `dry-run`, `discovery`, `stats` |
|
||||||
| `CHECK_TARGET` | - | Required if `RUN_MODE=check`. Table name to check (e.g. `history`). |
|
| `CHECK_TARGET` | - | Required if `RUN_MODE=stats`. Table name to check (e.g. `history`). |
|
||||||
| `PARTITION_DAILY_[TABLE]` | - | Custom daily retention (e.g., `PARTITION_DAILY_mytable=30d`) |
|
| `PARTITION_DAILY_[TABLE]` | - | Custom daily retention (e.g., `PARTITION_DAILY_mytable=30d`) |
|
||||||
| `PARTITION_WEEKLY_[TABLE]` | - | Custom weekly retention |
|
| `PARTITION_WEEKLY_[TABLE]` | - | Custom weekly retention |
|
||||||
| `PARTITION_MONTHLY_[TABLE]` | - | Custom monthly retention |
|
| `PARTITION_MONTHLY_[TABLE]` | - | Custom monthly retention |
|
||||||
@@ -282,36 +321,37 @@ docker run --rm \
|
|||||||
zabbix-partitioning
|
zabbix-partitioning
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Scenario G: Monitoring (Health Check)
|
#### Scenario G: Monitoring (Stats)
|
||||||
Check days remaining for a specific table (e.g., `history`). Returns integer days.
|
Get detailed JSON statistics for a specific table.
|
||||||
```bash
|
```bash
|
||||||
docker run --rm \
|
docker run --rm \
|
||||||
-e DB_HOST=zabbix-db \
|
-e DB_HOST=zabbix-db \
|
||||||
-e RUN_MODE=check \
|
-e RUN_MODE=stats \
|
||||||
-e CHECK_TARGET=history \
|
-e CHECK_TARGET=history \
|
||||||
zabbix-partitioning
|
zabbix-partitioning
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 10. Monitoring
|
## 11. Monitoring
|
||||||
The script includes built-in features for monitoring the health of your partitions via Zabbix.
|
The script includes built-in features for monitoring the health of your partitions via Zabbix.
|
||||||
|
|
||||||
### 10.1 CLI Usage
|
### 10.1 CLI Usage
|
||||||
- **Discovery (LLD)**:
|
- **Discovery (LLD)**:
|
||||||
|
Output the list of configured tables for Zabbix Discovery rules.
|
||||||
```bash
|
```bash
|
||||||
./zabbix_partitioning.py --discovery
|
./zabbix_partitioning.py --discovery
|
||||||
# Output: [{"{#TABLE}": "history", "{#PERIOD}": "daily"}, ...]
|
# Output: [{"{#TABLE}": "history", "{#PERIOD}": "daily"}, ...]
|
||||||
```
|
```
|
||||||
- **Check Days**:
|
- **Statistics (JSON)**:
|
||||||
|
Get detailed stats (Size, Partition Count, Days Remaining).
|
||||||
```bash
|
```bash
|
||||||
./zabbix_partitioning.py --check-days history
|
./zabbix_partitioning.py --stats history
|
||||||
# Output: 30 (integer days remaining)
|
# Output: {"table": "history", "size_bytes": 102400, "partition_count": 5, "days_left": 30}
|
||||||
```
|
```
|
||||||
- **Version**:
|
- **Version**:
|
||||||
```bash
|
```bash
|
||||||
./zabbix_partitioning.py --version
|
./zabbix_partitioning.py --version
|
||||||
# Output: zabbix_partitioning.py 0.3.1-test
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### 10.2 Zabbix Template
|
### 10.2 Zabbix Template
|
||||||
|
|||||||
@@ -11,6 +11,11 @@ database:
|
|||||||
user: zbx_part
|
user: zbx_part
|
||||||
passwd: <password>
|
passwd: <password>
|
||||||
db: zabbix
|
db: zabbix
|
||||||
|
# ssl: required # Use system CAs (Mandatory for Azure/AWS/GCP)
|
||||||
|
# ssl: # Or use custom certificates
|
||||||
|
# ca: /etc/ssl/certs/ca-cert.pem
|
||||||
|
# cert: /etc/ssl/certs/client-cert.pem
|
||||||
|
# key: /etc/ssl/certs/client-key.pem
|
||||||
|
|
||||||
# partitions: Define retention periods for tables.
|
# partitions: Define retention periods for tables.
|
||||||
# Format: table_name: duration (e.g., 14d, 12w, 1m, 1y)
|
# Format: table_name: duration (e.g., 14d, 12w, 1m, 1y)
|
||||||
|
|||||||
@@ -1,9 +1,8 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
# -*- coding: utf-8 -*-
|
# -*- coding: utf-8 -*-
|
||||||
"""
|
"""
|
||||||
Zabbix Database Partitioning Management Script
|
Zabbix MySQL Partitioning Script
|
||||||
|
|
||||||
Refactored for Zabbix 7.x compatibility, better maintainability, and standard logging.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import os
|
import os
|
||||||
@@ -21,7 +20,7 @@ from typing import Optional, Dict, List, Any, Union, Tuple
|
|||||||
from contextlib import contextmanager
|
from contextlib import contextmanager
|
||||||
|
|
||||||
# Semantic Versioning
|
# Semantic Versioning
|
||||||
VERSION = '0.4.1'
|
VERSION = '0.6.0'
|
||||||
|
|
||||||
# Constants
|
# Constants
|
||||||
PART_PERIOD_REGEX = r'([0-9]+)(h|d|m|y)'
|
PART_PERIOD_REGEX = r'([0-9]+)(h|d|m|y)'
|
||||||
@@ -74,6 +73,11 @@ class ZabbixPartitioner:
|
|||||||
connect_args['host'] = self.db_host
|
connect_args['host'] = self.db_host
|
||||||
|
|
||||||
if self.db_ssl:
|
if self.db_ssl:
|
||||||
|
if self.db_ssl is True or (isinstance(self.db_ssl, str) and self.db_ssl.lower() == 'required'):
|
||||||
|
# Enable SSL with default context (uses system CAs)
|
||||||
|
# Useful for Cloud DBs (Azure, AWS) that require TLS by default
|
||||||
|
connect_args['ssl'] = {}
|
||||||
|
else:
|
||||||
connect_args['ssl'] = self.db_ssl
|
connect_args['ssl'] = self.db_ssl
|
||||||
# PyMySQL SSL options
|
# PyMySQL SSL options
|
||||||
# Note: valid ssl keys for PyMySQL are 'ca', 'capath', 'cert', 'key', 'cipher', 'check_hostname'
|
# Note: valid ssl keys for PyMySQL are 'ca', 'capath', 'cert', 'key', 'cipher', 'check_hostname'
|
||||||
@@ -177,9 +181,11 @@ class ZabbixPartitioner:
|
|||||||
"""
|
"""
|
||||||
Calculate the retention date based on config string (e.g., "30d", "12m").
|
Calculate the retention date based on config string (e.g., "30d", "12m").
|
||||||
"""
|
"""
|
||||||
|
# Ensure string
|
||||||
|
period_str = str(period_str)
|
||||||
match = re.search(PART_PERIOD_REGEX, period_str)
|
match = re.search(PART_PERIOD_REGEX, period_str)
|
||||||
if not match:
|
if not match:
|
||||||
raise ConfigurationError(f"Invalid period format: {period_str}")
|
raise ConfigurationError(f"Invalid period format: '{period_str}' (Expected format like 30d, 12w, 1y)")
|
||||||
|
|
||||||
amount = int(match.group(1))
|
amount = int(match.group(1))
|
||||||
unit = match.group(2)
|
unit = match.group(2)
|
||||||
@@ -233,10 +239,9 @@ class ZabbixPartitioner:
|
|||||||
raise DatabaseError("Could not determine MySQL version")
|
raise DatabaseError("Could not determine MySQL version")
|
||||||
|
|
||||||
# MySQL 8.0+ supports partitioning natively
|
# MySQL 8.0+ supports partitioning natively
|
||||||
# (Assuming MySQL 8+ or MariaDB 10+ for modern Zabbix)
|
|
||||||
self.logger.info(f"MySQL Version: {version_str}")
|
self.logger.info(f"MySQL Version: {version_str}")
|
||||||
|
|
||||||
# 2. Check Zabbix DB Version (optional info)
|
# 2. Check Zabbix DB Version
|
||||||
try:
|
try:
|
||||||
mandatory = self.execute_query('SELECT `mandatory` FROM `dbversion`', fetch='one')
|
mandatory = self.execute_query('SELECT `mandatory` FROM `dbversion`', fetch='one')
|
||||||
if mandatory:
|
if mandatory:
|
||||||
@@ -244,6 +249,18 @@ class ZabbixPartitioner:
|
|||||||
except Exception:
|
except Exception:
|
||||||
self.logger.warning("Could not read 'dbversion' table. Is this a Zabbix DB?")
|
self.logger.warning("Could not read 'dbversion' table. Is this a Zabbix DB?")
|
||||||
|
|
||||||
|
def validate_table_exists(self, table: str) -> bool:
|
||||||
|
"""Return True if table exists in the database."""
|
||||||
|
# Use simple select from information_schema
|
||||||
|
exists = self.execute_query(
|
||||||
|
"SELECT 1 FROM information_schema.tables WHERE table_schema = %s AND table_name = %s",
|
||||||
|
(self.db_name, table), fetch='one'
|
||||||
|
)
|
||||||
|
if not exists:
|
||||||
|
self.logger.error(f"Table '{table}' does NOT exist in database '{self.db_name}'. Skipped.")
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
def get_table_min_clock(self, table: str) -> Optional[datetime]:
|
def get_table_min_clock(self, table: str) -> Optional[datetime]:
|
||||||
ts = self.execute_query(f"SELECT MIN(`clock`) FROM `{table}`", fetch='one')
|
ts = self.execute_query(f"SELECT MIN(`clock`) FROM `{table}`", fetch='one')
|
||||||
return datetime.fromtimestamp(int(ts)) if ts else None
|
return datetime.fromtimestamp(int(ts)) if ts else None
|
||||||
@@ -373,95 +390,131 @@ class ZabbixPartitioner:
|
|||||||
for name in to_drop:
|
for name in to_drop:
|
||||||
self.execute_query(f"ALTER TABLE `{table}` DROP PARTITION {name}")
|
self.execute_query(f"ALTER TABLE `{table}` DROP PARTITION {name}")
|
||||||
|
|
||||||
def initialize_partitioning(self, table: str, period: str, premake: int, retention_str: str):
|
|
||||||
"""Initial partitioning for a table (convert regular table to partitioned)."""
|
|
||||||
self.logger.info(f"Initializing partitioning for {table}")
|
|
||||||
|
|
||||||
|
|
||||||
|
def _check_init_prerequisites(self, table: str) -> bool:
|
||||||
|
"""Return True if partitioning can proceed."""
|
||||||
if self.has_incompatible_primary_key(table):
|
if self.has_incompatible_primary_key(table):
|
||||||
self.logger.error(f"Cannot partition {table}: Primary Key does not include 'clock' column.")
|
self.logger.error(f"Cannot partition {table}: Primary Key does not include 'clock' column.")
|
||||||
return
|
return False
|
||||||
|
|
||||||
# If already partitioned, skip
|
|
||||||
if self.get_existing_partitions(table):
|
if self.get_existing_partitions(table):
|
||||||
self.logger.info(f"Table {table} is already partitioned.")
|
self.logger.info(f"Table {table} is already partitioned.")
|
||||||
return
|
return False
|
||||||
|
|
||||||
# init_strategy = self.config.get('initial_partitioning_start', 'db_min') # Removed in favor of flag
|
# Disk Space & Lock Warning
|
||||||
# but flag needs to be passed to this method or accessed from somewhere.
|
msg = (
|
||||||
# Since I can't easily change signature without affecting calls, I'll pass it in kwargs or check self.fast_init if I add it to class.
|
f"WARNING: Partitioning table '{table}' requires creating a copy of the table.\n"
|
||||||
pass
|
f" Ensure you have free disk space >= Data_Length + Index_Length (approx 2x table size).\n"
|
||||||
|
f" For large databases, this operation may take SEVERAL HOURS to complete."
|
||||||
|
)
|
||||||
|
self.logger.warning(msg)
|
||||||
|
|
||||||
def initialize_partitioning(self, table: str, period: str, premake: int, retention_str: str, fast_init: bool = False):
|
# Interactive Check
|
||||||
"""Initial partitioning for a table (convert regular table to partitioned)."""
|
if sys.stdin.isatty():
|
||||||
self.logger.info(f"Initializing partitioning for {table}")
|
print(f"\n{msg}")
|
||||||
|
if input("Do you have enough free space and is Zabbix stopped? [y/N]: ").lower().strip() != 'y':
|
||||||
|
self.logger.error("Initialization aborted by user.")
|
||||||
|
return False
|
||||||
|
|
||||||
if self.has_incompatible_primary_key(table):
|
return True
|
||||||
self.logger.error(f"Cannot partition {table}: Primary Key does not include 'clock' column.")
|
|
||||||
return
|
|
||||||
|
|
||||||
# If already partitioned, skip
|
def _generate_init_sql(self, table: str, period: str, start_dt: datetime, premake: int, p_archive_ts: int = None):
|
||||||
if self.get_existing_partitions(table):
|
"""Generate and execute ALTER TABLE command."""
|
||||||
self.logger.info(f"Table {table} is already partitioned.")
|
|
||||||
return
|
|
||||||
|
|
||||||
start_dt = None
|
|
||||||
p_archive_ts = None
|
|
||||||
|
|
||||||
if fast_init:
|
|
||||||
self.logger.info(f"Strategy 'fast-init': Calculating start date from retention ({retention_str})")
|
|
||||||
retention_date = self.get_lookback_date(retention_str)
|
|
||||||
# Start granular partitions from the retention date
|
|
||||||
start_dt = self.truncate_date(retention_date, period)
|
|
||||||
# Create a catch-all for anything older
|
|
||||||
p_archive_ts = int(start_dt.timestamp())
|
|
||||||
else:
|
|
||||||
# Default 'db_min' strategy
|
|
||||||
self.logger.info("Strategy 'db_min': Querying table for minimum clock (may be slow)")
|
|
||||||
min_clock = self.get_table_min_clock(table)
|
|
||||||
|
|
||||||
if not min_clock:
|
|
||||||
# Empty table. Start from NOW
|
|
||||||
start_dt = self.truncate_date(datetime.now(), period)
|
|
||||||
else:
|
|
||||||
# Table has data.
|
|
||||||
start_dt = self.truncate_date(min_clock, period)
|
|
||||||
|
|
||||||
# Build list of partitions from start_dt up to NOW + premake
|
|
||||||
target_dt = self.get_next_date(self.truncate_date(datetime.now(), period), period, premake)
|
target_dt = self.get_next_date(self.truncate_date(datetime.now(), period), period, premake)
|
||||||
|
|
||||||
curr = start_dt
|
|
||||||
partitions_def = {}
|
|
||||||
|
|
||||||
# If we have an archive partition, add it first
|
|
||||||
if p_archive_ts:
|
|
||||||
partitions_def['p_archive'] = str(p_archive_ts)
|
|
||||||
|
|
||||||
while curr < target_dt:
|
|
||||||
name = self.get_partition_name(curr, period)
|
|
||||||
desc = self.get_partition_description(curr, period)
|
|
||||||
partitions_def[name] = desc
|
|
||||||
curr = self.get_next_date(curr, period, 1)
|
|
||||||
|
|
||||||
# Re-doing the loop to be cleaner on types
|
|
||||||
parts_sql = []
|
parts_sql = []
|
||||||
|
|
||||||
# 1. Archive Partition
|
|
||||||
if p_archive_ts:
|
if p_archive_ts:
|
||||||
parts_sql.append(f"PARTITION p_archive VALUES LESS THAN ({p_archive_ts}) ENGINE = InnoDB")
|
parts_sql.append(f"PARTITION p_archive VALUES LESS THAN ({p_archive_ts}) ENGINE = InnoDB")
|
||||||
|
|
||||||
# 2. Granular Partitions
|
|
||||||
# We need to iterate again from start_dt
|
|
||||||
curr = start_dt
|
curr = start_dt
|
||||||
while curr < target_dt:
|
while curr < target_dt:
|
||||||
name = self.get_partition_name(curr, period)
|
name = self.get_partition_name(curr, period)
|
||||||
desc_date_str = self.get_partition_description(curr, period) # Returns "YYYY-MM-DD HH:MM:SS"
|
desc_date_str = self.get_partition_description(curr, period)
|
||||||
parts_sql.append(PARTITION_TEMPLATE % (name, desc_date_str))
|
parts_sql.append(PARTITION_TEMPLATE % (name, desc_date_str))
|
||||||
curr = self.get_next_date(curr, period, 1)
|
curr = self.get_next_date(curr, period, 1)
|
||||||
|
|
||||||
|
if not parts_sql:
|
||||||
|
self.logger.warning(f"No partitions generated for {table}")
|
||||||
|
return
|
||||||
|
|
||||||
query = f"ALTER TABLE `{table}` PARTITION BY RANGE (`clock`) (\n" + ",\n".join(parts_sql) + "\n)"
|
query = f"ALTER TABLE `{table}` PARTITION BY RANGE (`clock`) (\n" + ",\n".join(parts_sql) + "\n)"
|
||||||
self.logger.info(f"Applying initial partitioning to {table} ({len(parts_sql)} partitions)")
|
self.logger.info(f"Applying initial partitioning to {table} ({len(parts_sql)} partitions)")
|
||||||
self.execute_query(query)
|
self.execute_query(query)
|
||||||
|
|
||||||
|
def _get_configured_tables(self) -> List[str]:
|
||||||
|
"""Return a list of all tables configured for partitioning."""
|
||||||
|
tables_set = set()
|
||||||
|
partitions_conf = self.config.get('partitions', {})
|
||||||
|
for tables in partitions_conf.values():
|
||||||
|
if not tables: continue
|
||||||
|
for item in tables:
|
||||||
|
tables_set.add(list(item.keys())[0])
|
||||||
|
return list(tables_set)
|
||||||
|
|
||||||
|
def clean_housekeeper_table(self):
|
||||||
|
"""
|
||||||
|
Clean up Zabbix housekeeper table during initialization.
|
||||||
|
Removes pending tasks ONLY for configured tables to prevent conflicts.
|
||||||
|
"""
|
||||||
|
configured_tables = self._get_configured_tables()
|
||||||
|
if not configured_tables:
|
||||||
|
return
|
||||||
|
|
||||||
|
self.logger.info(f"Cleaning up 'housekeeper' table for: {', '.join(configured_tables)}")
|
||||||
|
|
||||||
|
# Construct IN clause
|
||||||
|
placeholders = ', '.join(['%s'] * len(configured_tables))
|
||||||
|
query = f"DELETE FROM `housekeeper` WHERE `tablename` IN ({placeholders})"
|
||||||
|
self.execute_query(query, configured_tables)
|
||||||
|
|
||||||
|
def check_housekeeper_execution(self):
|
||||||
|
"""
|
||||||
|
Check if Zabbix internal housekeeper is running for configured partitioned tables.
|
||||||
|
"""
|
||||||
|
configured_tables = self._get_configured_tables()
|
||||||
|
if not configured_tables:
|
||||||
|
return
|
||||||
|
|
||||||
|
query = "SELECT DISTINCT `tablename` FROM `housekeeper` WHERE `tablename` != 'events'"
|
||||||
|
rows = self.execute_query(query, fetch='all')
|
||||||
|
|
||||||
|
if rows:
|
||||||
|
found_tables = {r[0] for r in rows}
|
||||||
|
configured_set = set(configured_tables)
|
||||||
|
conflicts = found_tables.intersection(configured_set)
|
||||||
|
|
||||||
|
if conflicts:
|
||||||
|
self.logger.error(f"CRITICAL: Found pending housekeeper tasks for partitioned tables: {', '.join(conflicts)}")
|
||||||
|
self.logger.error("Please DISABLE Zabbix internal housekeeper for these tables in Administration -> General -> Housekeeping!")
|
||||||
|
|
||||||
|
def initialize_partitioning_fast(self, table: str, period: str, premake: int, retention_str: str):
|
||||||
|
"""Fast Init: Use retention period to determine start date."""
|
||||||
|
self.logger.info(f"Initializing partitioning for {table} (Fast Mode)")
|
||||||
|
if not self._check_init_prerequisites(table): return
|
||||||
|
|
||||||
|
self.logger.info(f"Strategy 'fast-init': Calculating start date from retention ({retention_str})")
|
||||||
|
retention_date = self.get_lookback_date(retention_str)
|
||||||
|
start_dt = self.truncate_date(retention_date, period)
|
||||||
|
p_archive_ts = int(start_dt.timestamp())
|
||||||
|
|
||||||
|
self._generate_init_sql(table, period, start_dt, premake, p_archive_ts)
|
||||||
|
|
||||||
|
def initialize_partitioning_scan(self, table: str, period: str, premake: int):
|
||||||
|
"""Scan Init: Scan table for minimum clock value."""
|
||||||
|
self.logger.info(f"Initializing partitioning for {table} (Scan Mode)")
|
||||||
|
if not self._check_init_prerequisites(table): return
|
||||||
|
|
||||||
|
self.logger.info("Strategy 'db_min': Querying table for minimum clock (may be slow)")
|
||||||
|
min_clock = self.get_table_min_clock(table)
|
||||||
|
|
||||||
|
if not min_clock:
|
||||||
|
start_dt = self.truncate_date(datetime.now(), period)
|
||||||
|
else:
|
||||||
|
start_dt = self.truncate_date(min_clock, period)
|
||||||
|
|
||||||
|
self._generate_init_sql(table, period, start_dt, premake)
|
||||||
|
|
||||||
def discovery(self):
|
def discovery(self):
|
||||||
"""Output Zabbix Low-Level Discovery logic JSON."""
|
"""Output Zabbix Low-Level Discovery logic JSON."""
|
||||||
partitions_conf = self.config.get('partitions', {})
|
partitions_conf = self.config.get('partitions', {})
|
||||||
@@ -554,32 +607,6 @@ class ZabbixPartitioner:
|
|||||||
self.discovery()
|
self.discovery()
|
||||||
return
|
return
|
||||||
|
|
||||||
# --- Check Mode ---
|
|
||||||
if mode == 'check':
|
|
||||||
if not target_table:
|
|
||||||
# Check all and print simple status? Or error?
|
|
||||||
# Zabbix usually queries one by one.
|
|
||||||
# Implementing simple check which returns days for specific table
|
|
||||||
raise ConfigurationError("Target table required for check mode")
|
|
||||||
|
|
||||||
# Find period for table
|
|
||||||
found_period = None
|
|
||||||
for period, tables in partitions_conf.items():
|
|
||||||
for item in tables:
|
|
||||||
if list(item.keys())[0] == target_table:
|
|
||||||
found_period = period
|
|
||||||
break
|
|
||||||
if found_period: break
|
|
||||||
|
|
||||||
if not found_period:
|
|
||||||
# Table not in config?
|
|
||||||
print("-1") # Error code
|
|
||||||
return
|
|
||||||
|
|
||||||
days_left = self.check_partitions_coverage(target_table, found_period)
|
|
||||||
print(days_left)
|
|
||||||
return
|
|
||||||
|
|
||||||
# --- Stats Mode ---
|
# --- Stats Mode ---
|
||||||
if mode == 'stats':
|
if mode == 'stats':
|
||||||
if not target_table:
|
if not target_table:
|
||||||
@@ -593,6 +620,11 @@ class ZabbixPartitioner:
|
|||||||
self.check_compatibility()
|
self.check_compatibility()
|
||||||
premake = self.config.get('premake', 10)
|
premake = self.config.get('premake', 10)
|
||||||
|
|
||||||
|
if mode == 'init':
|
||||||
|
self.clean_housekeeper_table()
|
||||||
|
else:
|
||||||
|
self.check_housekeeper_execution()
|
||||||
|
|
||||||
for period, tables in partitions_conf.items():
|
for period, tables in partitions_conf.items():
|
||||||
if not tables:
|
if not tables:
|
||||||
continue
|
continue
|
||||||
@@ -601,12 +633,24 @@ class ZabbixPartitioner:
|
|||||||
table = list(item.keys())[0]
|
table = list(item.keys())[0]
|
||||||
retention = item[table]
|
retention = item[table]
|
||||||
|
|
||||||
|
if not self.validate_table_exists(table):
|
||||||
|
continue
|
||||||
|
|
||||||
if mode == 'init':
|
if mode == 'init':
|
||||||
self.initialize_partitioning(table, period, premake, retention, fast_init=self.fast_init)
|
try:
|
||||||
|
if self.fast_init:
|
||||||
|
self.initialize_partitioning_fast(table, period, premake, retention)
|
||||||
|
else:
|
||||||
|
self.initialize_partitioning_scan(table, period, premake)
|
||||||
|
except ConfigurationError as e:
|
||||||
|
raise ConfigurationError(f"Table '{table}': {e}")
|
||||||
else:
|
else:
|
||||||
# Maintenance mode (Add new, remove old)
|
# Maintenance mode (Add new, remove old)
|
||||||
|
try:
|
||||||
self.create_future_partitions(table, period, premake)
|
self.create_future_partitions(table, period, premake)
|
||||||
self.remove_old_partitions(table, retention)
|
self.remove_old_partitions(table, retention)
|
||||||
|
except ConfigurationError as e:
|
||||||
|
raise ConfigurationError(f"Table '{table}': {e}")
|
||||||
|
|
||||||
# Housekeeping extras
|
# Housekeeping extras
|
||||||
if mode != 'init' and not self.dry_run:
|
if mode != 'init' and not self.dry_run:
|
||||||
@@ -615,6 +659,31 @@ class ZabbixPartitioner:
|
|||||||
if mode != 'init' and not self.dry_run:
|
if mode != 'init' and not self.dry_run:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
def get_retention_input(prompt: str, default: str = None, allow_empty: bool = False) -> str:
|
||||||
|
"""Helper to get and validate retention period input."""
|
||||||
|
while True:
|
||||||
|
val = input(prompt).strip()
|
||||||
|
|
||||||
|
# Handle Empty Input
|
||||||
|
if not val:
|
||||||
|
if default:
|
||||||
|
return default
|
||||||
|
if allow_empty:
|
||||||
|
return ""
|
||||||
|
# If no default and not allow_empty, continue loop
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Handle Unit-less Input (Reject)
|
||||||
|
if val.isdigit():
|
||||||
|
print(f"Error: '{val}' is missing a unit. Please use 'd', 'w', 'm', or 'y' (e.g., {val}d).")
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Validate Format
|
||||||
|
if re.search(PART_PERIOD_REGEX, val):
|
||||||
|
return val
|
||||||
|
|
||||||
|
print("Invalid format. Use format like 30d, 12w, 1y.")
|
||||||
|
|
||||||
def run_wizard():
|
def run_wizard():
|
||||||
print("Welcome to Zabbix Partitioning Wizard")
|
print("Welcome to Zabbix Partitioning Wizard")
|
||||||
print("-------------------------------------")
|
print("-------------------------------------")
|
||||||
@@ -645,11 +714,26 @@ def run_wizard():
|
|||||||
config['database']['passwd'] = input("Database Password: ").strip()
|
config['database']['passwd'] = input("Database Password: ").strip()
|
||||||
config['database']['db'] = input("Database Name [zabbix]: ").strip() or 'zabbix'
|
config['database']['db'] = input("Database Name [zabbix]: ").strip() or 'zabbix'
|
||||||
|
|
||||||
|
# 1.1 SSL
|
||||||
|
if input("Use SSL/TLS for connection? [y/N]: ").lower().strip() == 'y':
|
||||||
|
print(" Mode 1: Managed/Cloud DB (Use system CAs, e.g. Azure/AWS)")
|
||||||
|
print(" Mode 2: Custom Certificates (Provide ca, cert, key)")
|
||||||
|
|
||||||
|
if input(" Use custom certificates? [y/N]: ").lower().strip() == 'y':
|
||||||
|
ssl_conf = {}
|
||||||
|
ssl_conf['ca'] = input(" CA Certificate Path: ").strip()
|
||||||
|
ssl_conf['cert'] = input(" Client Certificate Path: ").strip()
|
||||||
|
ssl_conf['key'] = input(" Client Key Path: ").strip()
|
||||||
|
# Filter empty
|
||||||
|
config['database']['ssl'] = {k: v for k, v in ssl_conf.items() if v}
|
||||||
|
else:
|
||||||
|
config['database']['ssl'] = 'required'
|
||||||
|
|
||||||
# 2. Auditlog
|
# 2. Auditlog
|
||||||
print("\n[Auditlog]")
|
print("\n[Auditlog]")
|
||||||
print("Note: To partition 'auditlog', ensure its Primary Key includes the 'clock' column.")
|
print("Note: To partition 'auditlog', ensure its Primary Key includes the 'clock' column.")
|
||||||
if input("Partition 'auditlog' table? [y/N]: ").lower().strip() == 'y':
|
if input("Partition 'auditlog' table? [y/N]: ").lower().strip() == 'y':
|
||||||
ret = input("Auditlog retention (e.g. 365d) [365d]: ").strip() or '365d'
|
ret = get_retention_input("Auditlog retention (e.g. 365d) [365d]: ", "365d")
|
||||||
config['partitions']['weekly'].append({'auditlog': ret})
|
config['partitions']['weekly'].append({'auditlog': ret})
|
||||||
|
|
||||||
# 3. History Tables
|
# 3. History Tables
|
||||||
@@ -659,12 +743,12 @@ def run_wizard():
|
|||||||
print("\n[History Tables]")
|
print("\n[History Tables]")
|
||||||
# Separate logic as requested
|
# Separate logic as requested
|
||||||
if input("Set SAME retention for all history tables? [Y/n]: ").lower().strip() != 'n':
|
if input("Set SAME retention for all history tables? [Y/n]: ").lower().strip() != 'n':
|
||||||
ret = input("Retention for all history tables (e.g. 30d) [30d]: ").strip() or '30d'
|
ret = get_retention_input("Retention for all history tables (e.g. 30d) [30d]: ", "30d")
|
||||||
for t in history_tables:
|
for t in history_tables:
|
||||||
config['partitions']['daily'].append({t: ret})
|
config['partitions']['daily'].append({t: ret})
|
||||||
else:
|
else:
|
||||||
for t in history_tables:
|
for t in history_tables:
|
||||||
ret = input(f"Retention for '{t}' (e.g. 30d, skip to ignore): ").strip()
|
ret = get_retention_input(f"Retention for '{t}' (e.g. 30d, skip to ignore): ", allow_empty=True)
|
||||||
if ret:
|
if ret:
|
||||||
config['partitions']['daily'].append({t: ret})
|
config['partitions']['daily'].append({t: ret})
|
||||||
|
|
||||||
@@ -672,12 +756,12 @@ def run_wizard():
|
|||||||
trends_tables = ['trends', 'trends_uint']
|
trends_tables = ['trends', 'trends_uint']
|
||||||
print("\n[Trends Tables]")
|
print("\n[Trends Tables]")
|
||||||
if input("Set SAME retention for all trends tables? [Y/n]: ").lower().strip() != 'n':
|
if input("Set SAME retention for all trends tables? [Y/n]: ").lower().strip() != 'n':
|
||||||
ret = input("Retention for all trends tables (e.g. 365d) [365d]: ").strip() or '365d'
|
ret = get_retention_input("Retention for all trends tables (e.g. 365d) [365d]: ", "365d")
|
||||||
for t in trends_tables:
|
for t in trends_tables:
|
||||||
config['partitions']['monthly'].append({t: ret})
|
config['partitions']['monthly'].append({t: ret})
|
||||||
else:
|
else:
|
||||||
for t in trends_tables:
|
for t in trends_tables:
|
||||||
ret = input(f"Retention for '{t}' (e.g. 365d, skip to ignore): ").strip()
|
ret = get_retention_input(f"Retention for '{t}' (e.g. 365d, skip to ignore): ", allow_empty=True)
|
||||||
if ret:
|
if ret:
|
||||||
config['partitions']['monthly'].append({t: ret})
|
config['partitions']['monthly'].append({t: ret})
|
||||||
|
|
||||||
@@ -731,22 +815,43 @@ def setup_logging(config_log_type: str, verbose: bool = False):
|
|||||||
logger.addHandler(handler)
|
logger.addHandler(handler)
|
||||||
|
|
||||||
def parse_args():
|
def parse_args():
|
||||||
parser = argparse.ArgumentParser(description='Zabbix Partitioning Manager')
|
parser = argparse.ArgumentParser(
|
||||||
parser.add_argument('-c', '--config', default='/etc/zabbix/zabbix_partitioning.conf', help='Config file path')
|
description='Zabbix Database Partitioning Script',
|
||||||
parser.add_argument('-i', '--init', action='store_true', help='Initialize partitions')
|
epilog='''
|
||||||
parser.add_argument('-r', '--dry-run', action='store_true', help='Simulate queries')
|
Examples:
|
||||||
parser.add_argument('-v', '--verbose', action='store_true', help='Enable debug logging')
|
# 1. Interactive Configuration (Initial config file creation)
|
||||||
|
%(prog)s --wizard
|
||||||
|
|
||||||
# Monitoring args
|
# 2. Initialization (First Run)
|
||||||
parser.add_argument('--discovery', action='store_true', help='Output Zabbix LLD JSON')
|
# A. Standard (Scans DB for oldest record - Recommended):
|
||||||
parser.add_argument('--check-days', type=str, help='Check days of future partitions left for table', metavar='TABLE')
|
%(prog)s --init
|
||||||
parser.add_argument('--stats', type=str, help='Output detailed table statistics in JSON', metavar='TABLE')
|
|
||||||
|
|
||||||
# Wizard & Flags
|
# B. Fast (Start from retention period - Best for large DBs):
|
||||||
parser.add_argument('--wizard', action='store_true', help='Launch interactive configuration wizard')
|
%(prog)s --fast-init
|
||||||
parser.add_argument('--fast-init', action='store_true', help='Skip MIN(clock) check during init, start from retention')
|
|
||||||
|
|
||||||
parser.add_argument('-V', '--version', action='version', version=f'%(prog)s {VERSION}', help='Show version and exit')
|
# 3. Regular Maintenance (Cron/Systemd)
|
||||||
|
# Creates future partitions and drops old ones.
|
||||||
|
%(prog)s
|
||||||
|
|
||||||
|
# 4. Monitoring (Zabbix Integration)
|
||||||
|
# Discovery (LLD): %(prog)s --discovery
|
||||||
|
# Statistics (JSON): %(prog)s --stats history
|
||||||
|
''',
|
||||||
|
formatter_class=argparse.RawDescriptionHelpFormatter
|
||||||
|
)
|
||||||
|
parser.add_argument('--config','-c', default='/etc/zabbix/zabbix_partitioning.conf', help='Path to configuration file')
|
||||||
|
|
||||||
|
# Mutually Exclusive Actions
|
||||||
|
group = parser.add_mutually_exclusive_group()
|
||||||
|
group.add_argument('--init', '-i', action='store_true', help='Initialize partitions (Standard: Scans DB for oldest record)')
|
||||||
|
group.add_argument('--fast-init', '-f', action='store_true', help='Initialize partitions (Fast: Starts FROM retention period, skips scan)')
|
||||||
|
group.add_argument('--discovery', '-d', action='store_true', help='Output Zabbix Low-Level Discovery (LLD) JSON')
|
||||||
|
group.add_argument('--stats', '-s', type=str, help='Output table statistics (Size, Count, Usage) in JSON', metavar='TABLE')
|
||||||
|
group.add_argument('--wizard', '-w', action='store_true', help='Launch interactive configuration wizard')
|
||||||
|
|
||||||
|
parser.add_argument('--version', '-V', action='version', version=f'%(prog)s {VERSION}', help='Show version and exit')
|
||||||
|
parser.add_argument('--verbose', '-vv', action='store_true', help='Enable debug logging')
|
||||||
|
parser.add_argument('--dry-run', '-r', action='store_true', help='Simulate queries without executing them')
|
||||||
|
|
||||||
return parser.parse_args()
|
return parser.parse_args()
|
||||||
|
|
||||||
@@ -781,16 +886,16 @@ def main():
|
|||||||
config['logging'] = 'console' # Force console for discovery? Or suppress?
|
config['logging'] = 'console' # Force console for discovery? Or suppress?
|
||||||
# actually we don't want logs mixing with JSON output
|
# actually we don't want logs mixing with JSON output
|
||||||
# so checking mode before setup logging
|
# so checking mode before setup logging
|
||||||
elif args.check_days:
|
|
||||||
mode = 'check'
|
|
||||||
target = args.check_days
|
|
||||||
elif args.stats:
|
elif args.stats:
|
||||||
mode = 'stats'
|
mode = 'stats'
|
||||||
target = args.stats
|
target = args.stats
|
||||||
elif args.init: mode = 'init'
|
elif args.init:
|
||||||
|
mode = 'init'
|
||||||
|
elif args.fast_init:
|
||||||
|
mode = 'init'
|
||||||
|
|
||||||
# Setup logging
|
# Setup logging
|
||||||
if mode in ['discovery', 'check', 'stats']:
|
if mode in ['discovery', 'stats']:
|
||||||
logging.basicConfig(level=logging.ERROR) # Only show critical errors
|
logging.basicConfig(level=logging.ERROR) # Only show critical errors
|
||||||
else:
|
else:
|
||||||
setup_logging(config.get('logging', 'console'), verbose=args.verbose)
|
setup_logging(config.get('logging', 'console'), verbose=args.verbose)
|
||||||
|
|||||||
Reference in New Issue
Block a user