Merge test branch changes excluding tests directory
This commit is contained in:
15
postgresql/CHANGELOG.md
Normal file
15
postgresql/CHANGELOG.md
Normal file
@@ -0,0 +1,15 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
## [7-2] - 2026-04-29
|
||||
|
||||
### Added
|
||||
- **Housekeeper Interceptor**: Added a PostgreSQL `BEFORE INSERT` trigger to the `housekeeper` table. This trigger dynamically checks `partitions.config` and silently discards Housekeeper tasks (inserts) for any tables managed by the partitioning system. This prevents Zabbix from filling the `housekeeper` table with unnecessary tasks, eliminating Sequential Scans and Table Bloat on the `housekeeper` table, while still allowing tasks for unpartitioned tables (like `events`) to be recorded normally.
|
||||
|
||||
## [7-1] - Previous Release
|
||||
|
||||
### Added
|
||||
- Zabbix 7.0 and 7.4 compatibility.
|
||||
- Special handling for the `auditlog` table primary key changes in Zabbix 7.0+.
|
||||
- Dynamic creation of partitions and retention management logic.
|
||||
@@ -15,6 +15,14 @@ CREATE TABLE IF NOT EXISTS partitions.config (
|
||||
PRIMARY KEY (table_name)
|
||||
);
|
||||
|
||||
-- Ensure the future_partitions column exists for users upgrading from older versions
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (SELECT 1 FROM information_schema.columns WHERE table_schema = 'partitions' AND table_name = 'config' AND column_name = 'future_partitions') THEN
|
||||
ALTER TABLE partitions.config ADD COLUMN future_partitions integer NOT NULL DEFAULT 5;
|
||||
END IF;
|
||||
END $$;
|
||||
|
||||
-- Table to track installed version of the partitioning solution
|
||||
CREATE TABLE IF NOT EXISTS partitions.version (
|
||||
version text PRIMARY KEY,
|
||||
|
||||
@@ -74,11 +74,24 @@ BEGIN
|
||||
END $$;
|
||||
|
||||
-- ==========================================================================
|
||||
-- IMPORTANT: If the Zabbix Server connects with a non-superuser (e.g., 'zabbix'),
|
||||
-- that user MUST have access to the partitions schema for the housekeeper trigger
|
||||
-- to work. Without these GRANTs, every INSERT into housekeeper will FAIL.
|
||||
-- Uncomment and adjust the username below:
|
||||
-- IMPORTANT: To ensure the Zabbix Server can insert into the housekeeper
|
||||
-- table without permission issues, we automatically grant access to the
|
||||
-- partitions schema to the owner of the housekeeper table.
|
||||
-- ==========================================================================
|
||||
-- GRANT USAGE ON SCHEMA partitions TO zabbix;
|
||||
-- GRANT SELECT ON partitions.config TO zabbix;
|
||||
DO $$
|
||||
DECLARE
|
||||
v_owner text;
|
||||
BEGIN
|
||||
SELECT pg_get_userbyid(c.relowner) INTO v_owner
|
||||
FROM pg_class c
|
||||
JOIN pg_namespace n ON n.oid = c.relnamespace
|
||||
WHERE c.relname = 'housekeeper' AND pg_table_is_visible(c.oid);
|
||||
|
||||
IF v_owner IS NOT NULL THEN
|
||||
EXECUTE format('GRANT USAGE ON SCHEMA partitions TO %I', v_owner);
|
||||
EXECUTE format('GRANT SELECT ON partitions.config TO %I', v_owner);
|
||||
RAISE NOTICE 'Granted partitions schema permissions to housekeeper owner: %', v_owner;
|
||||
ELSE
|
||||
RAISE WARNING 'housekeeper table not found, could not grant permissions automatically!';
|
||||
END IF;
|
||||
END $$;
|
||||
|
||||
@@ -31,3 +31,12 @@ LEFT JOIN pg_inherits ON pg_inherits.inhparent = parent.oid
|
||||
LEFT JOIN pg_class child ON pg_inherits.inhrelid = child.oid
|
||||
WHERE parent.relkind = 'p' -- Only partitioned tables
|
||||
GROUP BY parent.relname, c.table_name, c.period, c.keep_history, c.future_partitions, c.last_updated;
|
||||
|
||||
-- ============================================================================
|
||||
-- IMPORTANT: Since DROP VIEW / CREATE VIEW resets permissions, we automatically
|
||||
-- grant read access to PUBLIC. This ensures that whatever monitoring user
|
||||
-- (e.g. zbx_monitor) is connecting to the DB will always be able to read
|
||||
-- the monitoring stats without manual intervention after upgrades.
|
||||
-- ==========================================================================
|
||||
GRANT USAGE ON SCHEMA partitions TO PUBLIC;
|
||||
GRANT SELECT ON partitions.monitoring TO PUBLIC;
|
||||
|
||||
@@ -53,32 +53,34 @@ Once inside `psql`, connect to your Zabbix database (usually named `zabbix`):
|
||||
|
||||
## Step 3: Execute Installation Scripts
|
||||
|
||||
Run the scripts in the following exact order. You can use the `\i` command in `psql` if you are in the `procedures` directory, or specify the full path.
|
||||
Run the scripts in the following exact order. You can execute them directly from your bash terminal, from within the interactive `psql` console, or by pasting their contents into a GUI tool (like pgAdmin).
|
||||
|
||||
**1. Create the partitioning schema and config tables:**
|
||||
> [!NOTE]
|
||||
> **Zabbix 8.0+ Users:** Zabbix 8.0 introduced a new `history_json` table. Before running the script below, open `00_schema_create.sql` in a text editor and uncomment the lines specifically marked for Zabbix 8.0 at the end of the history tables block.
|
||||
> **Zabbix 8.0+ Users:** Zabbix 8.0 introduced a new `history_json` table. Before running the first script, open `00_schema_create.sql` in a text editor and uncomment the lines specifically marked for Zabbix 8.0 at the end of the history tables block.
|
||||
|
||||
### Option A: From bash terminal (Recommended)
|
||||
If you are in your normal Linux terminal, run these commands one by one:
|
||||
```bash
|
||||
psql -U postgres -d zabbix -f 00_schema_create.sql
|
||||
psql -U postgres -d zabbix -f 01_maintenance.sql
|
||||
# MIGRATION STEP: The next script renames your existing large tables to `_old`
|
||||
# and instantly creates new partitioned tables. This might take a few moments.
|
||||
psql -U postgres -d zabbix -f 02_enable_partitioning.sql
|
||||
psql -U postgres -d zabbix -f 03_monitoring_view.sql
|
||||
```
|
||||
|
||||
### Option B: Inside interactive `psql`
|
||||
If you already connected via `psql` (as in Step 2), use the `\i` command:
|
||||
```sql
|
||||
\i 00_schema_create.sql
|
||||
```
|
||||
|
||||
**2. Install the maintenance logic and functions:**
|
||||
```sql
|
||||
\i 01_maintenance.sql
|
||||
```
|
||||
|
||||
**3. Enable Partitioning (MIGRATION STEP):**
|
||||
*This step renames your existing large tables to `_old` and instantly creates new partitioned tables. This might take a few moments.*
|
||||
```sql
|
||||
\i 02_enable_partitioning.sql
|
||||
```
|
||||
|
||||
**4. Create the Monitoring View:**
|
||||
```sql
|
||||
\i 03_monitoring_view.sql
|
||||
```
|
||||
|
||||
### Option C: GUI Tools (pgAdmin, DBeaver, etc.)
|
||||
If you use a visual database manager, simply open each file in your query editor and execute them one by one in the exact order (00, 01, 02, 03).
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Schedule Automated Maintenance
|
||||
@@ -127,6 +129,7 @@ SELECT
|
||||
FROM partitions.monitoring;
|
||||
EOF
|
||||
```
|
||||
*(Note: If your Zabbix Agent is configured to look for custom queries in a different directory like `/etc/zabbix/postgresql/sql/`, modify the destination path above).*
|
||||
|
||||
2. Configure the PostgreSQL Plugin by editing `/etc/zabbix/zabbix_agent2.d/plugins.d/postgresql.conf`. Ensure you have defined a session (e.g., `MY_DB`) and enabled custom queries:
|
||||
|
||||
@@ -159,3 +162,20 @@ sudo systemctl restart zabbix-agent2
|
||||
8. Click **Update**.
|
||||
|
||||
**Congratulations!** Your Zabbix database is now fully partitioned, optimized, and monitored.
|
||||
|
||||
---
|
||||
|
||||
## Step 8: Upgrades & Updates
|
||||
|
||||
To apply future updates or fixes to the partitioning logic, the scripts are fully idempotent and safe to re-run on a live database.
|
||||
Simply connect to `psql` and re-execute the updated scripts in order:
|
||||
|
||||
```bash
|
||||
psql -U postgres -d zabbix -f 00_schema_create.sql
|
||||
psql -U postgres -d zabbix -f 01_maintenance.sql
|
||||
psql -U postgres -d zabbix -f 02_enable_partitioning.sql
|
||||
psql -U postgres -d zabbix -f 03_monitoring_view.sql
|
||||
```
|
||||
|
||||
> [!WARNING]
|
||||
> If you update the agent SQL query, always remember to restart the agent (`sudo systemctl restart zabbix-agent2`) so it flushes its cache!
|
||||
|
||||
@@ -231,8 +231,8 @@ GRANT USAGE ON SCHEMA partitions TO zbxpart_monitor;
|
||||
GRANT SELECT ON partitions.monitoring TO zbxpart_monitor;
|
||||
```
|
||||
|
||||
> [!WARNING]
|
||||
> Because `03_monitoring_view.sql` uses a `DROP VIEW` command to apply updates, re-running the script will destroy all previously assigned `GRANT` permissions. If you ever update the view script, you **must** manually re-run the `GRANT SELECT` command above to restore access for the `zbxpart_monitor` user!
|
||||
> [!NOTE]
|
||||
> The `03_monitoring_view.sql` script is designed to automatically execute `GRANT SELECT ON partitions.monitoring TO PUBLIC;` so that read access is never lost during upgrades when the view is recreated.
|
||||
|
||||
## Implementation Details
|
||||
|
||||
|
||||
@@ -57,7 +57,7 @@ zabbix_export:
|
||||
preprocessing:
|
||||
- type: JSONPATH
|
||||
parameters:
|
||||
- '$.[?(@.table_name == "{#TABLE_NAME}")].future_partitions.first()'
|
||||
- '$.[?(@.table_name == "{#TABLE_NAME}")].actual_future_partitions.first()'
|
||||
master_item:
|
||||
key: 'pgsql.custom.query["{$PG.CONNSTRING.AGENT2}",,,"{$PG.DBNAME}","partitions.get_all"]'
|
||||
tags:
|
||||
|
||||
Reference in New Issue
Block a user