pax_global_header00006660000000000000000000000064133607270740014523gustar00rootroot0000000000000052 comment=1b3c71bc1b8fe117ce087c31ee73c225d15d19a3 powa-archivist-REL_3_2_0/000077500000000000000000000000001336072707400153315ustar00rootroot00000000000000powa-archivist-REL_3_2_0/.gitignore000066400000000000000000000000301336072707400173120ustar00rootroot00000000000000*.o *.so *.zip results/ powa-archivist-REL_3_2_0/CHANGELOG.md000066400000000000000000000121101336072707400171350ustar00rootroot00000000000000## 3.2.0 (2018-10-14) - New features: - Add support for pg_wait_sampling extension (Julien Rouhaud) - Miscellaneous: - Reduce logs when PoWA is deactivated (Julien Rouhaud) - Bugfix: - Fix possible bug if an error happens during the stats retrieval (Julien Rouhaud) ## 3.1.2 (2018-05-30) - Miscellaneous: - Add pg11 compatibility (Julien Rouhaud) - Catch errors in Debian packaging test script (Christoph Berg, spotted by Niels Thykier) ## 3.1.1 (2017-09-19) - Bugfix: - Fix unsafe coding with sighup handler (Andreas Seltenreich, Julien Rouhaud) - Make sure we wait at least powa.frequency between two snapshot (Marc Cousin and Julien Rouhaud) - Fix win32 portability of compute_powa_frequeny() (Julien Rouhaud) - Don't try to read dbentry->tables if it's NULL (Julien Rouhaud) - Fix compilation for platform with HAVE_CLOCK_GETTIME (Julien Rouhaud, reported by Maxence Ahlouche) - Miscellaneous: - Add pg10 Compatibility (Julien Rouhaud) - Only execute once the powa_stat functions (Julien Rouhaud) ## 3.1.0 (2016-07-29) - Fix issue leading to impossibility to stop the worker without shutting down the database - Fix cluster wide statistics to get fresh values - Report PoWA collector activity in pg_stat_activity and process title - add a new powa.debug parameter - Purge at the same frequency as we coalesce. We just don't do both at the same iteration - Fix bloat issue - Add + and / operators on powa types to get delta and counters per second given two records ## 3.0.1 (2016-02-09) - Don't track 2PC related statements, as they're not normalized by pg_stat_statements. Upgrade script will do all the needed cleanup. - Restore the install_all.sql file to easily setup PoWA. - Maintain a cache of pg_database to allow seeing dropped database in the UI. See issue https://github.com/powa-team/powa/issues/63 - Don't try to load PoWA if it's not in shared_preload_libraries ## 3.0.0 (2015-11-06) Please not that there is not upgrade to switch to this version. You need to remove the old one and install the new 3.0.0 version. - Handle pg_qualtats 0.0.7 - Sample cluster wide statistics, for relations and functions - Fix the powa reset function, and rename it to powa_reset() - Add min/max records to improve performance when analyzing big time interval - Allow disabling some statistics sampling - Handle pg_track_settings extension - Add a GUC to ignore some users activity in sampled data ## 2.0.1 (2015-07-27) - Handle creation/suppression of supported extensions. - Remove the install_all script ## 2.0 (2015-02-06) Major rework of the extension. PoWA 2 is now only compatible with PostgreSQL version 9.4 and above. PoWA 2 is also now compatible with external extensions, such as [pg_qualstats](https://github.com/powa-team/pg_qualstats) or [pg_stat_kcache](https://github.com/powa-team/pg_stat_kcache). Third-part extensions can also now be implemented easily. The UI is also now in a [new repository](https://github.com/powa-team/powa-web), with more frequent release cycle. ## 1.2.1 (2015-01-16) No changes in core. New features and changes in UI : - UI is now compatible with mojolicious 5.0 and more - UI can now connect to multiple servers, and credentials can be specified for each server - Use ISO 8601 timestamp format - Add POWA_CONFIG_FILE variable to specify config file location - Better charts display on small screens When upgrading from 1.2: - No change on the extension - the format of the database section of the powa.conf has changed, to allow multiple servers specification. Please read INSTALL.md for more details about it. ## 1.2 (2014-10-27) News features and fixes in core : - Display more metrics : temporary data, I/O time, average runtime - Fix timestamp for snapshots - DEALLOCATE and BEGIN statements are now ignored - PoWA history tables are now marked as "to be dumped" by pg_dump - Improve performance for "per database aggregated stats" News features and changes in UI : - Follow the selected time interval between each page - Add a title to each page - Display metrics for each query page - Move database selector as a menu entry - Display human readable metrics - Fix empty graph bug When upgrading from older versions : - Upgrade the core with ALTER EXTENSION powa UPDATE. - The format of the database section of the powa.conf has changed. The new format is : "dbname" : "powa", "host" : "127.0.0.1", "port" : "5432", (instead of one line containing the dbi:Pg connection info) ## 1.1 (2014-08-18) **POWA is now production ready** Features: - Various UI improvments - More documentation - New demo mode - Plugin support - The code is now under the PostgreSQL license - New website - New logo Bug fixes: - Use a temporary table for unpacked records to avoid unnecessary bloat ## 1.0 (2014-06-13) **Hello World ! This is the first public release of POWA** Features: - Web UI based on Mojolicious - Graph and dynamic charts - Packed the code as an extension - PL functions powa-archivist-REL_3_2_0/CONTRIBUTORS.md000066400000000000000000000006731336072707400176160ustar00rootroot00000000000000Contributors to PoWA : * Marc Cousin * Julien Rouhaud * Damien Clochard * Thomas Reiss * Hyunjun Kim * Raghu Ram * Rodolphe Quiédeville * Ahmed Bessifi * Christopher Liu * menardorama * Victor D * Justin Miller * Arthur Lutz * Luis Pinto Da Costa * Ronan Dunklau * Christoph Berg * Palle Girgensohn * Maxence Ahlouche * Stéphane Tachoires * trourance * indreek * Andreas Seltenreich * edechaux powa-archivist-REL_3_2_0/INSTALL.md000066400000000000000000000134441336072707400167670ustar00rootroot00000000000000PostgreSQL Workload Analyzer detailled installation guide ========================================================= Read [README.md](https://github.com/powa-team/powa/blob/master/README.md) and [the official documentation](http://powa.readthedocs.io/) for further details about PoWA. PoWA requires PostgreSQL 9.4 or more. This documentation assumes you're using the 9.4 version of PostgreSQL. The following documentation describes the detailed installation steps to install PoWA. Download powa-archivist from the website ---------------------------------------- The latest stable version should be used. It can be downloaded from [github](https://github.com/powa-team/powa-archivist/releases/latest). This documentation assumes that the latest version is 3.0.0, and you downloaded the .zip file. Unpack the downloaded file -------------------------- ``` cd /usr/src unzip powa-REL_3_0_0.zip ``` Compile and install the software -------------------------------- Before proceeding, be sure to have a compiler installed and the appropriate PostgreSQL development packages. Something like ``` apt-get install postgresql-server-dev-9.4 ``` or ``` yum install postgresql94-devel ``` Then: ``` cd /usr/src/powa-REL_3_0_0 make ``` If everything goes fine, you will have this kind of output : ``` gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -fpic -I. -I. -I/home/thomas/postgresql/postgresql-9.3.4/include/server -I/home/thomas/postgresql/postgresql-9.4.4/include/internal -D_GNU_SOURCE -I/usr/include/libxml2 -c -o powa.o powa.c gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -fpic -L/home/thomas/postgresql/postgresql-9.3.4/lib -Wl,--as-needed -Wl,-rpath,'/home/thomas/postgresql/postgresql-9.4.4/lib',--enable-new-dtags -shared -o powa.so powa.o ``` Install the software : - This step has to be made with the user that has installed PostgreSQL. If you have used a package, it will be certainly be root. If so: ``` sudo make install ``` Else, sudo into the user that owns your PostgreSQL executables, and ``` make install ``` It should output something like the following : ``` /bin/mkdir -p '/usr/pgsql-9.4/share/extension' /bin/mkdir -p '/usr/pgsql-9.4/share/extension' /bin/mkdir -p '/usr/pgsql-9.4/lib' /bin/mkdir -p '/usr/pgsql-9.4/share/doc/extension' /usr/bin/install -c -m 644 ./powa.control '/usr/pgsql-9.4/share/extension/' /usr/bin/install -c -m 644 ./powa--2.0.1.sql ./powa--2.0-2.0.1.sql ./powa--3.0.0.sql '/usr/pgsql-9.4/share/extension/' /usr/bin/install -c -m 755 powa.so '/usr/pgsql-9.4/postgresql-9.4.4/lib/' /usr/bin/install -c -m 644 ./README.md '/usr/pgsql-9.4/share/doc/extension/' ``` Create a PoWA database and create required extensions ----------------------------------------------------- Note: if you are upgrading from a previous PoWA release, please consult the upgrading section at the end of this file. First, connect to PostgreSQL as administrator : ``` bash-4.1$ psql psql (9.3.5) Type "help" for help. postgres=# create database powa; CREATE DATABASE postgres=# \c powa You are now connected to database "powa" as user "postgres". powa=# create extension pg_stat_statements ; CREATE EXTENSION powa=# create extension btree_gist ; CREATE EXTENSION powa=# create extension powa; CREATE EXTENSION powa=# \dt List of relations Schema | Name | Type | Owner --------+---------------------------------+-------+---------- public | powa_functions | table | postgres public | powa_last_aggregation | table | postgres public | powa_last_purge | table | postgres public | powa_statements | table | postgres public | powa_statements_history | table | postgres public | powa_statements_history_current | table | postgres [...] ``` Modify the configuration files ------------------------------ In `postgresql.conf`: Change the `shared_preload_libraries` appropriately : ``` shared_preload_libraries = 'powa,pg_stat_statements'# (change requires restart) ``` If possible (check with pg_test_timing), activate track_io_timing on your instance, in postgresql.conf : ``` track_io_timing = on ``` Other GUC variables are available. Read [README.md](https://github.com/powa-team/powa/blob/master/README.md) for further details. In `pg_hba.conf`: Add an entry if needed for the PostgreSQL user(s) that need to connect on the GUI. For instance, assuming a `local connection` on database `powa`, allowing any user: `host powa all 127.0.0.1/32 md5` Restart PostgreSQL ------------------ As root, run the following command : ``` service postgresql-9.3 restart ``` PostgreSQL should output the following messages in the log files : ``` 2014-07-25 03:48:20 IST LOG: registering background worker "powa" 2014-07-25 03:48:20 IST LOG: loaded library "powa" 2014-07-25 03:48:20 IST LOG: loaded library "pg_stat_statements" ``` Upgrading from a previous version of PoWA ----------------------------------------- If you already have an older PoWA installation, you can simply upgrade PoWA with the following steps : First, connect to PostgreSQL as administrator and update the extension : ``` bash-4.1$ psql powa psql (9.3.5) Type "help" for help. powa=# ALTER EXTENSION powa UPDATE ; ALTER EXTENSION ``` However, due to a lot of changes in the data storage, it's not possible to update to PoWA 3.0.0. In this case, you need to drop and create the extension. Next, you will need to restart PostgreSQL in order to take account of the updated background worker. As root, run the following command : ``` service postgresql-9.4 restart ``` powa-archivist-REL_3_2_0/LICENSE.md000066400000000000000000000016511336072707400167400ustar00rootroot00000000000000Copyright (c) 2014-2017, DALIBO Copyright (c) 2018, The PoWA-team Permission to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a written agreement is hereby granted, provided that the above copyright notice and this paragraph and the following two paragraphs appear in all copies. IN NO EVENT SHALL DALIBO BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF DALIBO HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. DALIBO SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS, AND DALIBO HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. powa-archivist-REL_3_2_0/META.json000066400000000000000000000017171336072707400167600ustar00rootroot00000000000000{ "name": "powa", "abstract": "An extension gathering pg_stat_statements and other plugins statistics", "version": "__VERSION__", "maintainer": "Julien Rouhaud ", "license": "postgresql", "release_status": "stable", "provides": { "powa": { "abstract": "An extension gathering pg_stat_statements and other plugins statistics", "file": "powa.sql", "docfile": "README.md", "version": "__VERSION__" } }, "resources": { "bugtracker": { "web": "http://github.com/powa-team/powa/issues/" }, "repository": { "url": "git://github.com/powa-team/powa-archivist.git", "web": "http://github.com/powa-team/powa-archivist/", "type": "git" } }, "generated_by": "Julien Rouhaud", "meta-spec": { "version": "1.0.0", "url": "http://pgxn.org/meta/spec.txt" }, "tags": [ "monitoring", "workload" ] } powa-archivist-REL_3_2_0/Makefile000066400000000000000000000014441336072707400167740ustar00rootroot00000000000000EXTENSION = powa EXTVERSION = $(shell grep default_version $(EXTENSION).control | sed -e "s/default_version[[:space:]]*=[[:space:]]*'\([^']*\)'/\1/") TESTS = $(wildcard sql/*.sql) REGRESS = $(patsubst sql/%.sql,%,$(TESTS)) REGRESS_OPTS = --inputdir=test PG_CONFIG ?= pg_config MODULES = powa all: release-zip: all git archive --format zip --prefix=powa-${EXTVERSION}/ --output ./powa-${EXTVERSION}.zip HEAD unzip ./powa-$(EXTVERSION).zip rm ./powa-$(EXTVERSION).zip rm ./powa-$(EXTVERSION)/.gitignore rm ./powa-$(EXTVERSION)/reindent.sh sed -i -e "s/__VERSION__/$(EXTVERSION)/g" ./powa-$(EXTVERSION)/META.json zip -r ./powa-$(EXTVERSION).zip ./powa-$(EXTVERSION)/ rm ./powa-$(EXTVERSION) -rf DATA = $(wildcard *--*.sql) PGXS := $(shell $(PG_CONFIG) --pgxs) include $(PGXS) powa-archivist-REL_3_2_0/PL_funcs.md000066400000000000000000000036241336072707400173710ustar00rootroot00000000000000This is a list of all functions and what they are used for: * `powa_take_snapshot`: takes a snapshot. It means calling all the **snapshot** functions registered in the **powa_functions** table, then maybe do an **aggregate** and/or a **purge**, if conditions are met (these functions are also registered in powa_functions). * `powa_take_statements_snapshot`: takes a snapshot of pg_stat_statements. This is the included **snapshot** function. * `powa_statements_purge`: does a purge of collected data from pg_stat_statements. This is the included **purge** function. * `powa_statements_aggregate`: does an aggregate (putting individual records into arrays to save space) on collected data from pg_stat_statements. This is the included **aggregate** function. * `powa_stats_reset`: cleans-up pg_stat_staments collected data. **FIXME: Should be moved to dedicated functions, and stored in powa_functions**. * `powa_kcache_register`: Add the pg_stat_kcache snapshot, aggregate and purge functions to list of powa functions if pg_stat_kcache extension exists. * `powa_kcache_unregister`: Remove the pg_stat_kcache snapshot, aggregate and purge functions from list of powa functions. * `powa_kcache_snapshot`: Take a snapshot of pg_stat_kcache. * `powa_kcache_aggregate`: Does an aggregate on collected data from pg_stat_kcache. * `powa_kcache_purge`: Does a purge of collected data from pg_stat_kcache. * `powa_qualstats_register`: Add the pg_qualstats snapshot, aggregate and purge functions to list of powa functions if pg_qualstats extension exists. * `powa_qualstats_unregister`: Remove the pg_qualstats snapshot, aggregate and purge pg_qualstats functions from list of powa functions. * `powa_qualstats_snapshot`: Take a snapshot of pg_qualstats. * `powa_qualstats_aggregate`: Does an aggregate on collected data from pg_qualstats. * `powa_qualstats_purge`: Does a purge of collected data from pg_qualstats. powa-archivist-REL_3_2_0/README.md000066400000000000000000000011021336072707400166020ustar00rootroot00000000000000 ![PostgreSQL Workload Analyzer](https://github.com/powa-team/powa/blob/master/img/powa_logo.410x161.png) PoWA Archivist ============== This project is the core extension of the [PoWA](http://powa.readthedocs.io/) project, a PostgreSQL Workload Analyzer that gathers performance stats and provides real-time charts and graphs to help monitor and tune your PostgreSQL servers. For more information, please read the [PoWA-archivist documentation](http://powa.readthedocs.io/en/latest/powa-archivist/index.html): http://powa.readthedocs.io/en/latest/powa-archivist/index.html powa-archivist-REL_3_2_0/debian/000077500000000000000000000000001336072707400165535ustar00rootroot00000000000000powa-archivist-REL_3_2_0/debian/changelog000066400000000000000000000034171336072707400204320ustar00rootroot00000000000000powa-archivist (3.2.0-1) unstable; urgency=medium * New upstream version [ Christoph Berg ] * debian/tests: Depend on postgresql-server-dev-all for pg_buildext. * debian/tests: Use "pg_buildext installcheck". * Move maintainer address to team+postgresql@tracker.debian.org. -- Julien Rouhaud Sun, 14 Oct 2018 22:52:51 +0200 powa-archivist (3.1.2-1) unstable; urgency=medium * New upstream version with PG11 support. -- Christoph Berg Wed, 30 May 2018 08:55:39 +0200 powa-archivist (3.1.1-2) unstable; urgency=medium * Catch errors in autopkgtest script, thanks to Niels Thykier for spotting! Closes: #881695 * Use source format 1.0, works even better for building snapshots. -- Christoph Berg Sat, 19 May 2018 17:47:40 +0200 powa-archivist (3.1.1) unstable; urgency=medium * Team upload. * New upstream version. * Packaging has been merged into upstream git. * Use source format 3.0 (native) to ease building snapshots. * Bump Standards-Version to 4.1.0. -- Christoph Berg Tue, 19 Sep 2017 10:40:00 +0200 powa-archivist (3.1.0-1) unstable; urgency=medium * New upstream version. -- Christoph Berg Mon, 26 Sep 2016 22:50:34 +0200 powa-archivist (3.0.1-1) unstable; urgency=medium * New upstream version. -- Christoph Berg Fri, 19 Feb 2016 10:04:25 +0100 powa-archivist (3.0.0-2) unstable; urgency=medium * Upload to unstable with 9.5 support. * Add debian/tests/create-extension. -- Christoph Berg Wed, 13 Jan 2016 18:14:25 +0100 powa-archivist (3.0.0-1) unstable; urgency=medium * Initial release. -- Christoph Berg Wed, 11 Nov 2015 15:11:42 +0100 powa-archivist-REL_3_2_0/debian/compat000066400000000000000000000000021336072707400177510ustar00rootroot000000000000009 powa-archivist-REL_3_2_0/debian/control000066400000000000000000000014721336072707400201620ustar00rootroot00000000000000Source: powa-archivist Section: database Priority: optional Maintainer: Debian PostgreSQL Maintainers Uploaders: Christoph Berg Standards-Version: 4.1.4 Build-Depends: debhelper (>= 9~), postgresql-server-dev-all (>= 141~) Homepage: https://powa.readthedocs.io/ Vcs-Browser: https://github.com/powa-team/powa-archivist Vcs-Git: https://github.com/powa-team/powa-archivist.git Package: postgresql-10-powa Architecture: any Depends: ${misc:Depends}, ${shlibs:Depends}, postgresql-10 Description: PostgreSQL Workload Analyzer -- PostgreSQL 10 extension This package contains the core extension of the PoWA project, a PostgreSQL Workload Analyzer that gathers performance stats and provides real-time charts and graphs to help monitor and tune your PostgreSQL servers. powa-archivist-REL_3_2_0/debian/control.in000066400000000000000000000015161336072707400205660ustar00rootroot00000000000000Source: powa-archivist Section: database Priority: optional Maintainer: Debian PostgreSQL Maintainers Uploaders: Christoph Berg Standards-Version: 4.1.4 Build-Depends: debhelper (>= 9~), postgresql-server-dev-all (>= 141~) Homepage: https://powa.readthedocs.io/ Vcs-Browser: https://github.com/powa-team/powa-archivist Vcs-Git: https://github.com/powa-team/powa-archivist.git Package: postgresql-PGVERSION-powa Architecture: any Depends: ${misc:Depends}, ${shlibs:Depends}, postgresql-PGVERSION Description: PostgreSQL Workload Analyzer -- PostgreSQL PGVERSION extension This package contains the core extension of the PoWA project, a PostgreSQL Workload Analyzer that gathers performance stats and provides real-time charts and graphs to help monitor and tune your PostgreSQL servers. powa-archivist-REL_3_2_0/debian/copyright000066400000000000000000000026511336072707400205120ustar00rootroot00000000000000powa-archivist was debianized by Christoph Berg . Copyright (c) 2014-2017, DALIBO Copyright (c) 2018, The PoWA-team Permission to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a written agreement is hereby granted, provided that the above copyright notice and this paragraph and the following two paragraphs appear in all copies. IN NO EVENT SHALL DALIBO BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF DALIBO HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. DALIBO SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS, AND DALIBO HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. Contributors to PoWA : * Marc Cousin * Julien Rouhaud * Damien Clochard * Thomas Reiss * Hyunjun Kim * Raghu Ram * Rodolphe Quiédeville * Ahmed Bessifi * Christopher Liu * menardorama * Victor D * Justin Miller * Arthur Lutz * Luis Pinto Da Costa * Ronan Dunklau * Christoph Berg * Palle Girgensohn * Maxence Ahlouche * Stéphane Tachoires * trourance * indreek * Andreas Seltenreich * edechaux powa-archivist-REL_3_2_0/debian/pgversions000066400000000000000000000000051336072707400206700ustar00rootroot000000000000009.4+ powa-archivist-REL_3_2_0/debian/rules000077500000000000000000000006011336072707400176300ustar00rootroot00000000000000#!/usr/bin/make -f include /usr/share/postgresql-common/pgxs_debian_control.mk override_dh_auto_build: override_dh_auto_test: # nothing to do here, see debian/tests/* instead override_dh_auto_install: +pg_buildext loop postgresql-%v-powa override_dh_installdocs: dh_installdocs --all INSTALL.md PL_funcs.md README.md rm -rfv debian/*/usr/share/doc/postgresql-doc-* %: dh $@ powa-archivist-REL_3_2_0/debian/source/000077500000000000000000000000001336072707400200535ustar00rootroot00000000000000powa-archivist-REL_3_2_0/debian/source/format000066400000000000000000000000041336072707400212600ustar00rootroot000000000000001.0 powa-archivist-REL_3_2_0/debian/source/lintian-overrides000066400000000000000000000001751336072707400234370ustar00rootroot00000000000000# don't bug people uploading from @work source: changelog-should-mention-nmu source: source-nmu-has-incorrect-version-number powa-archivist-REL_3_2_0/debian/tests/000077500000000000000000000000001336072707400177155ustar00rootroot00000000000000powa-archivist-REL_3_2_0/debian/tests/control000066400000000000000000000001541336072707400213200ustar00rootroot00000000000000Depends: @, postgresql-contrib-10, postgresql-server-dev-all Tests: installcheck Restrictions: allow-stderr powa-archivist-REL_3_2_0/debian/tests/control.in000066400000000000000000000001631336072707400217250ustar00rootroot00000000000000Depends: @, postgresql-contrib-PGVERSION, postgresql-server-dev-all Tests: installcheck Restrictions: allow-stderr powa-archivist-REL_3_2_0/debian/tests/installcheck000077500000000000000000000001321336072707400223030ustar00rootroot00000000000000#!/bin/sh pg_buildext -o "shared_preload_libraries=powa,pg_stat_statements" installcheck powa-archivist-REL_3_2_0/debian/watch000066400000000000000000000001741336072707400176060ustar00rootroot00000000000000version=3 opts="uversionmangle=s/_/./g" \ https://github.com/powa-team/powa-archivist/releases .*/archive/REL_(.*).tar.gz powa-archivist-REL_3_2_0/expected/000077500000000000000000000000001336072707400171325ustar00rootroot00000000000000powa-archivist-REL_3_2_0/expected/powa-archivist.out000066400000000000000000000122561336072707400226310ustar00rootroot00000000000000--Setup extension CREATE EXTENSION pg_stat_statements; CREATE EXTENSION btree_gist; CREATE EXTENSION powa; -- Aggregate data every 5 snapshots SET powa.coalesce = 5; -- Test created ojects SELECT * FROM powa_functions ORDER BY module, operation; module | operation | function_name | added_manually | enabled --------------------------+-----------+-------------------------------+----------------+--------- pg_stat_statements | aggregate | powa_statements_aggregate | f | t pg_stat_statements | purge | powa_statements_purge | f | t pg_stat_statements | reset | powa_statements_reset | f | t pg_stat_statements | snapshot | powa_statements_snapshot | f | t powa_stat_all_relations | aggregate | powa_all_relations_aggregate | f | t powa_stat_all_relations | purge | powa_all_relations_purge | f | t powa_stat_all_relations | reset | powa_all_relations_reset | f | t powa_stat_all_relations | snapshot | powa_all_relations_snapshot | f | t powa_stat_user_functions | aggregate | powa_user_functions_aggregate | f | t powa_stat_user_functions | purge | powa_user_functions_purge | f | t powa_stat_user_functions | reset | powa_user_functions_reset | f | t powa_stat_user_functions | snapshot | powa_user_functions_snapshot | f | t (12 rows) -- test C SRFs SELECT COUNT(*) = 0 FROM pg_database, LATERAL powa_stat_user_functions(oid) f WHERE datname = current_database(); ?column? ---------- t (1 row) SELECT COUNT(*) > 10 FROM pg_database, LATERAL powa_stat_all_rel(oid) WHERE datname = current_database(); ?column? ---------- t (1 row) -- Test snapshot SELECT COUNT(*) = 0 FROM powa_user_functions_history_current; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_all_relations_history_current; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_statements_history_current; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_statements_history_current_db; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_user_functions_history; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_all_relations_history; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_statements_history; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_statements_history; ?column? ---------- t (1 row) SELECT powa_take_snapshot(); powa_take_snapshot -------------------- (1 row) SELECT COUNT(*) >= 0 FROM powa_user_functions_history_current; ?column? ---------- t (1 row) SELECT COUNT(*) > 0 FROM powa_all_relations_history_current; ?column? ---------- t (1 row) SELECT COUNT(*) > 0 FROM powa_statements_history_current; ?column? ---------- t (1 row) SELECT COUNT(*) > 0 FROM powa_statements_history_current_db; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_user_functions_history; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_all_relations_history; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_statements_history; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_statements_history; ?column? ---------- t (1 row) SELECT powa_take_snapshot(); powa_take_snapshot -------------------- (1 row) SELECT powa_take_snapshot(); powa_take_snapshot -------------------- (1 row) SELECT powa_take_snapshot(); powa_take_snapshot -------------------- (1 row) -- This snapshot will trigger the aggregate SELECT powa_take_snapshot(); powa_take_snapshot -------------------- (1 row) SELECT COUNT(*) = 0 FROM powa_user_functions_history_current; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_all_relations_history_current; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_statements_history_current; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_statements_history_current_db; ?column? ---------- t (1 row) SELECT COUNT(*) >= 0 FROM powa_user_functions_history; ?column? ---------- t (1 row) SELECT COUNT(*) > 0 FROM powa_all_relations_history; ?column? ---------- t (1 row) SELECT COUNT(*) > 0 FROM powa_statements_history; ?column? ---------- t (1 row) SELECT COUNT(*) > 0 FROM powa_statements_history; ?column? ---------- t (1 row) -- Test reset function SELECT * from powa_reset(); powa_reset ------------ t (1 row) SELECT COUNT(*) = 0 FROM powa_user_functions_history_current; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_all_relations_history_current; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_statements_history_current; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_statements_history_current_db; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_user_functions_history; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_all_relations_history; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_statements_history; ?column? ---------- t (1 row) SELECT COUNT(*) = 0 FROM powa_statements_history; ?column? ---------- t (1 row) powa-archivist-REL_3_2_0/install_all.sql000066400000000000000000000002671336072707400203550ustar00rootroot00000000000000CREATE DATABASE powa; \c powa CREATE EXTENSION btree_gist; CREATE EXTENSION pg_stat_statements; CREATE EXTENSION pg_qualstats; CREATE EXTENSION pg_stat_kcache; CREATE EXTENSION powa; powa-archivist-REL_3_2_0/powa--2.0--2.0.1.sql000066400000000000000000000230541336072707400201070ustar00rootroot00000000000000-- complain if script is sourced in psql, rather than via ALTER EXTENSION --\echo Use "ALTER EXTENSION powa" to load this file. \quit -- powa_functions now have an "unregister" operation ALTER TABLE public.powa_functions DROP CONSTRAINT powa_functions_operation_check; ALTER TABLE public.powa_functions ADD CHECK (operation IN ('snapshot','aggregate','purge','unregister')); -- Handle automatic extensions registering CREATE OR REPLACE FUNCTION public.powa_check_created_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE BEGIN /* We have for now no way for a proper handling of this event, * as we don't have a table with the list of supported extensions. * So just call every powa_*_register() function we know each time an * extension is created. Powa should be in a dedicated database and the * register function handle to be called several time, so it's not critical */ PERFORM public.powa_kcache_register(); PERFORM public.powa_qualstats_register(); END; $_$; CREATE EVENT TRIGGER powa_check_created_extensions ON ddl_command_end WHEN tag IN ('CREATE EXTENSION') EXECUTE PROCEDURE public.powa_check_created_extensions() ; CREATE OR REPLACE FUNCTION public.powa_check_dropped_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN WITH ext AS ( SELECT object_name FROM pg_event_trigger_dropped_objects() d WHERE d.object_type = 'extension' ) SELECT function_name INTO funcname FROM powa_functions f JOIN ext ON f.module = ext.object_name WHERE operation = 'unregister'; IF ( funcname IS NOT NULL ) THEN BEGIN RAISE DEBUG 'running %', funcname; EXECUTE 'SELECT ' || quote_ident(funcname) || '()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE WARNING 'powa_check_dropped_extensions(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END IF; END; $_$; -- Handle automatic extensions unregistering CREATE EVENT TRIGGER powa_check_dropped_extensions ON sql_drop WHEN tag IN ('DROP EXTENSION') EXECUTE PROCEDURE public.powa_check_dropped_extensions() ; -- New powa_kcache_register function CREATE OR REPLACE function public.powa_kcache_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_stat_kcache'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_stat_kcache'; IF ( NOT v_func_present) THEN INSERT INTO powa_functions (module, operation, function_name, added_manually) VALUES ('pg_stat_kcache', 'snapshot', 'powa_kcache_snapshot', false), ('pg_stat_kcache', 'aggregate', 'powa_kcache_aggregate', false), ('pg_stat_kcache', 'unregister', 'powa_kcache_unregister', false), ('pg_stat_kcache', 'purge', 'powa_kcache_purge', false); END IF; END IF; RETURN true; END; $_$ language plpgsql; -- New powa_qualstats_unregister function CREATE OR REPLACE function public.powa_qualstats_unregister() RETURNS bool AS $_$ BEGIN DELETE FROM public.powa_functions WHERE module = 'pg_qualstats'; RETURN true; END; $_$ language plpgsql; -- New powa_qualstats_register function CREATE OR REPLACE function public.powa_qualstats_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_qualstats'; IF ( v_ext_present) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE function_name IN ('powa_qualstats_snapshot', 'powa_qualstats_aggregate', 'powa_qualstats_purge'); IF ( NOT v_func_present) THEN INSERT INTO powa_functions (module, operation, function_name, added_manually) VALUES ('pg_qualstats', 'snapshot', 'powa_qualstats_snapshot', false), ('pg_qualstats', 'aggregate', 'powa_qualstats_aggregate', false), ('pg_qualstats', 'unregister', 'powa_qualstats_unregister', false), ('pg_qualstats', 'purge', 'powa_qualstats_purge', false); END IF; END IF; RETURN true; END; $_$ language plpgsql; -- Add the _unregister() function in powa_functions if the related extension exists WITH ext_exists AS ( SELECT 1 FROM pg_extension WHERE extname = 'pg_stat_kcache' ) INSERT INTO public.powa_functions (module, operation, function_name, added_manually) SELECT 'pg_stat_kcache', 'unregister', 'powa_kcache_unregister', false FROM ext_exists; WITH ext_exists AS ( SELECT 1 FROM pg_extension WHERE extname = 'pg_qualstats' ) INSERT INTO public.powa_functions (module, operation, function_name, added_manually) SELECT 'pg_qualstats', 'unregister', 'powa_qualstats_unregister', false FROM ext_exists; -- Fix the "added_manually" value for pg_stat_kcache extension UPDATE public.powa_functions SET added_manually = false WHERE module = 'pg_stat_kcache'; ----------------------------------------------------------- -- Fix the tstzrange inclusive upper bounds for -- * powa_kcache_aggregate() function -- * powa_qualstats_aggregate_constvalues_current view -- * powa_qualstats_aggregate() function ----------------------------------------------------------- CREATE OR REPLACE FUNCTION powa_kcache_aggregate() RETURNS void AS $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_kcache_aggregate'; -- aggregate metrics table LOCK TABLE powa_kcache_metrics_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics (coalesce_range, queryid, dbid, userid, metrics) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), queryid, dbid, userid, array_agg(metrics) FROM powa_kcache_metrics_current GROUP BY queryid, dbid, userid; TRUNCATE powa_kcache_metrics_current; -- aggregate metrics_db table LOCK TABLE powa_kcache_metrics_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics_db (coalesce_range, dbid, metrics) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), dbid, array_agg(metrics) FROM powa_kcache_metrics_current_db GROUP BY dbid; TRUNCATE powa_kcache_metrics_current_db; END $PROC$ language plpgsql; CREATE OR REPLACE VIEW powa_qualstats_aggregate_constvalues_current AS WITH consts AS ( SELECT qualid, queryid, dbid, userid, min(ts) as mints, max(ts) as maxts, sum(nbfiltered) as nbfiltered, sum(count) as count, constvalues FROM powa_qualstats_constvalues_history_current GROUP BY qualid, queryid, dbid, userid, constvalues ), groups AS ( SELECT qualid, queryid, dbid, userid, tstzrange(min(mints), max(maxts),'[]') FROM consts GROUP BY qualid, queryid, dbid, userid ) SELECT * FROM groups, LATERAL ( SELECT array_agg(constvalues) as mf FROM ( SELECT (constvalues, nbfiltered, count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN count = 0 THEN 0 ELSE nbfiltered / count::numeric END DESC LIMIT 20 ) s ) as mf, LATERAL ( SELECT array_agg(constvalues) as lf FROM ( SELECT (constvalues, nbfiltered, count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN count = 0 THEN 0 ELSE nbfiltered / count::numeric END DESC LIMIT 20 ) s ) as lf, LATERAL ( SELECT array_agg(constvalues) as me FROM ( SELECT (constvalues, nbfiltered, count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY count desc LIMIT 20 ) s ) as me; CREATE OR REPLACE FUNCTION powa_qualstats_aggregate() RETURNS void AS $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_qualstats_aggregate'; LOCK TABLE powa_qualstats_constvalues_history_current IN SHARE MODE; LOCK TABLE powa_qualstats_quals_history_current IN SHARE MODE; INSERT INTO powa_qualstats_constvalues_history ( qualid, queryid, dbid, userid, coalesce_range, most_filtering, least_filtering, most_executed) SELECT * FROM powa_qualstats_aggregate_constvalues_current; INSERT INTO powa_qualstats_quals_history (qualid, queryid, dbid, userid, coalesce_range, records) SELECT qualid, queryid, dbid, userid, tstzrange(min(ts), max(ts),'[]'), array_agg((ts, count, nbfiltered)::powa_qualstats_history_item) FROM powa_qualstats_quals_history_current GROUP BY qualid, queryid, dbid, userid; TRUNCATE powa_qualstats_constvalues_history_current; TRUNCATE powa_qualstats_quals_history_current; END $PROC$ language plpgsql; -- Try to register handled extensions SELECT * FROM public.powa_qualstats_register(); powa-archivist-REL_3_2_0/powa--2.0.1.sql000066400000000000000000000661061336072707400175420ustar00rootroot00000000000000-- complain if script is sourced in psql, rather than via CREATE EXTENSION --\echo Use "CREATE EXTENSION powa" to load this file. \quit SET statement_timeout = 0; SET client_encoding = 'UTF8'; SET standard_conforming_strings = on; SET check_function_bodies = false; SET client_min_messages = warning; SET escape_string_warning = off; SET search_path = public, pg_catalog; CREATE TYPE powa_statement_history_record AS ( ts timestamp with time zone, calls bigint, total_time double precision, rows bigint, shared_blks_hit bigint, shared_blks_read bigint, shared_blks_dirtied bigint, shared_blks_written bigint, local_blks_hit bigint, local_blks_read bigint, local_blks_dirtied bigint, local_blks_written bigint, temp_blks_read bigint, temp_blks_written bigint, blk_read_time double precision, blk_write_time double precision ); CREATE TABLE powa_last_aggregation ( aggts timestamp with time zone ); INSERT INTO powa_last_aggregation(aggts) VALUES (current_timestamp); CREATE TABLE powa_last_purge ( purgets timestamp with time zone ); INSERT INTO powa_last_purge (purgets) VALUES (current_timestamp); CREATE TABLE powa_statements ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, query text NOT NULL ); ALTER TABLE ONLY powa_statements ADD CONSTRAINT powa_statements_pkey PRIMARY KEY (queryid, dbid, userid); CREATE INDEX powa_statements_dbid_idx ON powa_statements(dbid); CREATE INDEX powa_statements_userid_idx ON powa_statements(userid); CREATE TABLE powa_statements_history ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statement_history_record[] NOT NULL ); CREATE INDEX powa_statements_history_query_ts ON powa_statements_history USING gist (queryid, coalesce_range); CREATE TABLE powa_statements_history_db ( dbid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statement_history_record[] NOT NULL ); CREATE INDEX powa_statements_history_db_ts ON powa_statements_history_db USING gist (dbid, coalesce_range); CREATE TABLE powa_statements_history_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, record powa_statement_history_record NOT NULL ); CREATE TABLE powa_statements_history_current_db ( dbid oid NOT NULL, record powa_statement_history_record NOT NULL ); CREATE SEQUENCE powa_coalesce_sequence INCREMENT BY 1 START WITH 1 CYCLE; CREATE TABLE powa_functions ( module text NOT NULL, operation text NOT NULL, function_name text NOT NULL, added_manually boolean NOT NULL default true, CHECK (operation IN ('snapshot','aggregate','purge','unregister')) ); INSERT INTO powa_functions (module, operation, function_name, added_manually) VALUES ('pg_stat_statements', 'snapshot', 'powa_statements_snapshot', false), ('pg_stat_statements', 'aggregate','powa_statements_aggregate', false), ('pg_stat_statements', 'purge', 'powa_statements_purge', false); /* pg_stat_kcache integration - part 1 */ CREATE TYPE public.kcache_type AS ( ts timestamptz, reads bigint, writes bigint, user_time double precision, system_time double precision ); CREATE TABLE public.powa_kcache_metrics ( coalesce_range tstzrange NOT NULL, queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics public.kcache_type[] NOT NULL, PRIMARY KEY (coalesce_range, queryid, dbid, userid) ); CREATE INDEX ON public.powa_kcache_metrics (queryid); CREATE TABLE public.powa_kcache_metrics_db ( coalesce_range tstzrange NOT NULL, dbid oid NOT NULL, metrics public.kcache_type[] NOT NULL, PRIMARY KEY (coalesce_range, dbid) ); CREATE TABLE public.powa_kcache_metrics_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics kcache_type NULL NULL ); CREATE TABLE public.powa_kcache_metrics_current_db ( dbid oid NOT NULL, metrics kcache_type NULL NULL ); /* end of pg_stat_kcache integration - part 1 */ /* pg_qualstats integration - part 1 */ CREATE TYPE public.qual_type AS ( relid oid, attnum integer, opno oid, eval_type "char" ); CREATE TYPE public.qual_values AS ( constants text[], count bigint, nbfiltered bigint ); CREATE TYPE powa_qualstats_history_item AS ( ts timestamptz, count bigint, nbfiltered bigint ); CREATE TABLE public.powa_qualstats_quals ( qualid bigint, queryid bigint, dbid oid, userid oid, quals public.qual_type[], PRIMARY KEY (qualid, queryid, dbid, userid), FOREIGN KEY (queryid, dbid, userid) REFERENCES powa_statements(queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, records powa_qualstats_history_item[], FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES powa_qualstats_quals(qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, most_filtering qual_values[], least_filtering qual_values[], most_executed qual_values[], FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, constvalues text[], count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE INDEX ON powa_qualstats_constvalues_history USING gist (queryid, qualid, coalesce_range); CREATE INDEX ON powa_qualstats_constvalues_history (qualid, queryid); CREATE INDEX ON powa_qualstats_quals(queryid); /* end of pg_qualstats_integration - part 1 */ -- Mark all of powa's tables as "to be dumped" SELECT pg_catalog.pg_extension_config_dump('powa_statements',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_functions','WHERE added_manually'); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history_current',''); CREATE OR REPLACE FUNCTION public.powa_check_created_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE BEGIN /* We have for now no way for a proper handling of this event, * as we don't have a table with the list of supported extensions. * So just call every powa_*_register() function we know each time an * extension is created. Powa should be in a dedicated database and the * register function handle to be called several time, so it's not critical */ PERFORM public.powa_kcache_register(); PERFORM public.powa_qualstats_register(); END; $_$; CREATE EVENT TRIGGER powa_check_created_extensions ON ddl_command_end WHEN tag IN ('CREATE EXTENSION') EXECUTE PROCEDURE public.powa_check_created_extensions() ; CREATE OR REPLACE FUNCTION public.powa_check_dropped_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN WITH ext AS ( SELECT object_name FROM pg_event_trigger_dropped_objects() d WHERE d.object_type = 'extension' ) SELECT function_name INTO funcname FROM powa_functions f JOIN ext ON f.module = ext.object_name WHERE operation = 'unregister'; IF ( funcname IS NOT NULL ) THEN BEGIN RAISE DEBUG 'running %', funcname; EXECUTE 'SELECT ' || quote_ident(funcname) || '()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE WARNING 'powa_check_dropped_extensions(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END IF; END; $_$; CREATE EVENT TRIGGER powa_check_dropped_extensions ON sql_drop WHEN tag IN ('DROP EXTENSION') EXECUTE PROCEDURE public.powa_check_dropped_extensions() ; CREATE OR REPLACE FUNCTION powa_take_snapshot() RETURNS void AS $PROC$ DECLARE purgets timestamp with time zone; purge_seq bigint; funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- For all snapshot functions in the powa_functions table, execute FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='snapshot' LOOP -- Call all of them, with no parameter RAISE debug 'fonction: %',funcname; BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; -- Coalesce datas if needed SELECT nextval('powa_coalesce_sequence'::regclass) INTO purge_seq; IF ( purge_seq % current_setting('powa.coalesce')::bigint ) = 0 THEN FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='aggregate' LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; UPDATE powa_last_aggregation SET aggts = now(); END IF; -- Once every 10 packs, we also purge IF ( purge_seq % (current_setting('powa.coalesce')::bigint *10) ) = 0 THEN FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='purge' LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; UPDATE powa_last_purge SET purgets = now(); END IF; END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_statements_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; ignore_regexp text:='^[[:space:]]*(BEGIN)'; -- Ignore begin at beginning of statement BEGIN -- In this function, we capture statements, and also aggregate counters by database -- so that the first screens of powa stay reactive even though there may be thousands -- of different statements RAISE DEBUG 'running powa_statements_snapshot'; WITH capture AS( SELECT pg_stat_statements.* FROM pg_stat_statements WHERE pg_stat_statements.query !~* ignore_regexp ), missing_statements AS( INSERT INTO powa_statements (queryid, dbid, userid, query) SELECT queryid, dbid, userid, query FROM capture c WHERE NOT EXISTS (SELECT 1 FROM powa_statements ps WHERE ps.queryid = c.queryid AND ps.dbid = c.dbid AND ps.userid = c.userid ) ), by_query AS ( INSERT INTO powa_statements_history_current SELECT queryid, dbid, userid, ROW( now(), calls, total_time, rows, shared_blks_hit, shared_blks_read, shared_blks_dirtied, shared_blks_written, local_blks_hit, local_blks_read, local_blks_dirtied, local_blks_written, temp_blks_read, temp_blks_written, blk_read_time, blk_write_time )::powa_statement_history_record AS record FROM capture ), by_database AS ( INSERT INTO powa_statements_history_current_db SELECT dbid, ROW( now(), sum(calls), sum(total_time), sum(rows), sum(shared_blks_hit), sum(shared_blks_read), sum(shared_blks_dirtied), sum(shared_blks_written), sum(local_blks_hit), sum(local_blks_read), sum(local_blks_dirtied), sum(local_blks_written), sum(temp_blks_read), sum(temp_blks_written), sum(blk_read_time), sum(blk_write_time) )::powa_statement_history_record AS record FROM capture GROUP BY dbid ) SELECT true::boolean INTO result; -- For now we don't care. What could we do on error except crash anyway? END; $PROC$ language plpgsql; CREATE OR REPLACE FUNCTION powa_statements_purge() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_statements_purge'; -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_statements_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); DELETE FROM powa_statements_history_db WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); -- FIXME maybe we should cleanup the powa_statements table ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_statements_aggregate() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_statements_aggregate'; -- aggregate statements table LOCK TABLE powa_statements_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history SELECT queryid, dbid, userid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record) FROM powa_statements_history_current GROUP BY queryid, dbid, userid; TRUNCATE powa_statements_history_current; -- aggregate db table LOCK TABLE powa_statements_history_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history_db SELECT dbid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record) FROM powa_statements_history_current_db GROUP BY dbid; TRUNCATE powa_statements_history_current_db; END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION public.powa_stats_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN TRUNCATE TABLE powa_statements_history; TRUNCATE TABLE powa_statements_history_current; TRUNCATE TABLE powa_statements_history_db; TRUNCATE TABLE powa_statements_history_current_db; TRUNCATE TABLE powa_statements; RETURN true; END; $function$; /* pg_stat_kcache integration - part 2 */ /* * register pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_stat_kcache'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_stat_kcache'; IF ( NOT v_func_present) THEN INSERT INTO powa_functions (module, operation, function_name, added_manually) VALUES ('pg_stat_kcache', 'snapshot', 'powa_kcache_snapshot', false), ('pg_stat_kcache', 'aggregate', 'powa_kcache_aggregate', false), ('pg_stat_kcache', 'unregister', 'powa_kcache_unregister', false), ('pg_stat_kcache', 'purge', 'powa_kcache_purge', false); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* * unregister pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_unregister() RETURNS bool AS $_$ BEGIN DELETE FROM public.powa_functions WHERE module = 'pg_stat_kcache'; RETURN true; END; $_$ language plpgsql; /* * powa_kcache snapshot collection. */ CREATE OR REPLACE FUNCTION powa_kcache_snapshot() RETURNS void as $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_kcache_snapshot'; WITH capture AS ( SELECT * FROM pg_stat_kcache() ), by_query AS ( INSERT INTO powa_kcache_metrics_current (queryid, dbid, userid, metrics) SELECT queryid, dbid, userid, (now(), reads, writes, user_time, system_time)::kcache_type FROM capture ), by_database AS ( INSERT INTO powa_kcache_metrics_current_db (dbid, metrics) SELECT dbid, (now(), sum(reads), sum(writes), sum(user_time), sum(system_time))::kcache_type FROM capture GROUP BY dbid ) SELECT true into result; END $PROC$ language plpgsql; /* * powa_kcache aggregation */ CREATE OR REPLACE FUNCTION powa_kcache_aggregate() RETURNS void AS $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_kcache_aggregate'; -- aggregate metrics table LOCK TABLE powa_kcache_metrics_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics (coalesce_range, queryid, dbid, userid, metrics) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), queryid, dbid, userid, array_agg(metrics) FROM powa_kcache_metrics_current GROUP BY queryid, dbid, userid; TRUNCATE powa_kcache_metrics_current; -- aggregate metrics_db table LOCK TABLE powa_kcache_metrics_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics_db (coalesce_range, dbid, metrics) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), dbid, array_agg(metrics) FROM powa_kcache_metrics_current_db GROUP BY dbid; TRUNCATE powa_kcache_metrics_current_db; END $PROC$ language plpgsql; /* * powa_kcache purge */ CREATE OR REPLACE FUNCTION powa_kcache_purge() RETURNS void as $PROC$ BEGIN RAISE DEBUG 'running powa_kcache_purge'; DELETE FROM powa_kcache_metrics WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); DELETE FROM powa_kcache_metrics_db WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); END; $PROC$ language plpgsql; -- By default, try to register pg_stat_kcache, in case it's alreay here SELECT * FROM public.powa_kcache_register(); /* end of pg_stat_kcache integration - part 2 */ /* pg_qualstats integration - part 2 */ /* * powa_qualstats_register */ CREATE OR REPLACE function public.powa_qualstats_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_qualstats'; IF ( v_ext_present) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE function_name IN ('powa_qualstats_snapshot', 'powa_qualstats_aggregate', 'powa_qualstats_purge'); IF ( NOT v_func_present) THEN INSERT INTO powa_functions (module, operation, function_name, added_manually) VALUES ('pg_qualstats', 'snapshot', 'powa_qualstats_snapshot', false), ('pg_qualstats', 'aggregate', 'powa_qualstats_aggregate', false), ('pg_qualstats', 'unregister', 'powa_qualstats_unregister', false), ('pg_qualstats', 'purge', 'powa_qualstats_purge', false); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* * powa_qualstats utility view for aggregating constvalues */ CREATE OR REPLACE VIEW powa_qualstats_aggregate_constvalues_current AS WITH consts AS ( SELECT qualid, queryid, dbid, userid, min(ts) as mints, max(ts) as maxts, sum(nbfiltered) as nbfiltered, sum(count) as count, constvalues FROM powa_qualstats_constvalues_history_current GROUP BY qualid, queryid, dbid, userid, constvalues ), groups AS ( SELECT qualid, queryid, dbid, userid, tstzrange(min(mints), max(maxts),'[]') FROM consts GROUP BY qualid, queryid, dbid, userid ) SELECT * FROM groups, LATERAL ( SELECT array_agg(constvalues) as mf FROM ( SELECT (constvalues, nbfiltered, count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN count = 0 THEN 0 ELSE nbfiltered / count::numeric END DESC LIMIT 20 ) s ) as mf, LATERAL ( SELECT array_agg(constvalues) as lf FROM ( SELECT (constvalues, nbfiltered, count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN count = 0 THEN 0 ELSE nbfiltered / count::numeric END DESC LIMIT 20 ) s ) as lf, LATERAL ( SELECT array_agg(constvalues) as me FROM ( SELECT (constvalues, nbfiltered, count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY count desc LIMIT 20 ) s ) as me; CREATE OR REPLACE FUNCTION powa_qualstats_snapshot() RETURNS void as $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_qualstats_snaphot'; WITH capture AS ( SELECT pgqs.*, s.query FROM pg_qualstats_by_query pgqs JOIN powa_statements s USING(queryid, dbid, userid) ), missing_quals AS ( INSERT INTO powa_qualstats_quals (qualid, queryid, dbid, userid, quals) SELECT DISTINCT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, array_agg(DISTINCT q::qual_type) FROM capture qs, LATERAL (SELECT (unnest(quals)).*) as q WHERE NOT EXISTS ( SELECT 1 FROM powa_qualstats_quals nh WHERE nh.qualid = qs.qualnodeid AND nh.queryid = qs.queryid AND nh.dbid = qs.dbid AND nh.userid = qs.userid ) GROUP BY qualnodeid, queryid, dbid, userid RETURNING * ), by_qual AS ( INSERT INTO powa_qualstats_quals_history_current (qualid, queryid, dbid, userid, ts, count, nbfiltered) SELECT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), sum(count), sum(nbfiltered) FROM capture as qs GROUP BY qualnodeid, qs.queryid, qs.dbid, qs.userid RETURNING * ), by_qual_with_const AS ( INSERT INTO powa_qualstats_constvalues_history_current(qualid, queryid, dbid, userid, ts, count, nbfiltered, constvalues) SELECT qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), count, nbfiltered, constvalues FROM capture as qs ) SELECT true into result; PERFORM pg_qualstats_reset(); END $PROC$ language plpgsql; /* * powa_qualstats aggregate */ CREATE OR REPLACE FUNCTION powa_qualstats_aggregate() RETURNS void AS $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_qualstats_aggregate'; LOCK TABLE powa_qualstats_constvalues_history_current IN SHARE MODE; LOCK TABLE powa_qualstats_quals_history_current IN SHARE MODE; INSERT INTO powa_qualstats_constvalues_history ( qualid, queryid, dbid, userid, coalesce_range, most_filtering, least_filtering, most_executed) SELECT * FROM powa_qualstats_aggregate_constvalues_current; INSERT INTO powa_qualstats_quals_history (qualid, queryid, dbid, userid, coalesce_range, records) SELECT qualid, queryid, dbid, userid, tstzrange(min(ts), max(ts),'[]'), array_agg((ts, count, nbfiltered)::powa_qualstats_history_item) FROM powa_qualstats_quals_history_current GROUP BY qualid, queryid, dbid, userid; TRUNCATE powa_qualstats_constvalues_history_current; TRUNCATE powa_qualstats_quals_history_current; END $PROC$ language plpgsql; /* * powa_qualstats_purge */ CREATE OR REPLACE FUNCTION powa_qualstats_purge() RETURNS void as $PROC$ BEGIN RAISE DEBUG 'running powa_qualstats_purge'; DELETE FROM powa_qualstats_constvalues_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); DELETE FROM powa_qualstats_quals_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); END; $PROC$ language plpgsql; /* * powa_qualstats_unregister */ CREATE OR REPLACE function public.powa_qualstats_unregister() RETURNS bool AS $_$ BEGIN DELETE FROM public.powa_functions WHERE module = 'pg_qualstats'; RETURN true; END; $_$ language plpgsql; SELECT * FROM public.powa_qualstats_register(); /* end of pg_qualstats_integration - part 2 */ powa-archivist-REL_3_2_0/powa--3.0.0--3.0.1.sql000066400000000000000000000175351336072707400202560ustar00rootroot00000000000000-- complain if script is sourced in psql, rather than via CREATE EXTENSION --\echo Use "ALTER EXTENSION powa" to load this file. \quit SET statement_timeout = 0; SET client_encoding = 'UTF8'; SET standard_conforming_strings = on; SET client_min_messages = warning; SET escape_string_warning = off; SET search_path = public, pg_catalog; CREATE TABLE powa_databases( oid oid PRIMARY KEY, datname name, dropped timestamp with time zone ); CREATE OR REPLACE FUNCTION powa_statements_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; ignore_regexp text:='^[[:space:]]*(DEALLOCATE|BEGIN|PREPARE TRANSACTION|COMMIT PREPARED|ROLLBACK PREPARED)'; BEGIN -- In this function, we capture statements, and also aggregate counters by database -- so that the first screens of powa stay reactive even though there may be thousands -- of different statements RAISE DEBUG 'running powa_statements_snapshot'; WITH capture AS( SELECT pgss.* FROM pg_stat_statements pgss JOIN pg_roles r ON pgss.userid = r.oid WHERE pgss.query !~* ignore_regexp AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_statements AS( INSERT INTO powa_statements (queryid, dbid, userid, query) SELECT queryid, dbid, userid, query FROM capture c WHERE NOT EXISTS (SELECT 1 FROM powa_statements ps WHERE ps.queryid = c.queryid AND ps.dbid = c.dbid AND ps.userid = c.userid ) ), by_query AS ( INSERT INTO powa_statements_history_current SELECT queryid, dbid, userid, ROW( now(), calls, total_time, rows, shared_blks_hit, shared_blks_read, shared_blks_dirtied, shared_blks_written, local_blks_hit, local_blks_read, local_blks_dirtied, local_blks_written, temp_blks_read, temp_blks_written, blk_read_time, blk_write_time )::powa_statements_history_record AS record FROM capture ), by_database AS ( INSERT INTO powa_statements_history_current_db SELECT dbid, ROW( now(), sum(calls), sum(total_time), sum(rows), sum(shared_blks_hit), sum(shared_blks_read), sum(shared_blks_dirtied), sum(shared_blks_written), sum(local_blks_hit), sum(local_blks_read), sum(local_blks_dirtied), sum(local_blks_written), sum(temp_blks_read), sum(temp_blks_written), sum(blk_read_time), sum(blk_write_time) )::powa_statements_history_record AS record FROM capture GROUP BY dbid ) SELECT true::boolean INTO result; -- For now we don't care. What could we do on error except crash anyway? END; $PROC$ language plpgsql; CREATE OR REPLACE FUNCTION powa_take_snapshot() RETURNS void AS $PROC$ DECLARE purgets timestamp with time zone; purge_seq bigint; funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- Keep track of existing databases WITH missing AS ( SELECT d.oid, d.datname FROM pg_database d LEFT JOIN powa_databases p ON d.oid = p.oid WHERE p.oid IS NULL ) INSERT INTO powa_databases SELECT * FROM missing; -- Keep track of renamed databases WITH renamed AS ( SELECT d.oid, d.datname FROM pg_database AS d JOIN powa_databases AS p ON d.oid = p.oid WHERE d.datname != p.datname ) UPDATE powa_databases AS p SET datname = r.datname FROM renamed AS r WHERE p.oid = r.oid; -- Keep track of when databases are dropped WITH dropped AS ( SELECT p.oid FROM powa_databases p LEFT JOIN pg_database d ON p.oid = d.oid WHERE d.oid IS NULL AND p.dropped IS NULL) UPDATE powa_databases p SET dropped = now() FROM dropped d WHERE p.oid = d.oid; -- For all enabled snapshot functions in the powa_functions table, execute FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='snapshot' AND enabled LOOP -- Call all of them, with no parameter RAISE debug 'fonction: %',funcname; BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; -- Coalesce datas if needed SELECT nextval('powa_coalesce_sequence'::regclass) INTO purge_seq; IF ( purge_seq % current_setting('powa.coalesce')::bigint ) = 0 THEN FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='aggregate' AND enabled LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; UPDATE powa_last_aggregation SET aggts = now(); END IF; -- Once every 10 packs, we also purge IF ( purge_seq % (current_setting('powa.coalesce')::bigint *10) ) = 0 THEN FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='purge' AND enabled LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; UPDATE powa_last_purge SET purgets = now(); END IF; END; $PROC$ LANGUAGE plpgsql; -- remove entries that should not have been stored DELETE FROM powa_statements_history_current pshc USING powa_statements ps WHERE pshc.queryid = ps.queryid AND pshc.dbid = ps.dbid AND pshc.userid = ps.userid AND ps.query ~* '^[[:space:]]*(DEALLOCATE|BEGIN|PREPARE TRANSACTION|COMMIT PREPARED|ROLLBACK PREPARED)'; DELETE FROM powa_statements_history psh USING powa_statements ps WHERE psh.queryid = ps.queryid AND psh.dbid = ps.dbid AND psh.userid = ps.userid AND ps.query ~* '^[[:space:]]*(DEALLOCATE|BEGIN|PREPARE TRANSACTION|COMMIT PREPARED|ROLLBACK PREPARED)'; DELETE FROM powa_statements WHERE query ~* '^[[:space:]]*(DEALLOCATE|BEGIN|PREPARE TRANSACTION|COMMIT PREPARED|ROLLBACK PREPARED)'; powa-archivist-REL_3_2_0/powa--3.0.0.sql000066400000000000000000001341071336072707400175370ustar00rootroot00000000000000-- complain if script is sourced in psql, rather than via CREATE EXTENSION --\echo Use "CREATE EXTENSION powa" to load this file. \quit SET statement_timeout = 0; SET client_encoding = 'UTF8'; SET standard_conforming_strings = on; SET client_min_messages = warning; SET escape_string_warning = off; SET search_path = public, pg_catalog; CREATE FUNCTION powa_stat_user_functions(IN dbid oid, OUT funcid oid, OUT calls bigint, OUT total_time double precision, OUT self_time double precision) RETURNS SETOF record LANGUAGE c COST 100 AS '$libdir/powa', 'powa_stat_user_functions'; CREATE FUNCTION powa_stat_all_rel(IN dbid oid, OUT relid oid, OUT numscan bigint, OUT tup_returned bigint, OUT tup_fetched bigint, OUT n_tup_ins bigint, OUT n_tup_upd bigint, OUT n_tup_del bigint, OUT n_tup_hot_upd bigint, OUT n_liv_tup bigint, OUT n_dead_tup bigint, OUT n_mod_since_analyze bigint, OUT blks_read bigint, OUT blks_hit bigint, OUT last_vacuum timestamp with time zone, OUT vacuum_count bigint, OUT last_autovacuum timestamp with time zone, OUT autovacuum_count bigint, OUT last_analyze timestamp with time zone, OUT analyze_count bigint, OUT last_autoanalyze timestamp with time zone, OUT autoanalyze_count bigint) RETURNS SETOF record LANGUAGE c COST 100 AS '$libdir/powa', 'powa_stat_all_rel'; CREATE TYPE powa_statements_history_record AS ( ts timestamp with time zone, calls bigint, total_time double precision, rows bigint, shared_blks_hit bigint, shared_blks_read bigint, shared_blks_dirtied bigint, shared_blks_written bigint, local_blks_hit bigint, local_blks_read bigint, local_blks_dirtied bigint, local_blks_written bigint, temp_blks_read bigint, temp_blks_written bigint, blk_read_time double precision, blk_write_time double precision ); CREATE TYPE powa_user_functions_history_record AS ( ts timestamp with time zone, calls bigint, total_time double precision, self_time double precision ); CREATE TYPE powa_all_relations_history_record AS ( ts timestamp with time zone, numscan bigint, tup_returned bigint, tup_fetched bigint, n_tup_ins bigint, n_tup_upd bigint, n_tup_del bigint, n_tup_hot_upd bigint, n_liv_tup bigint, n_dead_tup bigint, n_mod_since_analyze bigint, blks_read bigint, blks_hit bigint, last_vacuum timestamp with time zone, vacuum_count bigint, last_autovacuum timestamp with time zone, autovacuum_count bigint, last_analyze timestamp with time zone, analyze_count bigint, last_autoanalyze timestamp with time zone, autoanalyze_count bigint ); CREATE TABLE powa_last_aggregation ( aggts timestamp with time zone ); INSERT INTO powa_last_aggregation(aggts) VALUES (current_timestamp); CREATE TABLE powa_last_purge ( purgets timestamp with time zone ); INSERT INTO powa_last_purge (purgets) VALUES (current_timestamp); CREATE TABLE powa_statements ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, query text NOT NULL ); ALTER TABLE ONLY powa_statements ADD CONSTRAINT powa_statements_pkey PRIMARY KEY (queryid, dbid, userid); CREATE INDEX powa_statements_dbid_idx ON powa_statements(dbid); CREATE INDEX powa_statements_userid_idx ON powa_statements(userid); CREATE TABLE powa_statements_history ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statements_history_record[] NOT NULL, mins_in_range powa_statements_history_record NOT NULL, maxs_in_range powa_statements_history_record NOT NULL ); CREATE INDEX powa_statements_history_query_ts ON powa_statements_history USING gist (queryid, coalesce_range); CREATE TABLE powa_statements_history_db ( dbid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statements_history_record[] NOT NULL, mins_in_range powa_statements_history_record NOT NULL, maxs_in_range powa_statements_history_record NOT NULL ); CREATE INDEX powa_statements_history_db_ts ON powa_statements_history_db USING gist (dbid, coalesce_range); CREATE TABLE powa_statements_history_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, record powa_statements_history_record NOT NULL ); CREATE TABLE powa_statements_history_current_db ( dbid oid NOT NULL, record powa_statements_history_record NOT NULL ); CREATE TABLE powa_user_functions_history ( dbid oid NOT NULL, funcid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_user_functions_history_record[] NOT NULL, mins_in_range powa_user_functions_history_record NOT NULL, maxs_in_range powa_user_functions_history_record NOT NULL ); CREATE INDEX powa_user_functions_history_funcid_ts ON powa_user_functions_history USING gist (funcid, coalesce_range); CREATE TABLE powa_user_functions_history_current ( dbid oid NOT NULL, funcid oid NOT NULL, record powa_user_functions_history_record NOT NULL ); CREATE TABLE powa_all_relations_history ( dbid oid NOT NULL, relid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_all_relations_history_record[] NOT NULL, mins_in_range powa_all_relations_history_record NOT NULL, maxs_in_range powa_all_relations_history_record NOT NULL ); CREATE INDEX powa_all_relations_history_relid_ts ON powa_all_relations_history USING gist (relid, coalesce_range); CREATE TABLE powa_all_relations_history_current ( dbid oid NOT NULL, relid oid NOT NULL, record powa_all_relations_history_record NOT NULL ); CREATE SEQUENCE powa_coalesce_sequence INCREMENT BY 1 START WITH 1 CYCLE; CREATE TABLE powa_functions ( module text NOT NULL, operation text NOT NULL, function_name text NOT NULL, added_manually boolean NOT NULL default true, enabled boolean NOT NULL default true, CHECK (operation IN ('snapshot','aggregate','purge','unregister','reset')) ); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_stat_statements', 'snapshot', 'powa_statements_snapshot', false, true), ('powa_stat_user_functions', 'snapshot', 'powa_user_functions_snapshot', false, true), ('powa_stat_all_relations', 'snapshot', 'powa_all_relations_snapshot', false, true), ('pg_stat_statements', 'aggregate','powa_statements_aggregate', false, true), ('powa_stat_user_functions', 'aggregate','powa_user_functions_aggregate', false, true), ('powa_stat_all_relations', 'aggregate','powa_all_relations_aggregate', false, true), ('pg_stat_statements', 'purge', 'powa_statements_purge', false, true), ('powa_stat_user_functions', 'purge', 'powa_user_functions_purge', false, true), ('powa_stat_all_relations', 'purge', 'powa_all_relations_purge', false, true), ('pg_stat_statements', 'reset', 'powa_statements_reset', false, true), ('powa_stat_user_functions', 'reset', 'powa_user_functions_reset', false, true), ('powa_stat_all_relations', 'reset', 'powa_all_relations_reset', false, true); /* pg_stat_kcache integration - part 1 */ CREATE TYPE public.kcache_type AS ( ts timestamptz, reads bigint, writes bigint, user_time double precision, system_time double precision ); CREATE TABLE public.powa_kcache_metrics ( coalesce_range tstzrange NOT NULL, queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics public.kcache_type[] NOT NULL, mins_in_range public.kcache_type NOT NULL, maxs_in_range public.kcache_type NOT NULL, PRIMARY KEY (coalesce_range, queryid, dbid, userid) ); CREATE INDEX ON public.powa_kcache_metrics (queryid); CREATE TABLE public.powa_kcache_metrics_db ( coalesce_range tstzrange NOT NULL, dbid oid NOT NULL, metrics public.kcache_type[] NOT NULL, mins_in_range public.kcache_type NOT NULL, maxs_in_range public.kcache_type NOT NULL, PRIMARY KEY (coalesce_range, dbid) ); CREATE TABLE public.powa_kcache_metrics_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics kcache_type NULL NULL ); CREATE TABLE public.powa_kcache_metrics_current_db ( dbid oid NOT NULL, metrics kcache_type NULL NULL ); /* end of pg_stat_kcache integration - part 1 */ /* pg_qualstats integration - part 1 */ CREATE TYPE public.qual_type AS ( relid oid, attnum integer, opno oid, eval_type "char" ); CREATE TYPE public.qual_values AS ( constants text[], occurences bigint, execution_count bigint, nbfiltered bigint ); CREATE TYPE powa_qualstats_history_item AS ( ts timestamptz, occurences bigint, execution_count bigint, nbfiltered bigint ); CREATE TABLE public.powa_qualstats_quals ( qualid bigint, queryid bigint, dbid oid, userid oid, quals public.qual_type[], PRIMARY KEY (qualid, queryid, dbid, userid), FOREIGN KEY (queryid, dbid, userid) REFERENCES powa_statements(queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, records powa_qualstats_history_item[], mins_in_range powa_qualstats_history_item, maxs_in_range powa_qualstats_history_item, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, occurences bigint, execution_count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES powa_qualstats_quals(qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, most_used qual_values[], most_filtering qual_values[], least_filtering qual_values[], most_executed qual_values[], FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, constvalues text[], occurences bigint, execution_count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE INDEX ON powa_qualstats_constvalues_history USING gist (queryid, qualid, coalesce_range); CREATE INDEX ON powa_qualstats_constvalues_history (qualid, queryid); CREATE INDEX ON powa_qualstats_quals(queryid); /* end of pg_qualstats_integration - part 1 */ -- Mark all of powa's tables as "to be dumped" SELECT pg_catalog.pg_extension_config_dump('powa_statements',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_user_functions_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_user_functions_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_all_relations_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_all_relations_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_functions','WHERE added_manually'); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history_current',''); CREATE OR REPLACE FUNCTION public.powa_check_created_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE BEGIN /* We have for now no way for a proper handling of this event, * as we don't have a table with the list of supported extensions. * So just call every powa_*_register() function we know each time an * extension is created. Powa should be in a dedicated database and the * register function handle to be called several time, so it's not critical */ PERFORM public.powa_kcache_register(); PERFORM public.powa_qualstats_register(); PERFORM public.powa_track_settings_register(); END; $_$; CREATE EVENT TRIGGER powa_check_created_extensions ON ddl_command_end WHEN tag IN ('CREATE EXTENSION') EXECUTE PROCEDURE public.powa_check_created_extensions() ; CREATE OR REPLACE FUNCTION public.powa_check_dropped_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- We unregister extensions regardless the "enabled" field WITH ext AS ( SELECT object_name FROM pg_event_trigger_dropped_objects() d WHERE d.object_type = 'extension' ) SELECT function_name INTO funcname FROM powa_functions f JOIN ext ON f.module = ext.object_name WHERE operation = 'unregister'; IF ( funcname IS NOT NULL ) THEN BEGIN RAISE DEBUG 'running %', funcname; EXECUTE 'SELECT ' || quote_ident(funcname) || '()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE WARNING 'powa_check_dropped_extensions(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END IF; END; $_$; CREATE EVENT TRIGGER powa_check_dropped_extensions ON sql_drop WHEN tag IN ('DROP EXTENSION') EXECUTE PROCEDURE public.powa_check_dropped_extensions() ; CREATE OR REPLACE FUNCTION powa_take_snapshot() RETURNS void AS $PROC$ DECLARE purgets timestamp with time zone; purge_seq bigint; funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- For all enabled snapshot functions in the powa_functions table, execute FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='snapshot' AND enabled LOOP -- Call all of them, with no parameter RAISE debug 'fonction: %',funcname; BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; -- Coalesce datas if needed SELECT nextval('powa_coalesce_sequence'::regclass) INTO purge_seq; IF ( purge_seq % current_setting('powa.coalesce')::bigint ) = 0 THEN FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='aggregate' AND enabled LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; UPDATE powa_last_aggregation SET aggts = now(); END IF; -- Once every 10 packs, we also purge IF ( purge_seq % (current_setting('powa.coalesce')::bigint *10) ) = 0 THEN FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='purge' AND enabled LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; UPDATE powa_last_purge SET purgets = now(); END IF; END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_statements_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; ignore_regexp text:='^[[:space:]]*(BEGIN)'; -- Ignore begin at beginning of statement BEGIN -- In this function, we capture statements, and also aggregate counters by database -- so that the first screens of powa stay reactive even though there may be thousands -- of different statements RAISE DEBUG 'running powa_statements_snapshot'; WITH capture AS( SELECT pgss.* FROM pg_stat_statements pgss JOIN pg_roles r ON pgss.userid = r.oid WHERE pgss.query !~* ignore_regexp AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_statements AS( INSERT INTO powa_statements (queryid, dbid, userid, query) SELECT queryid, dbid, userid, query FROM capture c WHERE NOT EXISTS (SELECT 1 FROM powa_statements ps WHERE ps.queryid = c.queryid AND ps.dbid = c.dbid AND ps.userid = c.userid ) ), by_query AS ( INSERT INTO powa_statements_history_current SELECT queryid, dbid, userid, ROW( now(), calls, total_time, rows, shared_blks_hit, shared_blks_read, shared_blks_dirtied, shared_blks_written, local_blks_hit, local_blks_read, local_blks_dirtied, local_blks_written, temp_blks_read, temp_blks_written, blk_read_time, blk_write_time )::powa_statements_history_record AS record FROM capture ), by_database AS ( INSERT INTO powa_statements_history_current_db SELECT dbid, ROW( now(), sum(calls), sum(total_time), sum(rows), sum(shared_blks_hit), sum(shared_blks_read), sum(shared_blks_dirtied), sum(shared_blks_written), sum(local_blks_hit), sum(local_blks_read), sum(local_blks_dirtied), sum(local_blks_written), sum(temp_blks_read), sum(temp_blks_written), sum(blk_read_time), sum(blk_write_time) )::powa_statements_history_record AS record FROM capture GROUP BY dbid ) SELECT true::boolean INTO result; -- For now we don't care. What could we do on error except crash anyway? END; $PROC$ language plpgsql; CREATE OR REPLACE FUNCTION powa_user_functions_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; BEGIN RAISE DEBUG 'running powa_user_functions_snapshot'; -- Insert cluster-wide user function statistics WITH func(dbid,funcid, r) AS ( SELECT oid, (powa_stat_user_functions(oid)).funcid, powa_stat_user_functions(oid) FROM pg_database ) INSERT INTO powa_user_functions_history_current SELECT dbid, funcid, ROW(now(), (r).calls, (r).total_time, (r).self_time)::powa_user_functions_history_record AS record FROM func; result := true; END; $PROC$ language plpgsql; CREATE OR REPLACE FUNCTION powa_all_relations_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; BEGIN RAISE DEBUG 'running powa_all_relations_snapshot'; -- Insert cluster-wide relation statistics WITH rel(dbid, relid, r) AS ( SELECT oid, (powa_stat_all_rel(oid)).relid, powa_stat_all_rel(oid) FROM pg_database ) INSERT INTO powa_all_relations_history_current SELECT dbid, relid, ROW(now(),(r).numscan, (r).tup_returned, (r).tup_fetched, (r).n_tup_ins, (r).n_tup_upd, (r).n_tup_del, (r).n_tup_hot_upd, (r).n_liv_tup, (r).n_dead_tup, (r).n_mod_since_analyze, (r).blks_read, (r).blks_hit, (r).last_vacuum, (r).vacuum_count, (r).last_autovacuum, (r).autovacuum_count, (r).last_analyze, (r).analyze_count, (r).last_autoanalyze, (r).autoanalyze_count)::powa_all_relations_history_record AS record FROM rel; result := true; END; $PROC$ language plpgsql; CREATE OR REPLACE FUNCTION powa_statements_purge() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_statements_purge'; -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_statements_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); DELETE FROM powa_statements_history_db WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_user_functions_purge() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_user_functions_purge'; -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_user_functions_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_all_relations_purge() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_all_relations_purge'; -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_all_relations_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_statements_aggregate() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_statements_aggregate'; -- aggregate statements table LOCK TABLE powa_statements_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history SELECT queryid, dbid, userid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current GROUP BY queryid, dbid, userid; TRUNCATE powa_statements_history_current; -- aggregate db table LOCK TABLE powa_statements_history_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history_db SELECT dbid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current_db GROUP BY dbid; TRUNCATE powa_statements_history_current_db; END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_user_functions_aggregate() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_user_functions_aggregate'; -- aggregate user_functions table LOCK TABLE powa_user_functions_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_user_functions_history SELECT dbid, funcid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time), min((record).self_time))::powa_user_functions_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time), max((record).self_time))::powa_user_functions_history_record FROM powa_user_functions_history_current GROUP BY dbid, funcid; TRUNCATE powa_user_functions_history_current; END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_all_relations_aggregate() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_all_relations_aggregate'; -- aggregate all_relations table LOCK TABLE powa_all_relations_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_all_relations_history SELECT dbid, relid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).numscan),min((record).tup_returned),min((record).tup_fetched), min((record).n_tup_ins),min((record).n_tup_upd), min((record).n_tup_del),min((record).n_tup_hot_upd), min((record).n_liv_tup),min((record).n_dead_tup), min((record).n_mod_since_analyze),min((record).blks_read), min((record).blks_hit),min((record).last_vacuum), min((record).vacuum_count),min((record).last_autovacuum), min((record).autovacuum_count),min((record).last_analyze), min((record).analyze_count),min((record).last_autoanalyze), min((record).autoanalyze_count))::powa_all_relations_history_record, ROW(max((record).ts), max((record).numscan),max((record).tup_returned),max((record).tup_fetched), max((record).n_tup_ins),max((record).n_tup_upd), max((record).n_tup_del),max((record).n_tup_hot_upd), max((record).n_liv_tup),max((record).n_dead_tup), max((record).n_mod_since_analyze),max((record).blks_read), max((record).blks_hit),max((record).last_vacuum), max((record).vacuum_count),max((record).last_autovacuum), max((record).autovacuum_count),max((record).last_analyze), max((record).analyze_count),max((record).last_autoanalyze), max((record).autoanalyze_count))::powa_all_relations_history_record FROM powa_all_relations_history_current GROUP BY dbid, relid; TRUNCATE powa_all_relations_history_current; END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION public.powa_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- Find reset function for every supported datasource, including pgss -- Also call reset function even if they're not enabled FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='reset' LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_reset(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; RETURN true; END; $function$; CREATE OR REPLACE FUNCTION public.powa_statements_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN TRUNCATE TABLE powa_statements_history; TRUNCATE TABLE powa_statements_history_current; TRUNCATE TABLE powa_statements_history_db; TRUNCATE TABLE powa_statements_history_current_db; -- if 3rd part datasource has FK on it, throw everything away TRUNCATE TABLE powa_statements CASCADE; RETURN true; END; $function$; CREATE OR REPLACE FUNCTION public.powa_user_functions_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN TRUNCATE TABLE powa_user_functions_history; TRUNCATE TABLE powa_user_functions_history_current; RETURN true; END; $function$; CREATE OR REPLACE FUNCTION public.powa_all_relations_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN TRUNCATE TABLE powa_all_relations_history; TRUNCATE TABLE powa_all_relations_history_current; RETURN true; END; $function$; /* pg_stat_kcache integration - part 2 */ /* * register pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_stat_kcache'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_stat_kcache'; IF ( NOT v_func_present) THEN INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_stat_kcache', 'snapshot', 'powa_kcache_snapshot', false, true), ('pg_stat_kcache', 'aggregate', 'powa_kcache_aggregate', false, true), ('pg_stat_kcache', 'unregister', 'powa_kcache_unregister', false, true), ('pg_stat_kcache', 'purge', 'powa_kcache_purge', false, true), ('pg_stat_kcache', 'reset', 'powa_kcache_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* * unregister pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_unregister() RETURNS bool AS $_$ BEGIN DELETE FROM public.powa_functions WHERE module = 'pg_stat_kcache'; RETURN true; END; $_$ language plpgsql; /* * powa_kcache snapshot collection. */ CREATE OR REPLACE FUNCTION powa_kcache_snapshot() RETURNS void as $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_kcache_snapshot'; WITH capture AS ( SELECT * FROM pg_stat_kcache() k JOIN pg_roles r ON r.oid = k.userid WHERE NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), by_query AS ( INSERT INTO powa_kcache_metrics_current (queryid, dbid, userid, metrics) SELECT queryid, dbid, userid, (now(), reads, writes, user_time, system_time)::kcache_type FROM capture ), by_database AS ( INSERT INTO powa_kcache_metrics_current_db (dbid, metrics) SELECT dbid, (now(), sum(reads), sum(writes), sum(user_time), sum(system_time))::kcache_type FROM capture GROUP BY dbid ) SELECT true into result; END $PROC$ language plpgsql; /* * powa_kcache aggregation */ CREATE OR REPLACE FUNCTION powa_kcache_aggregate() RETURNS void AS $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_kcache_aggregate'; -- aggregate metrics table LOCK TABLE powa_kcache_metrics_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics (coalesce_range, queryid, dbid, userid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), queryid, dbid, userid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current GROUP BY queryid, dbid, userid; TRUNCATE powa_kcache_metrics_current; -- aggregate metrics_db table LOCK TABLE powa_kcache_metrics_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics_db (coalesce_range, dbid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), dbid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current_db GROUP BY dbid; TRUNCATE powa_kcache_metrics_current_db; END $PROC$ language plpgsql; /* * powa_kcache purge */ CREATE OR REPLACE FUNCTION powa_kcache_purge() RETURNS void as $PROC$ BEGIN RAISE DEBUG 'running powa_kcache_purge'; DELETE FROM powa_kcache_metrics WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); DELETE FROM powa_kcache_metrics_db WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); END; $PROC$ language plpgsql; /* * powa_kcache reset */ CREATE OR REPLACE FUNCTION powa_kcache_reset() RETURNS void as $PROC$ BEGIN RAISE DEBUG 'running powa_kcache_reset'; TRUNCATE TABLE powa_kcache_metrics; TRUNCATE TABLE powa_kcache_metrics_db; TRUNCATE TABLE powa_kcache_metrics_current; TRUNCATE TABLE powa_kcache_metrics_current_db; END; $PROC$ language plpgsql; -- By default, try to register pg_stat_kcache, in case it's alreay here SELECT * FROM public.powa_kcache_register(); /* end of pg_stat_kcache integration - part 2 */ /* pg_qualstats integration - part 2 */ /* * powa_qualstats_register */ CREATE OR REPLACE function public.powa_qualstats_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_qualstats'; IF ( v_ext_present) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE function_name IN ('powa_qualstats_snapshot', 'powa_qualstats_aggregate', 'powa_qualstats_purge'); IF ( NOT v_func_present) THEN INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_qualstats', 'snapshot', 'powa_qualstats_snapshot', false, true), ('pg_qualstats', 'aggregate', 'powa_qualstats_aggregate', false, true), ('pg_qualstats', 'unregister', 'powa_qualstats_unregister', false, true), ('pg_qualstats', 'purge', 'powa_qualstats_purge', false, true), ('pg_qualstats', 'reset', 'powa_qualstats_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* * powa_qualstats utility view for aggregating constvalues */ CREATE OR REPLACE VIEW powa_qualstats_aggregate_constvalues_current AS WITH consts AS ( SELECT qualid, queryid, dbid, userid, min(ts) as mints, max(ts) as maxts, sum(occurences) as occurences, sum(nbfiltered) as nbfiltered, sum(execution_count) as execution_count, constvalues FROM powa_qualstats_constvalues_history_current GROUP BY qualid, queryid, dbid, userid, constvalues ), groups AS ( SELECT qualid, queryid, dbid, userid, tstzrange(min(mints), max(maxts),'[]') FROM consts GROUP BY qualid, queryid, dbid, userid ) SELECT * FROM groups, LATERAL ( SELECT array_agg(constvalues) as mu FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY occurences desc LIMIT 20 ) s ) as mu, LATERAL ( SELECT array_agg(constvalues) as mf FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN execution_count = 0 THEN 0 ELSE nbfiltered / execution_count::numeric END DESC LIMIT 20 ) s ) as mf, LATERAL ( SELECT array_agg(constvalues) as lf FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN execution_count = 0 THEN 0 ELSE nbfiltered / execution_count::numeric END DESC LIMIT 20 ) s ) as lf, LATERAL ( SELECT array_agg(constvalues) as me FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY execution_count desc LIMIT 20 ) s ) as me; CREATE OR REPLACE FUNCTION powa_qualstats_snapshot() RETURNS void as $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_qualstats_snaphot'; WITH capture AS ( SELECT pgqs.*, s.query FROM pg_qualstats_by_query pgqs JOIN powa_statements s USING(queryid, dbid, userid) JOIN pg_roles r ON s.userid = r.oid AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_quals AS ( INSERT INTO powa_qualstats_quals (qualid, queryid, dbid, userid, quals) SELECT DISTINCT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, array_agg(DISTINCT q::qual_type) FROM capture qs, LATERAL (SELECT (unnest(quals)).*) as q WHERE NOT EXISTS ( SELECT 1 FROM powa_qualstats_quals nh WHERE nh.qualid = qs.qualnodeid AND nh.queryid = qs.queryid AND nh.dbid = qs.dbid AND nh.userid = qs.userid ) GROUP BY qualnodeid, queryid, dbid, userid RETURNING * ), by_qual AS ( INSERT INTO powa_qualstats_quals_history_current (qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered) SELECT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), sum(occurences), sum(execution_count), sum(nbfiltered) FROM capture as qs GROUP BY qualnodeid, qs.queryid, qs.dbid, qs.userid RETURNING * ), by_qual_with_const AS ( INSERT INTO powa_qualstats_constvalues_history_current(qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered, constvalues) SELECT qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), occurences, execution_count, nbfiltered, constvalues FROM capture as qs ) SELECT true into result; PERFORM pg_qualstats_reset(); END $PROC$ language plpgsql; /* * powa_qualstats aggregate */ CREATE OR REPLACE FUNCTION powa_qualstats_aggregate() RETURNS void AS $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_qualstats_aggregate'; LOCK TABLE powa_qualstats_constvalues_history_current IN SHARE MODE; LOCK TABLE powa_qualstats_quals_history_current IN SHARE MODE; INSERT INTO powa_qualstats_constvalues_history ( qualid, queryid, dbid, userid, coalesce_range, most_used, most_filtering, least_filtering, most_executed) SELECT * FROM powa_qualstats_aggregate_constvalues_current; INSERT INTO powa_qualstats_quals_history (qualid, queryid, dbid, userid, coalesce_range, records, mins_in_range, maxs_in_range) SELECT qualid, queryid, dbid, userid, tstzrange(min(ts), max(ts),'[]'), array_agg((ts, occurences, execution_count, nbfiltered)::powa_qualstats_history_item), ROW(min(ts), min(occurences), min(execution_count), min(nbfiltered))::powa_qualstats_history_item, ROW(max(ts), max(occurences), max(execution_count), max(nbfiltered))::powa_qualstats_history_item FROM powa_qualstats_quals_history_current GROUP BY qualid, queryid, dbid, userid; TRUNCATE powa_qualstats_constvalues_history_current; TRUNCATE powa_qualstats_quals_history_current; END $PROC$ language plpgsql; /* * powa_qualstats_purge */ CREATE OR REPLACE FUNCTION powa_qualstats_purge() RETURNS void as $PROC$ BEGIN RAISE DEBUG 'running powa_qualstats_purge'; DELETE FROM powa_qualstats_constvalues_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); DELETE FROM powa_qualstats_quals_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); END; $PROC$ language plpgsql; /* * powa_qualstats_reset */ CREATE OR REPLACE FUNCTION powa_qualstats_reset() RETURNS void as $PROC$ BEGIN RAISE DEBUG 'running powa_qualstats_reset'; TRUNCATE TABLE powa_qualstats_quals CASCADE; -- cascaded : -- powa_qualstats_quals_history -- powa_qualstats_quals_history_current -- powa_qualstats_constvalues_history -- powa_qualstats_constvalues_history_current END; $PROC$ language plpgsql; /* * powa_qualstats_unregister */ CREATE OR REPLACE function public.powa_qualstats_unregister() RETURNS bool AS $_$ BEGIN DELETE FROM public.powa_functions WHERE module = 'pg_qualstats'; RETURN true; END; $_$ language plpgsql; SELECT * FROM public.powa_qualstats_register(); /* end of pg_qualstats_integration - part 2 */ /* pg_track_settings integration */ CREATE OR REPLACE FUNCTION powa_track_settings_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_track_settings'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_track_settings'; IF ( NOT v_func_present) THEN -- This extension handles its own storage, just its snapshot -- function and an unregister function. INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_track_settings', 'snapshot', 'pg_track_settings_snapshot', false, true), ('pg_track_settings', 'unregister', 'powa_track_settings_unregister', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; CREATE OR REPLACE function public.powa_track_settings_unregister() RETURNS bool AS $_$ BEGIN DELETE FROM public.powa_functions WHERE module = 'pg_track_settings'; RETURN true; END; $_$ language plpgsql; -- By default, try to register pg_track_settings, in case it's alreay here SELECT * FROM public.powa_track_settings_register(); /* end pg_track_settings integration */ powa-archivist-REL_3_2_0/powa--3.0.1--3.1.0.sql000066400000000000000000001531611336072707400202530ustar00rootroot00000000000000-- complain if script is sourced in psql, rather than via CREATE EXTENSION --\echo Use "ALTER EXTENSION powa" to load this file. \quit SET statement_timeout = 0; SET client_encoding = 'UTF8'; SET standard_conforming_strings = on; SET client_min_messages = warning; SET escape_string_warning = off; SET search_path = public, pg_catalog; CREATE FUNCTION powa_log (msg text) RETURNS void LANGUAGE plpgsql AS $_$ BEGIN IF current_setting('powa.debug')::bool THEN RAISE WARNING '%', msg; ELSE RAISE DEBUG '%', msg; END IF; END; $_$; CREATE OR REPLACE FUNCTION public.powa_check_dropped_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- We unregister extensions regardless the "enabled" field WITH ext AS ( SELECT object_name FROM pg_event_trigger_dropped_objects() d WHERE d.object_type = 'extension' ) SELECT function_name INTO funcname FROM powa_functions f JOIN ext ON f.module = ext.object_name WHERE operation = 'unregister'; IF ( funcname IS NOT NULL ) THEN BEGIN PERFORM powa_log(format('running %I', funcname)); EXECUTE 'SELECT ' || quote_ident(funcname) || '()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE WARNING 'powa_check_dropped_extensions(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END IF; END; $_$; /* end of powa_check_dropped_extensions */ CREATE OR REPLACE FUNCTION powa_take_snapshot() RETURNS void AS $PROC$ DECLARE purgets timestamp with time zone; purge_seq bigint; funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; v_title text = 'PoWA - '; v_rowcount bigint; BEGIN PERFORM set_config('application_name', v_title || ' snapshot database list', false); PERFORM powa_log('start of powa_take_snapshot'); -- Keep track of existing databases PERFORM powa_log('Maintaining database list...'); WITH missing AS ( SELECT d.oid, d.datname FROM pg_database d LEFT JOIN powa_databases p ON d.oid = p.oid WHERE p.oid IS NULL ) INSERT INTO powa_databases SELECT * FROM missing; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('missing db: %s', v_rowcount)); -- Keep track of renamed databases WITH renamed AS ( SELECT d.oid, d.datname FROM pg_database AS d JOIN powa_databases AS p ON d.oid = p.oid WHERE d.datname != p.datname ) UPDATE powa_databases AS p SET datname = r.datname FROM renamed AS r WHERE p.oid = r.oid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('renamed db: %s', v_rowcount)); -- Keep track of when databases are dropped WITH dropped AS ( SELECT p.oid FROM powa_databases p LEFT JOIN pg_database d ON p.oid = d.oid WHERE d.oid IS NULL AND p.dropped IS NULL) UPDATE powa_databases p SET dropped = now() FROM dropped d WHERE p.oid = d.oid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('dropped db: %s', v_rowcount)); -- For all enabled snapshot functions in the powa_functions table, execute FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='snapshot' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling snapshot function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; -- Coalesce datas if needed SELECT nextval('powa_coalesce_sequence'::regclass) INTO purge_seq; PERFORM powa_log(format('powa_coalesce_sequence: %s', purge_seq)); IF ( purge_seq % current_setting('powa.coalesce')::bigint ) = 0 THEN PERFORM powa_log(format('coalesce needed, seq: %s coalesce seq: %s', purge_seq, current_setting('powa.coalesce')::bigint )); FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='aggregate' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling aggregate function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; UPDATE powa_last_aggregation SET aggts = now(); END IF; -- We also purge, at next pass IF ( purge_seq % (current_setting('powa.coalesce')::bigint ) ) = 1 THEN PERFORM powa_log(format('purge needed, seq: %s coalesce seq: %s', purge_seq, current_setting('powa.coalesce'))); FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='purge' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling purge function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; PERFORM set_config('application_name', v_title || 'UPDATE powa_last_purge', false); UPDATE powa_last_purge SET purgets = now(); END IF; PERFORM powa_log('end of powa_take_snapshot'); PERFORM set_config('application_name', v_title || 'snapshot finished', false); END; $PROC$ LANGUAGE plpgsql; /* end of powa_take_snapshot */ CREATE OR REPLACE FUNCTION powa_statements_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; ignore_regexp text :='^[[:space:]]*(DEALLOCATE|BEGIN|PREPARE TRANSACTION|COMMIT PREPARED|ROLLBACK PREPARED)'; v_funcname text := 'powa_statements_snapshot'; v_rowcount bigint; BEGIN -- In this function, we capture statements, and also aggregate counters by database -- so that the first screens of powa stay reactive even though there may be thousands -- of different statements PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS( SELECT pgss.* FROM pg_stat_statements pgss JOIN pg_roles r ON pgss.userid = r.oid WHERE pgss.query !~* ignore_regexp AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_statements AS( INSERT INTO powa_statements (queryid, dbid, userid, query) SELECT queryid, dbid, userid, query FROM capture c WHERE NOT EXISTS (SELECT 1 FROM powa_statements ps WHERE ps.queryid = c.queryid AND ps.dbid = c.dbid AND ps.userid = c.userid ) ), by_query AS ( INSERT INTO powa_statements_history_current SELECT queryid, dbid, userid, ROW( now(), calls, total_time, rows, shared_blks_hit, shared_blks_read, shared_blks_dirtied, shared_blks_written, local_blks_hit, local_blks_read, local_blks_dirtied, local_blks_written, temp_blks_read, temp_blks_written, blk_read_time, blk_write_time )::powa_statements_history_record AS record FROM capture ), by_database AS ( INSERT INTO powa_statements_history_current_db SELECT dbid, ROW( now(), sum(calls), sum(total_time), sum(rows), sum(shared_blks_hit), sum(shared_blks_read), sum(shared_blks_dirtied), sum(shared_blks_written), sum(local_blks_hit), sum(local_blks_read), sum(local_blks_dirtied), sum(local_blks_written), sum(temp_blks_read), sum(temp_blks_written), sum(blk_read_time), sum(blk_write_time) )::powa_statements_history_record AS record FROM capture GROUP BY dbid ) SELECT count(*) INTO v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; -- For now we don't care. What could we do on error except crash anyway? END; $PROC$ language plpgsql; /* end of powa_statements_snapshot */ CREATE OR REPLACE FUNCTION powa_user_functions_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_user_functions_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide user function statistics WITH func(dbid,funcid, r) AS ( SELECT oid, (powa_stat_user_functions(oid)).funcid, powa_stat_user_functions(oid) FROM pg_database ) INSERT INTO powa_user_functions_history_current SELECT dbid, funcid, ROW(now(), (r).calls, (r).total_time, (r).self_time)::powa_user_functions_history_record AS record FROM func; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_user_functions_snapshot */ CREATE OR REPLACE FUNCTION powa_all_relations_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_all_relations_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide relation statistics WITH rel(dbid, relid, r) AS ( SELECT oid, (powa_stat_all_rel(oid)).relid, powa_stat_all_rel(oid) FROM pg_database ) INSERT INTO powa_all_relations_history_current SELECT dbid, relid, ROW(now(),(r).numscan, (r).tup_returned, (r).tup_fetched, (r).n_tup_ins, (r).n_tup_upd, (r).n_tup_del, (r).n_tup_hot_upd, (r).n_liv_tup, (r).n_dead_tup, (r).n_mod_since_analyze, (r).blks_read, (r).blks_hit, (r).last_vacuum, (r).vacuum_count, (r).last_autovacuum, (r).autovacuum_count, (r).last_analyze, (r).analyze_count, (r).last_autoanalyze, (r).autoanalyze_count)::powa_all_relations_history_record AS record FROM rel; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_all_relations_snapshot */ CREATE OR REPLACE FUNCTION powa_statements_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_statements_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_statements_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_hitory) - rowcount: %s', v_funcname, v_rowcount)); DELETE FROM powa_statements_history_db WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history_db) - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_statements_purge */ CREATE OR REPLACE FUNCTION powa_user_functions_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_user_functions_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_user_functions_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_user_functions_purge */ CREATE OR REPLACE FUNCTION powa_all_relations_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_all_relations_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_all_relations_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_all_relations_purge */ CREATE OR REPLACE FUNCTION powa_statements_aggregate() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_statements_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate statements table LOCK TABLE powa_statements_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history SELECT queryid, dbid, userid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current GROUP BY queryid, dbid, userid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_statements_history_current; -- aggregate db table LOCK TABLE powa_statements_history_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history_db SELECT dbid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current_db GROUP BY dbid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history_db) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_statements_history_current_db; END; $PROC$ LANGUAGE plpgsql; /* end of powa_statements_aggregate */ CREATE OR REPLACE FUNCTION powa_user_functions_aggregate() RETURNS void AS $PROC$ BEGIN PERFORM powa_log('running powa_user_functions_aggregate'); -- aggregate user_functions table LOCK TABLE powa_user_functions_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_user_functions_history SELECT dbid, funcid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time), min((record).self_time))::powa_user_functions_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time), max((record).self_time))::powa_user_functions_history_record FROM powa_user_functions_history_current GROUP BY dbid, funcid; TRUNCATE powa_user_functions_history_current; END; $PROC$ LANGUAGE plpgsql; /* end of powa_user_functions_aggregate */ CREATE OR REPLACE FUNCTION powa_all_relations_aggregate() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_all_relations_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate all_relations table LOCK TABLE powa_all_relations_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_all_relations_history SELECT dbid, relid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).numscan),min((record).tup_returned),min((record).tup_fetched), min((record).n_tup_ins),min((record).n_tup_upd), min((record).n_tup_del),min((record).n_tup_hot_upd), min((record).n_liv_tup),min((record).n_dead_tup), min((record).n_mod_since_analyze),min((record).blks_read), min((record).blks_hit),min((record).last_vacuum), min((record).vacuum_count),min((record).last_autovacuum), min((record).autovacuum_count),min((record).last_analyze), min((record).analyze_count),min((record).last_autoanalyze), min((record).autoanalyze_count))::powa_all_relations_history_record, ROW(max((record).ts), max((record).numscan),max((record).tup_returned),max((record).tup_fetched), max((record).n_tup_ins),max((record).n_tup_upd), max((record).n_tup_del),max((record).n_tup_hot_upd), max((record).n_liv_tup),max((record).n_dead_tup), max((record).n_mod_since_analyze),max((record).blks_read), max((record).blks_hit),max((record).last_vacuum), max((record).vacuum_count),max((record).last_autovacuum), max((record).autovacuum_count),max((record).last_analyze), max((record).analyze_count),max((record).last_autoanalyze), max((record).autoanalyze_count))::powa_all_relations_history_record FROM powa_all_relations_history_current GROUP BY dbid, relid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_all_relations_history_current; END; $PROC$ LANGUAGE plpgsql; /* end of powa_all_relations_aggregate */ CREATE OR REPLACE FUNCTION public.powa_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- Find reset function for every supported datasource, including pgss -- Also call reset function even if they're not enabled FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='reset' LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_reset(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; RETURN true; END; $function$; /* end of powa_reset */ CREATE OR REPLACE FUNCTION public.powa_statements_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_statements_history'); TRUNCATE TABLE powa_statements_history; PERFORM powa_log('truncating powa_statements_history_current'); TRUNCATE TABLE powa_statements_history_current; PERFORM powa_log('truncating powa_statements_history_db'); TRUNCATE TABLE powa_statements_history_db; PERFORM powa_log('truncating powa_statements_history_current_db'); TRUNCATE TABLE powa_statements_history_current_db; PERFORM powa_log('truncating powa_statements'); -- if 3rd part datasource has FK on it, throw everything away TRUNCATE TABLE powa_statements CASCADE; RETURN true; END; $function$; /* end of powa_statements_reset */ CREATE OR REPLACE FUNCTION public.powa_user_functions_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_user_functions_history'); TRUNCATE TABLE powa_user_functions_history; PERFORM powa_log('truncating powa_user_functions_history_current'); TRUNCATE TABLE powa_user_functions_history_current; RETURN true; END; $function$; /* end of powa_user_functions_reset */ CREATE OR REPLACE FUNCTION public.powa_all_relations_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_all_relations_history'); TRUNCATE TABLE powa_all_relations_history; PERFORM powa_log('truncating powa_all_relations_history_current'); TRUNCATE TABLE powa_all_relations_history_current; RETURN true; END; $function$; /* end of powa_all_relations_reset */ /* * register pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_stat_kcache'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_stat_kcache'; IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_stat_kcache'); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_stat_kcache', 'snapshot', 'powa_kcache_snapshot', false, true), ('pg_stat_kcache', 'aggregate', 'powa_kcache_aggregate', false, true), ('pg_stat_kcache', 'unregister', 'powa_kcache_unregister', false, true), ('pg_stat_kcache', 'purge', 'powa_kcache_purge', false, true), ('pg_stat_kcache', 'reset', 'powa_kcache_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_kcache_register */ /* * unregister pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_stat_kcache'); DELETE FROM public.powa_functions WHERE module = 'pg_stat_kcache'; RETURN true; END; $_$ language plpgsql; /* * powa_kcache snapshot collection. */ CREATE OR REPLACE FUNCTION powa_kcache_snapshot() RETURNS void as $PROC$ DECLARE result bool; v_funcname text := 'powa_kcache_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS ( SELECT * FROM pg_stat_kcache() k JOIN pg_roles r ON r.oid = k.userid WHERE NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), by_query AS ( INSERT INTO powa_kcache_metrics_current (queryid, dbid, userid, metrics) SELECT queryid, dbid, userid, (now(), reads, writes, user_time, system_time)::kcache_type FROM capture ), by_database AS ( INSERT INTO powa_kcache_metrics_current_db (dbid, metrics) SELECT dbid, (now(), sum(reads), sum(writes), sum(user_time), sum(system_time))::kcache_type FROM capture GROUP BY dbid ) SELECT COUNT(*) into v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END $PROC$ language plpgsql; /* end of powa_kcache_unregister */ /* * powa_kcache aggregation */ CREATE OR REPLACE FUNCTION powa_kcache_aggregate() RETURNS void AS $PROC$ DECLARE result bool; v_funcname text := 'powa_kcache_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate metrics table LOCK TABLE powa_kcache_metrics_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics (coalesce_range, queryid, dbid, userid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), queryid, dbid, userid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current GROUP BY queryid, dbid, userid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_kcache_metrics_current; -- aggregate metrics_db table LOCK TABLE powa_kcache_metrics_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics_db (coalesce_range, dbid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), dbid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current_db GROUP BY dbid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics_db) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_kcache_metrics_current_db; END $PROC$ language plpgsql; /* end of powa_kcache_aggregate */ /* * powa_kcache purge */ CREATE OR REPLACE FUNCTION powa_kcache_purge() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_kcache_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); DELETE FROM powa_kcache_metrics WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics) - rowcount: %s', v_funcname, v_rowcount)); DELETE FROM powa_kcache_metrics_db WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics_db) - rowcount: %s', v_funcname, v_rowcount)); END; $PROC$ language plpgsql; /* end of powa_kcache_purge */ /* * powa_kcache reset */ CREATE OR REPLACE FUNCTION powa_kcache_reset() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_kcache_reset'; v_rowcount bigint; BEGIN PERFORM powa_log('running powa_kcache_reset'); PERFORM powa_log('truncating powa_kcache_metrics'); TRUNCATE TABLE powa_kcache_metrics; PERFORM powa_log('truncating powa_kcache_metrics_db'); TRUNCATE TABLE powa_kcache_metrics_db; PERFORM powa_log('truncating powa_kcache_metrics_current'); TRUNCATE TABLE powa_kcache_metrics_current; PERFORM powa_log('truncating powa_kcache_metrics_current_db'); TRUNCATE TABLE powa_kcache_metrics_current_db; END; $PROC$ language plpgsql; /* end of powa_kcache_reset */ /* * powa_qualstats_register */ CREATE OR REPLACE function public.powa_qualstats_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_qualstats'; IF ( v_ext_present) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE function_name IN ('powa_qualstats_snapshot', 'powa_qualstats_aggregate', 'powa_qualstats_purge'); IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_qualstats'); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_qualstats', 'snapshot', 'powa_qualstats_snapshot', false, true), ('pg_qualstats', 'aggregate', 'powa_qualstats_aggregate', false, true), ('pg_qualstats', 'unregister', 'powa_qualstats_unregister', false, true), ('pg_qualstats', 'purge', 'powa_qualstats_purge', false, true), ('pg_qualstats', 'reset', 'powa_qualstats_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_qualstats_register */ CREATE OR REPLACE FUNCTION powa_qualstats_snapshot() RETURNS void as $PROC$ DECLARE result bool; v_funcname text := 'powa_qualstats_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS ( SELECT pgqs.*, s.query FROM pg_qualstats_by_query pgqs JOIN powa_statements s USING(queryid, dbid, userid) JOIN pg_roles r ON s.userid = r.oid AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_quals AS ( INSERT INTO powa_qualstats_quals (qualid, queryid, dbid, userid, quals) SELECT DISTINCT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, array_agg(DISTINCT q::qual_type) FROM capture qs, LATERAL (SELECT (unnest(quals)).*) as q WHERE NOT EXISTS ( SELECT 1 FROM powa_qualstats_quals nh WHERE nh.qualid = qs.qualnodeid AND nh.queryid = qs.queryid AND nh.dbid = qs.dbid AND nh.userid = qs.userid ) GROUP BY qualnodeid, queryid, dbid, userid RETURNING * ), by_qual AS ( INSERT INTO powa_qualstats_quals_history_current (qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered) SELECT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), sum(occurences), sum(execution_count), sum(nbfiltered) FROM capture as qs GROUP BY qualnodeid, qs.queryid, qs.dbid, qs.userid RETURNING * ), by_qual_with_const AS ( INSERT INTO powa_qualstats_constvalues_history_current(qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered, constvalues) SELECT qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), occurences, execution_count, nbfiltered, constvalues FROM capture as qs ) SELECT COUNT(*) into v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; PERFORM pg_qualstats_reset(); END $PROC$ language plpgsql; /* end of powa_qualstats_snapshot */ /* * powa_qualstats aggregate */ CREATE OR REPLACE FUNCTION powa_qualstats_aggregate() RETURNS void AS $PROC$ DECLARE result bool; BEGIN PERFORM powa_log('running powa_qualstats_aggregate'); LOCK TABLE powa_qualstats_constvalues_history_current IN SHARE MODE; LOCK TABLE powa_qualstats_quals_history_current IN SHARE MODE; INSERT INTO powa_qualstats_constvalues_history ( qualid, queryid, dbid, userid, coalesce_range, most_used, most_filtering, least_filtering, most_executed) SELECT * FROM powa_qualstats_aggregate_constvalues_current; INSERT INTO powa_qualstats_quals_history (qualid, queryid, dbid, userid, coalesce_range, records, mins_in_range, maxs_in_range) SELECT qualid, queryid, dbid, userid, tstzrange(min(ts), max(ts),'[]'), array_agg((ts, occurences, execution_count, nbfiltered)::powa_qualstats_history_item), ROW(min(ts), min(occurences), min(execution_count), min(nbfiltered))::powa_qualstats_history_item, ROW(max(ts), max(occurences), max(execution_count), max(nbfiltered))::powa_qualstats_history_item FROM powa_qualstats_quals_history_current GROUP BY qualid, queryid, dbid, userid; TRUNCATE powa_qualstats_constvalues_history_current; TRUNCATE powa_qualstats_quals_history_current; END $PROC$ language plpgsql; /* end of powa_qualstats_aggregate */ /* * powa_qualstats_purge */ CREATE OR REPLACE FUNCTION powa_qualstats_purge() RETURNS void as $PROC$ BEGIN PERFORM powa_log('running powa_qualstats_purge'); DELETE FROM powa_qualstats_constvalues_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); DELETE FROM powa_qualstats_quals_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); END; $PROC$ language plpgsql; /* end of powa_qualstats_purge */ /* * powa_qualstats_reset */ CREATE OR REPLACE FUNCTION powa_qualstats_reset() RETURNS void as $PROC$ BEGIN PERFORM powa_log('running powa_qualstats_reset'); PERFORM powa_log('truncating powa_qualstats_quals'); TRUNCATE TABLE powa_qualstats_quals CASCADE; -- cascaded : -- powa_qualstats_quals_history -- powa_qualstats_quals_history_current -- powa_qualstats_constvalues_history -- powa_qualstats_constvalues_history_current END; $PROC$ language plpgsql; /* end of powa_qualstats_reset */ /* * powa_qualstats_unregister */ CREATE OR REPLACE function public.powa_qualstats_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_qualstats'); DELETE FROM public.powa_functions WHERE module = 'pg_qualstats'; RETURN true; END; $_$ language plpgsql; SELECT * FROM public.powa_qualstats_register(); /* end of pg_qualstats_integration - part 2 */ /* pg_track_settings integration */ CREATE OR REPLACE FUNCTION powa_track_settings_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_track_settings'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_track_settings'; IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_track_settings'); -- This extension handles its own storage, just its snapshot -- function and an unregister function. INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_track_settings', 'snapshot', 'pg_track_settings_snapshot', false, true), ('pg_track_settings', 'unregister', 'powa_track_settings_unregister', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_qualstats_unregister */ CREATE OR REPLACE function public.powa_track_settings_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_track_settings'); DELETE FROM public.powa_functions WHERE module = 'pg_track_settings'; RETURN true; END; $_$ language plpgsql; /* end of powa_track_settings_unregister */ /* pg_stat_statements operator support */ CREATE TYPE powa_statements_history_diff AS ( intvl interval, calls bigint, total_time double precision, rows bigint, shared_blks_hit bigint, shared_blks_read bigint, shared_blks_dirtied bigint, shared_blks_written bigint, local_blks_hit bigint, local_blks_read bigint, local_blks_dirtied bigint, local_blks_written bigint, temp_blks_read bigint, temp_blks_written bigint, blk_read_time double precision, blk_write_time double precision ); CREATE OR REPLACE FUNCTION powa_statements_history_mi( a powa_statements_history_record, b powa_statements_history_record) RETURNS powa_statements_history_diff AS $_$ DECLARE res powa_statements_history_diff; BEGIN res.intvl = a.ts - b.ts; res.calls = a.calls - b.calls; res.total_time = a.total_time - b.total_time; res.rows = a.rows - b.rows; res.shared_blks_hit = a.shared_blks_hit - b.shared_blks_hit; res.shared_blks_read = a.shared_blks_read - b.shared_blks_read; res.shared_blks_dirtied = a.shared_blks_dirtied - b.shared_blks_dirtied; res.shared_blks_written = a.shared_blks_written - b.shared_blks_written; res.local_blks_hit = a.local_blks_hit - b.local_blks_hit; res.local_blks_read = a.local_blks_read - b.local_blks_read; res.local_blks_dirtied = a.local_blks_dirtied - b.local_blks_dirtied; res.local_blks_written = a.local_blks_written - b.local_blks_written; res.temp_blks_read = a.temp_blks_read - b.temp_blks_read; res.temp_blks_written = a.temp_blks_written - b.temp_blks_written; res.blk_read_time = a.blk_read_time - b.blk_read_time; res.blk_write_time = a.blk_write_time - b.blk_write_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_statements_history_mi, LEFTARG = powa_statements_history_record, RIGHTARG = powa_statements_history_record ); CREATE TYPE powa_statements_history_rate AS ( sec integer, calls_per_sec double precision, runtime_per_sec double precision, rows_per_sec double precision, shared_blks_hit_per_sec double precision, shared_blks_read_per_sec double precision, shared_blks_dirtied_per_sec double precision, shared_blks_written_per_sec double precision, local_blks_hit_per_sec double precision, local_blks_read_per_sec double precision, local_blks_dirtied_per_sec double precision, local_blks_written_per_sec double precision, temp_blks_read_per_sec double precision, temp_blks_written_per_sec double precision, blk_read_time_per_sec double precision, blk_write_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_statements_history_div( a powa_statements_history_record, b powa_statements_history_record) RETURNS powa_statements_history_rate AS $_$ DECLARE res powa_statements_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.calls_per_sec = (a.calls - b.calls)::double precision / sec; res.runtime_per_sec = (a.total_time - b.total_time)::double precision / sec; res.rows_per_sec = (a.rows - b.rows)::double precision / sec; res.shared_blks_hit_per_sec = (a.shared_blks_hit - b.shared_blks_hit)::double precision / sec; res.shared_blks_read_per_sec = (a.shared_blks_read - b.shared_blks_read)::double precision / sec; res.shared_blks_dirtied_per_sec = (a.shared_blks_dirtied - b.shared_blks_dirtied)::double precision / sec; res.shared_blks_written_per_sec = (a.shared_blks_written - b.shared_blks_written)::double precision / sec; res.local_blks_hit_per_sec = (a.local_blks_hit - b.local_blks_hit)::double precision / sec; res.local_blks_read_per_sec = (a.local_blks_read - b.local_blks_read)::double precision / sec; res.local_blks_dirtied_per_sec = (a.local_blks_dirtied - b.local_blks_dirtied)::double precision / sec; res.local_blks_written_per_sec = (a.local_blks_written - b.local_blks_written)::double precision / sec; res.temp_blks_read_per_sec = (a.temp_blks_read - b.temp_blks_read)::double precision / sec; res.temp_blks_written_per_sec = (a.temp_blks_written - b.temp_blks_written)::double precision / sec; res.blk_read_time_per_sec = (a.blk_read_time - b.blk_read_time)::double precision / sec; res.blk_write_time_per_sec = (a.blk_write_time - b.blk_write_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_statements_history_div, LEFTARG = powa_statements_history_record, RIGHTARG = powa_statements_history_record ); /* end of pg_stat_statements operator support */ /* pg_stat_all_relations operator support */ CREATE TYPE powa_all_relations_history_diff AS ( intvl interval, numscan bigint, tup_returned bigint, tup_fetched bigint, n_tup_ins bigint, n_tup_upd bigint, n_tup_del bigint, n_tup_hot_upd bigint, n_liv_tup bigint, n_dead_tup bigint, n_mod_since_analyze bigint, blks_read bigint, blks_hit bigint, vacuum_count bigint, autovacuum_count bigint, analyze_count bigint, autoanalyze_count bigint ); CREATE OR REPLACE FUNCTION powa_all_relations_history_mi( a powa_all_relations_history_record, b powa_all_relations_history_record) RETURNS powa_all_relations_history_diff AS $_$ DECLARE res powa_all_relations_history_diff; BEGIN res.intvl = a.ts - b.ts; res.numscan = a.numscan - b.numscan; res.tup_returned = a.tup_returned - b.tup_returned; res.tup_fetched = a.tup_fetched - b.tup_fetched; res.n_tup_ins = a.n_tup_ins - b.n_tup_ins; res.n_tup_upd = a.n_tup_upd - b.n_tup_upd; res.n_tup_del = a.n_tup_del - b.n_tup_del; res.n_tup_hot_upd = a.n_tup_hot_upd - b.n_tup_hot_upd; res.n_liv_tup = a.n_liv_tup - b.n_liv_tup; res.n_dead_tup = a.n_dead_tup - b.n_dead_tup; res.n_mod_since_analyze = a.n_mod_since_analyze - b.n_mod_since_analyze; res.blks_read = a.blks_read - b.blks_read; res.blks_hit = a.blks_hit - b.blks_hit; res.vacuum_count = a.vacuum_count - b.vacuum_count; res.autovacuum_count = a.autovacuum_count - b.autovacuum_count; res.analyze_count = a.analyze_count - b.analyze_count; res.autoanalyze_count = a.autoanalyze_count - b.autoanalyze_count; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_all_relations_history_mi, LEFTARG = powa_all_relations_history_record, RIGHTARG = powa_all_relations_history_record ); CREATE TYPE powa_all_relations_history_rate AS ( sec integer, numscan_per_sec double precision, tup_returned_per_sec double precision, tup_fetched_per_sec double precision, n_tup_ins_per_sec double precision, n_tup_upd_per_sec double precision, n_tup_del_per_sec double precision, n_tup_hot_upd_per_sec double precision, n_liv_tup_per_sec double precision, n_dead_tup_per_sec double precision, n_mod_since_analyze_per_sec double precision, blks_read_per_sec double precision, blks_hit_per_sec double precision, vacuum_count_per_sec double precision, autovacuum_count_per_sec double precision, analyze_count_per_sec double precision, autoanalyze_count_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_all_relations_history_div( a powa_all_relations_history_record, b powa_all_relations_history_record) RETURNS powa_all_relations_history_rate AS $_$ DECLARE res powa_all_relations_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.numscan_per_sec = (a.numscan - b.numscan)::double precision / sec; res.tup_returned_per_sec = (a.tup_returned - b.tup_returned)::double precision / sec; res.tup_fetched_per_sec = (a.tup_fetched - b.tup_fetched)::double precision / sec; res.n_tup_ins_per_sec = (a.n_tup_ins - b.n_tup_ins)::double precision / sec; res.n_tup_upd_per_sec = (a.n_tup_upd - b.n_tup_upd)::double precision / sec; res.n_tup_del_per_sec = (a.n_tup_del - b.n_tup_del)::double precision / sec; res.n_tup_hot_upd_per_sec = (a.n_tup_hot_upd - b.n_tup_hot_upd)::double precision / sec; res.n_liv_tup_per_sec = (a.n_liv_tup - b.n_liv_tup)::double precision / sec; res.n_dead_tup_per_sec = (a.n_dead_tup - b.n_dead_tup)::double precision / sec; res.n_mod_since_analyze_per_sec = (a.n_mod_since_analyze - b.n_mod_since_analyze)::double precision / sec; res.blks_read_per_sec = (a.blks_read - b.blks_read)::double precision / sec; res.blks_hit_per_sec = (a.blks_hit - b.blks_hit)::double precision / sec; res.vacuum_count_per_sec = (a.vacuum_count - b.vacuum_count)::double precision / sec; res.autovacuum_count_per_sec = (a.autovacuum_count - b.autovacuum_count)::double precision / sec; res.analyze_count_per_sec = (a.analyze_count - b.analyze_count)::double precision / sec; res.autoanalyze_count_per_sec = (a.autoanalyze_count - b.autoanalyze_count)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_all_relations_history_div, LEFTARG = powa_all_relations_history_record, RIGHTARG = powa_all_relations_history_record ); /* end of pg_stat_all_relations operator support */ /* pg_stat_all_relations operator support */ CREATE TYPE powa_user_functions_history_diff AS ( intvl interval, calls bigint, total_time double precision, self_time double precision ); CREATE OR REPLACE FUNCTION powa_user_functions_history_mi( a powa_user_functions_history_record, b powa_user_functions_history_record) RETURNS powa_user_functions_history_diff AS $_$ DECLARE res powa_user_functions_history_diff; BEGIN res.intvl = a.ts - b.ts; res.calls = a.calls - b.calls; res.total_time = a.total_time - b.total_time; res.self_time = a.self_time - b.self_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_user_functions_history_mi, LEFTARG = powa_user_functions_history_record, RIGHTARG = powa_user_functions_history_record ); CREATE TYPE powa_user_functions_history_rate AS ( sec integer, calls_per_sec double precision, total_time_per_sec double precision, self_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_user_functions_history_div( a powa_user_functions_history_record, b powa_user_functions_history_record) RETURNS powa_user_functions_history_rate AS $_$ DECLARE res powa_user_functions_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.calls_per_sec = (a.calls - b.calls)::double precision / sec; res.total_time_per_sec = (a.total_time - b.total_time)::double precision / sec; res.self_time_per_sec = (a.self_time - b.self_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_user_functions_history_div, LEFTARG = powa_user_functions_history_record, RIGHTARG = powa_user_functions_history_record ); /* end of pg_stat_user_functions operator support */ /* pg_stat_kcache operator support */ CREATE TYPE powa_kcache_diff AS ( intvl interval, reads bigint, writes bigint, user_time double precision, system_time double precision ); CREATE OR REPLACE FUNCTION powa_kcache_mi( a kcache_type, b kcache_type) RETURNS powa_kcache_diff AS $_$ DECLARE res powa_kcache_diff; BEGIN res.intvl = a.ts - b.ts; res.reads = a.reads - b.reads; res.writes = a.writes - b.writes; res.user_time = a.user_time - b.user_time; res.system_time = a.system_time - b.system_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_kcache_mi, LEFTARG = kcache_type, RIGHTARG = kcache_type ); CREATE TYPE powa_kcache_rate AS ( sec integer, reads_per_sec double precision, writes_per_sec double precision, user_time_per_sec double precision, system_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_kcache_div( a kcache_type, b kcache_type) RETURNS powa_kcache_rate AS $_$ DECLARE res powa_kcache_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.reads_per_sec = (a.reads - b.reads)::double precision / sec; res.writes_per_sec = (a.writes - b.writes)::double precision / sec; res.user_time_per_sec = (a.user_time - b.user_time)::double precision / sec; res.system_time_per_sec = (a.system_time - b.system_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_kcache_div, LEFTARG = kcache_type, RIGHTARG = kcache_type ); /* end of pg_stat_kcache operator support */ /* pg_stat_qualstats operator support */ CREATE TYPE powa_qualstats_history_diff AS ( intvl interval, occurences bigint, execution_count bigint, nbfiltered bigint ); CREATE OR REPLACE FUNCTION powa_qualstats_history_mi( a powa_qualstats_history_item, b powa_qualstats_history_item) RETURNS powa_qualstats_history_diff AS $_$ DECLARE res powa_qualstats_history_diff; BEGIN res.intvl = a.ts - b.ts; res.occurences = a.occurences - b.occurences; res.execution_count = a.execution_count - b.execution_count; res.nbfiltered = a.nbfiltered - b.nbfiltered; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_qualstats_history_mi, LEFTARG = powa_qualstats_history_item, RIGHTARG = powa_qualstats_history_item ); CREATE TYPE powa_qualstats_history_rate AS ( sec integer, occurences_per_sec double precision, execution_count_per_sec double precision, nbfiltered_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_qualstats_history_div( a powa_qualstats_history_item, b powa_qualstats_history_item) RETURNS powa_qualstats_history_rate AS $_$ DECLARE res powa_qualstats_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.occurences_per_sec = (a.occurences - b.occurences)::double precision / sec; res.execution_count_per_sec = (a.execution_count - b.execution_count)::double precision / sec; res.nbfiltered_per_sec = (a.nbfiltered - b.nbfiltered)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_qualstats_history_div, LEFTARG = powa_qualstats_history_item, RIGHTARG = powa_qualstats_history_item ); /* end of pg_stat_qualstats operator support */ powa-archivist-REL_3_2_0/powa--3.0.1.sql000066400000000000000000001361461336072707400175450ustar00rootroot00000000000000-- complain if script is sourced in psql, rather than via CREATE EXTENSION --\echo Use "CREATE EXTENSION powa" to load this file. \quit SET statement_timeout = 0; SET client_encoding = 'UTF8'; SET standard_conforming_strings = on; SET client_min_messages = warning; SET escape_string_warning = off; SET search_path = public, pg_catalog; CREATE TABLE powa_databases( oid oid PRIMARY KEY, datname name, dropped timestamp with time zone ); CREATE FUNCTION powa_stat_user_functions(IN dbid oid, OUT funcid oid, OUT calls bigint, OUT total_time double precision, OUT self_time double precision) RETURNS SETOF record LANGUAGE c COST 100 AS '$libdir/powa', 'powa_stat_user_functions'; CREATE FUNCTION powa_stat_all_rel(IN dbid oid, OUT relid oid, OUT numscan bigint, OUT tup_returned bigint, OUT tup_fetched bigint, OUT n_tup_ins bigint, OUT n_tup_upd bigint, OUT n_tup_del bigint, OUT n_tup_hot_upd bigint, OUT n_liv_tup bigint, OUT n_dead_tup bigint, OUT n_mod_since_analyze bigint, OUT blks_read bigint, OUT blks_hit bigint, OUT last_vacuum timestamp with time zone, OUT vacuum_count bigint, OUT last_autovacuum timestamp with time zone, OUT autovacuum_count bigint, OUT last_analyze timestamp with time zone, OUT analyze_count bigint, OUT last_autoanalyze timestamp with time zone, OUT autoanalyze_count bigint) RETURNS SETOF record LANGUAGE c COST 100 AS '$libdir/powa', 'powa_stat_all_rel'; CREATE TYPE powa_statements_history_record AS ( ts timestamp with time zone, calls bigint, total_time double precision, rows bigint, shared_blks_hit bigint, shared_blks_read bigint, shared_blks_dirtied bigint, shared_blks_written bigint, local_blks_hit bigint, local_blks_read bigint, local_blks_dirtied bigint, local_blks_written bigint, temp_blks_read bigint, temp_blks_written bigint, blk_read_time double precision, blk_write_time double precision ); CREATE TYPE powa_user_functions_history_record AS ( ts timestamp with time zone, calls bigint, total_time double precision, self_time double precision ); CREATE TYPE powa_all_relations_history_record AS ( ts timestamp with time zone, numscan bigint, tup_returned bigint, tup_fetched bigint, n_tup_ins bigint, n_tup_upd bigint, n_tup_del bigint, n_tup_hot_upd bigint, n_liv_tup bigint, n_dead_tup bigint, n_mod_since_analyze bigint, blks_read bigint, blks_hit bigint, last_vacuum timestamp with time zone, vacuum_count bigint, last_autovacuum timestamp with time zone, autovacuum_count bigint, last_analyze timestamp with time zone, analyze_count bigint, last_autoanalyze timestamp with time zone, autoanalyze_count bigint ); CREATE TABLE powa_last_aggregation ( aggts timestamp with time zone ); INSERT INTO powa_last_aggregation(aggts) VALUES (current_timestamp); CREATE TABLE powa_last_purge ( purgets timestamp with time zone ); INSERT INTO powa_last_purge (purgets) VALUES (current_timestamp); CREATE TABLE powa_statements ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, query text NOT NULL ); ALTER TABLE ONLY powa_statements ADD CONSTRAINT powa_statements_pkey PRIMARY KEY (queryid, dbid, userid); CREATE INDEX powa_statements_dbid_idx ON powa_statements(dbid); CREATE INDEX powa_statements_userid_idx ON powa_statements(userid); CREATE TABLE powa_statements_history ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statements_history_record[] NOT NULL, mins_in_range powa_statements_history_record NOT NULL, maxs_in_range powa_statements_history_record NOT NULL ); CREATE INDEX powa_statements_history_query_ts ON powa_statements_history USING gist (queryid, coalesce_range); CREATE TABLE powa_statements_history_db ( dbid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statements_history_record[] NOT NULL, mins_in_range powa_statements_history_record NOT NULL, maxs_in_range powa_statements_history_record NOT NULL ); CREATE INDEX powa_statements_history_db_ts ON powa_statements_history_db USING gist (dbid, coalesce_range); CREATE TABLE powa_statements_history_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, record powa_statements_history_record NOT NULL ); CREATE TABLE powa_statements_history_current_db ( dbid oid NOT NULL, record powa_statements_history_record NOT NULL ); CREATE TABLE powa_user_functions_history ( dbid oid NOT NULL, funcid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_user_functions_history_record[] NOT NULL, mins_in_range powa_user_functions_history_record NOT NULL, maxs_in_range powa_user_functions_history_record NOT NULL ); CREATE INDEX powa_user_functions_history_funcid_ts ON powa_user_functions_history USING gist (funcid, coalesce_range); CREATE TABLE powa_user_functions_history_current ( dbid oid NOT NULL, funcid oid NOT NULL, record powa_user_functions_history_record NOT NULL ); CREATE TABLE powa_all_relations_history ( dbid oid NOT NULL, relid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_all_relations_history_record[] NOT NULL, mins_in_range powa_all_relations_history_record NOT NULL, maxs_in_range powa_all_relations_history_record NOT NULL ); CREATE INDEX powa_all_relations_history_relid_ts ON powa_all_relations_history USING gist (relid, coalesce_range); CREATE TABLE powa_all_relations_history_current ( dbid oid NOT NULL, relid oid NOT NULL, record powa_all_relations_history_record NOT NULL ); CREATE SEQUENCE powa_coalesce_sequence INCREMENT BY 1 START WITH 1 CYCLE; CREATE TABLE powa_functions ( module text NOT NULL, operation text NOT NULL, function_name text NOT NULL, added_manually boolean NOT NULL default true, enabled boolean NOT NULL default true, CHECK (operation IN ('snapshot','aggregate','purge','unregister','reset')) ); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_stat_statements', 'snapshot', 'powa_statements_snapshot', false, true), ('powa_stat_user_functions', 'snapshot', 'powa_user_functions_snapshot', false, true), ('powa_stat_all_relations', 'snapshot', 'powa_all_relations_snapshot', false, true), ('pg_stat_statements', 'aggregate','powa_statements_aggregate', false, true), ('powa_stat_user_functions', 'aggregate','powa_user_functions_aggregate', false, true), ('powa_stat_all_relations', 'aggregate','powa_all_relations_aggregate', false, true), ('pg_stat_statements', 'purge', 'powa_statements_purge', false, true), ('powa_stat_user_functions', 'purge', 'powa_user_functions_purge', false, true), ('powa_stat_all_relations', 'purge', 'powa_all_relations_purge', false, true), ('pg_stat_statements', 'reset', 'powa_statements_reset', false, true), ('powa_stat_user_functions', 'reset', 'powa_user_functions_reset', false, true), ('powa_stat_all_relations', 'reset', 'powa_all_relations_reset', false, true); /* pg_stat_kcache integration - part 1 */ CREATE TYPE public.kcache_type AS ( ts timestamptz, reads bigint, writes bigint, user_time double precision, system_time double precision ); CREATE TABLE public.powa_kcache_metrics ( coalesce_range tstzrange NOT NULL, queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics public.kcache_type[] NOT NULL, mins_in_range public.kcache_type NOT NULL, maxs_in_range public.kcache_type NOT NULL, PRIMARY KEY (coalesce_range, queryid, dbid, userid) ); CREATE INDEX ON public.powa_kcache_metrics (queryid); CREATE TABLE public.powa_kcache_metrics_db ( coalesce_range tstzrange NOT NULL, dbid oid NOT NULL, metrics public.kcache_type[] NOT NULL, mins_in_range public.kcache_type NOT NULL, maxs_in_range public.kcache_type NOT NULL, PRIMARY KEY (coalesce_range, dbid) ); CREATE TABLE public.powa_kcache_metrics_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics kcache_type NULL NULL ); CREATE TABLE public.powa_kcache_metrics_current_db ( dbid oid NOT NULL, metrics kcache_type NULL NULL ); /* end of pg_stat_kcache integration - part 1 */ /* pg_qualstats integration - part 1 */ CREATE TYPE public.qual_type AS ( relid oid, attnum integer, opno oid, eval_type "char" ); CREATE TYPE public.qual_values AS ( constants text[], occurences bigint, execution_count bigint, nbfiltered bigint ); CREATE TYPE powa_qualstats_history_item AS ( ts timestamptz, occurences bigint, execution_count bigint, nbfiltered bigint ); CREATE TABLE public.powa_qualstats_quals ( qualid bigint, queryid bigint, dbid oid, userid oid, quals public.qual_type[], PRIMARY KEY (qualid, queryid, dbid, userid), FOREIGN KEY (queryid, dbid, userid) REFERENCES powa_statements(queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, records powa_qualstats_history_item[], mins_in_range powa_qualstats_history_item, maxs_in_range powa_qualstats_history_item, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, occurences bigint, execution_count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES powa_qualstats_quals(qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, most_used qual_values[], most_filtering qual_values[], least_filtering qual_values[], most_executed qual_values[], FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, constvalues text[], occurences bigint, execution_count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE INDEX ON powa_qualstats_constvalues_history USING gist (queryid, qualid, coalesce_range); CREATE INDEX ON powa_qualstats_constvalues_history (qualid, queryid); CREATE INDEX ON powa_qualstats_quals(queryid); /* end of pg_qualstats_integration - part 1 */ -- Mark all of powa's tables as "to be dumped" SELECT pg_catalog.pg_extension_config_dump('powa_statements',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_user_functions_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_user_functions_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_all_relations_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_all_relations_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_functions','WHERE added_manually'); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history_current',''); CREATE OR REPLACE FUNCTION public.powa_check_created_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE BEGIN /* We have for now no way for a proper handling of this event, * as we don't have a table with the list of supported extensions. * So just call every powa_*_register() function we know each time an * extension is created. Powa should be in a dedicated database and the * register function handle to be called several time, so it's not critical */ PERFORM public.powa_kcache_register(); PERFORM public.powa_qualstats_register(); PERFORM public.powa_track_settings_register(); END; $_$; CREATE EVENT TRIGGER powa_check_created_extensions ON ddl_command_end WHEN tag IN ('CREATE EXTENSION') EXECUTE PROCEDURE public.powa_check_created_extensions() ; CREATE OR REPLACE FUNCTION public.powa_check_dropped_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- We unregister extensions regardless the "enabled" field WITH ext AS ( SELECT object_name FROM pg_event_trigger_dropped_objects() d WHERE d.object_type = 'extension' ) SELECT function_name INTO funcname FROM powa_functions f JOIN ext ON f.module = ext.object_name WHERE operation = 'unregister'; IF ( funcname IS NOT NULL ) THEN BEGIN RAISE DEBUG 'running %', funcname; EXECUTE 'SELECT ' || quote_ident(funcname) || '()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE WARNING 'powa_check_dropped_extensions(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END IF; END; $_$; CREATE EVENT TRIGGER powa_check_dropped_extensions ON sql_drop WHEN tag IN ('DROP EXTENSION') EXECUTE PROCEDURE public.powa_check_dropped_extensions() ; CREATE OR REPLACE FUNCTION powa_take_snapshot() RETURNS void AS $PROC$ DECLARE purgets timestamp with time zone; purge_seq bigint; funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- Keep track of existing databases WITH missing AS ( SELECT d.oid, d.datname FROM pg_database d LEFT JOIN powa_databases p ON d.oid = p.oid WHERE p.oid IS NULL ) INSERT INTO powa_databases SELECT * FROM missing; -- Keep track of renamed databases WITH renamed AS ( SELECT d.oid, d.datname FROM pg_database AS d JOIN powa_databases AS p ON d.oid = p.oid WHERE d.datname != p.datname ) UPDATE powa_databases AS p SET datname = r.datname FROM renamed AS r WHERE p.oid = r.oid; -- Keep track of when databases are dropped WITH dropped AS ( SELECT p.oid FROM powa_databases p LEFT JOIN pg_database d ON p.oid = d.oid WHERE d.oid IS NULL AND p.dropped IS NULL) UPDATE powa_databases p SET dropped = now() FROM dropped d WHERE p.oid = d.oid; -- For all enabled snapshot functions in the powa_functions table, execute FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='snapshot' AND enabled LOOP -- Call all of them, with no parameter RAISE debug 'fonction: %',funcname; BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; -- Coalesce datas if needed SELECT nextval('powa_coalesce_sequence'::regclass) INTO purge_seq; IF ( purge_seq % current_setting('powa.coalesce')::bigint ) = 0 THEN FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='aggregate' AND enabled LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; UPDATE powa_last_aggregation SET aggts = now(); END IF; -- Once every 10 packs, we also purge IF ( purge_seq % (current_setting('powa.coalesce')::bigint *10) ) = 0 THEN FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='purge' AND enabled LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; UPDATE powa_last_purge SET purgets = now(); END IF; END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_statements_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; ignore_regexp text:='^[[:space:]]*(DEALLOCATE|BEGIN|PREPARE TRANSACTION|COMMIT PREPARED|ROLLBACK PREPARED)'; BEGIN -- In this function, we capture statements, and also aggregate counters by database -- so that the first screens of powa stay reactive even though there may be thousands -- of different statements RAISE DEBUG 'running powa_statements_snapshot'; WITH capture AS( SELECT pgss.* FROM pg_stat_statements pgss JOIN pg_roles r ON pgss.userid = r.oid WHERE pgss.query !~* ignore_regexp AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_statements AS( INSERT INTO powa_statements (queryid, dbid, userid, query) SELECT queryid, dbid, userid, query FROM capture c WHERE NOT EXISTS (SELECT 1 FROM powa_statements ps WHERE ps.queryid = c.queryid AND ps.dbid = c.dbid AND ps.userid = c.userid ) ), by_query AS ( INSERT INTO powa_statements_history_current SELECT queryid, dbid, userid, ROW( now(), calls, total_time, rows, shared_blks_hit, shared_blks_read, shared_blks_dirtied, shared_blks_written, local_blks_hit, local_blks_read, local_blks_dirtied, local_blks_written, temp_blks_read, temp_blks_written, blk_read_time, blk_write_time )::powa_statements_history_record AS record FROM capture ), by_database AS ( INSERT INTO powa_statements_history_current_db SELECT dbid, ROW( now(), sum(calls), sum(total_time), sum(rows), sum(shared_blks_hit), sum(shared_blks_read), sum(shared_blks_dirtied), sum(shared_blks_written), sum(local_blks_hit), sum(local_blks_read), sum(local_blks_dirtied), sum(local_blks_written), sum(temp_blks_read), sum(temp_blks_written), sum(blk_read_time), sum(blk_write_time) )::powa_statements_history_record AS record FROM capture GROUP BY dbid ) SELECT true::boolean INTO result; -- For now we don't care. What could we do on error except crash anyway? END; $PROC$ language plpgsql; CREATE OR REPLACE FUNCTION powa_user_functions_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; BEGIN RAISE DEBUG 'running powa_user_functions_snapshot'; -- Insert cluster-wide user function statistics WITH func(dbid,funcid, r) AS ( SELECT oid, (powa_stat_user_functions(oid)).funcid, powa_stat_user_functions(oid) FROM pg_database ) INSERT INTO powa_user_functions_history_current SELECT dbid, funcid, ROW(now(), (r).calls, (r).total_time, (r).self_time)::powa_user_functions_history_record AS record FROM func; result := true; END; $PROC$ language plpgsql; CREATE OR REPLACE FUNCTION powa_all_relations_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; BEGIN RAISE DEBUG 'running powa_all_relations_snapshot'; -- Insert cluster-wide relation statistics WITH rel(dbid, relid, r) AS ( SELECT oid, (powa_stat_all_rel(oid)).relid, powa_stat_all_rel(oid) FROM pg_database ) INSERT INTO powa_all_relations_history_current SELECT dbid, relid, ROW(now(),(r).numscan, (r).tup_returned, (r).tup_fetched, (r).n_tup_ins, (r).n_tup_upd, (r).n_tup_del, (r).n_tup_hot_upd, (r).n_liv_tup, (r).n_dead_tup, (r).n_mod_since_analyze, (r).blks_read, (r).blks_hit, (r).last_vacuum, (r).vacuum_count, (r).last_autovacuum, (r).autovacuum_count, (r).last_analyze, (r).analyze_count, (r).last_autoanalyze, (r).autoanalyze_count)::powa_all_relations_history_record AS record FROM rel; result := true; END; $PROC$ language plpgsql; CREATE OR REPLACE FUNCTION powa_statements_purge() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_statements_purge'; -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_statements_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); DELETE FROM powa_statements_history_db WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_user_functions_purge() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_user_functions_purge'; -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_user_functions_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_all_relations_purge() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_all_relations_purge'; -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_all_relations_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_statements_aggregate() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_statements_aggregate'; -- aggregate statements table LOCK TABLE powa_statements_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history SELECT queryid, dbid, userid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current GROUP BY queryid, dbid, userid; TRUNCATE powa_statements_history_current; -- aggregate db table LOCK TABLE powa_statements_history_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history_db SELECT dbid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current_db GROUP BY dbid; TRUNCATE powa_statements_history_current_db; END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_user_functions_aggregate() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_user_functions_aggregate'; -- aggregate user_functions table LOCK TABLE powa_user_functions_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_user_functions_history SELECT dbid, funcid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time), min((record).self_time))::powa_user_functions_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time), max((record).self_time))::powa_user_functions_history_record FROM powa_user_functions_history_current GROUP BY dbid, funcid; TRUNCATE powa_user_functions_history_current; END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION powa_all_relations_aggregate() RETURNS void AS $PROC$ BEGIN RAISE DEBUG 'running powa_all_relations_aggregate'; -- aggregate all_relations table LOCK TABLE powa_all_relations_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_all_relations_history SELECT dbid, relid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).numscan),min((record).tup_returned),min((record).tup_fetched), min((record).n_tup_ins),min((record).n_tup_upd), min((record).n_tup_del),min((record).n_tup_hot_upd), min((record).n_liv_tup),min((record).n_dead_tup), min((record).n_mod_since_analyze),min((record).blks_read), min((record).blks_hit),min((record).last_vacuum), min((record).vacuum_count),min((record).last_autovacuum), min((record).autovacuum_count),min((record).last_analyze), min((record).analyze_count),min((record).last_autoanalyze), min((record).autoanalyze_count))::powa_all_relations_history_record, ROW(max((record).ts), max((record).numscan),max((record).tup_returned),max((record).tup_fetched), max((record).n_tup_ins),max((record).n_tup_upd), max((record).n_tup_del),max((record).n_tup_hot_upd), max((record).n_liv_tup),max((record).n_dead_tup), max((record).n_mod_since_analyze),max((record).blks_read), max((record).blks_hit),max((record).last_vacuum), max((record).vacuum_count),max((record).last_autovacuum), max((record).autovacuum_count),max((record).last_analyze), max((record).analyze_count),max((record).last_autoanalyze), max((record).autoanalyze_count))::powa_all_relations_history_record FROM powa_all_relations_history_current GROUP BY dbid, relid; TRUNCATE powa_all_relations_history_current; END; $PROC$ LANGUAGE plpgsql; CREATE OR REPLACE FUNCTION public.powa_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- Find reset function for every supported datasource, including pgss -- Also call reset function even if they're not enabled FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='reset' LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_reset(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; RETURN true; END; $function$; CREATE OR REPLACE FUNCTION public.powa_statements_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN TRUNCATE TABLE powa_statements_history; TRUNCATE TABLE powa_statements_history_current; TRUNCATE TABLE powa_statements_history_db; TRUNCATE TABLE powa_statements_history_current_db; -- if 3rd part datasource has FK on it, throw everything away TRUNCATE TABLE powa_statements CASCADE; RETURN true; END; $function$; CREATE OR REPLACE FUNCTION public.powa_user_functions_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN TRUNCATE TABLE powa_user_functions_history; TRUNCATE TABLE powa_user_functions_history_current; RETURN true; END; $function$; CREATE OR REPLACE FUNCTION public.powa_all_relations_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN TRUNCATE TABLE powa_all_relations_history; TRUNCATE TABLE powa_all_relations_history_current; RETURN true; END; $function$; /* pg_stat_kcache integration - part 2 */ /* * register pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_stat_kcache'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_stat_kcache'; IF ( NOT v_func_present) THEN INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_stat_kcache', 'snapshot', 'powa_kcache_snapshot', false, true), ('pg_stat_kcache', 'aggregate', 'powa_kcache_aggregate', false, true), ('pg_stat_kcache', 'unregister', 'powa_kcache_unregister', false, true), ('pg_stat_kcache', 'purge', 'powa_kcache_purge', false, true), ('pg_stat_kcache', 'reset', 'powa_kcache_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* * unregister pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_unregister() RETURNS bool AS $_$ BEGIN DELETE FROM public.powa_functions WHERE module = 'pg_stat_kcache'; RETURN true; END; $_$ language plpgsql; /* * powa_kcache snapshot collection. */ CREATE OR REPLACE FUNCTION powa_kcache_snapshot() RETURNS void as $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_kcache_snapshot'; WITH capture AS ( SELECT * FROM pg_stat_kcache() k JOIN pg_roles r ON r.oid = k.userid WHERE NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), by_query AS ( INSERT INTO powa_kcache_metrics_current (queryid, dbid, userid, metrics) SELECT queryid, dbid, userid, (now(), reads, writes, user_time, system_time)::kcache_type FROM capture ), by_database AS ( INSERT INTO powa_kcache_metrics_current_db (dbid, metrics) SELECT dbid, (now(), sum(reads), sum(writes), sum(user_time), sum(system_time))::kcache_type FROM capture GROUP BY dbid ) SELECT true into result; END $PROC$ language plpgsql; /* * powa_kcache aggregation */ CREATE OR REPLACE FUNCTION powa_kcache_aggregate() RETURNS void AS $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_kcache_aggregate'; -- aggregate metrics table LOCK TABLE powa_kcache_metrics_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics (coalesce_range, queryid, dbid, userid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), queryid, dbid, userid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current GROUP BY queryid, dbid, userid; TRUNCATE powa_kcache_metrics_current; -- aggregate metrics_db table LOCK TABLE powa_kcache_metrics_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics_db (coalesce_range, dbid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), dbid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current_db GROUP BY dbid; TRUNCATE powa_kcache_metrics_current_db; END $PROC$ language plpgsql; /* * powa_kcache purge */ CREATE OR REPLACE FUNCTION powa_kcache_purge() RETURNS void as $PROC$ BEGIN RAISE DEBUG 'running powa_kcache_purge'; DELETE FROM powa_kcache_metrics WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); DELETE FROM powa_kcache_metrics_db WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); END; $PROC$ language plpgsql; /* * powa_kcache reset */ CREATE OR REPLACE FUNCTION powa_kcache_reset() RETURNS void as $PROC$ BEGIN RAISE DEBUG 'running powa_kcache_reset'; TRUNCATE TABLE powa_kcache_metrics; TRUNCATE TABLE powa_kcache_metrics_db; TRUNCATE TABLE powa_kcache_metrics_current; TRUNCATE TABLE powa_kcache_metrics_current_db; END; $PROC$ language plpgsql; -- By default, try to register pg_stat_kcache, in case it's alreay here SELECT * FROM public.powa_kcache_register(); /* end of pg_stat_kcache integration - part 2 */ /* pg_qualstats integration - part 2 */ /* * powa_qualstats_register */ CREATE OR REPLACE function public.powa_qualstats_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_qualstats'; IF ( v_ext_present) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE function_name IN ('powa_qualstats_snapshot', 'powa_qualstats_aggregate', 'powa_qualstats_purge'); IF ( NOT v_func_present) THEN INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_qualstats', 'snapshot', 'powa_qualstats_snapshot', false, true), ('pg_qualstats', 'aggregate', 'powa_qualstats_aggregate', false, true), ('pg_qualstats', 'unregister', 'powa_qualstats_unregister', false, true), ('pg_qualstats', 'purge', 'powa_qualstats_purge', false, true), ('pg_qualstats', 'reset', 'powa_qualstats_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* * powa_qualstats utility view for aggregating constvalues */ CREATE OR REPLACE VIEW powa_qualstats_aggregate_constvalues_current AS WITH consts AS ( SELECT qualid, queryid, dbid, userid, min(ts) as mints, max(ts) as maxts, sum(occurences) as occurences, sum(nbfiltered) as nbfiltered, sum(execution_count) as execution_count, constvalues FROM powa_qualstats_constvalues_history_current GROUP BY qualid, queryid, dbid, userid, constvalues ), groups AS ( SELECT qualid, queryid, dbid, userid, tstzrange(min(mints), max(maxts),'[]') FROM consts GROUP BY qualid, queryid, dbid, userid ) SELECT * FROM groups, LATERAL ( SELECT array_agg(constvalues) as mu FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY occurences desc LIMIT 20 ) s ) as mu, LATERAL ( SELECT array_agg(constvalues) as mf FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN execution_count = 0 THEN 0 ELSE nbfiltered / execution_count::numeric END DESC LIMIT 20 ) s ) as mf, LATERAL ( SELECT array_agg(constvalues) as lf FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN execution_count = 0 THEN 0 ELSE nbfiltered / execution_count::numeric END DESC LIMIT 20 ) s ) as lf, LATERAL ( SELECT array_agg(constvalues) as me FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY execution_count desc LIMIT 20 ) s ) as me; CREATE OR REPLACE FUNCTION powa_qualstats_snapshot() RETURNS void as $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_qualstats_snaphot'; WITH capture AS ( SELECT pgqs.*, s.query FROM pg_qualstats_by_query pgqs JOIN powa_statements s USING(queryid, dbid, userid) JOIN pg_roles r ON s.userid = r.oid AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_quals AS ( INSERT INTO powa_qualstats_quals (qualid, queryid, dbid, userid, quals) SELECT DISTINCT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, array_agg(DISTINCT q::qual_type) FROM capture qs, LATERAL (SELECT (unnest(quals)).*) as q WHERE NOT EXISTS ( SELECT 1 FROM powa_qualstats_quals nh WHERE nh.qualid = qs.qualnodeid AND nh.queryid = qs.queryid AND nh.dbid = qs.dbid AND nh.userid = qs.userid ) GROUP BY qualnodeid, queryid, dbid, userid RETURNING * ), by_qual AS ( INSERT INTO powa_qualstats_quals_history_current (qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered) SELECT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), sum(occurences), sum(execution_count), sum(nbfiltered) FROM capture as qs GROUP BY qualnodeid, qs.queryid, qs.dbid, qs.userid RETURNING * ), by_qual_with_const AS ( INSERT INTO powa_qualstats_constvalues_history_current(qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered, constvalues) SELECT qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), occurences, execution_count, nbfiltered, constvalues FROM capture as qs ) SELECT true into result; PERFORM pg_qualstats_reset(); END $PROC$ language plpgsql; /* * powa_qualstats aggregate */ CREATE OR REPLACE FUNCTION powa_qualstats_aggregate() RETURNS void AS $PROC$ DECLARE result bool; BEGIN RAISE DEBUG 'running powa_qualstats_aggregate'; LOCK TABLE powa_qualstats_constvalues_history_current IN SHARE MODE; LOCK TABLE powa_qualstats_quals_history_current IN SHARE MODE; INSERT INTO powa_qualstats_constvalues_history ( qualid, queryid, dbid, userid, coalesce_range, most_used, most_filtering, least_filtering, most_executed) SELECT * FROM powa_qualstats_aggregate_constvalues_current; INSERT INTO powa_qualstats_quals_history (qualid, queryid, dbid, userid, coalesce_range, records, mins_in_range, maxs_in_range) SELECT qualid, queryid, dbid, userid, tstzrange(min(ts), max(ts),'[]'), array_agg((ts, occurences, execution_count, nbfiltered)::powa_qualstats_history_item), ROW(min(ts), min(occurences), min(execution_count), min(nbfiltered))::powa_qualstats_history_item, ROW(max(ts), max(occurences), max(execution_count), max(nbfiltered))::powa_qualstats_history_item FROM powa_qualstats_quals_history_current GROUP BY qualid, queryid, dbid, userid; TRUNCATE powa_qualstats_constvalues_history_current; TRUNCATE powa_qualstats_quals_history_current; END $PROC$ language plpgsql; /* * powa_qualstats_purge */ CREATE OR REPLACE FUNCTION powa_qualstats_purge() RETURNS void as $PROC$ BEGIN RAISE DEBUG 'running powa_qualstats_purge'; DELETE FROM powa_qualstats_constvalues_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); DELETE FROM powa_qualstats_quals_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); END; $PROC$ language plpgsql; /* * powa_qualstats_reset */ CREATE OR REPLACE FUNCTION powa_qualstats_reset() RETURNS void as $PROC$ BEGIN RAISE DEBUG 'running powa_qualstats_reset'; TRUNCATE TABLE powa_qualstats_quals CASCADE; -- cascaded : -- powa_qualstats_quals_history -- powa_qualstats_quals_history_current -- powa_qualstats_constvalues_history -- powa_qualstats_constvalues_history_current END; $PROC$ language plpgsql; /* * powa_qualstats_unregister */ CREATE OR REPLACE function public.powa_qualstats_unregister() RETURNS bool AS $_$ BEGIN DELETE FROM public.powa_functions WHERE module = 'pg_qualstats'; RETURN true; END; $_$ language plpgsql; SELECT * FROM public.powa_qualstats_register(); /* end of pg_qualstats_integration - part 2 */ /* pg_track_settings integration */ CREATE OR REPLACE FUNCTION powa_track_settings_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_track_settings'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_track_settings'; IF ( NOT v_func_present) THEN -- This extension handles its own storage, just its snapshot -- function and an unregister function. INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_track_settings', 'snapshot', 'pg_track_settings_snapshot', false, true), ('pg_track_settings', 'unregister', 'powa_track_settings_unregister', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; CREATE OR REPLACE function public.powa_track_settings_unregister() RETURNS bool AS $_$ BEGIN DELETE FROM public.powa_functions WHERE module = 'pg_track_settings'; RETURN true; END; $_$ language plpgsql; -- By default, try to register pg_track_settings, in case it's alreay here SELECT * FROM public.powa_track_settings_register(); /* end pg_track_settings integration */ powa-archivist-REL_3_2_0/powa--3.1.0--3.1.1.sql000066400000000000000000000043171336072707400202520ustar00rootroot00000000000000-- complain if script is sourced in psql, rather than via CREATE EXTENSION --\echo Use "ALTER EXTENSION powa" to load this file. \quit CREATE OR REPLACE FUNCTION powa_all_relations_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_all_relations_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide relation statistics WITH rel(dbid, r) AS ( SELECT oid, powa_stat_all_rel(oid) FROM pg_database ) INSERT INTO powa_all_relations_history_current SELECT dbid, (r).relid, ROW(now(),(r).numscan, (r).tup_returned, (r).tup_fetched, (r).n_tup_ins, (r).n_tup_upd, (r).n_tup_del, (r).n_tup_hot_upd, (r).n_liv_tup, (r).n_dead_tup, (r).n_mod_since_analyze, (r).blks_read, (r).blks_hit, (r).last_vacuum, (r).vacuum_count, (r).last_autovacuum, (r).autovacuum_count, (r).last_analyze, (r).analyze_count, (r).last_autoanalyze, (r).autoanalyze_count)::powa_all_relations_history_record AS record FROM rel; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_all_relations_snapshot */ CREATE OR REPLACE FUNCTION powa_user_functions_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_user_functions_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide user function statistics WITH func(dbid, r) AS ( SELECT oid, powa_stat_user_functions(oid) FROM pg_database ) INSERT INTO powa_user_functions_history_current SELECT dbid, (r).funcid, ROW(now(), (r).calls, (r).total_time, (r).self_time)::powa_user_functions_history_record AS record FROM func; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_user_functions_snapshot */ powa-archivist-REL_3_2_0/powa--3.1.0.sql000066400000000000000000002126371336072707400175450ustar00rootroot00000000000000-- complain if script is sourced in psql, rather than via CREATE EXTENSION --\echo Use "CREATE EXTENSION powa" to load this file. \quit SET statement_timeout = 0; SET client_encoding = 'UTF8'; SET standard_conforming_strings = on; SET client_min_messages = warning; SET escape_string_warning = off; SET search_path = public, pg_catalog; CREATE TABLE powa_databases( oid oid PRIMARY KEY, datname name, dropped timestamp with time zone ); CREATE FUNCTION powa_stat_user_functions(IN dbid oid, OUT funcid oid, OUT calls bigint, OUT total_time double precision, OUT self_time double precision) RETURNS SETOF record LANGUAGE c COST 100 AS '$libdir/powa', 'powa_stat_user_functions'; CREATE FUNCTION powa_stat_all_rel(IN dbid oid, OUT relid oid, OUT numscan bigint, OUT tup_returned bigint, OUT tup_fetched bigint, OUT n_tup_ins bigint, OUT n_tup_upd bigint, OUT n_tup_del bigint, OUT n_tup_hot_upd bigint, OUT n_liv_tup bigint, OUT n_dead_tup bigint, OUT n_mod_since_analyze bigint, OUT blks_read bigint, OUT blks_hit bigint, OUT last_vacuum timestamp with time zone, OUT vacuum_count bigint, OUT last_autovacuum timestamp with time zone, OUT autovacuum_count bigint, OUT last_analyze timestamp with time zone, OUT analyze_count bigint, OUT last_autoanalyze timestamp with time zone, OUT autoanalyze_count bigint) RETURNS SETOF record LANGUAGE c COST 100 AS '$libdir/powa', 'powa_stat_all_rel'; CREATE TYPE powa_statements_history_record AS ( ts timestamp with time zone, calls bigint, total_time double precision, rows bigint, shared_blks_hit bigint, shared_blks_read bigint, shared_blks_dirtied bigint, shared_blks_written bigint, local_blks_hit bigint, local_blks_read bigint, local_blks_dirtied bigint, local_blks_written bigint, temp_blks_read bigint, temp_blks_written bigint, blk_read_time double precision, blk_write_time double precision ); /* pg_stat_statements operator support */ CREATE TYPE powa_statements_history_diff AS ( intvl interval, calls bigint, total_time double precision, rows bigint, shared_blks_hit bigint, shared_blks_read bigint, shared_blks_dirtied bigint, shared_blks_written bigint, local_blks_hit bigint, local_blks_read bigint, local_blks_dirtied bigint, local_blks_written bigint, temp_blks_read bigint, temp_blks_written bigint, blk_read_time double precision, blk_write_time double precision ); CREATE OR REPLACE FUNCTION powa_statements_history_mi( a powa_statements_history_record, b powa_statements_history_record) RETURNS powa_statements_history_diff AS $_$ DECLARE res powa_statements_history_diff; BEGIN res.intvl = a.ts - b.ts; res.calls = a.calls - b.calls; res.total_time = a.total_time - b.total_time; res.rows = a.rows - b.rows; res.shared_blks_hit = a.shared_blks_hit - b.shared_blks_hit; res.shared_blks_read = a.shared_blks_read - b.shared_blks_read; res.shared_blks_dirtied = a.shared_blks_dirtied - b.shared_blks_dirtied; res.shared_blks_written = a.shared_blks_written - b.shared_blks_written; res.local_blks_hit = a.local_blks_hit - b.local_blks_hit; res.local_blks_read = a.local_blks_read - b.local_blks_read; res.local_blks_dirtied = a.local_blks_dirtied - b.local_blks_dirtied; res.local_blks_written = a.local_blks_written - b.local_blks_written; res.temp_blks_read = a.temp_blks_read - b.temp_blks_read; res.temp_blks_written = a.temp_blks_written - b.temp_blks_written; res.blk_read_time = a.blk_read_time - b.blk_read_time; res.blk_write_time = a.blk_write_time - b.blk_write_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_statements_history_mi, LEFTARG = powa_statements_history_record, RIGHTARG = powa_statements_history_record ); CREATE TYPE powa_statements_history_rate AS ( sec integer, calls_per_sec double precision, runtime_per_sec double precision, rows_per_sec double precision, shared_blks_hit_per_sec double precision, shared_blks_read_per_sec double precision, shared_blks_dirtied_per_sec double precision, shared_blks_written_per_sec double precision, local_blks_hit_per_sec double precision, local_blks_read_per_sec double precision, local_blks_dirtied_per_sec double precision, local_blks_written_per_sec double precision, temp_blks_read_per_sec double precision, temp_blks_written_per_sec double precision, blk_read_time_per_sec double precision, blk_write_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_statements_history_div( a powa_statements_history_record, b powa_statements_history_record) RETURNS powa_statements_history_rate AS $_$ DECLARE res powa_statements_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.calls_per_sec = (a.calls - b.calls)::double precision / sec; res.runtime_per_sec = (a.total_time - b.total_time)::double precision / sec; res.rows_per_sec = (a.rows - b.rows)::double precision / sec; res.shared_blks_hit_per_sec = (a.shared_blks_hit - b.shared_blks_hit)::double precision / sec; res.shared_blks_read_per_sec = (a.shared_blks_read - b.shared_blks_read)::double precision / sec; res.shared_blks_dirtied_per_sec = (a.shared_blks_dirtied - b.shared_blks_dirtied)::double precision / sec; res.shared_blks_written_per_sec = (a.shared_blks_written - b.shared_blks_written)::double precision / sec; res.local_blks_hit_per_sec = (a.local_blks_hit - b.local_blks_hit)::double precision / sec; res.local_blks_read_per_sec = (a.local_blks_read - b.local_blks_read)::double precision / sec; res.local_blks_dirtied_per_sec = (a.local_blks_dirtied - b.local_blks_dirtied)::double precision / sec; res.local_blks_written_per_sec = (a.local_blks_written - b.local_blks_written)::double precision / sec; res.temp_blks_read_per_sec = (a.temp_blks_read - b.temp_blks_read)::double precision / sec; res.temp_blks_written_per_sec = (a.temp_blks_written - b.temp_blks_written)::double precision / sec; res.blk_read_time_per_sec = (a.blk_read_time - b.blk_read_time)::double precision / sec; res.blk_write_time_per_sec = (a.blk_write_time - b.blk_write_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_statements_history_div, LEFTARG = powa_statements_history_record, RIGHTARG = powa_statements_history_record ); /* end of pg_stat_statements operator support */ CREATE TYPE powa_user_functions_history_record AS ( ts timestamp with time zone, calls bigint, total_time double precision, self_time double precision ); /* pg_stat_user_functions operator support */ CREATE TYPE powa_user_functions_history_diff AS ( intvl interval, calls bigint, total_time double precision, self_time double precision ); CREATE OR REPLACE FUNCTION powa_user_functions_history_mi( a powa_user_functions_history_record, b powa_user_functions_history_record) RETURNS powa_user_functions_history_diff AS $_$ DECLARE res powa_user_functions_history_diff; BEGIN res.intvl = a.ts - b.ts; res.calls = a.calls - b.calls; res.total_time = a.total_time - b.total_time; res.self_time = a.self_time - b.self_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_user_functions_history_mi, LEFTARG = powa_user_functions_history_record, RIGHTARG = powa_user_functions_history_record ); CREATE TYPE powa_user_functions_history_rate AS ( sec integer, calls_per_sec double precision, total_time_per_sec double precision, self_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_user_functions_history_div( a powa_user_functions_history_record, b powa_user_functions_history_record) RETURNS powa_user_functions_history_rate AS $_$ DECLARE res powa_user_functions_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.calls_per_sec = (a.calls - b.calls)::double precision / sec; res.total_time_per_sec = (a.total_time - b.total_time)::double precision / sec; res.self_time_per_sec = (a.self_time - b.self_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_user_functions_history_div, LEFTARG = powa_user_functions_history_record, RIGHTARG = powa_user_functions_history_record ); /* end of pg_stat_user_functions operator support */ CREATE TYPE powa_all_relations_history_record AS ( ts timestamp with time zone, numscan bigint, tup_returned bigint, tup_fetched bigint, n_tup_ins bigint, n_tup_upd bigint, n_tup_del bigint, n_tup_hot_upd bigint, n_liv_tup bigint, n_dead_tup bigint, n_mod_since_analyze bigint, blks_read bigint, blks_hit bigint, last_vacuum timestamp with time zone, vacuum_count bigint, last_autovacuum timestamp with time zone, autovacuum_count bigint, last_analyze timestamp with time zone, analyze_count bigint, last_autoanalyze timestamp with time zone, autoanalyze_count bigint ); /* pg_stat_all_relations operator support */ CREATE TYPE powa_all_relations_history_diff AS ( intvl interval, numscan bigint, tup_returned bigint, tup_fetched bigint, n_tup_ins bigint, n_tup_upd bigint, n_tup_del bigint, n_tup_hot_upd bigint, n_liv_tup bigint, n_dead_tup bigint, n_mod_since_analyze bigint, blks_read bigint, blks_hit bigint, vacuum_count bigint, autovacuum_count bigint, analyze_count bigint, autoanalyze_count bigint ); CREATE OR REPLACE FUNCTION powa_all_relations_history_mi( a powa_all_relations_history_record, b powa_all_relations_history_record) RETURNS powa_all_relations_history_diff AS $_$ DECLARE res powa_all_relations_history_diff; BEGIN res.intvl = a.ts - b.ts; res.numscan = a.numscan - b.numscan; res.tup_returned = a.tup_returned - b.tup_returned; res.tup_fetched = a.tup_fetched - b.tup_fetched; res.n_tup_ins = a.n_tup_ins - b.n_tup_ins; res.n_tup_upd = a.n_tup_upd - b.n_tup_upd; res.n_tup_del = a.n_tup_del - b.n_tup_del; res.n_tup_hot_upd = a.n_tup_hot_upd - b.n_tup_hot_upd; res.n_liv_tup = a.n_liv_tup - b.n_liv_tup; res.n_dead_tup = a.n_dead_tup - b.n_dead_tup; res.n_mod_since_analyze = a.n_mod_since_analyze - b.n_mod_since_analyze; res.blks_read = a.blks_read - b.blks_read; res.blks_hit = a.blks_hit - b.blks_hit; res.vacuum_count = a.vacuum_count - b.vacuum_count; res.autovacuum_count = a.autovacuum_count - b.autovacuum_count; res.analyze_count = a.analyze_count - b.analyze_count; res.autoanalyze_count = a.autoanalyze_count - b.autoanalyze_count; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_all_relations_history_mi, LEFTARG = powa_all_relations_history_record, RIGHTARG = powa_all_relations_history_record ); CREATE TYPE powa_all_relations_history_rate AS ( sec integer, numscan_per_sec double precision, tup_returned_per_sec double precision, tup_fetched_per_sec double precision, n_tup_ins_per_sec double precision, n_tup_upd_per_sec double precision, n_tup_del_per_sec double precision, n_tup_hot_upd_per_sec double precision, n_liv_tup_per_sec double precision, n_dead_tup_per_sec double precision, n_mod_since_analyze_per_sec double precision, blks_read_per_sec double precision, blks_hit_per_sec double precision, vacuum_count_per_sec double precision, autovacuum_count_per_sec double precision, analyze_count_per_sec double precision, autoanalyze_count_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_all_relations_history_div( a powa_all_relations_history_record, b powa_all_relations_history_record) RETURNS powa_all_relations_history_rate AS $_$ DECLARE res powa_all_relations_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.numscan_per_sec = (a.numscan - b.numscan)::double precision / sec; res.tup_returned_per_sec = (a.tup_returned - b.tup_returned)::double precision / sec; res.tup_fetched_per_sec = (a.tup_fetched - b.tup_fetched)::double precision / sec; res.n_tup_ins_per_sec = (a.n_tup_ins - b.n_tup_ins)::double precision / sec; res.n_tup_upd_per_sec = (a.n_tup_upd - b.n_tup_upd)::double precision / sec; res.n_tup_del_per_sec = (a.n_tup_del - b.n_tup_del)::double precision / sec; res.n_tup_hot_upd_per_sec = (a.n_tup_hot_upd - b.n_tup_hot_upd)::double precision / sec; res.n_liv_tup_per_sec = (a.n_liv_tup - b.n_liv_tup)::double precision / sec; res.n_dead_tup_per_sec = (a.n_dead_tup - b.n_dead_tup)::double precision / sec; res.n_mod_since_analyze_per_sec = (a.n_mod_since_analyze - b.n_mod_since_analyze)::double precision / sec; res.blks_read_per_sec = (a.blks_read - b.blks_read)::double precision / sec; res.blks_hit_per_sec = (a.blks_hit - b.blks_hit)::double precision / sec; res.vacuum_count_per_sec = (a.vacuum_count - b.vacuum_count)::double precision / sec; res.autovacuum_count_per_sec = (a.autovacuum_count - b.autovacuum_count)::double precision / sec; res.analyze_count_per_sec = (a.analyze_count - b.analyze_count)::double precision / sec; res.autoanalyze_count_per_sec = (a.autoanalyze_count - b.autoanalyze_count)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_all_relations_history_div, LEFTARG = powa_all_relations_history_record, RIGHTARG = powa_all_relations_history_record ); /* end of pg_stat_all_relations operator support */ CREATE TABLE powa_last_aggregation ( aggts timestamp with time zone ); INSERT INTO powa_last_aggregation(aggts) VALUES (current_timestamp); CREATE TABLE powa_last_purge ( purgets timestamp with time zone ); INSERT INTO powa_last_purge (purgets) VALUES (current_timestamp); CREATE TABLE powa_statements ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, query text NOT NULL ); ALTER TABLE ONLY powa_statements ADD CONSTRAINT powa_statements_pkey PRIMARY KEY (queryid, dbid, userid); CREATE INDEX powa_statements_dbid_idx ON powa_statements(dbid); CREATE INDEX powa_statements_userid_idx ON powa_statements(userid); CREATE TABLE powa_statements_history ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statements_history_record[] NOT NULL, mins_in_range powa_statements_history_record NOT NULL, maxs_in_range powa_statements_history_record NOT NULL ); CREATE INDEX powa_statements_history_query_ts ON powa_statements_history USING gist (queryid, coalesce_range); CREATE TABLE powa_statements_history_db ( dbid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statements_history_record[] NOT NULL, mins_in_range powa_statements_history_record NOT NULL, maxs_in_range powa_statements_history_record NOT NULL ); CREATE INDEX powa_statements_history_db_ts ON powa_statements_history_db USING gist (dbid, coalesce_range); CREATE TABLE powa_statements_history_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, record powa_statements_history_record NOT NULL ); CREATE TABLE powa_statements_history_current_db ( dbid oid NOT NULL, record powa_statements_history_record NOT NULL ); CREATE TABLE powa_user_functions_history ( dbid oid NOT NULL, funcid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_user_functions_history_record[] NOT NULL, mins_in_range powa_user_functions_history_record NOT NULL, maxs_in_range powa_user_functions_history_record NOT NULL ); CREATE INDEX powa_user_functions_history_funcid_ts ON powa_user_functions_history USING gist (funcid, coalesce_range); CREATE TABLE powa_user_functions_history_current ( dbid oid NOT NULL, funcid oid NOT NULL, record powa_user_functions_history_record NOT NULL ); CREATE TABLE powa_all_relations_history ( dbid oid NOT NULL, relid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_all_relations_history_record[] NOT NULL, mins_in_range powa_all_relations_history_record NOT NULL, maxs_in_range powa_all_relations_history_record NOT NULL ); CREATE INDEX powa_all_relations_history_relid_ts ON powa_all_relations_history USING gist (relid, coalesce_range); CREATE TABLE powa_all_relations_history_current ( dbid oid NOT NULL, relid oid NOT NULL, record powa_all_relations_history_record NOT NULL ); CREATE SEQUENCE powa_coalesce_sequence INCREMENT BY 1 START WITH 1 CYCLE; CREATE TABLE powa_functions ( module text NOT NULL, operation text NOT NULL, function_name text NOT NULL, added_manually boolean NOT NULL default true, enabled boolean NOT NULL default true, CHECK (operation IN ('snapshot','aggregate','purge','unregister','reset')) ); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_stat_statements', 'snapshot', 'powa_statements_snapshot', false, true), ('powa_stat_user_functions', 'snapshot', 'powa_user_functions_snapshot', false, true), ('powa_stat_all_relations', 'snapshot', 'powa_all_relations_snapshot', false, true), ('pg_stat_statements', 'aggregate','powa_statements_aggregate', false, true), ('powa_stat_user_functions', 'aggregate','powa_user_functions_aggregate', false, true), ('powa_stat_all_relations', 'aggregate','powa_all_relations_aggregate', false, true), ('pg_stat_statements', 'purge', 'powa_statements_purge', false, true), ('powa_stat_user_functions', 'purge', 'powa_user_functions_purge', false, true), ('powa_stat_all_relations', 'purge', 'powa_all_relations_purge', false, true), ('pg_stat_statements', 'reset', 'powa_statements_reset', false, true), ('powa_stat_user_functions', 'reset', 'powa_user_functions_reset', false, true), ('powa_stat_all_relations', 'reset', 'powa_all_relations_reset', false, true); CREATE FUNCTION powa_log (msg text) RETURNS void LANGUAGE plpgsql AS $_$ BEGIN IF current_setting('powa.debug')::bool THEN RAISE WARNING '%', msg; ELSE RAISE DEBUG '%', msg; END IF; END; $_$; /* pg_stat_kcache integration - part 1 */ CREATE TYPE public.kcache_type AS ( ts timestamptz, reads bigint, writes bigint, user_time double precision, system_time double precision ); /* pg_stat_kcache operator support */ CREATE TYPE powa_kcache_diff AS ( intvl interval, reads bigint, writes bigint, user_time double precision, system_time double precision ); CREATE OR REPLACE FUNCTION powa_kcache_mi( a kcache_type, b kcache_type) RETURNS powa_kcache_diff AS $_$ DECLARE res powa_kcache_diff; BEGIN res.intvl = a.ts - b.ts; res.reads = a.reads - b.reads; res.writes = a.writes - b.writes; res.user_time = a.user_time - b.user_time; res.system_time = a.system_time - b.system_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_kcache_mi, LEFTARG = kcache_type, RIGHTARG = kcache_type ); CREATE TYPE powa_kcache_rate AS ( sec integer, reads_per_sec double precision, writes_per_sec double precision, user_time_per_sec double precision, system_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_kcache_div( a kcache_type, b kcache_type) RETURNS powa_kcache_rate AS $_$ DECLARE res powa_kcache_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.reads_per_sec = (a.reads - b.reads)::double precision / sec; res.writes_per_sec = (a.writes - b.writes)::double precision / sec; res.user_time_per_sec = (a.user_time - b.user_time)::double precision / sec; res.system_time_per_sec = (a.system_time - b.system_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_kcache_div, LEFTARG = kcache_type, RIGHTARG = kcache_type ); /* end of pg_stat_kcache operator support */ CREATE TABLE public.powa_kcache_metrics ( coalesce_range tstzrange NOT NULL, queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics public.kcache_type[] NOT NULL, mins_in_range public.kcache_type NOT NULL, maxs_in_range public.kcache_type NOT NULL, PRIMARY KEY (coalesce_range, queryid, dbid, userid) ); CREATE INDEX ON public.powa_kcache_metrics (queryid); CREATE TABLE public.powa_kcache_metrics_db ( coalesce_range tstzrange NOT NULL, dbid oid NOT NULL, metrics public.kcache_type[] NOT NULL, mins_in_range public.kcache_type NOT NULL, maxs_in_range public.kcache_type NOT NULL, PRIMARY KEY (coalesce_range, dbid) ); CREATE TABLE public.powa_kcache_metrics_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics kcache_type NULL NULL ); CREATE TABLE public.powa_kcache_metrics_current_db ( dbid oid NOT NULL, metrics kcache_type NULL NULL ); /* end of pg_stat_kcache integration - part 1 */ /* pg_qualstats integration - part 1 */ CREATE TYPE public.qual_type AS ( relid oid, attnum integer, opno oid, eval_type "char" ); CREATE TYPE public.qual_values AS ( constants text[], occurences bigint, execution_count bigint, nbfiltered bigint ); CREATE TYPE powa_qualstats_history_item AS ( ts timestamptz, occurences bigint, execution_count bigint, nbfiltered bigint ); /* pg_stat_qualstats operator support */ CREATE TYPE powa_qualstats_history_diff AS ( intvl interval, occurences bigint, execution_count bigint, nbfiltered bigint ); CREATE OR REPLACE FUNCTION powa_qualstats_history_mi( a powa_qualstats_history_item, b powa_qualstats_history_item) RETURNS powa_qualstats_history_diff AS $_$ DECLARE res powa_qualstats_history_diff; BEGIN res.intvl = a.ts - b.ts; res.occurences = a.occurences - b.occurences; res.execution_count = a.execution_count - b.execution_count; res.nbfiltered = a.nbfiltered - b.nbfiltered; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_qualstats_history_mi, LEFTARG = powa_qualstats_history_item, RIGHTARG = powa_qualstats_history_item ); CREATE TYPE powa_qualstats_history_rate AS ( sec integer, occurences_per_sec double precision, execution_count_per_sec double precision, nbfiltered_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_qualstats_history_div( a powa_qualstats_history_item, b powa_qualstats_history_item) RETURNS powa_qualstats_history_rate AS $_$ DECLARE res powa_qualstats_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.occurences_per_sec = (a.occurences - b.occurences)::double precision / sec; res.execution_count_per_sec = (a.execution_count - b.execution_count)::double precision / sec; res.nbfiltered_per_sec = (a.nbfiltered - b.nbfiltered)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_qualstats_history_div, LEFTARG = powa_qualstats_history_item, RIGHTARG = powa_qualstats_history_item ); /* end of pg_stat_qualstats operator support */ CREATE TABLE public.powa_qualstats_quals ( qualid bigint, queryid bigint, dbid oid, userid oid, quals public.qual_type[], PRIMARY KEY (qualid, queryid, dbid, userid), FOREIGN KEY (queryid, dbid, userid) REFERENCES powa_statements(queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, records powa_qualstats_history_item[], mins_in_range powa_qualstats_history_item, maxs_in_range powa_qualstats_history_item, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, occurences bigint, execution_count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES powa_qualstats_quals(qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, most_used qual_values[], most_filtering qual_values[], least_filtering qual_values[], most_executed qual_values[], FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, constvalues text[], occurences bigint, execution_count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE INDEX ON powa_qualstats_constvalues_history USING gist (queryid, qualid, coalesce_range); CREATE INDEX ON powa_qualstats_constvalues_history (qualid, queryid); CREATE INDEX ON powa_qualstats_quals(queryid); /* end of pg_qualstats_integration - part 1 */ -- Mark all of powa's tables as "to be dumped" SELECT pg_catalog.pg_extension_config_dump('powa_statements',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_user_functions_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_user_functions_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_all_relations_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_all_relations_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_functions','WHERE added_manually'); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history_current',''); CREATE OR REPLACE FUNCTION public.powa_check_created_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE BEGIN /* We have for now no way for a proper handling of this event, * as we don't have a table with the list of supported extensions. * So just call every powa_*_register() function we know each time an * extension is created. Powa should be in a dedicated database and the * register function handle to be called several time, so it's not critical */ PERFORM public.powa_kcache_register(); PERFORM public.powa_qualstats_register(); PERFORM public.powa_track_settings_register(); END; $_$; CREATE EVENT TRIGGER powa_check_created_extensions ON ddl_command_end WHEN tag IN ('CREATE EXTENSION') EXECUTE PROCEDURE public.powa_check_created_extensions() ; CREATE OR REPLACE FUNCTION public.powa_check_dropped_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- We unregister extensions regardless the "enabled" field WITH ext AS ( SELECT object_name FROM pg_event_trigger_dropped_objects() d WHERE d.object_type = 'extension' ) SELECT function_name INTO funcname FROM powa_functions f JOIN ext ON f.module = ext.object_name WHERE operation = 'unregister'; IF ( funcname IS NOT NULL ) THEN BEGIN PERFORM powa_log(format('running %I', funcname)); EXECUTE 'SELECT ' || quote_ident(funcname) || '()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE WARNING 'powa_check_dropped_extensions(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END IF; END; $_$; /* end of powa_check_dropped_extensions */ CREATE EVENT TRIGGER powa_check_dropped_extensions ON sql_drop WHEN tag IN ('DROP EXTENSION') EXECUTE PROCEDURE public.powa_check_dropped_extensions() ; CREATE OR REPLACE FUNCTION powa_take_snapshot() RETURNS void AS $PROC$ DECLARE purgets timestamp with time zone; purge_seq bigint; funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; v_title text = 'PoWA - '; v_rowcount bigint; BEGIN PERFORM set_config('application_name', v_title || ' snapshot database list', false); PERFORM powa_log('start of powa_take_snapshot'); -- Keep track of existing databases PERFORM powa_log('Maintaining database list...'); WITH missing AS ( SELECT d.oid, d.datname FROM pg_database d LEFT JOIN powa_databases p ON d.oid = p.oid WHERE p.oid IS NULL ) INSERT INTO powa_databases SELECT * FROM missing; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('missing db: %s', v_rowcount)); -- Keep track of renamed databases WITH renamed AS ( SELECT d.oid, d.datname FROM pg_database AS d JOIN powa_databases AS p ON d.oid = p.oid WHERE d.datname != p.datname ) UPDATE powa_databases AS p SET datname = r.datname FROM renamed AS r WHERE p.oid = r.oid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('renamed db: %s', v_rowcount)); -- Keep track of when databases are dropped WITH dropped AS ( SELECT p.oid FROM powa_databases p LEFT JOIN pg_database d ON p.oid = d.oid WHERE d.oid IS NULL AND p.dropped IS NULL) UPDATE powa_databases p SET dropped = now() FROM dropped d WHERE p.oid = d.oid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('dropped db: %s', v_rowcount)); -- For all enabled snapshot functions in the powa_functions table, execute FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='snapshot' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling snapshot function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; -- Coalesce datas if needed SELECT nextval('powa_coalesce_sequence'::regclass) INTO purge_seq; PERFORM powa_log(format('powa_coalesce_sequence: %s', purge_seq)); IF ( purge_seq % current_setting('powa.coalesce')::bigint ) = 0 THEN PERFORM powa_log(format('coalesce needed, seq: %s coalesce seq: %s', purge_seq, current_setting('powa.coalesce')::bigint )); FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='aggregate' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling aggregate function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; UPDATE powa_last_aggregation SET aggts = now(); END IF; -- We also purge, at next pass IF ( purge_seq % (current_setting('powa.coalesce')::bigint ) ) = 1 THEN PERFORM powa_log(format('purge needed, seq: %s coalesce seq: %s', purge_seq, current_setting('powa.coalesce'))); FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='purge' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling purge function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; PERFORM set_config('application_name', v_title || 'UPDATE powa_last_purge', false); UPDATE powa_last_purge SET purgets = now(); END IF; PERFORM powa_log('end of powa_take_snapshot'); PERFORM set_config('application_name', v_title || 'snapshot finished', false); END; $PROC$ LANGUAGE plpgsql; /* end of powa_take_snapshot */ CREATE OR REPLACE FUNCTION powa_statements_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; ignore_regexp text :='^[[:space:]]*(DEALLOCATE|BEGIN|PREPARE TRANSACTION|COMMIT PREPARED|ROLLBACK PREPARED)'; v_funcname text := 'powa_statements_snapshot'; v_rowcount bigint; BEGIN -- In this function, we capture statements, and also aggregate counters by database -- so that the first screens of powa stay reactive even though there may be thousands -- of different statements PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS( SELECT pgss.* FROM pg_stat_statements pgss JOIN pg_roles r ON pgss.userid = r.oid WHERE pgss.query !~* ignore_regexp AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_statements AS( INSERT INTO powa_statements (queryid, dbid, userid, query) SELECT queryid, dbid, userid, query FROM capture c WHERE NOT EXISTS (SELECT 1 FROM powa_statements ps WHERE ps.queryid = c.queryid AND ps.dbid = c.dbid AND ps.userid = c.userid ) ), by_query AS ( INSERT INTO powa_statements_history_current SELECT queryid, dbid, userid, ROW( now(), calls, total_time, rows, shared_blks_hit, shared_blks_read, shared_blks_dirtied, shared_blks_written, local_blks_hit, local_blks_read, local_blks_dirtied, local_blks_written, temp_blks_read, temp_blks_written, blk_read_time, blk_write_time )::powa_statements_history_record AS record FROM capture ), by_database AS ( INSERT INTO powa_statements_history_current_db SELECT dbid, ROW( now(), sum(calls), sum(total_time), sum(rows), sum(shared_blks_hit), sum(shared_blks_read), sum(shared_blks_dirtied), sum(shared_blks_written), sum(local_blks_hit), sum(local_blks_read), sum(local_blks_dirtied), sum(local_blks_written), sum(temp_blks_read), sum(temp_blks_written), sum(blk_read_time), sum(blk_write_time) )::powa_statements_history_record AS record FROM capture GROUP BY dbid ) SELECT count(*) INTO v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; -- For now we don't care. What could we do on error except crash anyway? END; $PROC$ language plpgsql; /* end of powa_statements_snapshot */ CREATE OR REPLACE FUNCTION powa_user_functions_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_user_functions_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide user function statistics WITH func(dbid,funcid, r) AS ( SELECT oid, (powa_stat_user_functions(oid)).funcid, powa_stat_user_functions(oid) FROM pg_database ) INSERT INTO powa_user_functions_history_current SELECT dbid, funcid, ROW(now(), (r).calls, (r).total_time, (r).self_time)::powa_user_functions_history_record AS record FROM func; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_user_functions_snapshot */ CREATE OR REPLACE FUNCTION powa_all_relations_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_all_relations_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide relation statistics WITH rel(dbid, relid, r) AS ( SELECT oid, (powa_stat_all_rel(oid)).relid, powa_stat_all_rel(oid) FROM pg_database ) INSERT INTO powa_all_relations_history_current SELECT dbid, relid, ROW(now(),(r).numscan, (r).tup_returned, (r).tup_fetched, (r).n_tup_ins, (r).n_tup_upd, (r).n_tup_del, (r).n_tup_hot_upd, (r).n_liv_tup, (r).n_dead_tup, (r).n_mod_since_analyze, (r).blks_read, (r).blks_hit, (r).last_vacuum, (r).vacuum_count, (r).last_autovacuum, (r).autovacuum_count, (r).last_analyze, (r).analyze_count, (r).last_autoanalyze, (r).autoanalyze_count)::powa_all_relations_history_record AS record FROM rel; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_all_relations_snapshot */ CREATE OR REPLACE FUNCTION powa_statements_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_statements_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_statements_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_hitory) - rowcount: %s', v_funcname, v_rowcount)); DELETE FROM powa_statements_history_db WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history_db) - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_statements_purge */ CREATE OR REPLACE FUNCTION powa_user_functions_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_user_functions_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_user_functions_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_user_functions_purge */ CREATE OR REPLACE FUNCTION powa_all_relations_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_all_relations_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_all_relations_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_all_relations_purge */ CREATE OR REPLACE FUNCTION powa_statements_aggregate() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_statements_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate statements table LOCK TABLE powa_statements_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history SELECT queryid, dbid, userid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current GROUP BY queryid, dbid, userid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_statements_history_current; -- aggregate db table LOCK TABLE powa_statements_history_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history_db SELECT dbid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current_db GROUP BY dbid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history_db) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_statements_history_current_db; END; $PROC$ LANGUAGE plpgsql; /* end of powa_statements_aggregate */ CREATE OR REPLACE FUNCTION powa_user_functions_aggregate() RETURNS void AS $PROC$ BEGIN PERFORM powa_log('running powa_user_functions_aggregate'); -- aggregate user_functions table LOCK TABLE powa_user_functions_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_user_functions_history SELECT dbid, funcid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time), min((record).self_time))::powa_user_functions_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time), max((record).self_time))::powa_user_functions_history_record FROM powa_user_functions_history_current GROUP BY dbid, funcid; TRUNCATE powa_user_functions_history_current; END; $PROC$ LANGUAGE plpgsql; /* end of powa_user_functions_aggregate */ CREATE OR REPLACE FUNCTION powa_all_relations_aggregate() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_all_relations_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate all_relations table LOCK TABLE powa_all_relations_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_all_relations_history SELECT dbid, relid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).numscan),min((record).tup_returned),min((record).tup_fetched), min((record).n_tup_ins),min((record).n_tup_upd), min((record).n_tup_del),min((record).n_tup_hot_upd), min((record).n_liv_tup),min((record).n_dead_tup), min((record).n_mod_since_analyze),min((record).blks_read), min((record).blks_hit),min((record).last_vacuum), min((record).vacuum_count),min((record).last_autovacuum), min((record).autovacuum_count),min((record).last_analyze), min((record).analyze_count),min((record).last_autoanalyze), min((record).autoanalyze_count))::powa_all_relations_history_record, ROW(max((record).ts), max((record).numscan),max((record).tup_returned),max((record).tup_fetched), max((record).n_tup_ins),max((record).n_tup_upd), max((record).n_tup_del),max((record).n_tup_hot_upd), max((record).n_liv_tup),max((record).n_dead_tup), max((record).n_mod_since_analyze),max((record).blks_read), max((record).blks_hit),max((record).last_vacuum), max((record).vacuum_count),max((record).last_autovacuum), max((record).autovacuum_count),max((record).last_analyze), max((record).analyze_count),max((record).last_autoanalyze), max((record).autoanalyze_count))::powa_all_relations_history_record FROM powa_all_relations_history_current GROUP BY dbid, relid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_all_relations_history_current; END; $PROC$ LANGUAGE plpgsql; /* end of powa_all_relations_aggregate */ CREATE OR REPLACE FUNCTION public.powa_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- Find reset function for every supported datasource, including pgss -- Also call reset function even if they're not enabled FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='reset' LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_reset(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; RETURN true; END; $function$; /* end of powa_reset */ CREATE OR REPLACE FUNCTION public.powa_statements_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_statements_history'); TRUNCATE TABLE powa_statements_history; PERFORM powa_log('truncating powa_statements_history_current'); TRUNCATE TABLE powa_statements_history_current; PERFORM powa_log('truncating powa_statements_history_db'); TRUNCATE TABLE powa_statements_history_db; PERFORM powa_log('truncating powa_statements_history_current_db'); TRUNCATE TABLE powa_statements_history_current_db; PERFORM powa_log('truncating powa_statements'); -- if 3rd part datasource has FK on it, throw everything away TRUNCATE TABLE powa_statements CASCADE; RETURN true; END; $function$; /* end of powa_statements_reset */ CREATE OR REPLACE FUNCTION public.powa_user_functions_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_user_functions_history'); TRUNCATE TABLE powa_user_functions_history; PERFORM powa_log('truncating powa_user_functions_history_current'); TRUNCATE TABLE powa_user_functions_history_current; RETURN true; END; $function$; /* end of powa_user_functions_reset */ CREATE OR REPLACE FUNCTION public.powa_all_relations_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_all_relations_history'); TRUNCATE TABLE powa_all_relations_history; PERFORM powa_log('truncating powa_all_relations_history_current'); TRUNCATE TABLE powa_all_relations_history_current; RETURN true; END; $function$; /* end of powa_all_relations_reset */ /* pg_stat_kcache integration - part 2 */ /* * register pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_stat_kcache'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_stat_kcache'; IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_stat_kcache'); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_stat_kcache', 'snapshot', 'powa_kcache_snapshot', false, true), ('pg_stat_kcache', 'aggregate', 'powa_kcache_aggregate', false, true), ('pg_stat_kcache', 'unregister', 'powa_kcache_unregister', false, true), ('pg_stat_kcache', 'purge', 'powa_kcache_purge', false, true), ('pg_stat_kcache', 'reset', 'powa_kcache_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_kcache_register */ /* * unregister pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_stat_kcache'); DELETE FROM public.powa_functions WHERE module = 'pg_stat_kcache'; RETURN true; END; $_$ language plpgsql; /* * powa_kcache snapshot collection. */ CREATE OR REPLACE FUNCTION powa_kcache_snapshot() RETURNS void as $PROC$ DECLARE result bool; v_funcname text := 'powa_kcache_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS ( SELECT * FROM pg_stat_kcache() k JOIN pg_roles r ON r.oid = k.userid WHERE NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), by_query AS ( INSERT INTO powa_kcache_metrics_current (queryid, dbid, userid, metrics) SELECT queryid, dbid, userid, (now(), reads, writes, user_time, system_time)::kcache_type FROM capture ), by_database AS ( INSERT INTO powa_kcache_metrics_current_db (dbid, metrics) SELECT dbid, (now(), sum(reads), sum(writes), sum(user_time), sum(system_time))::kcache_type FROM capture GROUP BY dbid ) SELECT COUNT(*) into v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END $PROC$ language plpgsql; /* end of powa_kcache_unregister */ /* * powa_kcache aggregation */ CREATE OR REPLACE FUNCTION powa_kcache_aggregate() RETURNS void AS $PROC$ DECLARE result bool; v_funcname text := 'powa_kcache_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate metrics table LOCK TABLE powa_kcache_metrics_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics (coalesce_range, queryid, dbid, userid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), queryid, dbid, userid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current GROUP BY queryid, dbid, userid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_kcache_metrics_current; -- aggregate metrics_db table LOCK TABLE powa_kcache_metrics_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics_db (coalesce_range, dbid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), dbid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current_db GROUP BY dbid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics_db) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_kcache_metrics_current_db; END $PROC$ language plpgsql; /* end of powa_kcache_aggregate */ /* * powa_kcache purge */ CREATE OR REPLACE FUNCTION powa_kcache_purge() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_kcache_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); DELETE FROM powa_kcache_metrics WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics) - rowcount: %s', v_funcname, v_rowcount)); DELETE FROM powa_kcache_metrics_db WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics_db) - rowcount: %s', v_funcname, v_rowcount)); END; $PROC$ language plpgsql; /* end of powa_kcache_purge */ /* * powa_kcache reset */ CREATE OR REPLACE FUNCTION powa_kcache_reset() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_kcache_reset'; v_rowcount bigint; BEGIN PERFORM powa_log('running powa_kcache_reset'); PERFORM powa_log('truncating powa_kcache_metrics'); TRUNCATE TABLE powa_kcache_metrics; PERFORM powa_log('truncating powa_kcache_metrics_db'); TRUNCATE TABLE powa_kcache_metrics_db; PERFORM powa_log('truncating powa_kcache_metrics_current'); TRUNCATE TABLE powa_kcache_metrics_current; PERFORM powa_log('truncating powa_kcache_metrics_current_db'); TRUNCATE TABLE powa_kcache_metrics_current_db; END; $PROC$ language plpgsql; /* end of powa_kcache_reset */ -- By default, try to register pg_stat_kcache, in case it's alreay here SELECT * FROM public.powa_kcache_register(); /* end of pg_stat_kcache integration - part 2 */ /* pg_qualstats integration - part 2 */ /* * powa_qualstats_register */ CREATE OR REPLACE function public.powa_qualstats_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_qualstats'; IF ( v_ext_present) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE function_name IN ('powa_qualstats_snapshot', 'powa_qualstats_aggregate', 'powa_qualstats_purge'); IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_qualstats'); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_qualstats', 'snapshot', 'powa_qualstats_snapshot', false, true), ('pg_qualstats', 'aggregate', 'powa_qualstats_aggregate', false, true), ('pg_qualstats', 'unregister', 'powa_qualstats_unregister', false, true), ('pg_qualstats', 'purge', 'powa_qualstats_purge', false, true), ('pg_qualstats', 'reset', 'powa_qualstats_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_qualstats_register */ /* * powa_qualstats utility view for aggregating constvalues */ CREATE OR REPLACE VIEW powa_qualstats_aggregate_constvalues_current AS WITH consts AS ( SELECT qualid, queryid, dbid, userid, min(ts) as mints, max(ts) as maxts, sum(occurences) as occurences, sum(nbfiltered) as nbfiltered, sum(execution_count) as execution_count, constvalues FROM powa_qualstats_constvalues_history_current GROUP BY qualid, queryid, dbid, userid, constvalues ), groups AS ( SELECT qualid, queryid, dbid, userid, tstzrange(min(mints), max(maxts),'[]') FROM consts GROUP BY qualid, queryid, dbid, userid ) SELECT * FROM groups, LATERAL ( SELECT array_agg(constvalues) as mu FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY occurences desc LIMIT 20 ) s ) as mu, LATERAL ( SELECT array_agg(constvalues) as mf FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN execution_count = 0 THEN 0 ELSE nbfiltered / execution_count::numeric END DESC LIMIT 20 ) s ) as mf, LATERAL ( SELECT array_agg(constvalues) as lf FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN execution_count = 0 THEN 0 ELSE nbfiltered / execution_count::numeric END DESC LIMIT 20 ) s ) as lf, LATERAL ( SELECT array_agg(constvalues) as me FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY execution_count desc LIMIT 20 ) s ) as me; CREATE OR REPLACE FUNCTION powa_qualstats_snapshot() RETURNS void as $PROC$ DECLARE result bool; v_funcname text := 'powa_qualstats_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS ( SELECT pgqs.*, s.query FROM pg_qualstats_by_query pgqs JOIN powa_statements s USING(queryid, dbid, userid) JOIN pg_roles r ON s.userid = r.oid AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_quals AS ( INSERT INTO powa_qualstats_quals (qualid, queryid, dbid, userid, quals) SELECT DISTINCT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, array_agg(DISTINCT q::qual_type) FROM capture qs, LATERAL (SELECT (unnest(quals)).*) as q WHERE NOT EXISTS ( SELECT 1 FROM powa_qualstats_quals nh WHERE nh.qualid = qs.qualnodeid AND nh.queryid = qs.queryid AND nh.dbid = qs.dbid AND nh.userid = qs.userid ) GROUP BY qualnodeid, queryid, dbid, userid RETURNING * ), by_qual AS ( INSERT INTO powa_qualstats_quals_history_current (qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered) SELECT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), sum(occurences), sum(execution_count), sum(nbfiltered) FROM capture as qs GROUP BY qualnodeid, qs.queryid, qs.dbid, qs.userid RETURNING * ), by_qual_with_const AS ( INSERT INTO powa_qualstats_constvalues_history_current(qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered, constvalues) SELECT qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), occurences, execution_count, nbfiltered, constvalues FROM capture as qs ) SELECT COUNT(*) into v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; PERFORM pg_qualstats_reset(); END $PROC$ language plpgsql; /* end of powa_qualstats_snapshot */ /* * powa_qualstats aggregate */ CREATE OR REPLACE FUNCTION powa_qualstats_aggregate() RETURNS void AS $PROC$ DECLARE result bool; BEGIN PERFORM powa_log('running powa_qualstats_aggregate'); LOCK TABLE powa_qualstats_constvalues_history_current IN SHARE MODE; LOCK TABLE powa_qualstats_quals_history_current IN SHARE MODE; INSERT INTO powa_qualstats_constvalues_history ( qualid, queryid, dbid, userid, coalesce_range, most_used, most_filtering, least_filtering, most_executed) SELECT * FROM powa_qualstats_aggregate_constvalues_current; INSERT INTO powa_qualstats_quals_history (qualid, queryid, dbid, userid, coalesce_range, records, mins_in_range, maxs_in_range) SELECT qualid, queryid, dbid, userid, tstzrange(min(ts), max(ts),'[]'), array_agg((ts, occurences, execution_count, nbfiltered)::powa_qualstats_history_item), ROW(min(ts), min(occurences), min(execution_count), min(nbfiltered))::powa_qualstats_history_item, ROW(max(ts), max(occurences), max(execution_count), max(nbfiltered))::powa_qualstats_history_item FROM powa_qualstats_quals_history_current GROUP BY qualid, queryid, dbid, userid; TRUNCATE powa_qualstats_constvalues_history_current; TRUNCATE powa_qualstats_quals_history_current; END $PROC$ language plpgsql; /* end of powa_qualstats_aggregate */ /* * powa_qualstats_purge */ CREATE OR REPLACE FUNCTION powa_qualstats_purge() RETURNS void as $PROC$ BEGIN PERFORM powa_log('running powa_qualstats_purge'); DELETE FROM powa_qualstats_constvalues_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); DELETE FROM powa_qualstats_quals_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); END; $PROC$ language plpgsql; /* end of powa_qualstats_purge */ /* * powa_qualstats_reset */ CREATE OR REPLACE FUNCTION powa_qualstats_reset() RETURNS void as $PROC$ BEGIN PERFORM powa_log('running powa_qualstats_reset'); PERFORM powa_log('truncating powa_qualstats_quals'); TRUNCATE TABLE powa_qualstats_quals CASCADE; -- cascaded : -- powa_qualstats_quals_history -- powa_qualstats_quals_history_current -- powa_qualstats_constvalues_history -- powa_qualstats_constvalues_history_current END; $PROC$ language plpgsql; /* end of powa_qualstats_reset */ /* * powa_qualstats_unregister */ CREATE OR REPLACE function public.powa_qualstats_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_qualstats'); DELETE FROM public.powa_functions WHERE module = 'pg_qualstats'; RETURN true; END; $_$ language plpgsql; SELECT * FROM public.powa_qualstats_register(); /* end of pg_qualstats_integration - part 2 */ /* pg_track_settings integration */ CREATE OR REPLACE FUNCTION powa_track_settings_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_track_settings'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_track_settings'; IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_track_settings'); -- This extension handles its own storage, just its snapshot -- function and an unregister function. INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_track_settings', 'snapshot', 'pg_track_settings_snapshot', false, true), ('pg_track_settings', 'unregister', 'powa_track_settings_unregister', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_qualstats_unregister */ CREATE OR REPLACE function public.powa_track_settings_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_track_settings'); DELETE FROM public.powa_functions WHERE module = 'pg_track_settings'; RETURN true; END; $_$ language plpgsql; /* end of powa_track_settings_unregister */ -- By default, try to register pg_track_settings, in case it's alreay here SELECT * FROM public.powa_track_settings_register(); /* end pg_track_settings integration */ powa-archivist-REL_3_2_0/powa--3.1.1--3.1.2.sql000066400000000000000000000043171336072707400202540ustar00rootroot00000000000000-- complain if script is sourced in psql, rather than via CREATE EXTENSION --\echo Use "ALTER EXTENSION powa" to load this file. \quit CREATE OR REPLACE FUNCTION powa_all_relations_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_all_relations_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide relation statistics WITH rel(dbid, r) AS ( SELECT oid, powa_stat_all_rel(oid) FROM pg_database ) INSERT INTO powa_all_relations_history_current SELECT dbid, (r).relid, ROW(now(),(r).numscan, (r).tup_returned, (r).tup_fetched, (r).n_tup_ins, (r).n_tup_upd, (r).n_tup_del, (r).n_tup_hot_upd, (r).n_liv_tup, (r).n_dead_tup, (r).n_mod_since_analyze, (r).blks_read, (r).blks_hit, (r).last_vacuum, (r).vacuum_count, (r).last_autovacuum, (r).autovacuum_count, (r).last_analyze, (r).analyze_count, (r).last_autoanalyze, (r).autoanalyze_count)::powa_all_relations_history_record AS record FROM rel; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_all_relations_snapshot */ CREATE OR REPLACE FUNCTION powa_user_functions_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_user_functions_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide user function statistics WITH func(dbid, r) AS ( SELECT oid, powa_stat_user_functions(oid) FROM pg_database ) INSERT INTO powa_user_functions_history_current SELECT dbid, (r).funcid, ROW(now(), (r).calls, (r).total_time, (r).self_time)::powa_user_functions_history_record AS record FROM func; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_user_functions_snapshot */ powa-archivist-REL_3_2_0/powa--3.1.1.sql000066400000000000000000002124711336072707400175420ustar00rootroot00000000000000-- complain if script is sourced in psql, rather than via CREATE EXTENSION --\echo Use "CREATE EXTENSION powa" to load this file. \quit SET statement_timeout = 0; SET client_encoding = 'UTF8'; SET standard_conforming_strings = on; SET client_min_messages = warning; SET escape_string_warning = off; SET search_path = public, pg_catalog; CREATE TABLE powa_databases( oid oid PRIMARY KEY, datname name, dropped timestamp with time zone ); CREATE FUNCTION powa_stat_user_functions(IN dbid oid, OUT funcid oid, OUT calls bigint, OUT total_time double precision, OUT self_time double precision) RETURNS SETOF record LANGUAGE c COST 100 AS '$libdir/powa', 'powa_stat_user_functions'; CREATE FUNCTION powa_stat_all_rel(IN dbid oid, OUT relid oid, OUT numscan bigint, OUT tup_returned bigint, OUT tup_fetched bigint, OUT n_tup_ins bigint, OUT n_tup_upd bigint, OUT n_tup_del bigint, OUT n_tup_hot_upd bigint, OUT n_liv_tup bigint, OUT n_dead_tup bigint, OUT n_mod_since_analyze bigint, OUT blks_read bigint, OUT blks_hit bigint, OUT last_vacuum timestamp with time zone, OUT vacuum_count bigint, OUT last_autovacuum timestamp with time zone, OUT autovacuum_count bigint, OUT last_analyze timestamp with time zone, OUT analyze_count bigint, OUT last_autoanalyze timestamp with time zone, OUT autoanalyze_count bigint) RETURNS SETOF record LANGUAGE c COST 100 AS '$libdir/powa', 'powa_stat_all_rel'; CREATE TYPE powa_statements_history_record AS ( ts timestamp with time zone, calls bigint, total_time double precision, rows bigint, shared_blks_hit bigint, shared_blks_read bigint, shared_blks_dirtied bigint, shared_blks_written bigint, local_blks_hit bigint, local_blks_read bigint, local_blks_dirtied bigint, local_blks_written bigint, temp_blks_read bigint, temp_blks_written bigint, blk_read_time double precision, blk_write_time double precision ); /* pg_stat_statements operator support */ CREATE TYPE powa_statements_history_diff AS ( intvl interval, calls bigint, total_time double precision, rows bigint, shared_blks_hit bigint, shared_blks_read bigint, shared_blks_dirtied bigint, shared_blks_written bigint, local_blks_hit bigint, local_blks_read bigint, local_blks_dirtied bigint, local_blks_written bigint, temp_blks_read bigint, temp_blks_written bigint, blk_read_time double precision, blk_write_time double precision ); CREATE OR REPLACE FUNCTION powa_statements_history_mi( a powa_statements_history_record, b powa_statements_history_record) RETURNS powa_statements_history_diff AS $_$ DECLARE res powa_statements_history_diff; BEGIN res.intvl = a.ts - b.ts; res.calls = a.calls - b.calls; res.total_time = a.total_time - b.total_time; res.rows = a.rows - b.rows; res.shared_blks_hit = a.shared_blks_hit - b.shared_blks_hit; res.shared_blks_read = a.shared_blks_read - b.shared_blks_read; res.shared_blks_dirtied = a.shared_blks_dirtied - b.shared_blks_dirtied; res.shared_blks_written = a.shared_blks_written - b.shared_blks_written; res.local_blks_hit = a.local_blks_hit - b.local_blks_hit; res.local_blks_read = a.local_blks_read - b.local_blks_read; res.local_blks_dirtied = a.local_blks_dirtied - b.local_blks_dirtied; res.local_blks_written = a.local_blks_written - b.local_blks_written; res.temp_blks_read = a.temp_blks_read - b.temp_blks_read; res.temp_blks_written = a.temp_blks_written - b.temp_blks_written; res.blk_read_time = a.blk_read_time - b.blk_read_time; res.blk_write_time = a.blk_write_time - b.blk_write_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_statements_history_mi, LEFTARG = powa_statements_history_record, RIGHTARG = powa_statements_history_record ); CREATE TYPE powa_statements_history_rate AS ( sec integer, calls_per_sec double precision, runtime_per_sec double precision, rows_per_sec double precision, shared_blks_hit_per_sec double precision, shared_blks_read_per_sec double precision, shared_blks_dirtied_per_sec double precision, shared_blks_written_per_sec double precision, local_blks_hit_per_sec double precision, local_blks_read_per_sec double precision, local_blks_dirtied_per_sec double precision, local_blks_written_per_sec double precision, temp_blks_read_per_sec double precision, temp_blks_written_per_sec double precision, blk_read_time_per_sec double precision, blk_write_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_statements_history_div( a powa_statements_history_record, b powa_statements_history_record) RETURNS powa_statements_history_rate AS $_$ DECLARE res powa_statements_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.calls_per_sec = (a.calls - b.calls)::double precision / sec; res.runtime_per_sec = (a.total_time - b.total_time)::double precision / sec; res.rows_per_sec = (a.rows - b.rows)::double precision / sec; res.shared_blks_hit_per_sec = (a.shared_blks_hit - b.shared_blks_hit)::double precision / sec; res.shared_blks_read_per_sec = (a.shared_blks_read - b.shared_blks_read)::double precision / sec; res.shared_blks_dirtied_per_sec = (a.shared_blks_dirtied - b.shared_blks_dirtied)::double precision / sec; res.shared_blks_written_per_sec = (a.shared_blks_written - b.shared_blks_written)::double precision / sec; res.local_blks_hit_per_sec = (a.local_blks_hit - b.local_blks_hit)::double precision / sec; res.local_blks_read_per_sec = (a.local_blks_read - b.local_blks_read)::double precision / sec; res.local_blks_dirtied_per_sec = (a.local_blks_dirtied - b.local_blks_dirtied)::double precision / sec; res.local_blks_written_per_sec = (a.local_blks_written - b.local_blks_written)::double precision / sec; res.temp_blks_read_per_sec = (a.temp_blks_read - b.temp_blks_read)::double precision / sec; res.temp_blks_written_per_sec = (a.temp_blks_written - b.temp_blks_written)::double precision / sec; res.blk_read_time_per_sec = (a.blk_read_time - b.blk_read_time)::double precision / sec; res.blk_write_time_per_sec = (a.blk_write_time - b.blk_write_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_statements_history_div, LEFTARG = powa_statements_history_record, RIGHTARG = powa_statements_history_record ); /* end of pg_stat_statements operator support */ CREATE TYPE powa_user_functions_history_record AS ( ts timestamp with time zone, calls bigint, total_time double precision, self_time double precision ); /* pg_stat_user_functions operator support */ CREATE TYPE powa_user_functions_history_diff AS ( intvl interval, calls bigint, total_time double precision, self_time double precision ); CREATE OR REPLACE FUNCTION powa_user_functions_history_mi( a powa_user_functions_history_record, b powa_user_functions_history_record) RETURNS powa_user_functions_history_diff AS $_$ DECLARE res powa_user_functions_history_diff; BEGIN res.intvl = a.ts - b.ts; res.calls = a.calls - b.calls; res.total_time = a.total_time - b.total_time; res.self_time = a.self_time - b.self_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_user_functions_history_mi, LEFTARG = powa_user_functions_history_record, RIGHTARG = powa_user_functions_history_record ); CREATE TYPE powa_user_functions_history_rate AS ( sec integer, calls_per_sec double precision, total_time_per_sec double precision, self_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_user_functions_history_div( a powa_user_functions_history_record, b powa_user_functions_history_record) RETURNS powa_user_functions_history_rate AS $_$ DECLARE res powa_user_functions_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.calls_per_sec = (a.calls - b.calls)::double precision / sec; res.total_time_per_sec = (a.total_time - b.total_time)::double precision / sec; res.self_time_per_sec = (a.self_time - b.self_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_user_functions_history_div, LEFTARG = powa_user_functions_history_record, RIGHTARG = powa_user_functions_history_record ); /* end of pg_stat_user_functions operator support */ CREATE TYPE powa_all_relations_history_record AS ( ts timestamp with time zone, numscan bigint, tup_returned bigint, tup_fetched bigint, n_tup_ins bigint, n_tup_upd bigint, n_tup_del bigint, n_tup_hot_upd bigint, n_liv_tup bigint, n_dead_tup bigint, n_mod_since_analyze bigint, blks_read bigint, blks_hit bigint, last_vacuum timestamp with time zone, vacuum_count bigint, last_autovacuum timestamp with time zone, autovacuum_count bigint, last_analyze timestamp with time zone, analyze_count bigint, last_autoanalyze timestamp with time zone, autoanalyze_count bigint ); /* pg_stat_all_relations operator support */ CREATE TYPE powa_all_relations_history_diff AS ( intvl interval, numscan bigint, tup_returned bigint, tup_fetched bigint, n_tup_ins bigint, n_tup_upd bigint, n_tup_del bigint, n_tup_hot_upd bigint, n_liv_tup bigint, n_dead_tup bigint, n_mod_since_analyze bigint, blks_read bigint, blks_hit bigint, vacuum_count bigint, autovacuum_count bigint, analyze_count bigint, autoanalyze_count bigint ); CREATE OR REPLACE FUNCTION powa_all_relations_history_mi( a powa_all_relations_history_record, b powa_all_relations_history_record) RETURNS powa_all_relations_history_diff AS $_$ DECLARE res powa_all_relations_history_diff; BEGIN res.intvl = a.ts - b.ts; res.numscan = a.numscan - b.numscan; res.tup_returned = a.tup_returned - b.tup_returned; res.tup_fetched = a.tup_fetched - b.tup_fetched; res.n_tup_ins = a.n_tup_ins - b.n_tup_ins; res.n_tup_upd = a.n_tup_upd - b.n_tup_upd; res.n_tup_del = a.n_tup_del - b.n_tup_del; res.n_tup_hot_upd = a.n_tup_hot_upd - b.n_tup_hot_upd; res.n_liv_tup = a.n_liv_tup - b.n_liv_tup; res.n_dead_tup = a.n_dead_tup - b.n_dead_tup; res.n_mod_since_analyze = a.n_mod_since_analyze - b.n_mod_since_analyze; res.blks_read = a.blks_read - b.blks_read; res.blks_hit = a.blks_hit - b.blks_hit; res.vacuum_count = a.vacuum_count - b.vacuum_count; res.autovacuum_count = a.autovacuum_count - b.autovacuum_count; res.analyze_count = a.analyze_count - b.analyze_count; res.autoanalyze_count = a.autoanalyze_count - b.autoanalyze_count; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_all_relations_history_mi, LEFTARG = powa_all_relations_history_record, RIGHTARG = powa_all_relations_history_record ); CREATE TYPE powa_all_relations_history_rate AS ( sec integer, numscan_per_sec double precision, tup_returned_per_sec double precision, tup_fetched_per_sec double precision, n_tup_ins_per_sec double precision, n_tup_upd_per_sec double precision, n_tup_del_per_sec double precision, n_tup_hot_upd_per_sec double precision, n_liv_tup_per_sec double precision, n_dead_tup_per_sec double precision, n_mod_since_analyze_per_sec double precision, blks_read_per_sec double precision, blks_hit_per_sec double precision, vacuum_count_per_sec double precision, autovacuum_count_per_sec double precision, analyze_count_per_sec double precision, autoanalyze_count_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_all_relations_history_div( a powa_all_relations_history_record, b powa_all_relations_history_record) RETURNS powa_all_relations_history_rate AS $_$ DECLARE res powa_all_relations_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.numscan_per_sec = (a.numscan - b.numscan)::double precision / sec; res.tup_returned_per_sec = (a.tup_returned - b.tup_returned)::double precision / sec; res.tup_fetched_per_sec = (a.tup_fetched - b.tup_fetched)::double precision / sec; res.n_tup_ins_per_sec = (a.n_tup_ins - b.n_tup_ins)::double precision / sec; res.n_tup_upd_per_sec = (a.n_tup_upd - b.n_tup_upd)::double precision / sec; res.n_tup_del_per_sec = (a.n_tup_del - b.n_tup_del)::double precision / sec; res.n_tup_hot_upd_per_sec = (a.n_tup_hot_upd - b.n_tup_hot_upd)::double precision / sec; res.n_liv_tup_per_sec = (a.n_liv_tup - b.n_liv_tup)::double precision / sec; res.n_dead_tup_per_sec = (a.n_dead_tup - b.n_dead_tup)::double precision / sec; res.n_mod_since_analyze_per_sec = (a.n_mod_since_analyze - b.n_mod_since_analyze)::double precision / sec; res.blks_read_per_sec = (a.blks_read - b.blks_read)::double precision / sec; res.blks_hit_per_sec = (a.blks_hit - b.blks_hit)::double precision / sec; res.vacuum_count_per_sec = (a.vacuum_count - b.vacuum_count)::double precision / sec; res.autovacuum_count_per_sec = (a.autovacuum_count - b.autovacuum_count)::double precision / sec; res.analyze_count_per_sec = (a.analyze_count - b.analyze_count)::double precision / sec; res.autoanalyze_count_per_sec = (a.autoanalyze_count - b.autoanalyze_count)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_all_relations_history_div, LEFTARG = powa_all_relations_history_record, RIGHTARG = powa_all_relations_history_record ); /* end of pg_stat_all_relations operator support */ CREATE TABLE powa_last_aggregation ( aggts timestamp with time zone ); INSERT INTO powa_last_aggregation(aggts) VALUES (current_timestamp); CREATE TABLE powa_last_purge ( purgets timestamp with time zone ); INSERT INTO powa_last_purge (purgets) VALUES (current_timestamp); CREATE TABLE powa_statements ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, query text NOT NULL ); ALTER TABLE ONLY powa_statements ADD CONSTRAINT powa_statements_pkey PRIMARY KEY (queryid, dbid, userid); CREATE INDEX powa_statements_dbid_idx ON powa_statements(dbid); CREATE INDEX powa_statements_userid_idx ON powa_statements(userid); CREATE TABLE powa_statements_history ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statements_history_record[] NOT NULL, mins_in_range powa_statements_history_record NOT NULL, maxs_in_range powa_statements_history_record NOT NULL ); CREATE INDEX powa_statements_history_query_ts ON powa_statements_history USING gist (queryid, coalesce_range); CREATE TABLE powa_statements_history_db ( dbid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statements_history_record[] NOT NULL, mins_in_range powa_statements_history_record NOT NULL, maxs_in_range powa_statements_history_record NOT NULL ); CREATE INDEX powa_statements_history_db_ts ON powa_statements_history_db USING gist (dbid, coalesce_range); CREATE TABLE powa_statements_history_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, record powa_statements_history_record NOT NULL ); CREATE TABLE powa_statements_history_current_db ( dbid oid NOT NULL, record powa_statements_history_record NOT NULL ); CREATE TABLE powa_user_functions_history ( dbid oid NOT NULL, funcid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_user_functions_history_record[] NOT NULL, mins_in_range powa_user_functions_history_record NOT NULL, maxs_in_range powa_user_functions_history_record NOT NULL ); CREATE INDEX powa_user_functions_history_funcid_ts ON powa_user_functions_history USING gist (funcid, coalesce_range); CREATE TABLE powa_user_functions_history_current ( dbid oid NOT NULL, funcid oid NOT NULL, record powa_user_functions_history_record NOT NULL ); CREATE TABLE powa_all_relations_history ( dbid oid NOT NULL, relid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_all_relations_history_record[] NOT NULL, mins_in_range powa_all_relations_history_record NOT NULL, maxs_in_range powa_all_relations_history_record NOT NULL ); CREATE INDEX powa_all_relations_history_relid_ts ON powa_all_relations_history USING gist (relid, coalesce_range); CREATE TABLE powa_all_relations_history_current ( dbid oid NOT NULL, relid oid NOT NULL, record powa_all_relations_history_record NOT NULL ); CREATE SEQUENCE powa_coalesce_sequence INCREMENT BY 1 START WITH 1 CYCLE; CREATE TABLE powa_functions ( module text NOT NULL, operation text NOT NULL, function_name text NOT NULL, added_manually boolean NOT NULL default true, enabled boolean NOT NULL default true, CHECK (operation IN ('snapshot','aggregate','purge','unregister','reset')) ); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_stat_statements', 'snapshot', 'powa_statements_snapshot', false, true), ('powa_stat_user_functions', 'snapshot', 'powa_user_functions_snapshot', false, true), ('powa_stat_all_relations', 'snapshot', 'powa_all_relations_snapshot', false, true), ('pg_stat_statements', 'aggregate','powa_statements_aggregate', false, true), ('powa_stat_user_functions', 'aggregate','powa_user_functions_aggregate', false, true), ('powa_stat_all_relations', 'aggregate','powa_all_relations_aggregate', false, true), ('pg_stat_statements', 'purge', 'powa_statements_purge', false, true), ('powa_stat_user_functions', 'purge', 'powa_user_functions_purge', false, true), ('powa_stat_all_relations', 'purge', 'powa_all_relations_purge', false, true), ('pg_stat_statements', 'reset', 'powa_statements_reset', false, true), ('powa_stat_user_functions', 'reset', 'powa_user_functions_reset', false, true), ('powa_stat_all_relations', 'reset', 'powa_all_relations_reset', false, true); CREATE FUNCTION powa_log (msg text) RETURNS void LANGUAGE plpgsql AS $_$ BEGIN IF current_setting('powa.debug')::bool THEN RAISE WARNING '%', msg; ELSE RAISE DEBUG '%', msg; END IF; END; $_$; /* pg_stat_kcache integration - part 1 */ CREATE TYPE public.kcache_type AS ( ts timestamptz, reads bigint, writes bigint, user_time double precision, system_time double precision ); /* pg_stat_kcache operator support */ CREATE TYPE powa_kcache_diff AS ( intvl interval, reads bigint, writes bigint, user_time double precision, system_time double precision ); CREATE OR REPLACE FUNCTION powa_kcache_mi( a kcache_type, b kcache_type) RETURNS powa_kcache_diff AS $_$ DECLARE res powa_kcache_diff; BEGIN res.intvl = a.ts - b.ts; res.reads = a.reads - b.reads; res.writes = a.writes - b.writes; res.user_time = a.user_time - b.user_time; res.system_time = a.system_time - b.system_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_kcache_mi, LEFTARG = kcache_type, RIGHTARG = kcache_type ); CREATE TYPE powa_kcache_rate AS ( sec integer, reads_per_sec double precision, writes_per_sec double precision, user_time_per_sec double precision, system_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_kcache_div( a kcache_type, b kcache_type) RETURNS powa_kcache_rate AS $_$ DECLARE res powa_kcache_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.reads_per_sec = (a.reads - b.reads)::double precision / sec; res.writes_per_sec = (a.writes - b.writes)::double precision / sec; res.user_time_per_sec = (a.user_time - b.user_time)::double precision / sec; res.system_time_per_sec = (a.system_time - b.system_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_kcache_div, LEFTARG = kcache_type, RIGHTARG = kcache_type ); /* end of pg_stat_kcache operator support */ CREATE TABLE public.powa_kcache_metrics ( coalesce_range tstzrange NOT NULL, queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics public.kcache_type[] NOT NULL, mins_in_range public.kcache_type NOT NULL, maxs_in_range public.kcache_type NOT NULL, PRIMARY KEY (coalesce_range, queryid, dbid, userid) ); CREATE INDEX ON public.powa_kcache_metrics (queryid); CREATE TABLE public.powa_kcache_metrics_db ( coalesce_range tstzrange NOT NULL, dbid oid NOT NULL, metrics public.kcache_type[] NOT NULL, mins_in_range public.kcache_type NOT NULL, maxs_in_range public.kcache_type NOT NULL, PRIMARY KEY (coalesce_range, dbid) ); CREATE TABLE public.powa_kcache_metrics_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics kcache_type NULL NULL ); CREATE TABLE public.powa_kcache_metrics_current_db ( dbid oid NOT NULL, metrics kcache_type NULL NULL ); /* end of pg_stat_kcache integration - part 1 */ /* pg_qualstats integration - part 1 */ CREATE TYPE public.qual_type AS ( relid oid, attnum integer, opno oid, eval_type "char" ); CREATE TYPE public.qual_values AS ( constants text[], occurences bigint, execution_count bigint, nbfiltered bigint ); CREATE TYPE powa_qualstats_history_item AS ( ts timestamptz, occurences bigint, execution_count bigint, nbfiltered bigint ); /* pg_stat_qualstats operator support */ CREATE TYPE powa_qualstats_history_diff AS ( intvl interval, occurences bigint, execution_count bigint, nbfiltered bigint ); CREATE OR REPLACE FUNCTION powa_qualstats_history_mi( a powa_qualstats_history_item, b powa_qualstats_history_item) RETURNS powa_qualstats_history_diff AS $_$ DECLARE res powa_qualstats_history_diff; BEGIN res.intvl = a.ts - b.ts; res.occurences = a.occurences - b.occurences; res.execution_count = a.execution_count - b.execution_count; res.nbfiltered = a.nbfiltered - b.nbfiltered; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_qualstats_history_mi, LEFTARG = powa_qualstats_history_item, RIGHTARG = powa_qualstats_history_item ); CREATE TYPE powa_qualstats_history_rate AS ( sec integer, occurences_per_sec double precision, execution_count_per_sec double precision, nbfiltered_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_qualstats_history_div( a powa_qualstats_history_item, b powa_qualstats_history_item) RETURNS powa_qualstats_history_rate AS $_$ DECLARE res powa_qualstats_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.occurences_per_sec = (a.occurences - b.occurences)::double precision / sec; res.execution_count_per_sec = (a.execution_count - b.execution_count)::double precision / sec; res.nbfiltered_per_sec = (a.nbfiltered - b.nbfiltered)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_qualstats_history_div, LEFTARG = powa_qualstats_history_item, RIGHTARG = powa_qualstats_history_item ); /* end of pg_stat_qualstats operator support */ CREATE TABLE public.powa_qualstats_quals ( qualid bigint, queryid bigint, dbid oid, userid oid, quals public.qual_type[], PRIMARY KEY (qualid, queryid, dbid, userid), FOREIGN KEY (queryid, dbid, userid) REFERENCES powa_statements(queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, records powa_qualstats_history_item[], mins_in_range powa_qualstats_history_item, maxs_in_range powa_qualstats_history_item, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, occurences bigint, execution_count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES powa_qualstats_quals(qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, most_used qual_values[], most_filtering qual_values[], least_filtering qual_values[], most_executed qual_values[], FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, constvalues text[], occurences bigint, execution_count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE INDEX ON powa_qualstats_constvalues_history USING gist (queryid, qualid, coalesce_range); CREATE INDEX ON powa_qualstats_constvalues_history (qualid, queryid); CREATE INDEX ON powa_qualstats_quals(queryid); /* end of pg_qualstats_integration - part 1 */ -- Mark all of powa's tables as "to be dumped" SELECT pg_catalog.pg_extension_config_dump('powa_statements',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_user_functions_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_user_functions_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_all_relations_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_all_relations_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_functions','WHERE added_manually'); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history_current',''); CREATE OR REPLACE FUNCTION public.powa_check_created_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE BEGIN /* We have for now no way for a proper handling of this event, * as we don't have a table with the list of supported extensions. * So just call every powa_*_register() function we know each time an * extension is created. Powa should be in a dedicated database and the * register function handle to be called several time, so it's not critical */ PERFORM public.powa_kcache_register(); PERFORM public.powa_qualstats_register(); PERFORM public.powa_track_settings_register(); END; $_$; CREATE EVENT TRIGGER powa_check_created_extensions ON ddl_command_end WHEN tag IN ('CREATE EXTENSION') EXECUTE PROCEDURE public.powa_check_created_extensions() ; CREATE OR REPLACE FUNCTION public.powa_check_dropped_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- We unregister extensions regardless the "enabled" field WITH ext AS ( SELECT object_name FROM pg_event_trigger_dropped_objects() d WHERE d.object_type = 'extension' ) SELECT function_name INTO funcname FROM powa_functions f JOIN ext ON f.module = ext.object_name WHERE operation = 'unregister'; IF ( funcname IS NOT NULL ) THEN BEGIN PERFORM powa_log(format('running %I', funcname)); EXECUTE 'SELECT ' || quote_ident(funcname) || '()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE WARNING 'powa_check_dropped_extensions(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END IF; END; $_$; /* end of powa_check_dropped_extensions */ CREATE EVENT TRIGGER powa_check_dropped_extensions ON sql_drop WHEN tag IN ('DROP EXTENSION') EXECUTE PROCEDURE public.powa_check_dropped_extensions() ; CREATE OR REPLACE FUNCTION powa_take_snapshot() RETURNS void AS $PROC$ DECLARE purgets timestamp with time zone; purge_seq bigint; funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; v_title text = 'PoWA - '; v_rowcount bigint; BEGIN PERFORM set_config('application_name', v_title || ' snapshot database list', false); PERFORM powa_log('start of powa_take_snapshot'); -- Keep track of existing databases PERFORM powa_log('Maintaining database list...'); WITH missing AS ( SELECT d.oid, d.datname FROM pg_database d LEFT JOIN powa_databases p ON d.oid = p.oid WHERE p.oid IS NULL ) INSERT INTO powa_databases SELECT * FROM missing; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('missing db: %s', v_rowcount)); -- Keep track of renamed databases WITH renamed AS ( SELECT d.oid, d.datname FROM pg_database AS d JOIN powa_databases AS p ON d.oid = p.oid WHERE d.datname != p.datname ) UPDATE powa_databases AS p SET datname = r.datname FROM renamed AS r WHERE p.oid = r.oid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('renamed db: %s', v_rowcount)); -- Keep track of when databases are dropped WITH dropped AS ( SELECT p.oid FROM powa_databases p LEFT JOIN pg_database d ON p.oid = d.oid WHERE d.oid IS NULL AND p.dropped IS NULL) UPDATE powa_databases p SET dropped = now() FROM dropped d WHERE p.oid = d.oid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('dropped db: %s', v_rowcount)); -- For all enabled snapshot functions in the powa_functions table, execute FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='snapshot' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling snapshot function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; -- Coalesce datas if needed SELECT nextval('powa_coalesce_sequence'::regclass) INTO purge_seq; PERFORM powa_log(format('powa_coalesce_sequence: %s', purge_seq)); IF ( purge_seq % current_setting('powa.coalesce')::bigint ) = 0 THEN PERFORM powa_log(format('coalesce needed, seq: %s coalesce seq: %s', purge_seq, current_setting('powa.coalesce')::bigint )); FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='aggregate' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling aggregate function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; UPDATE powa_last_aggregation SET aggts = now(); END IF; -- We also purge, at next pass IF ( purge_seq % (current_setting('powa.coalesce')::bigint ) ) = 1 THEN PERFORM powa_log(format('purge needed, seq: %s coalesce seq: %s', purge_seq, current_setting('powa.coalesce'))); FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='purge' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling purge function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; PERFORM set_config('application_name', v_title || 'UPDATE powa_last_purge', false); UPDATE powa_last_purge SET purgets = now(); END IF; PERFORM powa_log('end of powa_take_snapshot'); PERFORM set_config('application_name', v_title || 'snapshot finished', false); END; $PROC$ LANGUAGE plpgsql; /* end of powa_take_snapshot */ CREATE OR REPLACE FUNCTION powa_statements_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; ignore_regexp text :='^[[:space:]]*(DEALLOCATE|BEGIN|PREPARE TRANSACTION|COMMIT PREPARED|ROLLBACK PREPARED)'; v_funcname text := 'powa_statements_snapshot'; v_rowcount bigint; BEGIN -- In this function, we capture statements, and also aggregate counters by database -- so that the first screens of powa stay reactive even though there may be thousands -- of different statements PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS( SELECT pgss.* FROM pg_stat_statements pgss JOIN pg_roles r ON pgss.userid = r.oid WHERE pgss.query !~* ignore_regexp AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_statements AS( INSERT INTO powa_statements (queryid, dbid, userid, query) SELECT queryid, dbid, userid, query FROM capture c WHERE NOT EXISTS (SELECT 1 FROM powa_statements ps WHERE ps.queryid = c.queryid AND ps.dbid = c.dbid AND ps.userid = c.userid ) ), by_query AS ( INSERT INTO powa_statements_history_current SELECT queryid, dbid, userid, ROW( now(), calls, total_time, rows, shared_blks_hit, shared_blks_read, shared_blks_dirtied, shared_blks_written, local_blks_hit, local_blks_read, local_blks_dirtied, local_blks_written, temp_blks_read, temp_blks_written, blk_read_time, blk_write_time )::powa_statements_history_record AS record FROM capture ), by_database AS ( INSERT INTO powa_statements_history_current_db SELECT dbid, ROW( now(), sum(calls), sum(total_time), sum(rows), sum(shared_blks_hit), sum(shared_blks_read), sum(shared_blks_dirtied), sum(shared_blks_written), sum(local_blks_hit), sum(local_blks_read), sum(local_blks_dirtied), sum(local_blks_written), sum(temp_blks_read), sum(temp_blks_written), sum(blk_read_time), sum(blk_write_time) )::powa_statements_history_record AS record FROM capture GROUP BY dbid ) SELECT count(*) INTO v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; -- For now we don't care. What could we do on error except crash anyway? END; $PROC$ language plpgsql; /* end of powa_statements_snapshot */ CREATE OR REPLACE FUNCTION powa_user_functions_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_user_functions_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide user function statistics WITH func(dbid, r) AS ( SELECT oid, powa_stat_user_functions(oid) FROM pg_database ) INSERT INTO powa_user_functions_history_current SELECT dbid, (r).funcid, ROW(now(), (r).calls, (r).total_time, (r).self_time)::powa_user_functions_history_record AS record FROM func; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_user_functions_snapshot */ CREATE OR REPLACE FUNCTION powa_all_relations_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_all_relations_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide relation statistics WITH rel(dbid, r) AS ( SELECT oid, powa_stat_all_rel(oid) FROM pg_database ) INSERT INTO powa_all_relations_history_current SELECT dbid, (r).relid, ROW(now(),(r).numscan, (r).tup_returned, (r).tup_fetched, (r).n_tup_ins, (r).n_tup_upd, (r).n_tup_del, (r).n_tup_hot_upd, (r).n_liv_tup, (r).n_dead_tup, (r).n_mod_since_analyze, (r).blks_read, (r).blks_hit, (r).last_vacuum, (r).vacuum_count, (r).last_autovacuum, (r).autovacuum_count, (r).last_analyze, (r).analyze_count, (r).last_autoanalyze, (r).autoanalyze_count)::powa_all_relations_history_record AS record FROM rel; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_all_relations_snapshot */ CREATE OR REPLACE FUNCTION powa_statements_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_statements_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_statements_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_hitory) - rowcount: %s', v_funcname, v_rowcount)); DELETE FROM powa_statements_history_db WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history_db) - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_statements_purge */ CREATE OR REPLACE FUNCTION powa_user_functions_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_user_functions_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_user_functions_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_user_functions_purge */ CREATE OR REPLACE FUNCTION powa_all_relations_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_all_relations_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_all_relations_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_all_relations_purge */ CREATE OR REPLACE FUNCTION powa_statements_aggregate() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_statements_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate statements table LOCK TABLE powa_statements_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history SELECT queryid, dbid, userid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current GROUP BY queryid, dbid, userid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_statements_history_current; -- aggregate db table LOCK TABLE powa_statements_history_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history_db SELECT dbid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current_db GROUP BY dbid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history_db) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_statements_history_current_db; END; $PROC$ LANGUAGE plpgsql; /* end of powa_statements_aggregate */ CREATE OR REPLACE FUNCTION powa_user_functions_aggregate() RETURNS void AS $PROC$ BEGIN PERFORM powa_log('running powa_user_functions_aggregate'); -- aggregate user_functions table LOCK TABLE powa_user_functions_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_user_functions_history SELECT dbid, funcid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time), min((record).self_time))::powa_user_functions_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time), max((record).self_time))::powa_user_functions_history_record FROM powa_user_functions_history_current GROUP BY dbid, funcid; TRUNCATE powa_user_functions_history_current; END; $PROC$ LANGUAGE plpgsql; /* end of powa_user_functions_aggregate */ CREATE OR REPLACE FUNCTION powa_all_relations_aggregate() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_all_relations_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate all_relations table LOCK TABLE powa_all_relations_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_all_relations_history SELECT dbid, relid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).numscan),min((record).tup_returned),min((record).tup_fetched), min((record).n_tup_ins),min((record).n_tup_upd), min((record).n_tup_del),min((record).n_tup_hot_upd), min((record).n_liv_tup),min((record).n_dead_tup), min((record).n_mod_since_analyze),min((record).blks_read), min((record).blks_hit),min((record).last_vacuum), min((record).vacuum_count),min((record).last_autovacuum), min((record).autovacuum_count),min((record).last_analyze), min((record).analyze_count),min((record).last_autoanalyze), min((record).autoanalyze_count))::powa_all_relations_history_record, ROW(max((record).ts), max((record).numscan),max((record).tup_returned),max((record).tup_fetched), max((record).n_tup_ins),max((record).n_tup_upd), max((record).n_tup_del),max((record).n_tup_hot_upd), max((record).n_liv_tup),max((record).n_dead_tup), max((record).n_mod_since_analyze),max((record).blks_read), max((record).blks_hit),max((record).last_vacuum), max((record).vacuum_count),max((record).last_autovacuum), max((record).autovacuum_count),max((record).last_analyze), max((record).analyze_count),max((record).last_autoanalyze), max((record).autoanalyze_count))::powa_all_relations_history_record FROM powa_all_relations_history_current GROUP BY dbid, relid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_all_relations_history_current; END; $PROC$ LANGUAGE plpgsql; /* end of powa_all_relations_aggregate */ CREATE OR REPLACE FUNCTION public.powa_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- Find reset function for every supported datasource, including pgss -- Also call reset function even if they're not enabled FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='reset' LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_reset(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; RETURN true; END; $function$; /* end of powa_reset */ CREATE OR REPLACE FUNCTION public.powa_statements_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_statements_history'); TRUNCATE TABLE powa_statements_history; PERFORM powa_log('truncating powa_statements_history_current'); TRUNCATE TABLE powa_statements_history_current; PERFORM powa_log('truncating powa_statements_history_db'); TRUNCATE TABLE powa_statements_history_db; PERFORM powa_log('truncating powa_statements_history_current_db'); TRUNCATE TABLE powa_statements_history_current_db; PERFORM powa_log('truncating powa_statements'); -- if 3rd part datasource has FK on it, throw everything away TRUNCATE TABLE powa_statements CASCADE; RETURN true; END; $function$; /* end of powa_statements_reset */ CREATE OR REPLACE FUNCTION public.powa_user_functions_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_user_functions_history'); TRUNCATE TABLE powa_user_functions_history; PERFORM powa_log('truncating powa_user_functions_history_current'); TRUNCATE TABLE powa_user_functions_history_current; RETURN true; END; $function$; /* end of powa_user_functions_reset */ CREATE OR REPLACE FUNCTION public.powa_all_relations_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_all_relations_history'); TRUNCATE TABLE powa_all_relations_history; PERFORM powa_log('truncating powa_all_relations_history_current'); TRUNCATE TABLE powa_all_relations_history_current; RETURN true; END; $function$; /* end of powa_all_relations_reset */ /* pg_stat_kcache integration - part 2 */ /* * register pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_stat_kcache'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_stat_kcache'; IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_stat_kcache'); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_stat_kcache', 'snapshot', 'powa_kcache_snapshot', false, true), ('pg_stat_kcache', 'aggregate', 'powa_kcache_aggregate', false, true), ('pg_stat_kcache', 'unregister', 'powa_kcache_unregister', false, true), ('pg_stat_kcache', 'purge', 'powa_kcache_purge', false, true), ('pg_stat_kcache', 'reset', 'powa_kcache_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_kcache_register */ /* * unregister pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_stat_kcache'); DELETE FROM public.powa_functions WHERE module = 'pg_stat_kcache'; RETURN true; END; $_$ language plpgsql; /* * powa_kcache snapshot collection. */ CREATE OR REPLACE FUNCTION powa_kcache_snapshot() RETURNS void as $PROC$ DECLARE result bool; v_funcname text := 'powa_kcache_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS ( SELECT * FROM pg_stat_kcache() k JOIN pg_roles r ON r.oid = k.userid WHERE NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), by_query AS ( INSERT INTO powa_kcache_metrics_current (queryid, dbid, userid, metrics) SELECT queryid, dbid, userid, (now(), reads, writes, user_time, system_time)::kcache_type FROM capture ), by_database AS ( INSERT INTO powa_kcache_metrics_current_db (dbid, metrics) SELECT dbid, (now(), sum(reads), sum(writes), sum(user_time), sum(system_time))::kcache_type FROM capture GROUP BY dbid ) SELECT COUNT(*) into v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END $PROC$ language plpgsql; /* end of powa_kcache_unregister */ /* * powa_kcache aggregation */ CREATE OR REPLACE FUNCTION powa_kcache_aggregate() RETURNS void AS $PROC$ DECLARE result bool; v_funcname text := 'powa_kcache_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate metrics table LOCK TABLE powa_kcache_metrics_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics (coalesce_range, queryid, dbid, userid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), queryid, dbid, userid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current GROUP BY queryid, dbid, userid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_kcache_metrics_current; -- aggregate metrics_db table LOCK TABLE powa_kcache_metrics_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics_db (coalesce_range, dbid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), dbid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current_db GROUP BY dbid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics_db) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_kcache_metrics_current_db; END $PROC$ language plpgsql; /* end of powa_kcache_aggregate */ /* * powa_kcache purge */ CREATE OR REPLACE FUNCTION powa_kcache_purge() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_kcache_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); DELETE FROM powa_kcache_metrics WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics) - rowcount: %s', v_funcname, v_rowcount)); DELETE FROM powa_kcache_metrics_db WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics_db) - rowcount: %s', v_funcname, v_rowcount)); END; $PROC$ language plpgsql; /* end of powa_kcache_purge */ /* * powa_kcache reset */ CREATE OR REPLACE FUNCTION powa_kcache_reset() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_kcache_reset'; v_rowcount bigint; BEGIN PERFORM powa_log('running powa_kcache_reset'); PERFORM powa_log('truncating powa_kcache_metrics'); TRUNCATE TABLE powa_kcache_metrics; PERFORM powa_log('truncating powa_kcache_metrics_db'); TRUNCATE TABLE powa_kcache_metrics_db; PERFORM powa_log('truncating powa_kcache_metrics_current'); TRUNCATE TABLE powa_kcache_metrics_current; PERFORM powa_log('truncating powa_kcache_metrics_current_db'); TRUNCATE TABLE powa_kcache_metrics_current_db; END; $PROC$ language plpgsql; /* end of powa_kcache_reset */ -- By default, try to register pg_stat_kcache, in case it's alreay here SELECT * FROM public.powa_kcache_register(); /* end of pg_stat_kcache integration - part 2 */ /* pg_qualstats integration - part 2 */ /* * powa_qualstats_register */ CREATE OR REPLACE function public.powa_qualstats_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_qualstats'; IF ( v_ext_present) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE function_name IN ('powa_qualstats_snapshot', 'powa_qualstats_aggregate', 'powa_qualstats_purge'); IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_qualstats'); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_qualstats', 'snapshot', 'powa_qualstats_snapshot', false, true), ('pg_qualstats', 'aggregate', 'powa_qualstats_aggregate', false, true), ('pg_qualstats', 'unregister', 'powa_qualstats_unregister', false, true), ('pg_qualstats', 'purge', 'powa_qualstats_purge', false, true), ('pg_qualstats', 'reset', 'powa_qualstats_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_qualstats_register */ /* * powa_qualstats utility view for aggregating constvalues */ CREATE OR REPLACE VIEW powa_qualstats_aggregate_constvalues_current AS WITH consts AS ( SELECT qualid, queryid, dbid, userid, min(ts) as mints, max(ts) as maxts, sum(occurences) as occurences, sum(nbfiltered) as nbfiltered, sum(execution_count) as execution_count, constvalues FROM powa_qualstats_constvalues_history_current GROUP BY qualid, queryid, dbid, userid, constvalues ), groups AS ( SELECT qualid, queryid, dbid, userid, tstzrange(min(mints), max(maxts),'[]') FROM consts GROUP BY qualid, queryid, dbid, userid ) SELECT * FROM groups, LATERAL ( SELECT array_agg(constvalues) as mu FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY occurences desc LIMIT 20 ) s ) as mu, LATERAL ( SELECT array_agg(constvalues) as mf FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN execution_count = 0 THEN 0 ELSE nbfiltered / execution_count::numeric END DESC LIMIT 20 ) s ) as mf, LATERAL ( SELECT array_agg(constvalues) as lf FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN execution_count = 0 THEN 0 ELSE nbfiltered / execution_count::numeric END DESC LIMIT 20 ) s ) as lf, LATERAL ( SELECT array_agg(constvalues) as me FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY execution_count desc LIMIT 20 ) s ) as me; CREATE OR REPLACE FUNCTION powa_qualstats_snapshot() RETURNS void as $PROC$ DECLARE result bool; v_funcname text := 'powa_qualstats_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS ( SELECT pgqs.*, s.query FROM pg_qualstats_by_query pgqs JOIN powa_statements s USING(queryid, dbid, userid) JOIN pg_roles r ON s.userid = r.oid AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_quals AS ( INSERT INTO powa_qualstats_quals (qualid, queryid, dbid, userid, quals) SELECT DISTINCT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, array_agg(DISTINCT q::qual_type) FROM capture qs, LATERAL (SELECT (unnest(quals)).*) as q WHERE NOT EXISTS ( SELECT 1 FROM powa_qualstats_quals nh WHERE nh.qualid = qs.qualnodeid AND nh.queryid = qs.queryid AND nh.dbid = qs.dbid AND nh.userid = qs.userid ) GROUP BY qualnodeid, queryid, dbid, userid RETURNING * ), by_qual AS ( INSERT INTO powa_qualstats_quals_history_current (qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered) SELECT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), sum(occurences), sum(execution_count), sum(nbfiltered) FROM capture as qs GROUP BY qualnodeid, qs.queryid, qs.dbid, qs.userid RETURNING * ), by_qual_with_const AS ( INSERT INTO powa_qualstats_constvalues_history_current(qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered, constvalues) SELECT qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), occurences, execution_count, nbfiltered, constvalues FROM capture as qs ) SELECT COUNT(*) into v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; PERFORM pg_qualstats_reset(); END $PROC$ language plpgsql; /* end of powa_qualstats_snapshot */ /* * powa_qualstats aggregate */ CREATE OR REPLACE FUNCTION powa_qualstats_aggregate() RETURNS void AS $PROC$ DECLARE result bool; BEGIN PERFORM powa_log('running powa_qualstats_aggregate'); LOCK TABLE powa_qualstats_constvalues_history_current IN SHARE MODE; LOCK TABLE powa_qualstats_quals_history_current IN SHARE MODE; INSERT INTO powa_qualstats_constvalues_history ( qualid, queryid, dbid, userid, coalesce_range, most_used, most_filtering, least_filtering, most_executed) SELECT * FROM powa_qualstats_aggregate_constvalues_current; INSERT INTO powa_qualstats_quals_history (qualid, queryid, dbid, userid, coalesce_range, records, mins_in_range, maxs_in_range) SELECT qualid, queryid, dbid, userid, tstzrange(min(ts), max(ts),'[]'), array_agg((ts, occurences, execution_count, nbfiltered)::powa_qualstats_history_item), ROW(min(ts), min(occurences), min(execution_count), min(nbfiltered))::powa_qualstats_history_item, ROW(max(ts), max(occurences), max(execution_count), max(nbfiltered))::powa_qualstats_history_item FROM powa_qualstats_quals_history_current GROUP BY qualid, queryid, dbid, userid; TRUNCATE powa_qualstats_constvalues_history_current; TRUNCATE powa_qualstats_quals_history_current; END $PROC$ language plpgsql; /* end of powa_qualstats_aggregate */ /* * powa_qualstats_purge */ CREATE OR REPLACE FUNCTION powa_qualstats_purge() RETURNS void as $PROC$ BEGIN PERFORM powa_log('running powa_qualstats_purge'); DELETE FROM powa_qualstats_constvalues_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); DELETE FROM powa_qualstats_quals_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); END; $PROC$ language plpgsql; /* end of powa_qualstats_purge */ /* * powa_qualstats_reset */ CREATE OR REPLACE FUNCTION powa_qualstats_reset() RETURNS void as $PROC$ BEGIN PERFORM powa_log('running powa_qualstats_reset'); PERFORM powa_log('truncating powa_qualstats_quals'); TRUNCATE TABLE powa_qualstats_quals CASCADE; -- cascaded : -- powa_qualstats_quals_history -- powa_qualstats_quals_history_current -- powa_qualstats_constvalues_history -- powa_qualstats_constvalues_history_current END; $PROC$ language plpgsql; /* end of powa_qualstats_reset */ /* * powa_qualstats_unregister */ CREATE OR REPLACE function public.powa_qualstats_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_qualstats'); DELETE FROM public.powa_functions WHERE module = 'pg_qualstats'; RETURN true; END; $_$ language plpgsql; SELECT * FROM public.powa_qualstats_register(); /* end of pg_qualstats_integration - part 2 */ /* pg_track_settings integration */ CREATE OR REPLACE FUNCTION powa_track_settings_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_track_settings'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_track_settings'; IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_track_settings'); -- This extension handles its own storage, just its snapshot -- function and an unregister function. INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_track_settings', 'snapshot', 'pg_track_settings_snapshot', false, true), ('pg_track_settings', 'unregister', 'powa_track_settings_unregister', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_qualstats_unregister */ CREATE OR REPLACE function public.powa_track_settings_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_track_settings'); DELETE FROM public.powa_functions WHERE module = 'pg_track_settings'; RETURN true; END; $_$ language plpgsql; /* end of powa_track_settings_unregister */ -- By default, try to register pg_track_settings, in case it's alreay here SELECT * FROM public.powa_track_settings_register(); /* end pg_track_settings integration */ powa-archivist-REL_3_2_0/powa--3.1.2--3.2.0.sql000066400000000000000000000264001336072707400202510ustar00rootroot00000000000000-- complain if script is sourced in psql, rather than via CREATE EXTENSION --\echo Use "ALTER EXTENSION powa" to load this file. \quit /* pg_wait_sampling integration - part 1 */ CREATE TYPE public.wait_sampling_type AS ( ts timestamptz, count bigint ); /* pg_wait_sampling operator support */ CREATE TYPE wait_sampling_diff AS ( intvl interval, count bigint ); CREATE OR REPLACE FUNCTION wait_sampling_mi( a wait_sampling_type, b wait_sampling_type) RETURNS wait_sampling_diff AS $_$ DECLARE res wait_sampling_diff; BEGIN res.intvl = a.ts - b.ts; res.count = a.count - b.count; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = wait_sampling_mi, LEFTARG = wait_sampling_type, RIGHTARG = wait_sampling_type ); CREATE TYPE wait_sampling_rate AS ( sec integer, count_per_sec double precision ); CREATE OR REPLACE FUNCTION wait_sampling_div( a wait_sampling_type, b wait_sampling_type) RETURNS wait_sampling_rate AS $_$ DECLARE res wait_sampling_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.count_per_sec = (a.count - b.count)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = wait_sampling_div, LEFTARG = wait_sampling_type, RIGHTARG = wait_sampling_type ); /* end of pg_wait_sampling operator support */ CREATE TABLE public.powa_wait_sampling_history ( coalesce_range tstzrange NOT NULL, queryid bigint NOT NULL, dbid oid NOT NULL, event_type text NOT NULL, event text NOT NULL, records public.wait_sampling_type[] NOT NULL, mins_in_range public.wait_sampling_type NOT NULL, maxs_in_range public.wait_sampling_type NOT NULL, PRIMARY KEY (coalesce_range, queryid, dbid, event_type, event) ); CREATE INDEX ON public.powa_wait_sampling_history (queryid); CREATE TABLE public.powa_wait_sampling_history_db ( coalesce_range tstzrange NOT NULL, dbid oid NOT NULL, event_type text NOT NULL, event text NOT NULL, records public.wait_sampling_type[] NOT NULL, mins_in_range public.wait_sampling_type NOT NULL, maxs_in_range public.wait_sampling_type NOT NULL, PRIMARY KEY (coalesce_range, dbid, event_type, event) ); CREATE TABLE public.powa_wait_sampling_history_current ( queryid bigint NOT NULL, dbid oid NOT NULL, event_type text NOT NULL, event text NOT NULL, record wait_sampling_type NOT NULL ); CREATE TABLE public.powa_wait_sampling_history_current_db ( dbid oid NOT NULL, event_type text NOT NULL, event text NOT NULL, record wait_sampling_type NOT NULL ); /* end of pg_wait_sampling integration - part 1 */ SELECT pg_catalog.pg_extension_config_dump('powa_wait_sampling_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_wait_sampling_history_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_wait_sampling_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_wait_sampling_history_current_db',''); CREATE OR REPLACE FUNCTION public.powa_check_created_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE BEGIN /* We have for now no way for a proper handling of this event, * as we don't have a table with the list of supported extensions. * So just call every powa_*_register() function we know each time an * extension is created. Powa should be in a dedicated database and the * register function handle to be called several time, so it's not critical */ PERFORM public.powa_kcache_register(); PERFORM public.powa_qualstats_register(); PERFORM public.powa_track_settings_register(); PERFORM public.powa_wait_sampling_register(); END; $_$; /* end of powa_check_created_extensions */ /* end pg_track_settings integration */ /* pg_wait_sampling integration - part 2 */ /* * register pg_wait_sampling extension */ CREATE OR REPLACE function public.powa_wait_sampling_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_wait_sampling'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_wait_sampling'; IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_wait_sampling'); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_wait_sampling', 'snapshot', 'powa_wait_sampling_snapshot', false, true), ('pg_wait_sampling', 'aggregate', 'powa_wait_sampling_aggregate', false, true), ('pg_wait_sampling', 'unregister', 'powa_wait_sampling_unregister', false, true), ('pg_wait_sampling', 'purge', 'powa_wait_sampling_purge', false, true), ('pg_wait_sampling', 'reset', 'powa_wait_sampling_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_wait_sampling_register */ /* * unregister pg_wait_sampling extension */ CREATE OR REPLACE function public.powa_wait_sampling_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_wait_sampling'); DELETE FROM public.powa_functions WHERE module = 'pg_wait_sampling'; RETURN true; END; $_$ language plpgsql; /* * powa_wait_sampling snapshot collection. */ CREATE OR REPLACE FUNCTION powa_wait_sampling_snapshot() RETURNS void as $PROC$ DECLARE result bool; v_funcname text := 'powa_wait_sampling_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS ( -- the various background processes report wait events but don't have -- associated queryid. Gather them all under a fake 0 dbid SELECT COALESCE(pgss.dbid, 0) AS dbid, s.event_type, s.event, s.queryid, sum(s.count) as count FROM pg_wait_sampling_profile s -- pg_wait_sampling doesn't offer a per (userid, dbid, queryid) view, -- only per pid, but pid can be reused for different databases or users -- so we cannot deduce db or user from it. However, queryid should be -- unique across differet databases, so we retrieve the dbid this way. LEFT JOIN pg_stat_statements(false) pgss ON pgss.queryid = s.queryid WHERE event_type IS NOT NULL AND event IS NOT NULL GROUP BY pgss.dbid, s.event_type, s.event, s.queryid ), by_query AS ( INSERT INTO powa_wait_sampling_history_current (queryid, dbid, event_type, event, record) SELECT queryid, dbid, event_type, event, (now(), count)::wait_sampling_type FROM capture ), by_database AS ( INSERT INTO powa_wait_sampling_history_current_db (dbid, event_type, event, record) SELECT dbid, event_type, event, (now(), sum(count))::wait_sampling_type FROM capture GROUP BY dbid, event_type, event ) SELECT COUNT(*) into v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END $PROC$ language plpgsql; /* end of powa_wait_sampling_snapshot */ /* * powa_wait_sampling aggregation */ CREATE OR REPLACE FUNCTION powa_wait_sampling_aggregate() RETURNS void AS $PROC$ DECLARE result bool; v_funcname text := 'powa_wait_sampling_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate history table LOCK TABLE powa_wait_sampling_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_wait_sampling_history (coalesce_range, queryid, dbid, event_type, event, records, mins_in_range, maxs_in_range) SELECT tstzrange(min((record).ts), max((record).ts),'[]'), queryid, dbid, event_type, event, array_agg(record), ROW(min((record).ts), min((record).count))::wait_sampling_type, ROW(max((record).ts), max((record).count))::wait_sampling_type FROM powa_wait_sampling_history_current GROUP BY queryid, dbid, event_type, event; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_wait_sampling_history) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_wait_sampling_history_current; -- aggregate history_db table LOCK TABLE powa_wait_sampling_history_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_wait_sampling_history_db (coalesce_range, dbid, event_type, event, records, mins_in_range, maxs_in_range) SELECT tstzrange(min((record).ts), max((record).ts),'[]'), dbid, event_type, event, array_agg(record), ROW(min((record).ts), min((record).count))::wait_sampling_type, ROW(max((record).ts), max((record).count))::wait_sampling_type FROM powa_wait_sampling_history_current_db GROUP BY dbid, event_type, event; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_wait_sampling_history_db) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_wait_sampling_history_current_db; END $PROC$ language plpgsql; /* end of powa_wait_sampling_aggregate */ /* * powa_wait_sampling purge */ CREATE OR REPLACE FUNCTION powa_wait_sampling_purge() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_wait_sampling_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); DELETE FROM powa_wait_sampling_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_wait_sampling_history) - rowcount: %s', v_funcname, v_rowcount)); DELETE FROM powa_wait_sampling_history_db WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_wait_sampling_history_db) - rowcount: %s', v_funcname, v_rowcount)); END; $PROC$ language plpgsql; /* end of powa_wait_sampling_purge */ /* * powa_wait_sampling reset */ CREATE OR REPLACE FUNCTION powa_wait_sampling_reset() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_wait_sampling_reset'; v_rowcount bigint; BEGIN PERFORM powa_log('running powa_wait_sampling_reset'); PERFORM powa_log('truncating powa_wait_sampling_history'); TRUNCATE TABLE powa_wait_sampling_history; PERFORM powa_log('truncating powa_wait_sampling_history_db'); TRUNCATE TABLE powa_wait_sampling_history_db; PERFORM powa_log('truncating powa_wait_sampling_history_current'); TRUNCATE TABLE powa_wait_sampling_history_current; PERFORM powa_log('truncating powa_wait_sampling_history_current_db'); TRUNCATE TABLE powa_wait_sampling_history_current_db; END; $PROC$ language plpgsql; /* end of powa_wait_sampling_reset */ -- By default, try to register pg_wait_sampling, in case it's alreay here SELECT * FROM public.powa_wait_sampling_register(); /* end of pg_wait_sampling integration - part 2 */ powa-archivist-REL_3_2_0/powa--3.1.2.sql000066400000000000000000002124711336072707400175430ustar00rootroot00000000000000-- complain if script is sourced in psql, rather than via CREATE EXTENSION --\echo Use "CREATE EXTENSION powa" to load this file. \quit SET statement_timeout = 0; SET client_encoding = 'UTF8'; SET standard_conforming_strings = on; SET client_min_messages = warning; SET escape_string_warning = off; SET search_path = public, pg_catalog; CREATE TABLE powa_databases( oid oid PRIMARY KEY, datname name, dropped timestamp with time zone ); CREATE FUNCTION powa_stat_user_functions(IN dbid oid, OUT funcid oid, OUT calls bigint, OUT total_time double precision, OUT self_time double precision) RETURNS SETOF record LANGUAGE c COST 100 AS '$libdir/powa', 'powa_stat_user_functions'; CREATE FUNCTION powa_stat_all_rel(IN dbid oid, OUT relid oid, OUT numscan bigint, OUT tup_returned bigint, OUT tup_fetched bigint, OUT n_tup_ins bigint, OUT n_tup_upd bigint, OUT n_tup_del bigint, OUT n_tup_hot_upd bigint, OUT n_liv_tup bigint, OUT n_dead_tup bigint, OUT n_mod_since_analyze bigint, OUT blks_read bigint, OUT blks_hit bigint, OUT last_vacuum timestamp with time zone, OUT vacuum_count bigint, OUT last_autovacuum timestamp with time zone, OUT autovacuum_count bigint, OUT last_analyze timestamp with time zone, OUT analyze_count bigint, OUT last_autoanalyze timestamp with time zone, OUT autoanalyze_count bigint) RETURNS SETOF record LANGUAGE c COST 100 AS '$libdir/powa', 'powa_stat_all_rel'; CREATE TYPE powa_statements_history_record AS ( ts timestamp with time zone, calls bigint, total_time double precision, rows bigint, shared_blks_hit bigint, shared_blks_read bigint, shared_blks_dirtied bigint, shared_blks_written bigint, local_blks_hit bigint, local_blks_read bigint, local_blks_dirtied bigint, local_blks_written bigint, temp_blks_read bigint, temp_blks_written bigint, blk_read_time double precision, blk_write_time double precision ); /* pg_stat_statements operator support */ CREATE TYPE powa_statements_history_diff AS ( intvl interval, calls bigint, total_time double precision, rows bigint, shared_blks_hit bigint, shared_blks_read bigint, shared_blks_dirtied bigint, shared_blks_written bigint, local_blks_hit bigint, local_blks_read bigint, local_blks_dirtied bigint, local_blks_written bigint, temp_blks_read bigint, temp_blks_written bigint, blk_read_time double precision, blk_write_time double precision ); CREATE OR REPLACE FUNCTION powa_statements_history_mi( a powa_statements_history_record, b powa_statements_history_record) RETURNS powa_statements_history_diff AS $_$ DECLARE res powa_statements_history_diff; BEGIN res.intvl = a.ts - b.ts; res.calls = a.calls - b.calls; res.total_time = a.total_time - b.total_time; res.rows = a.rows - b.rows; res.shared_blks_hit = a.shared_blks_hit - b.shared_blks_hit; res.shared_blks_read = a.shared_blks_read - b.shared_blks_read; res.shared_blks_dirtied = a.shared_blks_dirtied - b.shared_blks_dirtied; res.shared_blks_written = a.shared_blks_written - b.shared_blks_written; res.local_blks_hit = a.local_blks_hit - b.local_blks_hit; res.local_blks_read = a.local_blks_read - b.local_blks_read; res.local_blks_dirtied = a.local_blks_dirtied - b.local_blks_dirtied; res.local_blks_written = a.local_blks_written - b.local_blks_written; res.temp_blks_read = a.temp_blks_read - b.temp_blks_read; res.temp_blks_written = a.temp_blks_written - b.temp_blks_written; res.blk_read_time = a.blk_read_time - b.blk_read_time; res.blk_write_time = a.blk_write_time - b.blk_write_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_statements_history_mi, LEFTARG = powa_statements_history_record, RIGHTARG = powa_statements_history_record ); CREATE TYPE powa_statements_history_rate AS ( sec integer, calls_per_sec double precision, runtime_per_sec double precision, rows_per_sec double precision, shared_blks_hit_per_sec double precision, shared_blks_read_per_sec double precision, shared_blks_dirtied_per_sec double precision, shared_blks_written_per_sec double precision, local_blks_hit_per_sec double precision, local_blks_read_per_sec double precision, local_blks_dirtied_per_sec double precision, local_blks_written_per_sec double precision, temp_blks_read_per_sec double precision, temp_blks_written_per_sec double precision, blk_read_time_per_sec double precision, blk_write_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_statements_history_div( a powa_statements_history_record, b powa_statements_history_record) RETURNS powa_statements_history_rate AS $_$ DECLARE res powa_statements_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.calls_per_sec = (a.calls - b.calls)::double precision / sec; res.runtime_per_sec = (a.total_time - b.total_time)::double precision / sec; res.rows_per_sec = (a.rows - b.rows)::double precision / sec; res.shared_blks_hit_per_sec = (a.shared_blks_hit - b.shared_blks_hit)::double precision / sec; res.shared_blks_read_per_sec = (a.shared_blks_read - b.shared_blks_read)::double precision / sec; res.shared_blks_dirtied_per_sec = (a.shared_blks_dirtied - b.shared_blks_dirtied)::double precision / sec; res.shared_blks_written_per_sec = (a.shared_blks_written - b.shared_blks_written)::double precision / sec; res.local_blks_hit_per_sec = (a.local_blks_hit - b.local_blks_hit)::double precision / sec; res.local_blks_read_per_sec = (a.local_blks_read - b.local_blks_read)::double precision / sec; res.local_blks_dirtied_per_sec = (a.local_blks_dirtied - b.local_blks_dirtied)::double precision / sec; res.local_blks_written_per_sec = (a.local_blks_written - b.local_blks_written)::double precision / sec; res.temp_blks_read_per_sec = (a.temp_blks_read - b.temp_blks_read)::double precision / sec; res.temp_blks_written_per_sec = (a.temp_blks_written - b.temp_blks_written)::double precision / sec; res.blk_read_time_per_sec = (a.blk_read_time - b.blk_read_time)::double precision / sec; res.blk_write_time_per_sec = (a.blk_write_time - b.blk_write_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_statements_history_div, LEFTARG = powa_statements_history_record, RIGHTARG = powa_statements_history_record ); /* end of pg_stat_statements operator support */ CREATE TYPE powa_user_functions_history_record AS ( ts timestamp with time zone, calls bigint, total_time double precision, self_time double precision ); /* pg_stat_user_functions operator support */ CREATE TYPE powa_user_functions_history_diff AS ( intvl interval, calls bigint, total_time double precision, self_time double precision ); CREATE OR REPLACE FUNCTION powa_user_functions_history_mi( a powa_user_functions_history_record, b powa_user_functions_history_record) RETURNS powa_user_functions_history_diff AS $_$ DECLARE res powa_user_functions_history_diff; BEGIN res.intvl = a.ts - b.ts; res.calls = a.calls - b.calls; res.total_time = a.total_time - b.total_time; res.self_time = a.self_time - b.self_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_user_functions_history_mi, LEFTARG = powa_user_functions_history_record, RIGHTARG = powa_user_functions_history_record ); CREATE TYPE powa_user_functions_history_rate AS ( sec integer, calls_per_sec double precision, total_time_per_sec double precision, self_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_user_functions_history_div( a powa_user_functions_history_record, b powa_user_functions_history_record) RETURNS powa_user_functions_history_rate AS $_$ DECLARE res powa_user_functions_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.calls_per_sec = (a.calls - b.calls)::double precision / sec; res.total_time_per_sec = (a.total_time - b.total_time)::double precision / sec; res.self_time_per_sec = (a.self_time - b.self_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_user_functions_history_div, LEFTARG = powa_user_functions_history_record, RIGHTARG = powa_user_functions_history_record ); /* end of pg_stat_user_functions operator support */ CREATE TYPE powa_all_relations_history_record AS ( ts timestamp with time zone, numscan bigint, tup_returned bigint, tup_fetched bigint, n_tup_ins bigint, n_tup_upd bigint, n_tup_del bigint, n_tup_hot_upd bigint, n_liv_tup bigint, n_dead_tup bigint, n_mod_since_analyze bigint, blks_read bigint, blks_hit bigint, last_vacuum timestamp with time zone, vacuum_count bigint, last_autovacuum timestamp with time zone, autovacuum_count bigint, last_analyze timestamp with time zone, analyze_count bigint, last_autoanalyze timestamp with time zone, autoanalyze_count bigint ); /* pg_stat_all_relations operator support */ CREATE TYPE powa_all_relations_history_diff AS ( intvl interval, numscan bigint, tup_returned bigint, tup_fetched bigint, n_tup_ins bigint, n_tup_upd bigint, n_tup_del bigint, n_tup_hot_upd bigint, n_liv_tup bigint, n_dead_tup bigint, n_mod_since_analyze bigint, blks_read bigint, blks_hit bigint, vacuum_count bigint, autovacuum_count bigint, analyze_count bigint, autoanalyze_count bigint ); CREATE OR REPLACE FUNCTION powa_all_relations_history_mi( a powa_all_relations_history_record, b powa_all_relations_history_record) RETURNS powa_all_relations_history_diff AS $_$ DECLARE res powa_all_relations_history_diff; BEGIN res.intvl = a.ts - b.ts; res.numscan = a.numscan - b.numscan; res.tup_returned = a.tup_returned - b.tup_returned; res.tup_fetched = a.tup_fetched - b.tup_fetched; res.n_tup_ins = a.n_tup_ins - b.n_tup_ins; res.n_tup_upd = a.n_tup_upd - b.n_tup_upd; res.n_tup_del = a.n_tup_del - b.n_tup_del; res.n_tup_hot_upd = a.n_tup_hot_upd - b.n_tup_hot_upd; res.n_liv_tup = a.n_liv_tup - b.n_liv_tup; res.n_dead_tup = a.n_dead_tup - b.n_dead_tup; res.n_mod_since_analyze = a.n_mod_since_analyze - b.n_mod_since_analyze; res.blks_read = a.blks_read - b.blks_read; res.blks_hit = a.blks_hit - b.blks_hit; res.vacuum_count = a.vacuum_count - b.vacuum_count; res.autovacuum_count = a.autovacuum_count - b.autovacuum_count; res.analyze_count = a.analyze_count - b.analyze_count; res.autoanalyze_count = a.autoanalyze_count - b.autoanalyze_count; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_all_relations_history_mi, LEFTARG = powa_all_relations_history_record, RIGHTARG = powa_all_relations_history_record ); CREATE TYPE powa_all_relations_history_rate AS ( sec integer, numscan_per_sec double precision, tup_returned_per_sec double precision, tup_fetched_per_sec double precision, n_tup_ins_per_sec double precision, n_tup_upd_per_sec double precision, n_tup_del_per_sec double precision, n_tup_hot_upd_per_sec double precision, n_liv_tup_per_sec double precision, n_dead_tup_per_sec double precision, n_mod_since_analyze_per_sec double precision, blks_read_per_sec double precision, blks_hit_per_sec double precision, vacuum_count_per_sec double precision, autovacuum_count_per_sec double precision, analyze_count_per_sec double precision, autoanalyze_count_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_all_relations_history_div( a powa_all_relations_history_record, b powa_all_relations_history_record) RETURNS powa_all_relations_history_rate AS $_$ DECLARE res powa_all_relations_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.numscan_per_sec = (a.numscan - b.numscan)::double precision / sec; res.tup_returned_per_sec = (a.tup_returned - b.tup_returned)::double precision / sec; res.tup_fetched_per_sec = (a.tup_fetched - b.tup_fetched)::double precision / sec; res.n_tup_ins_per_sec = (a.n_tup_ins - b.n_tup_ins)::double precision / sec; res.n_tup_upd_per_sec = (a.n_tup_upd - b.n_tup_upd)::double precision / sec; res.n_tup_del_per_sec = (a.n_tup_del - b.n_tup_del)::double precision / sec; res.n_tup_hot_upd_per_sec = (a.n_tup_hot_upd - b.n_tup_hot_upd)::double precision / sec; res.n_liv_tup_per_sec = (a.n_liv_tup - b.n_liv_tup)::double precision / sec; res.n_dead_tup_per_sec = (a.n_dead_tup - b.n_dead_tup)::double precision / sec; res.n_mod_since_analyze_per_sec = (a.n_mod_since_analyze - b.n_mod_since_analyze)::double precision / sec; res.blks_read_per_sec = (a.blks_read - b.blks_read)::double precision / sec; res.blks_hit_per_sec = (a.blks_hit - b.blks_hit)::double precision / sec; res.vacuum_count_per_sec = (a.vacuum_count - b.vacuum_count)::double precision / sec; res.autovacuum_count_per_sec = (a.autovacuum_count - b.autovacuum_count)::double precision / sec; res.analyze_count_per_sec = (a.analyze_count - b.analyze_count)::double precision / sec; res.autoanalyze_count_per_sec = (a.autoanalyze_count - b.autoanalyze_count)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_all_relations_history_div, LEFTARG = powa_all_relations_history_record, RIGHTARG = powa_all_relations_history_record ); /* end of pg_stat_all_relations operator support */ CREATE TABLE powa_last_aggregation ( aggts timestamp with time zone ); INSERT INTO powa_last_aggregation(aggts) VALUES (current_timestamp); CREATE TABLE powa_last_purge ( purgets timestamp with time zone ); INSERT INTO powa_last_purge (purgets) VALUES (current_timestamp); CREATE TABLE powa_statements ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, query text NOT NULL ); ALTER TABLE ONLY powa_statements ADD CONSTRAINT powa_statements_pkey PRIMARY KEY (queryid, dbid, userid); CREATE INDEX powa_statements_dbid_idx ON powa_statements(dbid); CREATE INDEX powa_statements_userid_idx ON powa_statements(userid); CREATE TABLE powa_statements_history ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statements_history_record[] NOT NULL, mins_in_range powa_statements_history_record NOT NULL, maxs_in_range powa_statements_history_record NOT NULL ); CREATE INDEX powa_statements_history_query_ts ON powa_statements_history USING gist (queryid, coalesce_range); CREATE TABLE powa_statements_history_db ( dbid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statements_history_record[] NOT NULL, mins_in_range powa_statements_history_record NOT NULL, maxs_in_range powa_statements_history_record NOT NULL ); CREATE INDEX powa_statements_history_db_ts ON powa_statements_history_db USING gist (dbid, coalesce_range); CREATE TABLE powa_statements_history_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, record powa_statements_history_record NOT NULL ); CREATE TABLE powa_statements_history_current_db ( dbid oid NOT NULL, record powa_statements_history_record NOT NULL ); CREATE TABLE powa_user_functions_history ( dbid oid NOT NULL, funcid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_user_functions_history_record[] NOT NULL, mins_in_range powa_user_functions_history_record NOT NULL, maxs_in_range powa_user_functions_history_record NOT NULL ); CREATE INDEX powa_user_functions_history_funcid_ts ON powa_user_functions_history USING gist (funcid, coalesce_range); CREATE TABLE powa_user_functions_history_current ( dbid oid NOT NULL, funcid oid NOT NULL, record powa_user_functions_history_record NOT NULL ); CREATE TABLE powa_all_relations_history ( dbid oid NOT NULL, relid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_all_relations_history_record[] NOT NULL, mins_in_range powa_all_relations_history_record NOT NULL, maxs_in_range powa_all_relations_history_record NOT NULL ); CREATE INDEX powa_all_relations_history_relid_ts ON powa_all_relations_history USING gist (relid, coalesce_range); CREATE TABLE powa_all_relations_history_current ( dbid oid NOT NULL, relid oid NOT NULL, record powa_all_relations_history_record NOT NULL ); CREATE SEQUENCE powa_coalesce_sequence INCREMENT BY 1 START WITH 1 CYCLE; CREATE TABLE powa_functions ( module text NOT NULL, operation text NOT NULL, function_name text NOT NULL, added_manually boolean NOT NULL default true, enabled boolean NOT NULL default true, CHECK (operation IN ('snapshot','aggregate','purge','unregister','reset')) ); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_stat_statements', 'snapshot', 'powa_statements_snapshot', false, true), ('powa_stat_user_functions', 'snapshot', 'powa_user_functions_snapshot', false, true), ('powa_stat_all_relations', 'snapshot', 'powa_all_relations_snapshot', false, true), ('pg_stat_statements', 'aggregate','powa_statements_aggregate', false, true), ('powa_stat_user_functions', 'aggregate','powa_user_functions_aggregate', false, true), ('powa_stat_all_relations', 'aggregate','powa_all_relations_aggregate', false, true), ('pg_stat_statements', 'purge', 'powa_statements_purge', false, true), ('powa_stat_user_functions', 'purge', 'powa_user_functions_purge', false, true), ('powa_stat_all_relations', 'purge', 'powa_all_relations_purge', false, true), ('pg_stat_statements', 'reset', 'powa_statements_reset', false, true), ('powa_stat_user_functions', 'reset', 'powa_user_functions_reset', false, true), ('powa_stat_all_relations', 'reset', 'powa_all_relations_reset', false, true); CREATE FUNCTION powa_log (msg text) RETURNS void LANGUAGE plpgsql AS $_$ BEGIN IF current_setting('powa.debug')::bool THEN RAISE WARNING '%', msg; ELSE RAISE DEBUG '%', msg; END IF; END; $_$; /* pg_stat_kcache integration - part 1 */ CREATE TYPE public.kcache_type AS ( ts timestamptz, reads bigint, writes bigint, user_time double precision, system_time double precision ); /* pg_stat_kcache operator support */ CREATE TYPE powa_kcache_diff AS ( intvl interval, reads bigint, writes bigint, user_time double precision, system_time double precision ); CREATE OR REPLACE FUNCTION powa_kcache_mi( a kcache_type, b kcache_type) RETURNS powa_kcache_diff AS $_$ DECLARE res powa_kcache_diff; BEGIN res.intvl = a.ts - b.ts; res.reads = a.reads - b.reads; res.writes = a.writes - b.writes; res.user_time = a.user_time - b.user_time; res.system_time = a.system_time - b.system_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_kcache_mi, LEFTARG = kcache_type, RIGHTARG = kcache_type ); CREATE TYPE powa_kcache_rate AS ( sec integer, reads_per_sec double precision, writes_per_sec double precision, user_time_per_sec double precision, system_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_kcache_div( a kcache_type, b kcache_type) RETURNS powa_kcache_rate AS $_$ DECLARE res powa_kcache_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.reads_per_sec = (a.reads - b.reads)::double precision / sec; res.writes_per_sec = (a.writes - b.writes)::double precision / sec; res.user_time_per_sec = (a.user_time - b.user_time)::double precision / sec; res.system_time_per_sec = (a.system_time - b.system_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_kcache_div, LEFTARG = kcache_type, RIGHTARG = kcache_type ); /* end of pg_stat_kcache operator support */ CREATE TABLE public.powa_kcache_metrics ( coalesce_range tstzrange NOT NULL, queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics public.kcache_type[] NOT NULL, mins_in_range public.kcache_type NOT NULL, maxs_in_range public.kcache_type NOT NULL, PRIMARY KEY (coalesce_range, queryid, dbid, userid) ); CREATE INDEX ON public.powa_kcache_metrics (queryid); CREATE TABLE public.powa_kcache_metrics_db ( coalesce_range tstzrange NOT NULL, dbid oid NOT NULL, metrics public.kcache_type[] NOT NULL, mins_in_range public.kcache_type NOT NULL, maxs_in_range public.kcache_type NOT NULL, PRIMARY KEY (coalesce_range, dbid) ); CREATE TABLE public.powa_kcache_metrics_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics kcache_type NULL NULL ); CREATE TABLE public.powa_kcache_metrics_current_db ( dbid oid NOT NULL, metrics kcache_type NULL NULL ); /* end of pg_stat_kcache integration - part 1 */ /* pg_qualstats integration - part 1 */ CREATE TYPE public.qual_type AS ( relid oid, attnum integer, opno oid, eval_type "char" ); CREATE TYPE public.qual_values AS ( constants text[], occurences bigint, execution_count bigint, nbfiltered bigint ); CREATE TYPE powa_qualstats_history_item AS ( ts timestamptz, occurences bigint, execution_count bigint, nbfiltered bigint ); /* pg_stat_qualstats operator support */ CREATE TYPE powa_qualstats_history_diff AS ( intvl interval, occurences bigint, execution_count bigint, nbfiltered bigint ); CREATE OR REPLACE FUNCTION powa_qualstats_history_mi( a powa_qualstats_history_item, b powa_qualstats_history_item) RETURNS powa_qualstats_history_diff AS $_$ DECLARE res powa_qualstats_history_diff; BEGIN res.intvl = a.ts - b.ts; res.occurences = a.occurences - b.occurences; res.execution_count = a.execution_count - b.execution_count; res.nbfiltered = a.nbfiltered - b.nbfiltered; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_qualstats_history_mi, LEFTARG = powa_qualstats_history_item, RIGHTARG = powa_qualstats_history_item ); CREATE TYPE powa_qualstats_history_rate AS ( sec integer, occurences_per_sec double precision, execution_count_per_sec double precision, nbfiltered_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_qualstats_history_div( a powa_qualstats_history_item, b powa_qualstats_history_item) RETURNS powa_qualstats_history_rate AS $_$ DECLARE res powa_qualstats_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.occurences_per_sec = (a.occurences - b.occurences)::double precision / sec; res.execution_count_per_sec = (a.execution_count - b.execution_count)::double precision / sec; res.nbfiltered_per_sec = (a.nbfiltered - b.nbfiltered)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_qualstats_history_div, LEFTARG = powa_qualstats_history_item, RIGHTARG = powa_qualstats_history_item ); /* end of pg_stat_qualstats operator support */ CREATE TABLE public.powa_qualstats_quals ( qualid bigint, queryid bigint, dbid oid, userid oid, quals public.qual_type[], PRIMARY KEY (qualid, queryid, dbid, userid), FOREIGN KEY (queryid, dbid, userid) REFERENCES powa_statements(queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, records powa_qualstats_history_item[], mins_in_range powa_qualstats_history_item, maxs_in_range powa_qualstats_history_item, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, occurences bigint, execution_count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES powa_qualstats_quals(qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, most_used qual_values[], most_filtering qual_values[], least_filtering qual_values[], most_executed qual_values[], FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, constvalues text[], occurences bigint, execution_count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE INDEX ON powa_qualstats_constvalues_history USING gist (queryid, qualid, coalesce_range); CREATE INDEX ON powa_qualstats_constvalues_history (qualid, queryid); CREATE INDEX ON powa_qualstats_quals(queryid); /* end of pg_qualstats_integration - part 1 */ -- Mark all of powa's tables as "to be dumped" SELECT pg_catalog.pg_extension_config_dump('powa_statements',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_user_functions_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_user_functions_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_all_relations_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_all_relations_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_functions','WHERE added_manually'); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history_current',''); CREATE OR REPLACE FUNCTION public.powa_check_created_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE BEGIN /* We have for now no way for a proper handling of this event, * as we don't have a table with the list of supported extensions. * So just call every powa_*_register() function we know each time an * extension is created. Powa should be in a dedicated database and the * register function handle to be called several time, so it's not critical */ PERFORM public.powa_kcache_register(); PERFORM public.powa_qualstats_register(); PERFORM public.powa_track_settings_register(); END; $_$; CREATE EVENT TRIGGER powa_check_created_extensions ON ddl_command_end WHEN tag IN ('CREATE EXTENSION') EXECUTE PROCEDURE public.powa_check_created_extensions() ; CREATE OR REPLACE FUNCTION public.powa_check_dropped_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- We unregister extensions regardless the "enabled" field WITH ext AS ( SELECT object_name FROM pg_event_trigger_dropped_objects() d WHERE d.object_type = 'extension' ) SELECT function_name INTO funcname FROM powa_functions f JOIN ext ON f.module = ext.object_name WHERE operation = 'unregister'; IF ( funcname IS NOT NULL ) THEN BEGIN PERFORM powa_log(format('running %I', funcname)); EXECUTE 'SELECT ' || quote_ident(funcname) || '()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE WARNING 'powa_check_dropped_extensions(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END IF; END; $_$; /* end of powa_check_dropped_extensions */ CREATE EVENT TRIGGER powa_check_dropped_extensions ON sql_drop WHEN tag IN ('DROP EXTENSION') EXECUTE PROCEDURE public.powa_check_dropped_extensions() ; CREATE OR REPLACE FUNCTION powa_take_snapshot() RETURNS void AS $PROC$ DECLARE purgets timestamp with time zone; purge_seq bigint; funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; v_title text = 'PoWA - '; v_rowcount bigint; BEGIN PERFORM set_config('application_name', v_title || ' snapshot database list', false); PERFORM powa_log('start of powa_take_snapshot'); -- Keep track of existing databases PERFORM powa_log('Maintaining database list...'); WITH missing AS ( SELECT d.oid, d.datname FROM pg_database d LEFT JOIN powa_databases p ON d.oid = p.oid WHERE p.oid IS NULL ) INSERT INTO powa_databases SELECT * FROM missing; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('missing db: %s', v_rowcount)); -- Keep track of renamed databases WITH renamed AS ( SELECT d.oid, d.datname FROM pg_database AS d JOIN powa_databases AS p ON d.oid = p.oid WHERE d.datname != p.datname ) UPDATE powa_databases AS p SET datname = r.datname FROM renamed AS r WHERE p.oid = r.oid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('renamed db: %s', v_rowcount)); -- Keep track of when databases are dropped WITH dropped AS ( SELECT p.oid FROM powa_databases p LEFT JOIN pg_database d ON p.oid = d.oid WHERE d.oid IS NULL AND p.dropped IS NULL) UPDATE powa_databases p SET dropped = now() FROM dropped d WHERE p.oid = d.oid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('dropped db: %s', v_rowcount)); -- For all enabled snapshot functions in the powa_functions table, execute FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='snapshot' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling snapshot function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; -- Coalesce datas if needed SELECT nextval('powa_coalesce_sequence'::regclass) INTO purge_seq; PERFORM powa_log(format('powa_coalesce_sequence: %s', purge_seq)); IF ( purge_seq % current_setting('powa.coalesce')::bigint ) = 0 THEN PERFORM powa_log(format('coalesce needed, seq: %s coalesce seq: %s', purge_seq, current_setting('powa.coalesce')::bigint )); FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='aggregate' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling aggregate function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; UPDATE powa_last_aggregation SET aggts = now(); END IF; -- We also purge, at next pass IF ( purge_seq % (current_setting('powa.coalesce')::bigint ) ) = 1 THEN PERFORM powa_log(format('purge needed, seq: %s coalesce seq: %s', purge_seq, current_setting('powa.coalesce'))); FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='purge' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling purge function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; PERFORM set_config('application_name', v_title || 'UPDATE powa_last_purge', false); UPDATE powa_last_purge SET purgets = now(); END IF; PERFORM powa_log('end of powa_take_snapshot'); PERFORM set_config('application_name', v_title || 'snapshot finished', false); END; $PROC$ LANGUAGE plpgsql; /* end of powa_take_snapshot */ CREATE OR REPLACE FUNCTION powa_statements_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; ignore_regexp text :='^[[:space:]]*(DEALLOCATE|BEGIN|PREPARE TRANSACTION|COMMIT PREPARED|ROLLBACK PREPARED)'; v_funcname text := 'powa_statements_snapshot'; v_rowcount bigint; BEGIN -- In this function, we capture statements, and also aggregate counters by database -- so that the first screens of powa stay reactive even though there may be thousands -- of different statements PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS( SELECT pgss.* FROM pg_stat_statements pgss JOIN pg_roles r ON pgss.userid = r.oid WHERE pgss.query !~* ignore_regexp AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_statements AS( INSERT INTO powa_statements (queryid, dbid, userid, query) SELECT queryid, dbid, userid, query FROM capture c WHERE NOT EXISTS (SELECT 1 FROM powa_statements ps WHERE ps.queryid = c.queryid AND ps.dbid = c.dbid AND ps.userid = c.userid ) ), by_query AS ( INSERT INTO powa_statements_history_current SELECT queryid, dbid, userid, ROW( now(), calls, total_time, rows, shared_blks_hit, shared_blks_read, shared_blks_dirtied, shared_blks_written, local_blks_hit, local_blks_read, local_blks_dirtied, local_blks_written, temp_blks_read, temp_blks_written, blk_read_time, blk_write_time )::powa_statements_history_record AS record FROM capture ), by_database AS ( INSERT INTO powa_statements_history_current_db SELECT dbid, ROW( now(), sum(calls), sum(total_time), sum(rows), sum(shared_blks_hit), sum(shared_blks_read), sum(shared_blks_dirtied), sum(shared_blks_written), sum(local_blks_hit), sum(local_blks_read), sum(local_blks_dirtied), sum(local_blks_written), sum(temp_blks_read), sum(temp_blks_written), sum(blk_read_time), sum(blk_write_time) )::powa_statements_history_record AS record FROM capture GROUP BY dbid ) SELECT count(*) INTO v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; -- For now we don't care. What could we do on error except crash anyway? END; $PROC$ language plpgsql; /* end of powa_statements_snapshot */ CREATE OR REPLACE FUNCTION powa_user_functions_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_user_functions_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide user function statistics WITH func(dbid, r) AS ( SELECT oid, powa_stat_user_functions(oid) FROM pg_database ) INSERT INTO powa_user_functions_history_current SELECT dbid, (r).funcid, ROW(now(), (r).calls, (r).total_time, (r).self_time)::powa_user_functions_history_record AS record FROM func; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_user_functions_snapshot */ CREATE OR REPLACE FUNCTION powa_all_relations_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_all_relations_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide relation statistics WITH rel(dbid, r) AS ( SELECT oid, powa_stat_all_rel(oid) FROM pg_database ) INSERT INTO powa_all_relations_history_current SELECT dbid, (r).relid, ROW(now(),(r).numscan, (r).tup_returned, (r).tup_fetched, (r).n_tup_ins, (r).n_tup_upd, (r).n_tup_del, (r).n_tup_hot_upd, (r).n_liv_tup, (r).n_dead_tup, (r).n_mod_since_analyze, (r).blks_read, (r).blks_hit, (r).last_vacuum, (r).vacuum_count, (r).last_autovacuum, (r).autovacuum_count, (r).last_analyze, (r).analyze_count, (r).last_autoanalyze, (r).autoanalyze_count)::powa_all_relations_history_record AS record FROM rel; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_all_relations_snapshot */ CREATE OR REPLACE FUNCTION powa_statements_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_statements_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_statements_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_hitory) - rowcount: %s', v_funcname, v_rowcount)); DELETE FROM powa_statements_history_db WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history_db) - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_statements_purge */ CREATE OR REPLACE FUNCTION powa_user_functions_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_user_functions_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_user_functions_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_user_functions_purge */ CREATE OR REPLACE FUNCTION powa_all_relations_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_all_relations_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_all_relations_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_all_relations_purge */ CREATE OR REPLACE FUNCTION powa_statements_aggregate() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_statements_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate statements table LOCK TABLE powa_statements_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history SELECT queryid, dbid, userid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current GROUP BY queryid, dbid, userid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_statements_history_current; -- aggregate db table LOCK TABLE powa_statements_history_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history_db SELECT dbid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current_db GROUP BY dbid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history_db) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_statements_history_current_db; END; $PROC$ LANGUAGE plpgsql; /* end of powa_statements_aggregate */ CREATE OR REPLACE FUNCTION powa_user_functions_aggregate() RETURNS void AS $PROC$ BEGIN PERFORM powa_log('running powa_user_functions_aggregate'); -- aggregate user_functions table LOCK TABLE powa_user_functions_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_user_functions_history SELECT dbid, funcid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time), min((record).self_time))::powa_user_functions_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time), max((record).self_time))::powa_user_functions_history_record FROM powa_user_functions_history_current GROUP BY dbid, funcid; TRUNCATE powa_user_functions_history_current; END; $PROC$ LANGUAGE plpgsql; /* end of powa_user_functions_aggregate */ CREATE OR REPLACE FUNCTION powa_all_relations_aggregate() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_all_relations_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate all_relations table LOCK TABLE powa_all_relations_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_all_relations_history SELECT dbid, relid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).numscan),min((record).tup_returned),min((record).tup_fetched), min((record).n_tup_ins),min((record).n_tup_upd), min((record).n_tup_del),min((record).n_tup_hot_upd), min((record).n_liv_tup),min((record).n_dead_tup), min((record).n_mod_since_analyze),min((record).blks_read), min((record).blks_hit),min((record).last_vacuum), min((record).vacuum_count),min((record).last_autovacuum), min((record).autovacuum_count),min((record).last_analyze), min((record).analyze_count),min((record).last_autoanalyze), min((record).autoanalyze_count))::powa_all_relations_history_record, ROW(max((record).ts), max((record).numscan),max((record).tup_returned),max((record).tup_fetched), max((record).n_tup_ins),max((record).n_tup_upd), max((record).n_tup_del),max((record).n_tup_hot_upd), max((record).n_liv_tup),max((record).n_dead_tup), max((record).n_mod_since_analyze),max((record).blks_read), max((record).blks_hit),max((record).last_vacuum), max((record).vacuum_count),max((record).last_autovacuum), max((record).autovacuum_count),max((record).last_analyze), max((record).analyze_count),max((record).last_autoanalyze), max((record).autoanalyze_count))::powa_all_relations_history_record FROM powa_all_relations_history_current GROUP BY dbid, relid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_all_relations_history_current; END; $PROC$ LANGUAGE plpgsql; /* end of powa_all_relations_aggregate */ CREATE OR REPLACE FUNCTION public.powa_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- Find reset function for every supported datasource, including pgss -- Also call reset function even if they're not enabled FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='reset' LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_reset(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; RETURN true; END; $function$; /* end of powa_reset */ CREATE OR REPLACE FUNCTION public.powa_statements_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_statements_history'); TRUNCATE TABLE powa_statements_history; PERFORM powa_log('truncating powa_statements_history_current'); TRUNCATE TABLE powa_statements_history_current; PERFORM powa_log('truncating powa_statements_history_db'); TRUNCATE TABLE powa_statements_history_db; PERFORM powa_log('truncating powa_statements_history_current_db'); TRUNCATE TABLE powa_statements_history_current_db; PERFORM powa_log('truncating powa_statements'); -- if 3rd part datasource has FK on it, throw everything away TRUNCATE TABLE powa_statements CASCADE; RETURN true; END; $function$; /* end of powa_statements_reset */ CREATE OR REPLACE FUNCTION public.powa_user_functions_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_user_functions_history'); TRUNCATE TABLE powa_user_functions_history; PERFORM powa_log('truncating powa_user_functions_history_current'); TRUNCATE TABLE powa_user_functions_history_current; RETURN true; END; $function$; /* end of powa_user_functions_reset */ CREATE OR REPLACE FUNCTION public.powa_all_relations_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_all_relations_history'); TRUNCATE TABLE powa_all_relations_history; PERFORM powa_log('truncating powa_all_relations_history_current'); TRUNCATE TABLE powa_all_relations_history_current; RETURN true; END; $function$; /* end of powa_all_relations_reset */ /* pg_stat_kcache integration - part 2 */ /* * register pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_stat_kcache'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_stat_kcache'; IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_stat_kcache'); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_stat_kcache', 'snapshot', 'powa_kcache_snapshot', false, true), ('pg_stat_kcache', 'aggregate', 'powa_kcache_aggregate', false, true), ('pg_stat_kcache', 'unregister', 'powa_kcache_unregister', false, true), ('pg_stat_kcache', 'purge', 'powa_kcache_purge', false, true), ('pg_stat_kcache', 'reset', 'powa_kcache_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_kcache_register */ /* * unregister pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_stat_kcache'); DELETE FROM public.powa_functions WHERE module = 'pg_stat_kcache'; RETURN true; END; $_$ language plpgsql; /* * powa_kcache snapshot collection. */ CREATE OR REPLACE FUNCTION powa_kcache_snapshot() RETURNS void as $PROC$ DECLARE result bool; v_funcname text := 'powa_kcache_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS ( SELECT * FROM pg_stat_kcache() k JOIN pg_roles r ON r.oid = k.userid WHERE NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), by_query AS ( INSERT INTO powa_kcache_metrics_current (queryid, dbid, userid, metrics) SELECT queryid, dbid, userid, (now(), reads, writes, user_time, system_time)::kcache_type FROM capture ), by_database AS ( INSERT INTO powa_kcache_metrics_current_db (dbid, metrics) SELECT dbid, (now(), sum(reads), sum(writes), sum(user_time), sum(system_time))::kcache_type FROM capture GROUP BY dbid ) SELECT COUNT(*) into v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END $PROC$ language plpgsql; /* end of powa_kcache_unregister */ /* * powa_kcache aggregation */ CREATE OR REPLACE FUNCTION powa_kcache_aggregate() RETURNS void AS $PROC$ DECLARE result bool; v_funcname text := 'powa_kcache_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate metrics table LOCK TABLE powa_kcache_metrics_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics (coalesce_range, queryid, dbid, userid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), queryid, dbid, userid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current GROUP BY queryid, dbid, userid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_kcache_metrics_current; -- aggregate metrics_db table LOCK TABLE powa_kcache_metrics_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics_db (coalesce_range, dbid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), dbid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current_db GROUP BY dbid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics_db) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_kcache_metrics_current_db; END $PROC$ language plpgsql; /* end of powa_kcache_aggregate */ /* * powa_kcache purge */ CREATE OR REPLACE FUNCTION powa_kcache_purge() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_kcache_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); DELETE FROM powa_kcache_metrics WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics) - rowcount: %s', v_funcname, v_rowcount)); DELETE FROM powa_kcache_metrics_db WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics_db) - rowcount: %s', v_funcname, v_rowcount)); END; $PROC$ language plpgsql; /* end of powa_kcache_purge */ /* * powa_kcache reset */ CREATE OR REPLACE FUNCTION powa_kcache_reset() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_kcache_reset'; v_rowcount bigint; BEGIN PERFORM powa_log('running powa_kcache_reset'); PERFORM powa_log('truncating powa_kcache_metrics'); TRUNCATE TABLE powa_kcache_metrics; PERFORM powa_log('truncating powa_kcache_metrics_db'); TRUNCATE TABLE powa_kcache_metrics_db; PERFORM powa_log('truncating powa_kcache_metrics_current'); TRUNCATE TABLE powa_kcache_metrics_current; PERFORM powa_log('truncating powa_kcache_metrics_current_db'); TRUNCATE TABLE powa_kcache_metrics_current_db; END; $PROC$ language plpgsql; /* end of powa_kcache_reset */ -- By default, try to register pg_stat_kcache, in case it's alreay here SELECT * FROM public.powa_kcache_register(); /* end of pg_stat_kcache integration - part 2 */ /* pg_qualstats integration - part 2 */ /* * powa_qualstats_register */ CREATE OR REPLACE function public.powa_qualstats_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_qualstats'; IF ( v_ext_present) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE function_name IN ('powa_qualstats_snapshot', 'powa_qualstats_aggregate', 'powa_qualstats_purge'); IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_qualstats'); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_qualstats', 'snapshot', 'powa_qualstats_snapshot', false, true), ('pg_qualstats', 'aggregate', 'powa_qualstats_aggregate', false, true), ('pg_qualstats', 'unregister', 'powa_qualstats_unregister', false, true), ('pg_qualstats', 'purge', 'powa_qualstats_purge', false, true), ('pg_qualstats', 'reset', 'powa_qualstats_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_qualstats_register */ /* * powa_qualstats utility view for aggregating constvalues */ CREATE OR REPLACE VIEW powa_qualstats_aggregate_constvalues_current AS WITH consts AS ( SELECT qualid, queryid, dbid, userid, min(ts) as mints, max(ts) as maxts, sum(occurences) as occurences, sum(nbfiltered) as nbfiltered, sum(execution_count) as execution_count, constvalues FROM powa_qualstats_constvalues_history_current GROUP BY qualid, queryid, dbid, userid, constvalues ), groups AS ( SELECT qualid, queryid, dbid, userid, tstzrange(min(mints), max(maxts),'[]') FROM consts GROUP BY qualid, queryid, dbid, userid ) SELECT * FROM groups, LATERAL ( SELECT array_agg(constvalues) as mu FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY occurences desc LIMIT 20 ) s ) as mu, LATERAL ( SELECT array_agg(constvalues) as mf FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN execution_count = 0 THEN 0 ELSE nbfiltered / execution_count::numeric END DESC LIMIT 20 ) s ) as mf, LATERAL ( SELECT array_agg(constvalues) as lf FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN execution_count = 0 THEN 0 ELSE nbfiltered / execution_count::numeric END DESC LIMIT 20 ) s ) as lf, LATERAL ( SELECT array_agg(constvalues) as me FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY execution_count desc LIMIT 20 ) s ) as me; CREATE OR REPLACE FUNCTION powa_qualstats_snapshot() RETURNS void as $PROC$ DECLARE result bool; v_funcname text := 'powa_qualstats_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS ( SELECT pgqs.*, s.query FROM pg_qualstats_by_query pgqs JOIN powa_statements s USING(queryid, dbid, userid) JOIN pg_roles r ON s.userid = r.oid AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_quals AS ( INSERT INTO powa_qualstats_quals (qualid, queryid, dbid, userid, quals) SELECT DISTINCT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, array_agg(DISTINCT q::qual_type) FROM capture qs, LATERAL (SELECT (unnest(quals)).*) as q WHERE NOT EXISTS ( SELECT 1 FROM powa_qualstats_quals nh WHERE nh.qualid = qs.qualnodeid AND nh.queryid = qs.queryid AND nh.dbid = qs.dbid AND nh.userid = qs.userid ) GROUP BY qualnodeid, queryid, dbid, userid RETURNING * ), by_qual AS ( INSERT INTO powa_qualstats_quals_history_current (qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered) SELECT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), sum(occurences), sum(execution_count), sum(nbfiltered) FROM capture as qs GROUP BY qualnodeid, qs.queryid, qs.dbid, qs.userid RETURNING * ), by_qual_with_const AS ( INSERT INTO powa_qualstats_constvalues_history_current(qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered, constvalues) SELECT qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), occurences, execution_count, nbfiltered, constvalues FROM capture as qs ) SELECT COUNT(*) into v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; PERFORM pg_qualstats_reset(); END $PROC$ language plpgsql; /* end of powa_qualstats_snapshot */ /* * powa_qualstats aggregate */ CREATE OR REPLACE FUNCTION powa_qualstats_aggregate() RETURNS void AS $PROC$ DECLARE result bool; BEGIN PERFORM powa_log('running powa_qualstats_aggregate'); LOCK TABLE powa_qualstats_constvalues_history_current IN SHARE MODE; LOCK TABLE powa_qualstats_quals_history_current IN SHARE MODE; INSERT INTO powa_qualstats_constvalues_history ( qualid, queryid, dbid, userid, coalesce_range, most_used, most_filtering, least_filtering, most_executed) SELECT * FROM powa_qualstats_aggregate_constvalues_current; INSERT INTO powa_qualstats_quals_history (qualid, queryid, dbid, userid, coalesce_range, records, mins_in_range, maxs_in_range) SELECT qualid, queryid, dbid, userid, tstzrange(min(ts), max(ts),'[]'), array_agg((ts, occurences, execution_count, nbfiltered)::powa_qualstats_history_item), ROW(min(ts), min(occurences), min(execution_count), min(nbfiltered))::powa_qualstats_history_item, ROW(max(ts), max(occurences), max(execution_count), max(nbfiltered))::powa_qualstats_history_item FROM powa_qualstats_quals_history_current GROUP BY qualid, queryid, dbid, userid; TRUNCATE powa_qualstats_constvalues_history_current; TRUNCATE powa_qualstats_quals_history_current; END $PROC$ language plpgsql; /* end of powa_qualstats_aggregate */ /* * powa_qualstats_purge */ CREATE OR REPLACE FUNCTION powa_qualstats_purge() RETURNS void as $PROC$ BEGIN PERFORM powa_log('running powa_qualstats_purge'); DELETE FROM powa_qualstats_constvalues_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); DELETE FROM powa_qualstats_quals_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); END; $PROC$ language plpgsql; /* end of powa_qualstats_purge */ /* * powa_qualstats_reset */ CREATE OR REPLACE FUNCTION powa_qualstats_reset() RETURNS void as $PROC$ BEGIN PERFORM powa_log('running powa_qualstats_reset'); PERFORM powa_log('truncating powa_qualstats_quals'); TRUNCATE TABLE powa_qualstats_quals CASCADE; -- cascaded : -- powa_qualstats_quals_history -- powa_qualstats_quals_history_current -- powa_qualstats_constvalues_history -- powa_qualstats_constvalues_history_current END; $PROC$ language plpgsql; /* end of powa_qualstats_reset */ /* * powa_qualstats_unregister */ CREATE OR REPLACE function public.powa_qualstats_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_qualstats'); DELETE FROM public.powa_functions WHERE module = 'pg_qualstats'; RETURN true; END; $_$ language plpgsql; SELECT * FROM public.powa_qualstats_register(); /* end of pg_qualstats_integration - part 2 */ /* pg_track_settings integration */ CREATE OR REPLACE FUNCTION powa_track_settings_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_track_settings'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_track_settings'; IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_track_settings'); -- This extension handles its own storage, just its snapshot -- function and an unregister function. INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_track_settings', 'snapshot', 'pg_track_settings_snapshot', false, true), ('pg_track_settings', 'unregister', 'powa_track_settings_unregister', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_qualstats_unregister */ CREATE OR REPLACE function public.powa_track_settings_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_track_settings'); DELETE FROM public.powa_functions WHERE module = 'pg_track_settings'; RETURN true; END; $_$ language plpgsql; /* end of powa_track_settings_unregister */ -- By default, try to register pg_track_settings, in case it's alreay here SELECT * FROM public.powa_track_settings_register(); /* end pg_track_settings integration */ powa-archivist-REL_3_2_0/powa--3.2.0.sql000066400000000000000000002376001336072707400175430ustar00rootroot00000000000000-- complain if script is sourced in psql, rather than via CREATE EXTENSION --\echo Use "CREATE EXTENSION powa" to load this file. \quit SET statement_timeout = 0; SET client_encoding = 'UTF8'; SET standard_conforming_strings = on; SET client_min_messages = warning; SET escape_string_warning = off; SET search_path = public, pg_catalog; CREATE TABLE powa_databases( oid oid PRIMARY KEY, datname name, dropped timestamp with time zone ); CREATE FUNCTION powa_stat_user_functions(IN dbid oid, OUT funcid oid, OUT calls bigint, OUT total_time double precision, OUT self_time double precision) RETURNS SETOF record LANGUAGE c COST 100 AS '$libdir/powa', 'powa_stat_user_functions'; CREATE FUNCTION powa_stat_all_rel(IN dbid oid, OUT relid oid, OUT numscan bigint, OUT tup_returned bigint, OUT tup_fetched bigint, OUT n_tup_ins bigint, OUT n_tup_upd bigint, OUT n_tup_del bigint, OUT n_tup_hot_upd bigint, OUT n_liv_tup bigint, OUT n_dead_tup bigint, OUT n_mod_since_analyze bigint, OUT blks_read bigint, OUT blks_hit bigint, OUT last_vacuum timestamp with time zone, OUT vacuum_count bigint, OUT last_autovacuum timestamp with time zone, OUT autovacuum_count bigint, OUT last_analyze timestamp with time zone, OUT analyze_count bigint, OUT last_autoanalyze timestamp with time zone, OUT autoanalyze_count bigint) RETURNS SETOF record LANGUAGE c COST 100 AS '$libdir/powa', 'powa_stat_all_rel'; CREATE TYPE powa_statements_history_record AS ( ts timestamp with time zone, calls bigint, total_time double precision, rows bigint, shared_blks_hit bigint, shared_blks_read bigint, shared_blks_dirtied bigint, shared_blks_written bigint, local_blks_hit bigint, local_blks_read bigint, local_blks_dirtied bigint, local_blks_written bigint, temp_blks_read bigint, temp_blks_written bigint, blk_read_time double precision, blk_write_time double precision ); /* pg_stat_statements operator support */ CREATE TYPE powa_statements_history_diff AS ( intvl interval, calls bigint, total_time double precision, rows bigint, shared_blks_hit bigint, shared_blks_read bigint, shared_blks_dirtied bigint, shared_blks_written bigint, local_blks_hit bigint, local_blks_read bigint, local_blks_dirtied bigint, local_blks_written bigint, temp_blks_read bigint, temp_blks_written bigint, blk_read_time double precision, blk_write_time double precision ); CREATE OR REPLACE FUNCTION powa_statements_history_mi( a powa_statements_history_record, b powa_statements_history_record) RETURNS powa_statements_history_diff AS $_$ DECLARE res powa_statements_history_diff; BEGIN res.intvl = a.ts - b.ts; res.calls = a.calls - b.calls; res.total_time = a.total_time - b.total_time; res.rows = a.rows - b.rows; res.shared_blks_hit = a.shared_blks_hit - b.shared_blks_hit; res.shared_blks_read = a.shared_blks_read - b.shared_blks_read; res.shared_blks_dirtied = a.shared_blks_dirtied - b.shared_blks_dirtied; res.shared_blks_written = a.shared_blks_written - b.shared_blks_written; res.local_blks_hit = a.local_blks_hit - b.local_blks_hit; res.local_blks_read = a.local_blks_read - b.local_blks_read; res.local_blks_dirtied = a.local_blks_dirtied - b.local_blks_dirtied; res.local_blks_written = a.local_blks_written - b.local_blks_written; res.temp_blks_read = a.temp_blks_read - b.temp_blks_read; res.temp_blks_written = a.temp_blks_written - b.temp_blks_written; res.blk_read_time = a.blk_read_time - b.blk_read_time; res.blk_write_time = a.blk_write_time - b.blk_write_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_statements_history_mi, LEFTARG = powa_statements_history_record, RIGHTARG = powa_statements_history_record ); CREATE TYPE powa_statements_history_rate AS ( sec integer, calls_per_sec double precision, runtime_per_sec double precision, rows_per_sec double precision, shared_blks_hit_per_sec double precision, shared_blks_read_per_sec double precision, shared_blks_dirtied_per_sec double precision, shared_blks_written_per_sec double precision, local_blks_hit_per_sec double precision, local_blks_read_per_sec double precision, local_blks_dirtied_per_sec double precision, local_blks_written_per_sec double precision, temp_blks_read_per_sec double precision, temp_blks_written_per_sec double precision, blk_read_time_per_sec double precision, blk_write_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_statements_history_div( a powa_statements_history_record, b powa_statements_history_record) RETURNS powa_statements_history_rate AS $_$ DECLARE res powa_statements_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.calls_per_sec = (a.calls - b.calls)::double precision / sec; res.runtime_per_sec = (a.total_time - b.total_time)::double precision / sec; res.rows_per_sec = (a.rows - b.rows)::double precision / sec; res.shared_blks_hit_per_sec = (a.shared_blks_hit - b.shared_blks_hit)::double precision / sec; res.shared_blks_read_per_sec = (a.shared_blks_read - b.shared_blks_read)::double precision / sec; res.shared_blks_dirtied_per_sec = (a.shared_blks_dirtied - b.shared_blks_dirtied)::double precision / sec; res.shared_blks_written_per_sec = (a.shared_blks_written - b.shared_blks_written)::double precision / sec; res.local_blks_hit_per_sec = (a.local_blks_hit - b.local_blks_hit)::double precision / sec; res.local_blks_read_per_sec = (a.local_blks_read - b.local_blks_read)::double precision / sec; res.local_blks_dirtied_per_sec = (a.local_blks_dirtied - b.local_blks_dirtied)::double precision / sec; res.local_blks_written_per_sec = (a.local_blks_written - b.local_blks_written)::double precision / sec; res.temp_blks_read_per_sec = (a.temp_blks_read - b.temp_blks_read)::double precision / sec; res.temp_blks_written_per_sec = (a.temp_blks_written - b.temp_blks_written)::double precision / sec; res.blk_read_time_per_sec = (a.blk_read_time - b.blk_read_time)::double precision / sec; res.blk_write_time_per_sec = (a.blk_write_time - b.blk_write_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_statements_history_div, LEFTARG = powa_statements_history_record, RIGHTARG = powa_statements_history_record ); /* end of pg_stat_statements operator support */ CREATE TYPE powa_user_functions_history_record AS ( ts timestamp with time zone, calls bigint, total_time double precision, self_time double precision ); /* pg_stat_user_functions operator support */ CREATE TYPE powa_user_functions_history_diff AS ( intvl interval, calls bigint, total_time double precision, self_time double precision ); CREATE OR REPLACE FUNCTION powa_user_functions_history_mi( a powa_user_functions_history_record, b powa_user_functions_history_record) RETURNS powa_user_functions_history_diff AS $_$ DECLARE res powa_user_functions_history_diff; BEGIN res.intvl = a.ts - b.ts; res.calls = a.calls - b.calls; res.total_time = a.total_time - b.total_time; res.self_time = a.self_time - b.self_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_user_functions_history_mi, LEFTARG = powa_user_functions_history_record, RIGHTARG = powa_user_functions_history_record ); CREATE TYPE powa_user_functions_history_rate AS ( sec integer, calls_per_sec double precision, total_time_per_sec double precision, self_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_user_functions_history_div( a powa_user_functions_history_record, b powa_user_functions_history_record) RETURNS powa_user_functions_history_rate AS $_$ DECLARE res powa_user_functions_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.calls_per_sec = (a.calls - b.calls)::double precision / sec; res.total_time_per_sec = (a.total_time - b.total_time)::double precision / sec; res.self_time_per_sec = (a.self_time - b.self_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_user_functions_history_div, LEFTARG = powa_user_functions_history_record, RIGHTARG = powa_user_functions_history_record ); /* end of pg_stat_user_functions operator support */ CREATE TYPE powa_all_relations_history_record AS ( ts timestamp with time zone, numscan bigint, tup_returned bigint, tup_fetched bigint, n_tup_ins bigint, n_tup_upd bigint, n_tup_del bigint, n_tup_hot_upd bigint, n_liv_tup bigint, n_dead_tup bigint, n_mod_since_analyze bigint, blks_read bigint, blks_hit bigint, last_vacuum timestamp with time zone, vacuum_count bigint, last_autovacuum timestamp with time zone, autovacuum_count bigint, last_analyze timestamp with time zone, analyze_count bigint, last_autoanalyze timestamp with time zone, autoanalyze_count bigint ); /* pg_stat_all_relations operator support */ CREATE TYPE powa_all_relations_history_diff AS ( intvl interval, numscan bigint, tup_returned bigint, tup_fetched bigint, n_tup_ins bigint, n_tup_upd bigint, n_tup_del bigint, n_tup_hot_upd bigint, n_liv_tup bigint, n_dead_tup bigint, n_mod_since_analyze bigint, blks_read bigint, blks_hit bigint, vacuum_count bigint, autovacuum_count bigint, analyze_count bigint, autoanalyze_count bigint ); CREATE OR REPLACE FUNCTION powa_all_relations_history_mi( a powa_all_relations_history_record, b powa_all_relations_history_record) RETURNS powa_all_relations_history_diff AS $_$ DECLARE res powa_all_relations_history_diff; BEGIN res.intvl = a.ts - b.ts; res.numscan = a.numscan - b.numscan; res.tup_returned = a.tup_returned - b.tup_returned; res.tup_fetched = a.tup_fetched - b.tup_fetched; res.n_tup_ins = a.n_tup_ins - b.n_tup_ins; res.n_tup_upd = a.n_tup_upd - b.n_tup_upd; res.n_tup_del = a.n_tup_del - b.n_tup_del; res.n_tup_hot_upd = a.n_tup_hot_upd - b.n_tup_hot_upd; res.n_liv_tup = a.n_liv_tup - b.n_liv_tup; res.n_dead_tup = a.n_dead_tup - b.n_dead_tup; res.n_mod_since_analyze = a.n_mod_since_analyze - b.n_mod_since_analyze; res.blks_read = a.blks_read - b.blks_read; res.blks_hit = a.blks_hit - b.blks_hit; res.vacuum_count = a.vacuum_count - b.vacuum_count; res.autovacuum_count = a.autovacuum_count - b.autovacuum_count; res.analyze_count = a.analyze_count - b.analyze_count; res.autoanalyze_count = a.autoanalyze_count - b.autoanalyze_count; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_all_relations_history_mi, LEFTARG = powa_all_relations_history_record, RIGHTARG = powa_all_relations_history_record ); CREATE TYPE powa_all_relations_history_rate AS ( sec integer, numscan_per_sec double precision, tup_returned_per_sec double precision, tup_fetched_per_sec double precision, n_tup_ins_per_sec double precision, n_tup_upd_per_sec double precision, n_tup_del_per_sec double precision, n_tup_hot_upd_per_sec double precision, n_liv_tup_per_sec double precision, n_dead_tup_per_sec double precision, n_mod_since_analyze_per_sec double precision, blks_read_per_sec double precision, blks_hit_per_sec double precision, vacuum_count_per_sec double precision, autovacuum_count_per_sec double precision, analyze_count_per_sec double precision, autoanalyze_count_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_all_relations_history_div( a powa_all_relations_history_record, b powa_all_relations_history_record) RETURNS powa_all_relations_history_rate AS $_$ DECLARE res powa_all_relations_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.numscan_per_sec = (a.numscan - b.numscan)::double precision / sec; res.tup_returned_per_sec = (a.tup_returned - b.tup_returned)::double precision / sec; res.tup_fetched_per_sec = (a.tup_fetched - b.tup_fetched)::double precision / sec; res.n_tup_ins_per_sec = (a.n_tup_ins - b.n_tup_ins)::double precision / sec; res.n_tup_upd_per_sec = (a.n_tup_upd - b.n_tup_upd)::double precision / sec; res.n_tup_del_per_sec = (a.n_tup_del - b.n_tup_del)::double precision / sec; res.n_tup_hot_upd_per_sec = (a.n_tup_hot_upd - b.n_tup_hot_upd)::double precision / sec; res.n_liv_tup_per_sec = (a.n_liv_tup - b.n_liv_tup)::double precision / sec; res.n_dead_tup_per_sec = (a.n_dead_tup - b.n_dead_tup)::double precision / sec; res.n_mod_since_analyze_per_sec = (a.n_mod_since_analyze - b.n_mod_since_analyze)::double precision / sec; res.blks_read_per_sec = (a.blks_read - b.blks_read)::double precision / sec; res.blks_hit_per_sec = (a.blks_hit - b.blks_hit)::double precision / sec; res.vacuum_count_per_sec = (a.vacuum_count - b.vacuum_count)::double precision / sec; res.autovacuum_count_per_sec = (a.autovacuum_count - b.autovacuum_count)::double precision / sec; res.analyze_count_per_sec = (a.analyze_count - b.analyze_count)::double precision / sec; res.autoanalyze_count_per_sec = (a.autoanalyze_count - b.autoanalyze_count)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_all_relations_history_div, LEFTARG = powa_all_relations_history_record, RIGHTARG = powa_all_relations_history_record ); /* end of pg_stat_all_relations operator support */ CREATE TABLE powa_last_aggregation ( aggts timestamp with time zone ); INSERT INTO powa_last_aggregation(aggts) VALUES (current_timestamp); CREATE TABLE powa_last_purge ( purgets timestamp with time zone ); INSERT INTO powa_last_purge (purgets) VALUES (current_timestamp); CREATE TABLE powa_statements ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, query text NOT NULL ); ALTER TABLE ONLY powa_statements ADD CONSTRAINT powa_statements_pkey PRIMARY KEY (queryid, dbid, userid); CREATE INDEX powa_statements_dbid_idx ON powa_statements(dbid); CREATE INDEX powa_statements_userid_idx ON powa_statements(userid); CREATE TABLE powa_statements_history ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statements_history_record[] NOT NULL, mins_in_range powa_statements_history_record NOT NULL, maxs_in_range powa_statements_history_record NOT NULL ); CREATE INDEX powa_statements_history_query_ts ON powa_statements_history USING gist (queryid, coalesce_range); CREATE TABLE powa_statements_history_db ( dbid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_statements_history_record[] NOT NULL, mins_in_range powa_statements_history_record NOT NULL, maxs_in_range powa_statements_history_record NOT NULL ); CREATE INDEX powa_statements_history_db_ts ON powa_statements_history_db USING gist (dbid, coalesce_range); CREATE TABLE powa_statements_history_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, record powa_statements_history_record NOT NULL ); CREATE TABLE powa_statements_history_current_db ( dbid oid NOT NULL, record powa_statements_history_record NOT NULL ); CREATE TABLE powa_user_functions_history ( dbid oid NOT NULL, funcid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_user_functions_history_record[] NOT NULL, mins_in_range powa_user_functions_history_record NOT NULL, maxs_in_range powa_user_functions_history_record NOT NULL ); CREATE INDEX powa_user_functions_history_funcid_ts ON powa_user_functions_history USING gist (funcid, coalesce_range); CREATE TABLE powa_user_functions_history_current ( dbid oid NOT NULL, funcid oid NOT NULL, record powa_user_functions_history_record NOT NULL ); CREATE TABLE powa_all_relations_history ( dbid oid NOT NULL, relid oid NOT NULL, coalesce_range tstzrange NOT NULL, records powa_all_relations_history_record[] NOT NULL, mins_in_range powa_all_relations_history_record NOT NULL, maxs_in_range powa_all_relations_history_record NOT NULL ); CREATE INDEX powa_all_relations_history_relid_ts ON powa_all_relations_history USING gist (relid, coalesce_range); CREATE TABLE powa_all_relations_history_current ( dbid oid NOT NULL, relid oid NOT NULL, record powa_all_relations_history_record NOT NULL ); CREATE SEQUENCE powa_coalesce_sequence INCREMENT BY 1 START WITH 1 CYCLE; CREATE TABLE powa_functions ( module text NOT NULL, operation text NOT NULL, function_name text NOT NULL, added_manually boolean NOT NULL default true, enabled boolean NOT NULL default true, CHECK (operation IN ('snapshot','aggregate','purge','unregister','reset')) ); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_stat_statements', 'snapshot', 'powa_statements_snapshot', false, true), ('powa_stat_user_functions', 'snapshot', 'powa_user_functions_snapshot', false, true), ('powa_stat_all_relations', 'snapshot', 'powa_all_relations_snapshot', false, true), ('pg_stat_statements', 'aggregate','powa_statements_aggregate', false, true), ('powa_stat_user_functions', 'aggregate','powa_user_functions_aggregate', false, true), ('powa_stat_all_relations', 'aggregate','powa_all_relations_aggregate', false, true), ('pg_stat_statements', 'purge', 'powa_statements_purge', false, true), ('powa_stat_user_functions', 'purge', 'powa_user_functions_purge', false, true), ('powa_stat_all_relations', 'purge', 'powa_all_relations_purge', false, true), ('pg_stat_statements', 'reset', 'powa_statements_reset', false, true), ('powa_stat_user_functions', 'reset', 'powa_user_functions_reset', false, true), ('powa_stat_all_relations', 'reset', 'powa_all_relations_reset', false, true); CREATE FUNCTION powa_log (msg text) RETURNS void LANGUAGE plpgsql AS $_$ BEGIN IF current_setting('powa.debug')::bool THEN RAISE WARNING '%', msg; ELSE RAISE DEBUG '%', msg; END IF; END; $_$; /* pg_stat_kcache integration - part 1 */ CREATE TYPE public.kcache_type AS ( ts timestamptz, reads bigint, writes bigint, user_time double precision, system_time double precision ); /* pg_stat_kcache operator support */ CREATE TYPE powa_kcache_diff AS ( intvl interval, reads bigint, writes bigint, user_time double precision, system_time double precision ); CREATE OR REPLACE FUNCTION powa_kcache_mi( a kcache_type, b kcache_type) RETURNS powa_kcache_diff AS $_$ DECLARE res powa_kcache_diff; BEGIN res.intvl = a.ts - b.ts; res.reads = a.reads - b.reads; res.writes = a.writes - b.writes; res.user_time = a.user_time - b.user_time; res.system_time = a.system_time - b.system_time; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_kcache_mi, LEFTARG = kcache_type, RIGHTARG = kcache_type ); CREATE TYPE powa_kcache_rate AS ( sec integer, reads_per_sec double precision, writes_per_sec double precision, user_time_per_sec double precision, system_time_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_kcache_div( a kcache_type, b kcache_type) RETURNS powa_kcache_rate AS $_$ DECLARE res powa_kcache_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.reads_per_sec = (a.reads - b.reads)::double precision / sec; res.writes_per_sec = (a.writes - b.writes)::double precision / sec; res.user_time_per_sec = (a.user_time - b.user_time)::double precision / sec; res.system_time_per_sec = (a.system_time - b.system_time)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_kcache_div, LEFTARG = kcache_type, RIGHTARG = kcache_type ); /* end of pg_stat_kcache operator support */ CREATE TABLE public.powa_kcache_metrics ( coalesce_range tstzrange NOT NULL, queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics public.kcache_type[] NOT NULL, mins_in_range public.kcache_type NOT NULL, maxs_in_range public.kcache_type NOT NULL, PRIMARY KEY (coalesce_range, queryid, dbid, userid) ); CREATE INDEX ON public.powa_kcache_metrics (queryid); CREATE TABLE public.powa_kcache_metrics_db ( coalesce_range tstzrange NOT NULL, dbid oid NOT NULL, metrics public.kcache_type[] NOT NULL, mins_in_range public.kcache_type NOT NULL, maxs_in_range public.kcache_type NOT NULL, PRIMARY KEY (coalesce_range, dbid) ); CREATE TABLE public.powa_kcache_metrics_current ( queryid bigint NOT NULL, dbid oid NOT NULL, userid oid NOT NULL, metrics kcache_type NULL NULL ); CREATE TABLE public.powa_kcache_metrics_current_db ( dbid oid NOT NULL, metrics kcache_type NULL NULL ); /* end of pg_stat_kcache integration - part 1 */ /* pg_qualstats integration - part 1 */ CREATE TYPE public.qual_type AS ( relid oid, attnum integer, opno oid, eval_type "char" ); CREATE TYPE public.qual_values AS ( constants text[], occurences bigint, execution_count bigint, nbfiltered bigint ); CREATE TYPE powa_qualstats_history_item AS ( ts timestamptz, occurences bigint, execution_count bigint, nbfiltered bigint ); /* pg_stat_qualstats operator support */ CREATE TYPE powa_qualstats_history_diff AS ( intvl interval, occurences bigint, execution_count bigint, nbfiltered bigint ); CREATE OR REPLACE FUNCTION powa_qualstats_history_mi( a powa_qualstats_history_item, b powa_qualstats_history_item) RETURNS powa_qualstats_history_diff AS $_$ DECLARE res powa_qualstats_history_diff; BEGIN res.intvl = a.ts - b.ts; res.occurences = a.occurences - b.occurences; res.execution_count = a.execution_count - b.execution_count; res.nbfiltered = a.nbfiltered - b.nbfiltered; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = powa_qualstats_history_mi, LEFTARG = powa_qualstats_history_item, RIGHTARG = powa_qualstats_history_item ); CREATE TYPE powa_qualstats_history_rate AS ( sec integer, occurences_per_sec double precision, execution_count_per_sec double precision, nbfiltered_per_sec double precision ); CREATE OR REPLACE FUNCTION powa_qualstats_history_div( a powa_qualstats_history_item, b powa_qualstats_history_item) RETURNS powa_qualstats_history_rate AS $_$ DECLARE res powa_qualstats_history_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.occurences_per_sec = (a.occurences - b.occurences)::double precision / sec; res.execution_count_per_sec = (a.execution_count - b.execution_count)::double precision / sec; res.nbfiltered_per_sec = (a.nbfiltered - b.nbfiltered)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = powa_qualstats_history_div, LEFTARG = powa_qualstats_history_item, RIGHTARG = powa_qualstats_history_item ); /* end of pg_stat_qualstats operator support */ CREATE TABLE public.powa_qualstats_quals ( qualid bigint, queryid bigint, dbid oid, userid oid, quals public.qual_type[], PRIMARY KEY (qualid, queryid, dbid, userid), FOREIGN KEY (queryid, dbid, userid) REFERENCES powa_statements(queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, records powa_qualstats_history_item[], mins_in_range powa_qualstats_history_item, maxs_in_range powa_qualstats_history_item, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_quals_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, occurences bigint, execution_count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES powa_qualstats_quals(qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history ( qualid bigint, queryid bigint, dbid oid, userid oid, coalesce_range tstzrange, most_used qual_values[], most_filtering qual_values[], least_filtering qual_values[], most_executed qual_values[], FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE TABLE public.powa_qualstats_constvalues_history_current ( qualid bigint, queryid bigint, dbid oid, userid oid, ts timestamptz, constvalues text[], occurences bigint, execution_count bigint, nbfiltered bigint, FOREIGN KEY (qualid, queryid, dbid, userid) REFERENCES public.powa_qualstats_quals (qualid, queryid, dbid, userid) MATCH FULL ON UPDATE CASCADE ON DELETE CASCADE ); CREATE INDEX ON powa_qualstats_constvalues_history USING gist (queryid, qualid, coalesce_range); CREATE INDEX ON powa_qualstats_constvalues_history (qualid, queryid); CREATE INDEX ON powa_qualstats_quals(queryid); /* end of pg_qualstats_integration - part 1 */ /* pg_wait_sampling integration - part 1 */ CREATE TYPE public.wait_sampling_type AS ( ts timestamptz, count bigint ); /* pg_wait_sampling operator support */ CREATE TYPE wait_sampling_diff AS ( intvl interval, count bigint ); CREATE OR REPLACE FUNCTION wait_sampling_mi( a wait_sampling_type, b wait_sampling_type) RETURNS wait_sampling_diff AS $_$ DECLARE res wait_sampling_diff; BEGIN res.intvl = a.ts - b.ts; res.count = a.count - b.count; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR - ( PROCEDURE = wait_sampling_mi, LEFTARG = wait_sampling_type, RIGHTARG = wait_sampling_type ); CREATE TYPE wait_sampling_rate AS ( sec integer, count_per_sec double precision ); CREATE OR REPLACE FUNCTION wait_sampling_div( a wait_sampling_type, b wait_sampling_type) RETURNS wait_sampling_rate AS $_$ DECLARE res wait_sampling_rate; sec integer; BEGIN res.sec = extract(EPOCH FROM (a.ts - b.ts)); IF res.sec = 0 THEN sec = 1; ELSE sec = res.sec; END IF; res.count_per_sec = (a.count - b.count)::double precision / sec; return res; END; $_$ LANGUAGE plpgsql IMMUTABLE STRICT; CREATE OPERATOR / ( PROCEDURE = wait_sampling_div, LEFTARG = wait_sampling_type, RIGHTARG = wait_sampling_type ); /* end of pg_wait_sampling operator support */ CREATE TABLE public.powa_wait_sampling_history ( coalesce_range tstzrange NOT NULL, queryid bigint NOT NULL, dbid oid NOT NULL, event_type text NOT NULL, event text NOT NULL, records public.wait_sampling_type[] NOT NULL, mins_in_range public.wait_sampling_type NOT NULL, maxs_in_range public.wait_sampling_type NOT NULL, PRIMARY KEY (coalesce_range, queryid, dbid, event_type, event) ); CREATE INDEX ON public.powa_wait_sampling_history (queryid); CREATE TABLE public.powa_wait_sampling_history_db ( coalesce_range tstzrange NOT NULL, dbid oid NOT NULL, event_type text NOT NULL, event text NOT NULL, records public.wait_sampling_type[] NOT NULL, mins_in_range public.wait_sampling_type NOT NULL, maxs_in_range public.wait_sampling_type NOT NULL, PRIMARY KEY (coalesce_range, dbid, event_type, event) ); CREATE TABLE public.powa_wait_sampling_history_current ( queryid bigint NOT NULL, dbid oid NOT NULL, event_type text NOT NULL, event text NOT NULL, record wait_sampling_type NOT NULL ); CREATE TABLE public.powa_wait_sampling_history_current_db ( dbid oid NOT NULL, event_type text NOT NULL, event text NOT NULL, record wait_sampling_type NOT NULL ); /* end of pg_wait_sampling integration - part 1 */ -- Mark all of powa's tables as "to be dumped" SELECT pg_catalog.pg_extension_config_dump('powa_statements',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_statements_history_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_user_functions_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_user_functions_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_all_relations_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_all_relations_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_functions','WHERE added_manually'); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_kcache_metrics_current_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_quals_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_qualstats_constvalues_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_wait_sampling_history',''); SELECT pg_catalog.pg_extension_config_dump('powa_wait_sampling_history_db',''); SELECT pg_catalog.pg_extension_config_dump('powa_wait_sampling_history_current',''); SELECT pg_catalog.pg_extension_config_dump('powa_wait_sampling_history_current_db',''); CREATE OR REPLACE FUNCTION public.powa_check_created_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE BEGIN /* We have for now no way for a proper handling of this event, * as we don't have a table with the list of supported extensions. * So just call every powa_*_register() function we know each time an * extension is created. Powa should be in a dedicated database and the * register function handle to be called several time, so it's not critical */ PERFORM public.powa_kcache_register(); PERFORM public.powa_qualstats_register(); PERFORM public.powa_track_settings_register(); PERFORM public.powa_wait_sampling_register(); END; $_$; /* end of powa_check_created_extensions */ CREATE EVENT TRIGGER powa_check_created_extensions ON ddl_command_end WHEN tag IN ('CREATE EXTENSION') EXECUTE PROCEDURE public.powa_check_created_extensions() ; CREATE OR REPLACE FUNCTION public.powa_check_dropped_extensions() RETURNS event_trigger LANGUAGE plpgsql AS $_$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- We unregister extensions regardless the "enabled" field WITH ext AS ( SELECT object_name FROM pg_event_trigger_dropped_objects() d WHERE d.object_type = 'extension' ) SELECT function_name INTO funcname FROM powa_functions f JOIN ext ON f.module = ext.object_name WHERE operation = 'unregister'; IF ( funcname IS NOT NULL ) THEN BEGIN PERFORM powa_log(format('running %I', funcname)); EXECUTE 'SELECT ' || quote_ident(funcname) || '()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE WARNING 'powa_check_dropped_extensions(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END IF; END; $_$; /* end of powa_check_dropped_extensions */ CREATE EVENT TRIGGER powa_check_dropped_extensions ON sql_drop WHEN tag IN ('DROP EXTENSION') EXECUTE PROCEDURE public.powa_check_dropped_extensions() ; CREATE OR REPLACE FUNCTION powa_take_snapshot() RETURNS void AS $PROC$ DECLARE purgets timestamp with time zone; purge_seq bigint; funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; v_title text = 'PoWA - '; v_rowcount bigint; BEGIN PERFORM set_config('application_name', v_title || ' snapshot database list', false); PERFORM powa_log('start of powa_take_snapshot'); -- Keep track of existing databases PERFORM powa_log('Maintaining database list...'); WITH missing AS ( SELECT d.oid, d.datname FROM pg_database d LEFT JOIN powa_databases p ON d.oid = p.oid WHERE p.oid IS NULL ) INSERT INTO powa_databases SELECT * FROM missing; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('missing db: %s', v_rowcount)); -- Keep track of renamed databases WITH renamed AS ( SELECT d.oid, d.datname FROM pg_database AS d JOIN powa_databases AS p ON d.oid = p.oid WHERE d.datname != p.datname ) UPDATE powa_databases AS p SET datname = r.datname FROM renamed AS r WHERE p.oid = r.oid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('renamed db: %s', v_rowcount)); -- Keep track of when databases are dropped WITH dropped AS ( SELECT p.oid FROM powa_databases p LEFT JOIN pg_database d ON p.oid = d.oid WHERE d.oid IS NULL AND p.dropped IS NULL) UPDATE powa_databases p SET dropped = now() FROM dropped d WHERE p.oid = d.oid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('dropped db: %s', v_rowcount)); -- For all enabled snapshot functions in the powa_functions table, execute FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='snapshot' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling snapshot function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; -- Coalesce datas if needed SELECT nextval('powa_coalesce_sequence'::regclass) INTO purge_seq; PERFORM powa_log(format('powa_coalesce_sequence: %s', purge_seq)); IF ( purge_seq % current_setting('powa.coalesce')::bigint ) = 0 THEN PERFORM powa_log(format('coalesce needed, seq: %s coalesce seq: %s', purge_seq, current_setting('powa.coalesce')::bigint )); FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='aggregate' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling aggregate function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; UPDATE powa_last_aggregation SET aggts = now(); END IF; -- We also purge, at next pass IF ( purge_seq % (current_setting('powa.coalesce')::bigint ) ) = 1 THEN PERFORM powa_log(format('purge needed, seq: %s coalesce seq: %s', purge_seq, current_setting('powa.coalesce'))); FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='purge' AND enabled LOOP -- Call all of them, with no parameter BEGIN PERFORM powa_log(format('calling purge function: %I', funcname)); PERFORM set_config('application_name', v_title || quote_ident(funcname) || '()', false); EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_take_snapshot(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; PERFORM set_config('application_name', v_title || 'UPDATE powa_last_purge', false); UPDATE powa_last_purge SET purgets = now(); END IF; PERFORM powa_log('end of powa_take_snapshot'); PERFORM set_config('application_name', v_title || 'snapshot finished', false); END; $PROC$ LANGUAGE plpgsql; /* end of powa_take_snapshot */ CREATE OR REPLACE FUNCTION powa_statements_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; ignore_regexp text :='^[[:space:]]*(DEALLOCATE|BEGIN|PREPARE TRANSACTION|COMMIT PREPARED|ROLLBACK PREPARED)'; v_funcname text := 'powa_statements_snapshot'; v_rowcount bigint; BEGIN -- In this function, we capture statements, and also aggregate counters by database -- so that the first screens of powa stay reactive even though there may be thousands -- of different statements PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS( SELECT pgss.* FROM pg_stat_statements pgss JOIN pg_roles r ON pgss.userid = r.oid WHERE pgss.query !~* ignore_regexp AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_statements AS( INSERT INTO powa_statements (queryid, dbid, userid, query) SELECT queryid, dbid, userid, query FROM capture c WHERE NOT EXISTS (SELECT 1 FROM powa_statements ps WHERE ps.queryid = c.queryid AND ps.dbid = c.dbid AND ps.userid = c.userid ) ), by_query AS ( INSERT INTO powa_statements_history_current SELECT queryid, dbid, userid, ROW( now(), calls, total_time, rows, shared_blks_hit, shared_blks_read, shared_blks_dirtied, shared_blks_written, local_blks_hit, local_blks_read, local_blks_dirtied, local_blks_written, temp_blks_read, temp_blks_written, blk_read_time, blk_write_time )::powa_statements_history_record AS record FROM capture ), by_database AS ( INSERT INTO powa_statements_history_current_db SELECT dbid, ROW( now(), sum(calls), sum(total_time), sum(rows), sum(shared_blks_hit), sum(shared_blks_read), sum(shared_blks_dirtied), sum(shared_blks_written), sum(local_blks_hit), sum(local_blks_read), sum(local_blks_dirtied), sum(local_blks_written), sum(temp_blks_read), sum(temp_blks_written), sum(blk_read_time), sum(blk_write_time) )::powa_statements_history_record AS record FROM capture GROUP BY dbid ) SELECT count(*) INTO v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; -- For now we don't care. What could we do on error except crash anyway? END; $PROC$ language plpgsql; /* end of powa_statements_snapshot */ CREATE OR REPLACE FUNCTION powa_user_functions_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_user_functions_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide user function statistics WITH func(dbid, r) AS ( SELECT oid, powa_stat_user_functions(oid) FROM pg_database ) INSERT INTO powa_user_functions_history_current SELECT dbid, (r).funcid, ROW(now(), (r).calls, (r).total_time, (r).self_time)::powa_user_functions_history_record AS record FROM func; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_user_functions_snapshot */ CREATE OR REPLACE FUNCTION powa_all_relations_snapshot() RETURNS void AS $PROC$ DECLARE result boolean; v_funcname text := 'powa_all_relations_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Insert cluster-wide relation statistics WITH rel(dbid, r) AS ( SELECT oid, powa_stat_all_rel(oid) FROM pg_database ) INSERT INTO powa_all_relations_history_current SELECT dbid, (r).relid, ROW(now(),(r).numscan, (r).tup_returned, (r).tup_fetched, (r).n_tup_ins, (r).n_tup_upd, (r).n_tup_del, (r).n_tup_hot_upd, (r).n_liv_tup, (r).n_dead_tup, (r).n_mod_since_analyze, (r).blks_read, (r).blks_hit, (r).last_vacuum, (r).vacuum_count, (r).last_autovacuum, (r).autovacuum_count, (r).last_analyze, (r).analyze_count, (r).last_autoanalyze, (r).autoanalyze_count)::powa_all_relations_history_record AS record FROM rel; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END; $PROC$ language plpgsql; /* end of powa_all_relations_snapshot */ CREATE OR REPLACE FUNCTION powa_statements_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_statements_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_statements_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_hitory) - rowcount: %s', v_funcname, v_rowcount)); DELETE FROM powa_statements_history_db WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history_db) - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_statements_purge */ CREATE OR REPLACE FUNCTION powa_user_functions_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_user_functions_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_user_functions_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_user_functions_purge */ CREATE OR REPLACE FUNCTION powa_all_relations_purge() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_all_relations_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- Delete obsolete datas. We only bother with already coalesced data DELETE FROM powa_all_relations_history WHERE upper(coalesce_range)< (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); -- FIXME maybe we should cleanup the powa_*_history tables ? But it will take a while: unnest all records... END; $PROC$ LANGUAGE plpgsql; /* end of powa_all_relations_purge */ CREATE OR REPLACE FUNCTION powa_statements_aggregate() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_statements_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate statements table LOCK TABLE powa_statements_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history SELECT queryid, dbid, userid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current GROUP BY queryid, dbid, userid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_statements_history_current; -- aggregate db table LOCK TABLE powa_statements_history_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_statements_history_db SELECT dbid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time),min((record).rows), min((record).shared_blks_hit),min((record).shared_blks_read), min((record).shared_blks_dirtied),min((record).shared_blks_written), min((record).local_blks_hit),min((record).local_blks_read), min((record).local_blks_dirtied),min((record).local_blks_written), min((record).temp_blks_read),min((record).temp_blks_written), min((record).blk_read_time),min((record).blk_write_time))::powa_statements_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time),max((record).rows), max((record).shared_blks_hit),max((record).shared_blks_read), max((record).shared_blks_dirtied),max((record).shared_blks_written), max((record).local_blks_hit),max((record).local_blks_read), max((record).local_blks_dirtied),max((record).local_blks_written), max((record).temp_blks_read),max((record).temp_blks_written), max((record).blk_read_time),max((record).blk_write_time))::powa_statements_history_record FROM powa_statements_history_current_db GROUP BY dbid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_statements_history_db) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_statements_history_current_db; END; $PROC$ LANGUAGE plpgsql; /* end of powa_statements_aggregate */ CREATE OR REPLACE FUNCTION powa_user_functions_aggregate() RETURNS void AS $PROC$ BEGIN PERFORM powa_log('running powa_user_functions_aggregate'); -- aggregate user_functions table LOCK TABLE powa_user_functions_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_user_functions_history SELECT dbid, funcid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).calls),min((record).total_time), min((record).self_time))::powa_user_functions_history_record, ROW(max((record).ts), max((record).calls),max((record).total_time), max((record).self_time))::powa_user_functions_history_record FROM powa_user_functions_history_current GROUP BY dbid, funcid; TRUNCATE powa_user_functions_history_current; END; $PROC$ LANGUAGE plpgsql; /* end of powa_user_functions_aggregate */ CREATE OR REPLACE FUNCTION powa_all_relations_aggregate() RETURNS void AS $PROC$ DECLARE v_funcname text := 'powa_all_relations_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate all_relations table LOCK TABLE powa_all_relations_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_all_relations_history SELECT dbid, relid, tstzrange(min((record).ts), max((record).ts),'[]'), array_agg(record), ROW(min((record).ts), min((record).numscan),min((record).tup_returned),min((record).tup_fetched), min((record).n_tup_ins),min((record).n_tup_upd), min((record).n_tup_del),min((record).n_tup_hot_upd), min((record).n_liv_tup),min((record).n_dead_tup), min((record).n_mod_since_analyze),min((record).blks_read), min((record).blks_hit),min((record).last_vacuum), min((record).vacuum_count),min((record).last_autovacuum), min((record).autovacuum_count),min((record).last_analyze), min((record).analyze_count),min((record).last_autoanalyze), min((record).autoanalyze_count))::powa_all_relations_history_record, ROW(max((record).ts), max((record).numscan),max((record).tup_returned),max((record).tup_fetched), max((record).n_tup_ins),max((record).n_tup_upd), max((record).n_tup_del),max((record).n_tup_hot_upd), max((record).n_liv_tup),max((record).n_dead_tup), max((record).n_mod_since_analyze),max((record).blks_read), max((record).blks_hit),max((record).last_vacuum), max((record).vacuum_count),max((record).last_autovacuum), max((record).autovacuum_count),max((record).last_analyze), max((record).analyze_count),max((record).last_autoanalyze), max((record).autoanalyze_count))::powa_all_relations_history_record FROM powa_all_relations_history_current GROUP BY dbid, relid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_all_relations_history_current; END; $PROC$ LANGUAGE plpgsql; /* end of powa_all_relations_aggregate */ CREATE OR REPLACE FUNCTION public.powa_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ DECLARE funcname text; v_state text; v_msg text; v_detail text; v_hint text; v_context text; BEGIN -- Find reset function for every supported datasource, including pgss -- Also call reset function even if they're not enabled FOR funcname IN SELECT function_name FROM powa_functions WHERE operation='reset' LOOP -- Call all of them, with no parameter BEGIN EXECUTE 'SELECT ' || quote_ident(funcname)||'()'; EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS v_state = RETURNED_SQLSTATE, v_msg = MESSAGE_TEXT, v_detail = PG_EXCEPTION_DETAIL, v_hint = PG_EXCEPTION_HINT, v_context = PG_EXCEPTION_CONTEXT; RAISE warning 'powa_reset(): function "%" failed: state : % message: % detail : % hint : % context: %', funcname, v_state, v_msg, v_detail, v_hint, v_context; END; END LOOP; RETURN true; END; $function$; /* end of powa_reset */ CREATE OR REPLACE FUNCTION public.powa_statements_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_statements_history'); TRUNCATE TABLE powa_statements_history; PERFORM powa_log('truncating powa_statements_history_current'); TRUNCATE TABLE powa_statements_history_current; PERFORM powa_log('truncating powa_statements_history_db'); TRUNCATE TABLE powa_statements_history_db; PERFORM powa_log('truncating powa_statements_history_current_db'); TRUNCATE TABLE powa_statements_history_current_db; PERFORM powa_log('truncating powa_statements'); -- if 3rd part datasource has FK on it, throw everything away TRUNCATE TABLE powa_statements CASCADE; RETURN true; END; $function$; /* end of powa_statements_reset */ CREATE OR REPLACE FUNCTION public.powa_user_functions_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_user_functions_history'); TRUNCATE TABLE powa_user_functions_history; PERFORM powa_log('truncating powa_user_functions_history_current'); TRUNCATE TABLE powa_user_functions_history_current; RETURN true; END; $function$; /* end of powa_user_functions_reset */ CREATE OR REPLACE FUNCTION public.powa_all_relations_reset() RETURNS boolean LANGUAGE plpgsql AS $function$ BEGIN PERFORM powa_log('truncating powa_all_relations_history'); TRUNCATE TABLE powa_all_relations_history; PERFORM powa_log('truncating powa_all_relations_history_current'); TRUNCATE TABLE powa_all_relations_history_current; RETURN true; END; $function$; /* end of powa_all_relations_reset */ /* pg_stat_kcache integration - part 2 */ /* * register pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_stat_kcache'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_stat_kcache'; IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_stat_kcache'); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_stat_kcache', 'snapshot', 'powa_kcache_snapshot', false, true), ('pg_stat_kcache', 'aggregate', 'powa_kcache_aggregate', false, true), ('pg_stat_kcache', 'unregister', 'powa_kcache_unregister', false, true), ('pg_stat_kcache', 'purge', 'powa_kcache_purge', false, true), ('pg_stat_kcache', 'reset', 'powa_kcache_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_kcache_register */ /* * unregister pg_stat_kcache extension */ CREATE OR REPLACE function public.powa_kcache_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_stat_kcache'); DELETE FROM public.powa_functions WHERE module = 'pg_stat_kcache'; RETURN true; END; $_$ language plpgsql; /* end of powa_kcache_unregister */ /* * powa_kcache snapshot collection. */ CREATE OR REPLACE FUNCTION powa_kcache_snapshot() RETURNS void as $PROC$ DECLARE result bool; v_funcname text := 'powa_kcache_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS ( SELECT * FROM pg_stat_kcache() k JOIN pg_roles r ON r.oid = k.userid WHERE NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), by_query AS ( INSERT INTO powa_kcache_metrics_current (queryid, dbid, userid, metrics) SELECT queryid, dbid, userid, (now(), reads, writes, user_time, system_time)::kcache_type FROM capture ), by_database AS ( INSERT INTO powa_kcache_metrics_current_db (dbid, metrics) SELECT dbid, (now(), sum(reads), sum(writes), sum(user_time), sum(system_time))::kcache_type FROM capture GROUP BY dbid ) SELECT COUNT(*) into v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END $PROC$ language plpgsql; /* end of powa_kcache_snapshot */ /* * powa_kcache aggregation */ CREATE OR REPLACE FUNCTION powa_kcache_aggregate() RETURNS void AS $PROC$ DECLARE result bool; v_funcname text := 'powa_kcache_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate metrics table LOCK TABLE powa_kcache_metrics_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics (coalesce_range, queryid, dbid, userid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), queryid, dbid, userid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current GROUP BY queryid, dbid, userid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_kcache_metrics_current; -- aggregate metrics_db table LOCK TABLE powa_kcache_metrics_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_kcache_metrics_db (coalesce_range, dbid, metrics, mins_in_range, maxs_in_range) SELECT tstzrange(min((metrics).ts), max((metrics).ts),'[]'), dbid, array_agg(metrics), ROW(min((metrics).ts), min((metrics).reads),min((metrics).writes),min((metrics).user_time), min((metrics).system_time))::kcache_type, ROW(max((metrics).ts), max((metrics).reads),max((metrics).writes),max((metrics).user_time), max((metrics).system_time))::kcache_type FROM powa_kcache_metrics_current_db GROUP BY dbid; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics_db) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_kcache_metrics_current_db; END $PROC$ language plpgsql; /* end of powa_kcache_aggregate */ /* * powa_kcache purge */ CREATE OR REPLACE FUNCTION powa_kcache_purge() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_kcache_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); DELETE FROM powa_kcache_metrics WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics) - rowcount: %s', v_funcname, v_rowcount)); DELETE FROM powa_kcache_metrics_db WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_kcache_metrics_db) - rowcount: %s', v_funcname, v_rowcount)); END; $PROC$ language plpgsql; /* end of powa_kcache_purge */ /* * powa_kcache reset */ CREATE OR REPLACE FUNCTION powa_kcache_reset() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_kcache_reset'; v_rowcount bigint; BEGIN PERFORM powa_log('running powa_kcache_reset'); PERFORM powa_log('truncating powa_kcache_metrics'); TRUNCATE TABLE powa_kcache_metrics; PERFORM powa_log('truncating powa_kcache_metrics_db'); TRUNCATE TABLE powa_kcache_metrics_db; PERFORM powa_log('truncating powa_kcache_metrics_current'); TRUNCATE TABLE powa_kcache_metrics_current; PERFORM powa_log('truncating powa_kcache_metrics_current_db'); TRUNCATE TABLE powa_kcache_metrics_current_db; END; $PROC$ language plpgsql; /* end of powa_kcache_reset */ -- By default, try to register pg_stat_kcache, in case it's alreay here SELECT * FROM public.powa_kcache_register(); /* end of pg_stat_kcache integration - part 2 */ /* pg_qualstats integration - part 2 */ /* * powa_qualstats_register */ CREATE OR REPLACE function public.powa_qualstats_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_qualstats'; IF ( v_ext_present) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE function_name IN ('powa_qualstats_snapshot', 'powa_qualstats_aggregate', 'powa_qualstats_purge'); IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_qualstats'); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_qualstats', 'snapshot', 'powa_qualstats_snapshot', false, true), ('pg_qualstats', 'aggregate', 'powa_qualstats_aggregate', false, true), ('pg_qualstats', 'unregister', 'powa_qualstats_unregister', false, true), ('pg_qualstats', 'purge', 'powa_qualstats_purge', false, true), ('pg_qualstats', 'reset', 'powa_qualstats_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_qualstats_register */ /* * powa_qualstats utility view for aggregating constvalues */ CREATE OR REPLACE VIEW powa_qualstats_aggregate_constvalues_current AS WITH consts AS ( SELECT qualid, queryid, dbid, userid, min(ts) as mints, max(ts) as maxts, sum(occurences) as occurences, sum(nbfiltered) as nbfiltered, sum(execution_count) as execution_count, constvalues FROM powa_qualstats_constvalues_history_current GROUP BY qualid, queryid, dbid, userid, constvalues ), groups AS ( SELECT qualid, queryid, dbid, userid, tstzrange(min(mints), max(maxts),'[]') FROM consts GROUP BY qualid, queryid, dbid, userid ) SELECT * FROM groups, LATERAL ( SELECT array_agg(constvalues) as mu FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY occurences desc LIMIT 20 ) s ) as mu, LATERAL ( SELECT array_agg(constvalues) as mf FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN execution_count = 0 THEN 0 ELSE nbfiltered / execution_count::numeric END DESC LIMIT 20 ) s ) as mf, LATERAL ( SELECT array_agg(constvalues) as lf FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY CASE WHEN execution_count = 0 THEN 0 ELSE nbfiltered / execution_count::numeric END DESC LIMIT 20 ) s ) as lf, LATERAL ( SELECT array_agg(constvalues) as me FROM ( SELECT (constvalues, occurences, nbfiltered, execution_count)::qual_values as constvalues FROM consts WHERE consts.qualid = groups.qualid AND consts.queryid = groups.queryid AND consts.dbid = groups.dbid AND consts.userid = groups.userid ORDER BY execution_count desc LIMIT 20 ) s ) as me; /* end of powa_qualstats_aggregate_constvalues_current */ CREATE OR REPLACE FUNCTION powa_qualstats_snapshot() RETURNS void as $PROC$ DECLARE result bool; v_funcname text := 'powa_qualstats_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS ( SELECT pgqs.*, s.query FROM pg_qualstats_by_query pgqs JOIN powa_statements s USING(queryid, dbid, userid) JOIN pg_roles r ON s.userid = r.oid AND NOT (r.rolname = ANY (string_to_array(current_setting('powa.ignored_users'),','))) ), missing_quals AS ( INSERT INTO powa_qualstats_quals (qualid, queryid, dbid, userid, quals) SELECT DISTINCT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, array_agg(DISTINCT q::qual_type) FROM capture qs, LATERAL (SELECT (unnest(quals)).*) as q WHERE NOT EXISTS ( SELECT 1 FROM powa_qualstats_quals nh WHERE nh.qualid = qs.qualnodeid AND nh.queryid = qs.queryid AND nh.dbid = qs.dbid AND nh.userid = qs.userid ) GROUP BY qualnodeid, queryid, dbid, userid RETURNING * ), by_qual AS ( INSERT INTO powa_qualstats_quals_history_current (qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered) SELECT qs.qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), sum(occurences), sum(execution_count), sum(nbfiltered) FROM capture as qs GROUP BY qualnodeid, qs.queryid, qs.dbid, qs.userid RETURNING * ), by_qual_with_const AS ( INSERT INTO powa_qualstats_constvalues_history_current(qualid, queryid, dbid, userid, ts, occurences, execution_count, nbfiltered, constvalues) SELECT qualnodeid, qs.queryid, qs.dbid, qs.userid, now(), occurences, execution_count, nbfiltered, constvalues FROM capture as qs ) SELECT COUNT(*) into v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; PERFORM pg_qualstats_reset(); END $PROC$ language plpgsql; /* end of powa_qualstats_snapshot */ /* * powa_qualstats aggregate */ CREATE OR REPLACE FUNCTION powa_qualstats_aggregate() RETURNS void AS $PROC$ DECLARE result bool; BEGIN PERFORM powa_log('running powa_qualstats_aggregate'); LOCK TABLE powa_qualstats_constvalues_history_current IN SHARE MODE; LOCK TABLE powa_qualstats_quals_history_current IN SHARE MODE; INSERT INTO powa_qualstats_constvalues_history ( qualid, queryid, dbid, userid, coalesce_range, most_used, most_filtering, least_filtering, most_executed) SELECT * FROM powa_qualstats_aggregate_constvalues_current; INSERT INTO powa_qualstats_quals_history (qualid, queryid, dbid, userid, coalesce_range, records, mins_in_range, maxs_in_range) SELECT qualid, queryid, dbid, userid, tstzrange(min(ts), max(ts),'[]'), array_agg((ts, occurences, execution_count, nbfiltered)::powa_qualstats_history_item), ROW(min(ts), min(occurences), min(execution_count), min(nbfiltered))::powa_qualstats_history_item, ROW(max(ts), max(occurences), max(execution_count), max(nbfiltered))::powa_qualstats_history_item FROM powa_qualstats_quals_history_current GROUP BY qualid, queryid, dbid, userid; TRUNCATE powa_qualstats_constvalues_history_current; TRUNCATE powa_qualstats_quals_history_current; END $PROC$ language plpgsql; /* end of powa_qualstats_aggregate */ /* * powa_qualstats_purge */ CREATE OR REPLACE FUNCTION powa_qualstats_purge() RETURNS void as $PROC$ BEGIN PERFORM powa_log('running powa_qualstats_purge'); DELETE FROM powa_qualstats_constvalues_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); DELETE FROM powa_qualstats_quals_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); END; $PROC$ language plpgsql; /* end of powa_qualstats_purge */ /* * powa_qualstats_reset */ CREATE OR REPLACE FUNCTION powa_qualstats_reset() RETURNS void as $PROC$ BEGIN PERFORM powa_log('running powa_qualstats_reset'); PERFORM powa_log('truncating powa_qualstats_quals'); TRUNCATE TABLE powa_qualstats_quals CASCADE; -- cascaded : -- powa_qualstats_quals_history -- powa_qualstats_quals_history_current -- powa_qualstats_constvalues_history -- powa_qualstats_constvalues_history_current END; $PROC$ language plpgsql; /* end of powa_qualstats_reset */ /* * powa_qualstats_unregister */ CREATE OR REPLACE function public.powa_qualstats_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_qualstats'); DELETE FROM public.powa_functions WHERE module = 'pg_qualstats'; RETURN true; END; $_$ language plpgsql; /* end of powa_qualstats_unregister */ SELECT * FROM public.powa_qualstats_register(); /* end of pg_qualstats integration - part 2 */ /* pg_track_settings integration */ CREATE OR REPLACE FUNCTION powa_track_settings_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_track_settings'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_track_settings'; IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_track_settings'); -- This extension handles its own storage, just its snapshot -- function and an unregister function. INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_track_settings', 'snapshot', 'pg_track_settings_snapshot', false, true), ('pg_track_settings', 'unregister', 'powa_track_settings_unregister', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of pg_track_settings_register */ CREATE OR REPLACE function public.powa_track_settings_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_track_settings'); DELETE FROM public.powa_functions WHERE module = 'pg_track_settings'; RETURN true; END; $_$ language plpgsql; /* end of powa_track_settings_unregister */ -- By default, try to register pg_track_settings, in case it's alreay here SELECT * FROM public.powa_track_settings_register(); /* end pg_track_settings integration */ /* pg_wait_sampling integration - part 2 */ /* * register pg_wait_sampling extension */ CREATE OR REPLACE function public.powa_wait_sampling_register() RETURNS bool AS $_$ DECLARE v_func_present bool; v_ext_present bool; BEGIN SELECT COUNT(*) = 1 INTO v_ext_present FROM pg_extension WHERE extname = 'pg_wait_sampling'; IF ( v_ext_present ) THEN SELECT COUNT(*) > 0 INTO v_func_present FROM public.powa_functions WHERE module = 'pg_wait_sampling'; IF ( NOT v_func_present) THEN PERFORM powa_log('registering pg_wait_sampling'); INSERT INTO powa_functions (module, operation, function_name, added_manually, enabled) VALUES ('pg_wait_sampling', 'snapshot', 'powa_wait_sampling_snapshot', false, true), ('pg_wait_sampling', 'aggregate', 'powa_wait_sampling_aggregate', false, true), ('pg_wait_sampling', 'unregister', 'powa_wait_sampling_unregister', false, true), ('pg_wait_sampling', 'purge', 'powa_wait_sampling_purge', false, true), ('pg_wait_sampling', 'reset', 'powa_wait_sampling_reset', false, true); END IF; END IF; RETURN true; END; $_$ language plpgsql; /* end of powa_wait_sampling_register */ /* * unregister pg_wait_sampling extension */ CREATE OR REPLACE function public.powa_wait_sampling_unregister() RETURNS bool AS $_$ BEGIN PERFORM powa_log('unregistering pg_wait_sampling'); DELETE FROM public.powa_functions WHERE module = 'pg_wait_sampling'; RETURN true; END; $_$ language plpgsql; /* * powa_wait_sampling snapshot collection. */ CREATE OR REPLACE FUNCTION powa_wait_sampling_snapshot() RETURNS void as $PROC$ DECLARE result bool; v_funcname text := 'powa_wait_sampling_snapshot'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); WITH capture AS ( -- the various background processes report wait events but don't have -- associated queryid. Gather them all under a fake 0 dbid SELECT COALESCE(pgss.dbid, 0) AS dbid, s.event_type, s.event, s.queryid, sum(s.count) as count FROM pg_wait_sampling_profile s -- pg_wait_sampling doesn't offer a per (userid, dbid, queryid) view, -- only per pid, but pid can be reused for different databases or users -- so we cannot deduce db or user from it. However, queryid should be -- unique across differet databases, so we retrieve the dbid this way. LEFT JOIN pg_stat_statements(false) pgss ON pgss.queryid = s.queryid WHERE event_type IS NOT NULL AND event IS NOT NULL GROUP BY pgss.dbid, s.event_type, s.event, s.queryid ), by_query AS ( INSERT INTO powa_wait_sampling_history_current (queryid, dbid, event_type, event, record) SELECT queryid, dbid, event_type, event, (now(), count)::wait_sampling_type FROM capture ), by_database AS ( INSERT INTO powa_wait_sampling_history_current_db (dbid, event_type, event, record) SELECT dbid, event_type, event, (now(), sum(count))::wait_sampling_type FROM capture GROUP BY dbid, event_type, event ) SELECT COUNT(*) into v_rowcount FROM capture; perform powa_log(format('%I - rowcount: %s', v_funcname, v_rowcount)); result := true; END $PROC$ language plpgsql; /* end of powa_wait_sampling_snapshot */ /* * powa_wait_sampling aggregation */ CREATE OR REPLACE FUNCTION powa_wait_sampling_aggregate() RETURNS void AS $PROC$ DECLARE result bool; v_funcname text := 'powa_wait_sampling_aggregate'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); -- aggregate history table LOCK TABLE powa_wait_sampling_history_current IN SHARE MODE; -- prevent any other update INSERT INTO powa_wait_sampling_history (coalesce_range, queryid, dbid, event_type, event, records, mins_in_range, maxs_in_range) SELECT tstzrange(min((record).ts), max((record).ts),'[]'), queryid, dbid, event_type, event, array_agg(record), ROW(min((record).ts), min((record).count))::wait_sampling_type, ROW(max((record).ts), max((record).count))::wait_sampling_type FROM powa_wait_sampling_history_current GROUP BY queryid, dbid, event_type, event; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_wait_sampling_history) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_wait_sampling_history_current; -- aggregate history_db table LOCK TABLE powa_wait_sampling_history_current_db IN SHARE MODE; -- prevent any other update INSERT INTO powa_wait_sampling_history_db (coalesce_range, dbid, event_type, event, records, mins_in_range, maxs_in_range) SELECT tstzrange(min((record).ts), max((record).ts),'[]'), dbid, event_type, event, array_agg(record), ROW(min((record).ts), min((record).count))::wait_sampling_type, ROW(max((record).ts), max((record).count))::wait_sampling_type FROM powa_wait_sampling_history_current_db GROUP BY dbid, event_type, event; GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_wait_sampling_history_db) - rowcount: %s', v_funcname, v_rowcount)); TRUNCATE powa_wait_sampling_history_current_db; END $PROC$ language plpgsql; /* end of powa_wait_sampling_aggregate */ /* * powa_wait_sampling purge */ CREATE OR REPLACE FUNCTION powa_wait_sampling_purge() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_wait_sampling_purge'; v_rowcount bigint; BEGIN PERFORM powa_log(format('running %I', v_funcname)); DELETE FROM powa_wait_sampling_history WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_wait_sampling_history) - rowcount: %s', v_funcname, v_rowcount)); DELETE FROM powa_wait_sampling_history_db WHERE upper(coalesce_range) < (now() - current_setting('powa.retention')::interval); GET DIAGNOSTICS v_rowcount = ROW_COUNT; perform powa_log(format('%I (powa_wait_sampling_history_db) - rowcount: %s', v_funcname, v_rowcount)); END; $PROC$ language plpgsql; /* end of powa_wait_sampling_purge */ /* * powa_wait_sampling reset */ CREATE OR REPLACE FUNCTION powa_wait_sampling_reset() RETURNS void as $PROC$ DECLARE v_funcname text := 'powa_wait_sampling_reset'; v_rowcount bigint; BEGIN PERFORM powa_log('running powa_wait_sampling_reset'); PERFORM powa_log('truncating powa_wait_sampling_history'); TRUNCATE TABLE powa_wait_sampling_history; PERFORM powa_log('truncating powa_wait_sampling_history_db'); TRUNCATE TABLE powa_wait_sampling_history_db; PERFORM powa_log('truncating powa_wait_sampling_history_current'); TRUNCATE TABLE powa_wait_sampling_history_current; PERFORM powa_log('truncating powa_wait_sampling_history_current_db'); TRUNCATE TABLE powa_wait_sampling_history_current_db; END; $PROC$ language plpgsql; /* end of powa_wait_sampling_reset */ -- By default, try to register pg_wait_sampling, in case it's alreay here SELECT * FROM public.powa_wait_sampling_register(); /* end of pg_wait_sampling integration - part 2 */ powa-archivist-REL_3_2_0/powa.c000066400000000000000000000434521336072707400164530ustar00rootroot00000000000000/*------------------------------------------------------------------------- * * powa.c: PoWA background worker * * This program is open source, licensed under the PostgreSQL license. * For license terms, see the LICENSE file. * *------------------------------------------------------------------------- */ #include "postgres.h" #if PG_VERSION_NUM < 90400 #error "PoWA requires PostgreSQL 9.4 or later" #endif /* For a bgworker */ #include "miscadmin.h" #include "postmaster/bgworker.h" #include "storage/ipc.h" #include "storage/latch.h" #include "storage/lwlock.h" #include "storage/proc.h" #include "storage/shmem.h" /* Access a database */ #include "access/xact.h" #include "executor/spi.h" #include "fmgr.h" #include "lib/stringinfo.h" #include "utils/builtins.h" #include "utils/snapmgr.h" /* Some catalog elements */ #include "catalog/pg_type.h" #include "utils/timestamp.h" /* There is a GUC */ #include "utils/guc.h" /* We use tuplestore */ #include "funcapi.h" /* pgsats access */ #include "pgstat.h" /* rename process */ #include "utils/ps_status.h" PG_MODULE_MAGIC; #define POWA_STAT_FUNC_COLS 4 /* # of cols for functions stat SRF */ #define POWA_STAT_TAB_COLS 21 /* # of cols for relations stat SRF */ #define MIN_POWA_FREQUENCY 5000 /* minimum ms between two snapshots */ typedef enum { POWA_STAT_FUNCTION, POWA_STAT_TABLE } PowaStatKind; void _PG_init(void); static bool powa_check_frequency_hook(int *newval, void **extra, GucSource source); static void compute_powa_frequency(void); static int64 compute_next_wakeup(void); Datum powa_stat_user_functions(PG_FUNCTION_ARGS); Datum powa_stat_all_rel(PG_FUNCTION_ARGS); static Datum powa_stat_common(PG_FUNCTION_ARGS, PowaStatKind kind); PG_FUNCTION_INFO_V1(powa_stat_user_functions); PG_FUNCTION_INFO_V1(powa_stat_all_rel); #if (PG_VERSION_NUM >= 90500) void powa_main(Datum main_arg) pg_attribute_noreturn(); #else void powa_main(Datum main_arg) __attribute__((noreturn)); #endif static void powa_sighup(SIGNAL_ARGS); static void powa_process_sighup(void); static instr_time last_start; /* last snapshot start */ static int powa_frequency; /* powa.frequency GUC */ static instr_time time_powa_frequency; /* same in instr_time format */ static bool force_snapshot = false; /* used to force snapshot after enabling powa */ static int powa_retention; /* powa.retention GUC */ static int powa_coalesce; /* powa.coalesce GUC */ static char *powa_database = NULL; /* powa.database GUC */ static char *powa_ignored_users = NULL; /* powa.ignored_users GUC */ static bool powa_debug = false; /* powa.debug GUC */ /* flags set by signal handlers */ static volatile sig_atomic_t got_sighup = false; static bool powa_check_frequency_hook(int *newval, void **extra, GucSource source) { if (*newval >= MIN_POWA_FREQUENCY || *newval == -1) return true; return false; } static void compute_powa_frequency(void) { int local_frequency = powa_frequency; /* * If PoWA is deactivated, artitrarily set a sleep time of one hour to save * resources. The actual sleep time is not problematic since the * reactivation can only be done with a sighup, which will set the latch * used for the sleep. */ if (powa_frequency == -1) local_frequency = 3600000; /* Initialize time_powa_frequency to do maths with it */ #ifndef WIN32 #ifdef HAVE_CLOCK_GETTIME time_powa_frequency.tv_sec = local_frequency / 1000; /* Seconds */ time_powa_frequency.tv_nsec = 0; #else /* !HAVE_CLOCK_GETTIME */ time_powa_frequency.tv_sec = local_frequency / 1000; /* Seconds */ time_powa_frequency.tv_usec = 0; #endif /* HAVE_CLOCK_GETTIME */ #else /* WIN32 */ time_powa_frequency.QuadPart = local_frequency / 1000 * GetTimerFrequency(); #endif /* WIN32 */ } static int64 compute_next_wakeup(void) { instr_time time_to_wait, now; /* * If powa was deactivated and is now reactivated, we force an immediate * snapshot, and start usual interval sleep. In order to do that, we have * to setup a new last_start reference to now minus powa_frequency so that * the next calls to this function will return expected sleep time. */ if (force_snapshot) { force_snapshot = false; INSTR_TIME_SET_CURRENT(last_start); INSTR_TIME_SUBTRACT(last_start, time_powa_frequency); return 0; } memcpy(&time_to_wait, &last_start, sizeof(last_start)); INSTR_TIME_ADD(time_to_wait, time_powa_frequency); INSTR_TIME_SET_CURRENT(now); INSTR_TIME_SUBTRACT(time_to_wait, now); return INSTR_TIME_GET_MICROSEC(time_to_wait); } void _PG_init(void) { BackgroundWorker worker; if (!process_shared_preload_libraries_in_progress) { elog(ERROR, "This module can only be loaded via shared_preload_libraries"); return; } DefineCustomIntVariable("powa.frequency", "Defines the frequency in seconds of the snapshots", NULL, &powa_frequency, 300000, -1, INT_MAX / 1000, PGC_SUSET, GUC_UNIT_MS, powa_check_frequency_hook, NULL, NULL); DefineCustomIntVariable("powa.coalesce", "Defines the amount of records to group together in the table (more compact)", NULL, &powa_coalesce, 100, 5, INT_MAX, PGC_SUSET, 0, NULL, NULL, NULL); DefineCustomIntVariable("powa.retention", "Automatically purge data older than N minutes", NULL, &powa_retention, HOURS_PER_DAY * MINS_PER_HOUR, 0, INT_MAX / SECS_PER_MINUTE, PGC_SUSET, GUC_UNIT_MIN, NULL, NULL, NULL); DefineCustomStringVariable("powa.database", "Defines the database of the workload repository", NULL, &powa_database, "powa", PGC_POSTMASTER, 0, NULL, NULL, NULL); DefineCustomStringVariable("powa.ignored_users", "Defines a coma-separated list of users to ignore when taking activity snapshot", NULL, &powa_ignored_users, NULL, PGC_SIGHUP, 0, NULL, NULL, NULL); DefineCustomBoolVariable("powa.debug", "Provide logs to help troubleshooting issues", NULL, &powa_debug, false, PGC_USERSET, 0, NULL, NULL, NULL); EmitWarningsOnPlaceholders("powa"); /* * Register the worker processes */ worker.bgw_flags = BGWORKER_SHMEM_ACCESS | BGWORKER_BACKEND_DATABASE_CONNECTION; worker.bgw_start_time = BgWorkerStart_RecoveryFinished; /* Must write to the * database */ #if (PG_VERSION_NUM >= 100000) snprintf(worker.bgw_library_name, BGW_MAXLEN, "powa"); snprintf(worker.bgw_function_name, BGW_MAXLEN, "powa_main"); #else worker.bgw_main = powa_main; #endif snprintf(worker.bgw_name, BGW_MAXLEN, "powa"); worker.bgw_restart_time = 10; worker.bgw_main_arg = (Datum) 0; #if (PG_VERSION_NUM >= 90400) worker.bgw_notify_pid = 0; #endif RegisterBackgroundWorker(&worker); } void powa_main(Datum main_arg) { char *query_snapshot = "SELECT powa_take_snapshot()"; static char *query_appname = "SET application_name = 'PoWA collector'"; int64 us_to_wait; /* Should be uint64 per postgresql's spec, but we may have negative result, in our tests */ /* check powa_frequency validity, and if powa is enabled */ compute_powa_frequency(); /* * Set up signal handler, then unblock signals */ pqsignal(SIGHUP, powa_sighup); BackgroundWorkerUnblockSignals(); /* Define the snapshot reference time when the bgworker start */ INSTR_TIME_SET_CURRENT(last_start); /* Connect to POWA database */ #if PG_VERSION_NUM >= 110000 BackgroundWorkerInitializeConnection(powa_database, NULL, 0); #else BackgroundWorkerInitializeConnection(powa_database, NULL); #endif elog(LOG, "POWA connected to database %s", quote_identifier(powa_database)); set_ps_display("init", false); StartTransactionCommand(); SetCurrentStatementStartTimestamp(); SPI_connect(); PushActiveSnapshot(GetTransactionSnapshot()); pgstat_report_activity(STATE_RUNNING, query_appname); SPI_execute(query_appname, false, 0); SPI_finish(); PopActiveSnapshot(); CommitTransactionCommand(); pgstat_report_activity(STATE_IDLE, NULL); set_ps_display("idle", false); /*------------------ * Main loop of POWA * We exit from here if: * - we got a SIGINT (default bgworker sig handlers) * - powa.frequency becomes < 0 (change config and SIGHUP) */ for (;;) { /* Check if a SIGHUP has been received */ powa_process_sighup(); if (powa_frequency != -1) { set_ps_display("snapshot", false); SetCurrentStatementStartTimestamp(); StartTransactionCommand(); SPI_connect(); PushActiveSnapshot(GetTransactionSnapshot()); pgstat_report_activity(STATE_RUNNING, query_snapshot); SPI_execute(query_snapshot, false, 0); pgstat_report_activity(STATE_RUNNING, query_appname); SPI_execute(query_appname, false, 0); SPI_finish(); PopActiveSnapshot(); CommitTransactionCommand(); pgstat_report_stat(false); pgstat_report_activity(STATE_IDLE, NULL); set_ps_display("idle", false); } /* sleep loop */ for (;;) { StringInfoData buf; /* Check if a SIGHUP has been received */ powa_process_sighup(); /* * Compute if there is still some time to wait (we could have been * woken up by a latch, or snapshot took more than frequency) */ us_to_wait = compute_next_wakeup(); if (us_to_wait <= 0) break; /* Tell the world we are waiting */ elog(DEBUG1, "Waiting for %li milliseconds", us_to_wait/1000); initStringInfo(&buf); appendStringInfo(&buf, "-- sleeping for %li seconds", us_to_wait / 1000000); pgstat_report_activity(STATE_IDLE, buf.data); pfree(buf.data); /* sleep */ WaitLatch(&MyProc->procLatch, WL_LATCH_SET | WL_TIMEOUT | WL_POSTMASTER_DEATH, us_to_wait/1000 #if PG_VERSION_NUM >= 100000 ,PG_WAIT_EXTENSION #endif ); ResetLatch(&MyProc->procLatch); } /* end of sleep loop */ /* * We've stop waiting. Let's increment the snapshot reference time to * it's ideal target, not to now, so errors don't add up */ INSTR_TIME_ADD(last_start, time_powa_frequency); } /* end of snapshot loop */ } /* * Signal handler for SIGHUP * Set a flag to tell the main loop to reread the config file, do sanity * check, recompute the frequency and set our latch to wake it up. */ static void powa_sighup(SIGNAL_ARGS) { int save_errno = errno; got_sighup = true; if (MyProc) SetLatch(&MyProc->procLatch); errno = save_errno; } /* * Do all the needed work if a SIGHUP has been received * - reread the config file * - exit the bgworker if the frequency is invalid * - compute the time_powa_frequency var */ static void powa_process_sighup(void) { if (got_sighup) { int old_powa_frequency = powa_frequency; got_sighup = false; ProcessConfigFile(PGC_SIGHUP); /* setup force_snapshot if powa is reactivated */ if (old_powa_frequency == -1 && powa_frequency != -1) { elog(LOG, "PoWA is activated"); force_snapshot = (old_powa_frequency == -1 && powa_frequency != -1); } else if (old_powa_frequency != -1 && powa_frequency == -1) elog(LOG, "PoWA is deactivated"); compute_powa_frequency(); } } Datum powa_stat_user_functions(PG_FUNCTION_ARGS) { return powa_stat_common(fcinfo, POWA_STAT_FUNCTION); } Datum powa_stat_all_rel(PG_FUNCTION_ARGS) { return powa_stat_common(fcinfo, POWA_STAT_TABLE); } static Datum powa_stat_common(PG_FUNCTION_ARGS, PowaStatKind kind) { Oid dbid = PG_GETARG_OID(0); Oid backend_dbid; ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; MemoryContext per_query_ctx; MemoryContext oldcontext; TupleDesc tupdesc; Tuplestorestate *tupstore; PgStat_StatDBEntry *dbentry; HASH_SEQ_STATUS hash_seq; /* check to see if caller supports us returning a tuplestore */ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo)) ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("set-valued function called in context that cannot accept a set"))); if (!(rsinfo->allowedModes & SFRM_Materialize)) ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("materialize mode required, but it is not " \ "allowed in this context"))); per_query_ctx = rsinfo->econtext->ecxt_per_query_memory; oldcontext = MemoryContextSwitchTo(per_query_ctx); /* Build a tuple descriptor for our result type */ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE) elog(ERROR, "return type must be a row type"); tupstore = tuplestore_begin_heap(true, false, work_mem); rsinfo->returnMode = SFRM_Materialize; rsinfo->setResult = tupstore; rsinfo->setDesc = tupdesc; MemoryContextSwitchTo(oldcontext); /* ----------------------------------------------------- * Force deep statistics retrieval of specified database. * * Deep means to also include tables and functions HTAB, which is what we * want here. * * The stat collector isn't suppose to act this way, since a backend can't * access data outside the database it's connected to. It's not a problem * here since we only need the identifier that are stored in the pgstats, * the UI will connect to the database to do the lookup. * * So, to ensure we'll have fresh statitics of the wanted database, we have * to do following (ugly) tricks: * * - clear the current statistics cache. If a previous function already * asked for statistics in the same transaction, calling * pgstat_fetch_stat_dbentry would() just return the cache, which would * probably belong to another database. As the powa snapshot works inside * a function, we have the guarantee that this function will be called for * all the databases in a single transaction anyway. * * - change the global var MyDatabaseId to the wanted databaseid. pgstat * is designed to only retrieve statistics for current database, so we * need to fool it. * * - call pgstat_fetch_stat_dbentry(). * * - restore MyDatabaseId. * * - and finally clear again the statistics cache, to make sure any further * statement in the transaction will see the data related to the right * database. * * The pgstat_fetch_stat_dbentry() has to be done inside a PG_TRY bloc so * we make sure that MyDatabaseId is restored and the statistics cache is * cleared if an error happens during the call. */ pgstat_clear_snapshot(); backend_dbid = MyDatabaseId; MyDatabaseId = dbid; PG_TRY(); { dbentry = pgstat_fetch_stat_dbentry(dbid); } PG_CATCH(); { MyDatabaseId = backend_dbid; pgstat_clear_snapshot(); PG_RE_THROW(); } PG_END_TRY(); MyDatabaseId = backend_dbid; if (dbentry != NULL && dbentry->functions != NULL && dbentry->tables != NULL) { switch (kind) { case POWA_STAT_FUNCTION: { PgStat_StatFuncEntry *funcentry = NULL; hash_seq_init(&hash_seq, dbentry->functions); while ((funcentry = hash_seq_search(&hash_seq)) != NULL) { Datum values[POWA_STAT_FUNC_COLS]; bool nulls[POWA_STAT_FUNC_COLS]; int i = 0; memset(values, 0, sizeof(values)); memset(nulls, 0, sizeof(nulls)); values[i++] = ObjectIdGetDatum(funcentry->functionid); values[i++] = Int64GetDatum(funcentry->f_numcalls); values[i++] = Float8GetDatum(((double) funcentry->f_total_time) / 1000.0); values[i++] = Float8GetDatum(((double) funcentry->f_self_time) / 1000.0); Assert(i == POWA_STAT_FUNC_COLS); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } break; } case POWA_STAT_TABLE: { PgStat_StatTabEntry *tabentry = NULL; hash_seq_init(&hash_seq, dbentry->tables); while ((tabentry = hash_seq_search(&hash_seq)) != NULL) { Datum values[POWA_STAT_TAB_COLS]; bool nulls[POWA_STAT_TAB_COLS]; int i = 0; memset(values, 0, sizeof(values)); memset(nulls, 0, sizeof(nulls)); /* Oid of the table (or index) */ values[i++] = ObjectIdGetDatum(tabentry->tableid); values[i++] = Int64GetDatum((int64) tabentry->numscans); values[i++] = Int64GetDatum((int64) tabentry->tuples_returned); values[i++] = Int64GetDatum((int64) tabentry->tuples_fetched); values[i++] = Int64GetDatum((int64) tabentry->tuples_inserted); values[i++] = Int64GetDatum((int64) tabentry->tuples_updated); values[i++] = Int64GetDatum((int64) tabentry->tuples_deleted); values[i++] = Int64GetDatum((int64) tabentry->tuples_hot_updated); values[i++] = Int64GetDatum((int64) tabentry->n_live_tuples); values[i++] = Int64GetDatum((int64) tabentry->n_dead_tuples); values[i++] = Int64GetDatum((int64) tabentry->changes_since_analyze); values[i++] = Int64GetDatum((int64) (tabentry->blocks_fetched - tabentry->blocks_hit)); values[i++] = Int64GetDatum((int64) tabentry->blocks_hit); /* last vacuum */ if (tabentry->vacuum_timestamp == 0) nulls[i++] = true; else values[i++] = TimestampTzGetDatum(tabentry->vacuum_timestamp); values[i++] = Int64GetDatum((int64) tabentry->vacuum_count); /* last_autovacuum */ if (tabentry->autovac_vacuum_timestamp == 0) nulls[i++] = true; else values[i++] = TimestampTzGetDatum(tabentry->autovac_vacuum_timestamp); values[i++] = Int64GetDatum((int64) tabentry->autovac_vacuum_count); /* last_analyze */ if (tabentry->analyze_timestamp == 0) nulls[i++] = true; else values[i++] = TimestampTzGetDatum(tabentry->analyze_timestamp); values[i++] = Int64GetDatum((int64) tabentry->analyze_count); /* last_autoanalyze */ if (tabentry->autovac_analyze_timestamp == 0) nulls[i++] = true; else values[i++] = TimestampTzGetDatum(tabentry->autovac_analyze_timestamp); values[i++] = Int64GetDatum((int64) tabentry->autovac_analyze_count); Assert(i == POWA_STAT_TAB_COLS); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } break; } } } /* * Make sure any subsequent statistic retrieving will not see the one we * just fetched */ pgstat_clear_snapshot(); /* clean up and return the tuplestore */ tuplestore_donestoring(tupstore); return (Datum) 0; } powa-archivist-REL_3_2_0/powa.control000066400000000000000000000003461336072707400177040ustar00rootroot00000000000000# powa extension comment = 'PostgreSQL Workload Analyser-core' default_version = '3.2.0' module_pathname = '$libdir/powa' requires = 'plpgsql, pg_stat_statements, btree_gist' superuser = true relocatable = false schema = 'public' powa-archivist-REL_3_2_0/reindent.sh000077500000000000000000000005571336072707400175070ustar00rootroot00000000000000#/bin/sh # This reindents all C files in the directory following PG rules (ported to gnu indent) # as some of us have tab=8 spaces, and others tab=4 spaces indent -bad -bap -bbo -bbb -bc -bl -brs -c33 -cd33 -cdb -nce -ci4 -cli0 \ -cp33 -cs -d0 -di12 -nfc1 -nfca -nfc1 -i4 -nip -l79 -lp -nip -npcs \ -nprs -npsl -saf -sai -saw -nsc -nsob -nss -nut *.c powa-archivist-REL_3_2_0/sql/000077500000000000000000000000001336072707400161305ustar00rootroot00000000000000powa-archivist-REL_3_2_0/sql/powa-archivist.sql000066400000000000000000000047621336072707400216220ustar00rootroot00000000000000--Setup extension CREATE EXTENSION pg_stat_statements; CREATE EXTENSION btree_gist; CREATE EXTENSION powa; -- Aggregate data every 5 snapshots SET powa.coalesce = 5; -- Test created ojects SELECT * FROM powa_functions ORDER BY module, operation; -- test C SRFs SELECT COUNT(*) = 0 FROM pg_database, LATERAL powa_stat_user_functions(oid) f WHERE datname = current_database(); SELECT COUNT(*) > 10 FROM pg_database, LATERAL powa_stat_all_rel(oid) WHERE datname = current_database(); -- Test snapshot SELECT COUNT(*) = 0 FROM powa_user_functions_history_current; SELECT COUNT(*) = 0 FROM powa_all_relations_history_current; SELECT COUNT(*) = 0 FROM powa_statements_history_current; SELECT COUNT(*) = 0 FROM powa_statements_history_current_db; SELECT COUNT(*) = 0 FROM powa_user_functions_history; SELECT COUNT(*) = 0 FROM powa_all_relations_history; SELECT COUNT(*) = 0 FROM powa_statements_history; SELECT COUNT(*) = 0 FROM powa_statements_history; SELECT powa_take_snapshot(); SELECT COUNT(*) >= 0 FROM powa_user_functions_history_current; SELECT COUNT(*) > 0 FROM powa_all_relations_history_current; SELECT COUNT(*) > 0 FROM powa_statements_history_current; SELECT COUNT(*) > 0 FROM powa_statements_history_current_db; SELECT COUNT(*) = 0 FROM powa_user_functions_history; SELECT COUNT(*) = 0 FROM powa_all_relations_history; SELECT COUNT(*) = 0 FROM powa_statements_history; SELECT COUNT(*) = 0 FROM powa_statements_history; SELECT powa_take_snapshot(); SELECT powa_take_snapshot(); SELECT powa_take_snapshot(); -- This snapshot will trigger the aggregate SELECT powa_take_snapshot(); SELECT COUNT(*) = 0 FROM powa_user_functions_history_current; SELECT COUNT(*) = 0 FROM powa_all_relations_history_current; SELECT COUNT(*) = 0 FROM powa_statements_history_current; SELECT COUNT(*) = 0 FROM powa_statements_history_current_db; SELECT COUNT(*) >= 0 FROM powa_user_functions_history; SELECT COUNT(*) > 0 FROM powa_all_relations_history; SELECT COUNT(*) > 0 FROM powa_statements_history; SELECT COUNT(*) > 0 FROM powa_statements_history; -- Test reset function SELECT * from powa_reset(); SELECT COUNT(*) = 0 FROM powa_user_functions_history_current; SELECT COUNT(*) = 0 FROM powa_all_relations_history_current; SELECT COUNT(*) = 0 FROM powa_statements_history_current; SELECT COUNT(*) = 0 FROM powa_statements_history_current_db; SELECT COUNT(*) = 0 FROM powa_user_functions_history; SELECT COUNT(*) = 0 FROM powa_all_relations_history; SELECT COUNT(*) = 0 FROM powa_statements_history; SELECT COUNT(*) = 0 FROM powa_statements_history;