As part of a new and fairly large project I have a need to partition a few postgres tables and have a rolling daily window. That is.. I want to organize data by a timestamp storing each day in its own partition and maintain 90 days of historical data. Doing this is possible in Postgres but it’s not pretty or very clean to set it up. To simplify the process I wrote this perl script that (when run daily) will pre-create a certain number of empty partitions into the future and remove the oldest partitions from your window.
The script is generalized so as to be easy to modify and there isn’t much here that’s specific to postgres.. so it could easily be adapted for use with other systems like Oracle. You will need to put in the DDL for the sub tables you will create but otherwise it’s pretty straight forward. Please let me know if you find this useful as I couldn’t find anything else out there like it.
Expand to see how you call it.
This script implements a rolling window of _daily_ partitions in a postgres
database. This means.. you partition a table on some timestamp/date column
and maintain a fixed number of days into the past and pre-create empty
partitions x days into the future. It's intended to be called daily to
drop the oldest partition and create a new one for future use. However,
by calling it on a different schedule and with larger windows it could be
used for weekly or monthly schedules etc.
To use this you will need to customize the routine that creates your child
table(s) along with their associated indexes and other parameters. The
script is made to be generalized.. so that you may pre-define several child
table templates and then call this script from cron with different
window periods for each table.
Syntax
pgDynamicPartitions.pl [-h dbhost] [-d database] [-t table] [-p past] [-f future]
-h [host] DB Host to operate on
-d [database] Database to connect to
-t [table] Parent table name to partition
-p [days] How many daily partitions to keep into the past
Ones older than this will be dropped.
-f [days] How many empty daily partitions to be created into the future.
-n Don't make changes just show what would be done.