When installing PostgreSQL 9.3 onto a CentOS 6 system, you may notice that some postgres commands appear to be missing (like pg_ctl, initdb, and pg_config). However, they actually are on your system but just not in your path. You should be able to find them in /usr/pgsql-9.3/bin/. This can be frustrating if you don't know that they are there.

To solve the problem, you could just use full paths to these commands, like /usr/pgsql-9.3/bin/initdb, but that may get ugly quick depending on how you are calling these commands. Instead, we can add them to the path.

You could just copy them to /usr/bin/ or create symlinks for them, but both of these methods are hack-ish and could have unintended consequences. Another option is to add /usr/pgsql-9.3/bin/ to your path manually, or to the path for all users by adding it to /etc/bashrc. But again, that seems hack-ish and when you upgrade postgres to 9.4 down the road things will break again.

So instead, let's look at how Postgres' other commands get installed when you install the rpm. When you run "yum install postgresql93" the rpm contains not only all the files for that package but it also includes some scripts to run (in this case one at install time and another at uninstall time). To view everything that is getting installed and to see the scripts that are run use this command: "rpm -qilv --scripts postgresql93". There will be a lot of output, but in there you will see:

...
postinstall scriptlet (using /bin/sh):
/usr/sbin/update-alternatives --install /usr/bin/psql pgsql-psql /usr/pgsql-9.3/bin/psql 930
/usr/sbin/update-alternatives --install /usr/bin/clusterdb  pgsql-clusterdb  /usr/pgsql-9.3/bin/clusterdb 930
/usr/sbin/update-alternatives --install /usr/bin/createdb   pgsql-createdb   /usr/pgsql-9.3/bin/createdb 930
...

That line "postinstall scriptlet (using /bin/sh):" marks the beginning of the list of commands that are run at install time. Ah ha! It runs update-alternatives! If you'r note familiar with alternatives, the short description is that it keeps track of different versions of things installed on your system and automatically manages symlinks to the version you want to run.

Now, not all the commands we're interested in are installed by the package "postgresql93"- you can search through the output and see that pg_config gets installed but is not set up in alternatives. The commands initdb and pg_ctl are part of the package "postgresql93-server". If we run the same command to view its files and scripts we'll see something interesting- it doesn't set up any of its commands using alternatives! Grrr. :-(

In the postgresql93-server package the preinstall and postinstall scripts only set up the postgres user on the system, set up /var/log/pgsql, add postgres to the init scripts, and set up the postgres user's .bash_profile. That's it. But, now that we know what commands are run for getting psql, clusterdb, and createdb into the path, we can manually run the same commands for the postgres commands that we need. Like this:

/usr/sbin/update-alternatives --install /usr/bin/initdb pgsql-initdb /usr/pgsql-9.3/bin/initdb 930
/usr/sbin/update-alternatives --install /usr/bin/pg_ctl pgsql-pg_ctl /usr/pgsql-9.3/bin/pg_ctl 930
/usr/sbin/update-alternatives --install /usr/bin/pg_config pgsql-pg_config /usr/pgsql-9.3/bin/pg_config 930

These commands should be available in your path now and are set up the same as all your other postgres commands. Now, the question about why these commands are not added to alternatives like all the others is a good one. I don't know. If you have an idea, please leave it in the comments. But at least now you have a decent work-around.

Seriously, you should. You should submit a talk proposal to PGConf US 2015– the worst thing that will happen is the talk committee will say “no” and offer a bunch of reasons to help you get your talk approved next year! Believe it or not, speaking at a PostgreSQL conference is a great way to help the community at large, and I hope this personal story I am going to share will shed some light as to why.

read more

PGDay 2015 San Francisco will be held March 10th 2015 in Hyatt, San Franciso Airport, Burlingame, CA (Just outside of San Francisco). This year PGDay will be hosted along-side Free and Open Source Geospatial North America (FOSS4GNA) conference 2015 which runs March 9th-12th and EclipseCon NA 2015. Speaker submissions for FOSS4GNA 2015 and EclipseCon NA 2015 will end this Monday November 17th, 2015.

It's that time of the year again - we've wrapped PGConf.EU 2014, and I've just closed the feedback system, so it's time to take a look at what's been said.

We're keeping fairly consistent numbers with previous years, which is something we are definitely happy with. We did have a slight drop in "overall view", since this year we had 8% ranking us as 3, a worse score than we saw last year, and we had a couple of fewer people voting 5. And a slight shift from 5 to 4 on the programme. The numbers are still good of course, but since we had a tiny drop last year as well, we need to step our game back up for next year!

This year we had a slightly bigger spread of how users identify themselves, seeing most categories chip away a little on DBAs and Developers, but they are still definitely the dominating categories. We also have a lot of returning developers - it's cool to see so many people who have been to every one of our events so far, combined with a full 25% being first-time attendees!

JD wrote:

A couple of weeks ago I spoke at Bellingham Linux User Group with a talk entitled: An Evening with PostgreSQL. It was an enlightening talk for me because it was the first time, in a long time, that I have spoke to a non-postgresql community. Most of the people in attendance were Linux Users of course but also a few Mongo as well as MySQL users. I was asked questions such as, "Why would I use PostgreSQL over MySQL?". To be honest, I didn't even realize that was still a question but it opens up a huge advocacy opportunity.

read more

SFPUG has a new home page, here. This page will be used to post our meetups, as well as local events including the upcoming pgDay SF. It runs on Ghost and PostgreSQL 9.3.

Stay tuned!

SFPUG has a new home page, here. This page will be used to post our meetups, as well as local events including the upcoming pgDay SF. It runs on Ghost and PostgreSQL 9.3.

Stay tuned!

On Linux, a "compiler" is usually a synonym to gcc, but clang is gaining more and more adoption. Over the years, phoronix published severalarticlescomparing of performance of various clang and gcc versions, suggesting that while clang improves over time, gcc still wins in most benchmarks - except maybe "compilation time" where clang is a clear winner. But none of the benchmarks is really a database-style application, so the question is how much difference can you get by switching a compiler (or a compiler version). So I did a bunch of tests, with gcc versions 4.1-4.9, clang 3.1-3.5, and just for fun with icc 2013 and 2015. And here are the results.

I did two usual types of tests - pgbench, representing a transactional workload (lots of small queries), and a subset of TPC-DS benchmark, representing analytical workloads (a few queries chewing large amounts of data).

I'll present results from a machine with i5-2500k CPU, 8GB RAM and an SSD drive, running Gentoo with kernel 3.12.20. I did rudimentary PostgreSQL tuning, mostly by tweaking postgresql.conf like this:

shared_buffers=1GBwork_mem=128MBmaintenance_work_mem=256MBcheckpoint_segments=64effective_io_concurrency=32

I do have results from another machine, but in general it confirms the results presented here. The PostgreSQL was compiled like this

./configure--prefix=...makeinstall

i.e. nothing special (no custom tweaks, etc.). The rest of the system is compiled with gcc 4.7.

pgbench

I did pgbench with three dataset sizes - small (~150MB), medium (~25% RAM) and large (~200% RAM). For each scale I ran pgbench with 4 clients (which is the number of cores on the CPU) for 15 minutes, repeated 3x, and averaged the results. And all this in read-write and read-only mode.

The first observation is that once you start hitting the drives, compiler makes absolutely no measurable difference. That makes results from all the read-write tests (for all scales) uninteresting, as well as the read-only test on large dataset - for all these tests the I/O is the main bottleneck (and that's something the compiler can't really influence).

So we're left with just the read-only benchmark on small and medium datasets, where the results look like this:

Let's use the gcc 4.1.2 results as a baseline, and express the other results as a percentage of the baseline. So 100 means "same as gcc 4.1.2", 90 means "10% slower than gcc 4.1.2" and so on. On a chart it then looks like this (the higher the number, the better):

Not really a dramatic difference:

• gcc 4.9 and clang 3.5 are winners, with ~4-5% improvement over gcc 4.1.2
• gcc improves over time, with the exception of 4.3/4.4, where the performance dropped below 4.1
• clang is very fast right from 3.1, peaking at 3.2 (which is slightly better than 3.5)
• surprisingly, icc gives the worst results here

TPC-DS

Now, the data warehouse benchmark. I've used a small dataset (1GB), so that it fits into memory - otherwise we'd hit the I/O bottlenecks and the compilers would make no difference. First, lest's load the data - the script performs these operations:

• COPY data into all the tables
• create indexes
• VACUUM FULL (not really necessary)
• VACUUM FREEZE
• ANALYZE

The results (in seconds) look like this:

According to the totals, the difference between the slowest (gcc 4.1.2) and fastest (gcc 4.9.2) is ~10%. Again, gcc continuously improves, which is nice. Clang actually slightly slows down since 3.2, which is not so nice, and clang 3.5 is ~6.5% slower than gcc 4.9.2. And icc is somewhere in between, with a nice speedup between 2013 and 2015 versions.

But that was just loading the data, what about the actual queries? TPC-DS specifies 99 query templates. Some of those use features not yet available in PostgreSQL, leaving us with 61 PostgreSQL-compatible templates. Sadly 2 of those did not complete within 30 minutes on the 1GB dataset (clearly, room for improvement), so the actual benchmark consists of 59 queries.

Chart of total duration of three runs per query, using gcc 4.1.2 as a baseline (just like the pgbench, but this time lower numbers are better) looks like this:

Clearly, the differences are much more significant than in the pgbench results. Again, gcc continuously improves over time, with 4.9.2 being the winner here - the difference between 4.1.2 and 4.9.2 is astonishing ~15%. That's pretty significant improvement - good work, GCC developers!

Clang results fluctuate a lot - 3.1, 3.3 and 3.5 are quite good (not as good as gcc 4.9.2, though).

And icc is again somewhere in the middle - faster than gcc 4.1.2 but nowehere as fast as gcc 4.9.2 or the "good" clang versions. And this time 2015 actually slowed down (contrary to the previous results).

Summary

If your workload is transactional (pgbench-like), the compiler does not matter that much - either you're hitting disks (and the compiler does not matter at all), or the differences are within 5% from gcc 4.1.2. But if a gain this small is significant for you enough to warrant switching a compiler, you should probably consider getting a slightly more powerful hardware (CPU with more cores, faster RAM, better storage, ...).

Analytical workloads are a different case - gcc is a clear winner, and if you're using an ancient version (say, 4.3 or older), you can get ~10% speedup by switching to 4.7, or ~15% to 4.9. In any case, the newer the version, the better.

Tuesday night David Fetter and Josh Berkus are going to present all about the cool new aggregation features in 9.4 and 9.5, including WITHIN GROUP, FILTER, GROUPING SETS, and more.

Tuesday night David Fetter and I are going to present all about the cool new aggregation features in 9.4 and 9.5, including WITHIN GROUP, FILTER, GROUPING SETS, and more.

Use Postgres? In the San Francisco Bay Area? Maybe you're planning to attend FOSS4G-NA or EclipseCon? Well, join us as well! pgDay SF 2015 will be on March 10th in Burlingame, CA. We are currently looking for speakers and sponsors for the event.

pgDaySF will be a one-day, one track event held alongside FOSS4G North America and EclipseCon, allowing for cross-pollination among geo geeks, Java programmers, and PostgreSQL fans. We are looking for both user-oriented and advanced database talks, with a slant towards PostGIS. Interested? Submit a talk now. Submissions for full talks close on December 9th.

If your company uses or supports PostgreSQL, or markets products to PostgreSQL and PostGIS users, then you may want to sponsor the pgDay as well. Currently we're looking for one Sponsor and up to five Supporters. This is especially good for companies looking to hire PostgreSQL DBAs.

pgDay SF 2015 is sponsored by Google and PostgreSQL Experts Inc..

One common source of query problems in PostgreSQL results an unexpectedly-bad query plan when a LIMIT clause is included in a query. The typical symptom is that PostgreSQL picks an index-based plan that actually takes much, much longer than if a different index, or no index at all, had been used.

Here’s an example. First, we create a simple table and an index on it:

xof=# CREATE TABLE sample (
xof(#   i INTEGER,
xof(#   f FLOAT
xof(# );
CREATE TABLE
xof=# CREATE INDEX ON sample(f);
CREATE INDEX

And fill it with some data:

xof=# INSERT INTO sample SELECT 0, random() FROM generate_series(1, 10000000);
INSERT 0 10000000
xof=# ANALYZE;
ANALYZE

Then, for about 5% of the table, we set i to 1:

UPDATE sample SET i=1 WHERE f<.0.05;
ANALYZE;

Now, let’s find all of the entires where i is 1, in descending order of f.

xof=# EXPLAIN ANALYZE SELECT * FROM sample WHERE i=1 ORDER BY f DESC;
                                                         QUERY PLAN                                                         
----------------------------------------------------------------------------------------------------------------------------
 Sort  (cost=399309.76..401406.04 rows=838509 width=12) (actual time=1415.166..1511.202 rows=499607 loops=1)
   Sort Key: f
   Sort Method: quicksort  Memory: 35708kB
   ->  Seq Scan on sample  (cost=0.00..316811.10 rows=838509 width=12) (actual time=1101.836..1173.262 rows=499607 loops=1)
         Filter: (i = 1)
         Rows Removed by Filter: 9500393
 Total runtime: 1542.529 ms
(7 rows)

So, 1.5 seconds to do a sequential scan on the whole table. So, just getting the first 10 entries from that should be much faster, right?

xof=# EXPLAIN ANALYZE SELECT * FROM sample WHERE i=1 ORDER BY f DESC LIMIT 10;
                                                                        QUERY PLAN                                                                        
----------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=0.43..277.33 rows=10 width=12) (actual time=12710.612..12710.685 rows=10 loops=1)
   ->  Index Scan Backward using sample_f_idx on sample  (cost=0.43..23218083.52 rows=838509 width=12) (actual time=12710.610..12710.682 rows=10 loops=1)
         Filter: (i = 1)
         Rows Removed by Filter: 9500393
 Total runtime: 12710.714 ms
(5 rows)

Oh. 12.7 seconds. What happened?

PostgreSQL doesn’t keep correlated statistics about columns; each column’s statistics are kept independently. Thus, PostgreSQL made an assumption about the distribution of values of i in the table: they were scattered more or less evenly throughout. Thus, walking the index backwards meant that, to get 10 “hits,” it would have to scan about 100 index entries… and the index scan would be a big win.

It was wrong, however, because all of the i=1 values were clustered right at the beginning. If we reverse the order of the scan, we can see that was a much more efficient plan:

xof=# EXPLAIN ANALYZE SELECT * FROM sample WHERE i=1 ORDER BY f  LIMIT 10;
                                                               QUERY PLAN                                                                
-----------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=0.43..277.33 rows=10 width=12) (actual time=0.029..0.046 rows=10 loops=1)
   ->  Index Scan using sample_f_idx on sample  (cost=0.43..23218083.52 rows=838509 width=12) (actual time=0.027..0.044 rows=10 loops=1)
         Filter: (i = 1)
 Total runtime: 0.071 ms
(4 rows)

So, what do we do? There’s no way of telling PostgreSQL directly to pick one plan over the other. We could just turn off index scans for the query, but that could well have bad side effects.

In this particular case, where a predicate (like the WHERE i=1) picks up a relatively small number of entries, we can use a Common Table Expression, or CTE. Here’s the example rewritten using a CTE:

xof=# EXPLAIN ANALYZE
xof-# WITH inner_query AS (
xof(#    SELECT * FROM sample WHERE i=1 
xof(# )
xof-# SELECT * FROM inner_query ORDER BY f  LIMIT 10;
                                                              QUERY PLAN                                                              
--------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=351701.16..351701.18 rows=10 width=12) (actual time=1371.946..1371.949 rows=10 loops=1)
   CTE inner_query
     ->  Seq Scan on sample  (cost=0.00..316811.10 rows=838509 width=12) (actual time=1168.034..1244.785 rows=499607 loops=1)
           Filter: (i = 1)
           Rows Removed by Filter: 9500393
   ->  Sort  (cost=34890.06..36986.33 rows=838509 width=12) (actual time=1371.944..1371.944 rows=10 loops=1)
         Sort Key: inner_query.f
         Sort Method: top-N heapsort  Memory: 25kB
         ->  CTE Scan on inner_query  (cost=0.00..16770.18 rows=838509 width=12) (actual time=1168.040..1325.496 rows=499607 loops=1)
 Total runtime: 1381.472 ms
(10 rows)

A CTE is an “optimization fence”: The planner is prohibited from pushing the ORDER BY or LIMIT down into the CTE. In this case, that means that it is also prohibited from picking the index scan, and we’re back to the sequential scan.

So, when you see a query come completely apart, and it has a LIMIT clause, check to see if PostgreSQL is guessing wrong about the distribution of data. If the total number of hits before the LIMIT are relatively small, you can often use a CTE to isolate that part, and only apply the LIMIT thereafter. (Of course, you might be better off just doing the LIMIT operation in your application!)

• Kernel 3.2 is in fact lethally bad.
• Kernel 3.13 is the best out of kernel 3.X so far.  I hope that this can be credited to the PostgreSQL team's work with the LFS/MM group.
• No 3.X kernel yet has quite the throughput of 2.6.32, at least at moderate memory sizes and core counts.
• However, kernel 3.13 has substantially lower write volumes at almost the same throughput.  This means that if you are write-bound on IO, 3.13 will improve your performance considerably.
• If your database is mostly-reads and highly concurrent, consider enabling
  

  kernel.sched_autogroup_enabled.

A couple of days ago the following feature related to pg_dump has been committed and will be in Postgres 9.5:

commit: be1cc8f46f57a04e69d9e4dd268d34da885fe6eb
author: Simon Riggs <simon@2ndQuadrant.com>
date: Mon, 17 Nov 2014 22:15:07 +0000
Add pg_dump --snapshot option

Allows pg_dump to use a snapshot previously defined by a concurrent
session that has either used pg_export_snapshot() or obtained a
snapshot when creating a logical slot. When this option is used with
parallel pg_dump, the snapshot defined by this option is used and no
new snapshot is taken.

Simon Riggs and Michael Paquier

First, let's talk briefly about exported snapshots, a feature that has been introduced in PostgreSQL 9.2. With it, it is possible to export a snapshot from a first session with pg_export_snapshot, and by reusing this snapshot in transactions of other sessions all the transactions can share exactly the same state image of the database. When using this feature something like that needs to be done for the first session exporting the snapshot:

=# BEGIN;
BEGIN
=# SELECT pg_export_snapshot();
 pg_export_snapshot
--------------------
 000003F1-1
(1 row)

Then other sessions in parallel can use SET TRANSACTION SNAPSHOT to import back the snapshot and share the same database view as all the other transactions using this snapshot (be it the transaction exporting the snapshot or the other sessions that already imported it).

=# BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
BEGIN
=# SET TRANSACTION SNAPSHOT '000003F1-1';
SET
=# -- Do stuff
[...]
=# COMMIT;
COMMIT

Note that the transaction that exported the snapshot needs to remain active as long as the other sessions have not consumed it with SET TRANSACTION. This snapshot export and import dance is actually used by pg_dump since 9.3 for parallel dumps to make consistent the dump acquisition across the threads, whose number is defined by --jobs, doing the work.

Now, this commit adding the option --snapshot is simply what a transaction importing a snapshot does: caller can export a snapshot within the transaction of a session and then re-use it with pg_dump to take an image of a given database consistent with the previous session transaction. Well, doing only that is not that useful in itself. The fun begins actually by knowing that there is a different situation where a caller can get back a snapshot name, and this situation exists since 9.4 because it is the moment a logical slot. is created through a replication connection.

$ psql "replication=database dbname=ioltas"
[...]
=# CREATE_REPLICATION_SLOT foo3 LOGICAL test_decoding;
 slot_name | consistent_point | snapshot_name | output_plugin
-----------+------------------+---------------+---------------
 foo       | 0/16ED738        | 000003E9-1    | test_decoding
(1 row)

See "000003E9-1" in the field snapshot_name? That is the target. Note a couple of things as well at this point:

• The creation of a physical slot does not return back a snapshot.
• The creation of a logical slot using pg_create_logical_replication_slot with a normal connection (let's say non-replication) does not give back a snapshot name.
• The snapshot is alive as long as the replication connection is kept. That is different of pg_export_snapshot called in the context of a non-replication connection where the snapshot remains alive as long as the transaction that called it is not committed (or aborted).

This is where this feature takes all its sense: it is possible to get an image of the database at the moment the slot has been created, or putting it in other words before any changes in the replication slot have been consumed, something aimed to be extremely useful for replication solutions or cases like online migration/upgrade of databases because it means that the dump can be used as a base image on which changes could be replayed without data lost. Then, the dump can simply be done like that:

pg_dump --snapshot 000003E9-1

When doing a parallel dump with a snapshot name, the snapshot specified is used for all the jobs and is not enforced by the first worker as it would be the case when a snapshot name is not specified, or when pg_dump would work in 9.3 and 9.4. Note as well that it is possible to use a newer version of pg_dump on older servers so it is fine to take a dump with an exported snapshot with 9.5's pg_dump from a 9.4 instance of Postgres, meaning that the door of a live upgrade solution of a single database is not closed (combined with the fact that a client application consuming changes from a logical replication slot can behave as a synchronous standby).