In this post I'll talk about setting up two PgBouncers, one at the application layer, and the other at the database layer. One might wonder the purpose of setting up multiple connection pooling within the same project or website. The answer is simple - increased pooling.
For really large applications with a lot of application servers, a single PgBouncer at the database layer might not provide desired amount of pooling. Hence, having PgBouncer at the app layer will first pool connections coming from those servers, and send these to the PgBouncer at the database layer. This will further reduce connections before connecting to the underlying database.
A simple setup of such a system only requires providing IP and port of the database layer's PgBouncer to the application layer's PgBouncer:
Application layer PgBouncer:
vagrant@precise64:~/pgbouncer/bin$ cat config.ini
[databases]
* = host=192.168.19.100 port=5433 user=postgres
* = host=192.168.19.100 port=5433 user=vagrant
[pgbouncer]
listen_port = 5434
listen_addr = *
auth_type = any
logfile = pgbouncer.log
pidfile = pgbouncer.pid
unix_socket_dir = /var/run/postgresql
auth_file = auth.txt
Database layer PgBouncer:
vagrant@precise64:~/pgbouncer/bin$ cat config.ini
[databases]
* = host=localhost port=5432 user=postgres
* = host=localhost port=5432 user=vagrant
[pgbouncer]
listen_port = 5433
listen_addr = *
auth_type = any
logfile = pgbouncer.log
pidfile = pgbouncer.pid
unix_socket_dir = /var/run/postgresql
auth_file = auth.txt
And now connection to the database with psql from the client box will show the following in logs:
App layer PgBouncer log:
2016-03-27 03:04:21.628 19320 LOG C-0x1754320: (nodb)/(nouser)@unix(19445):5434 registered new auto-database: db = postgres
2016-03-27 03:04:21.628 19320 LOG C-0x1754320: postgres/vagrant@unix(19445):5434 login attempt: db=postgres user=vagrant tls=no
2016-03-27 03:04:21.629 19320 LOG S-0x1772300: postgres/vagrant@192.168.19.100:5433 new connection to server (from 192.168.19.101:46604)
DB Layer PgBouncer:
2016-03-27 03:04:21.618 27179 LOG C-0x2397480: (nodb)/(nouser)@192.168.19.101:46604 registered new auto-database: db = postgres
2016-03-27 03:04:21.619 27179 LOG C-0x2397480: postgres/vagrant@192.168.19.101:46604 login attempt: db=postgres user=vagrant tls=no
2016-03-27 03:04:21.619 27179 LOG S-0x23b52e0: postgres/vagrant@127.0.0.1:5432 new connection to server (from 127.0.0.1:57681)
For really large applications with a lot of application servers, a single PgBouncer at the database layer might not provide desired amount of pooling. Hence, having PgBouncer at the app layer will first pool connections coming from those servers, and send these to the PgBouncer at the database layer. This will further reduce connections before connecting to the underlying database.
A simple setup of such a system only requires providing IP and port of the database layer's PgBouncer to the application layer's PgBouncer:
Application layer PgBouncer:
vagrant@precise64:~/pgbouncer/bin$ cat config.ini
[databases]
* = host=192.168.19.100 port=5433 user=postgres
* = host=192.168.19.100 port=5433 user=vagrant
[pgbouncer]
listen_port = 5434
listen_addr = *
auth_type = any
logfile = pgbouncer.log
pidfile = pgbouncer.pid
unix_socket_dir = /var/run/postgresql
auth_file = auth.txt
Database layer PgBouncer:
vagrant@precise64:~/pgbouncer/bin$ cat config.ini
[databases]
* = host=localhost port=5432 user=postgres
* = host=localhost port=5432 user=vagrant
[pgbouncer]
listen_port = 5433
listen_addr = *
auth_type = any
logfile = pgbouncer.log
pidfile = pgbouncer.pid
unix_socket_dir = /var/run/postgresql
auth_file = auth.txt
And now connection to the database with psql from the client box will show the following in logs:
App layer PgBouncer log:
2016-03-27 03:04:21.628 19320 LOG C-0x1754320: (nodb)/(nouser)@unix(19445):5434 registered new auto-database: db = postgres
2016-03-27 03:04:21.628 19320 LOG C-0x1754320: postgres/vagrant@unix(19445):5434 login attempt: db=postgres user=vagrant tls=no
2016-03-27 03:04:21.629 19320 LOG S-0x1772300: postgres/vagrant@192.168.19.100:5433 new connection to server (from 192.168.19.101:46604)
DB Layer PgBouncer:
2016-03-27 03:04:21.618 27179 LOG C-0x2397480: (nodb)/(nouser)@192.168.19.101:46604 registered new auto-database: db = postgres
2016-03-27 03:04:21.619 27179 LOG C-0x2397480: postgres/vagrant@192.168.19.101:46604 login attempt: db=postgres user=vagrant tls=no
2016-03-27 03:04:21.619 27179 LOG S-0x23b52e0: postgres/vagrant@127.0.0.1:5432 new connection to server (from 127.0.0.1:57681)