Ernst-Georg Schmid: pg_strom - The rough road ahead

Inspired by thosearticles on pg_strom, the CUDA based GPGPU accelerator for PostgreSQL, I decided to repeat the tests.

First a caveat. I managed to destroy my Ubuntu 14.04 LTS beyond recoverability with the native Nvidia drivers and had to reinstall from scratch.

At least if you run a dual GPU setup like on my mobile workstation (Intel HD4600 and Nvidia Quadro K1100M) install the Nvidia CUDA package from the Nvidia repository on a fresh Ubuntu installation - and if it works, don't touch again.

If it works, it works good, you can even switch between Intel and Nvidia graphics without rebooting.

OK, here is the output of deviceQuery:

Device 0: "Quadro K1100M"
  CUDA Driver Version / Runtime Version          7.5 / 7.5
  CUDA Capability Major/Minor version number:    3.0
  Total amount of global memory:                 2048 MBytes (2147352576 bytes)
  ( 2) Multiprocessors, (192) CUDA Cores/MP:     384 CUDA Cores
  GPU Max Clock rate:                            706 MHz (0.71 GHz)
  Memory Clock rate:                             1400 Mhz
  Memory Bus Width:                              128-bit
  L2 Cache Size:                                 262144 bytes
  Maximum Texture Dimension Size (x,y,z)         1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
  Maximum Layered 1D Texture Size, (num) layers  1D=(16384), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(16384, 16384), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  2048
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z): (2147483647, 65535, 65535)
  Maximum memory pitch:                          2147483647 bytes
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 1 copy engine(s)
  Run time limit on kernels:                     Yes
  Integrated GPU sharing Host Memory:            No
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Disabled
  Device supports Unified Addressing (UVA):      Yes
  Device PCI Domain ID / Bus ID / location ID:   0 / 1 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 7.5, CUDA Runtime Version = 7.5, NumDevs = 1, 
Device0 = Quadro K1100M

Result = PASS
PostgreSQL says:

LOG:  CUDA Runtime version: 7.5.0
LOG:  NVIDIA driver version: 352.39
LOG:  GPU0 Quadro K1100M (384 CUDA cores, 705MHz), L2 256KB, RAM 2047MB (128bits, 1400MHz), capability 3.0
LOG:  NVRTC - CUDA Runtime Compilation vertion 7.5

Now the tests:

CREATE TABLE t_test AS
SELECT x, 'a'::char(100) AS y, 'b'::char(100) AS z
FROM generate_series(1, 5000000) AS x
ORDER BY random();

SET pg_strom.enabled = OFF;

EXPLAIN ANALYZE SELECT count(*)
FROM   t_test
WHERE sqrt(x) > 0

GROUP BY y;

HashAggregate  (cost=242892.34..242892.35 rows=1 width=101) (actual time=2550.064..2550.064 rows=1 loops=1)
  Group Key: y
  ->  Seq Scan on t_test  (cost=0.00..234559.00 rows=1666667 width=101) (actual time=0.016..779.110 rows=5000000 loops=1)
        Filter: (sqrt((x)::double precision) > '0'::double precision)"
Planning time: 0.104 ms

Execution time: 2550.131 ms

SET pg_strom.enabled = ON;

EXPLAIN ANALYZE SELECT count(*)
FROM   t_test
WHERE sqrt(x) > 0

GROUP BY y;

HashAggregate  (cost=177230.88..177230.89 rows=1 width=101) (actual time=25393.766..25393.767 rows=1 loops=1)
  Group Key: y
  ->  Custom Scan (GpuPreAgg)  (cost=13929.24..173681.39 rows=260 width=408) (actual time=348.584..25393.123 rows=76 loops=1)
        Bulkload: On (density: 100.00%)"
        Reduction: Local + Global
        Device Filter: (sqrt((x)::double precision) > '0'::double precision)"
        ->  Custom Scan (BulkScan) on t_test  (cost=9929.24..168897.54 rows=5000000 width=101) (actual time=4.336..628.920 rows=5000000 loops=1)"
Planning time: 0.330 ms

Execution time: 25488.189 ms

Whoa, pg_strom is 10x slower for me. Why? I don't know.

It could be a driver issue, because I see heavy CPU spikes during the query - up to 100% on some cores. My driver version is 352.39 instead of 352.30.

It could also be that in the original test a comparatively weak CPU (unspecified 4 GHz AMD, I therefore assume a FX8350) with a powerful GPU (Nvidia GTX970, 4GB) were used, while my test used a comparatively powerful CPU (Intel 4700MQ) and a weak GPU (Nvidia Quadro K1100M, 2GB).

But does that explain the CPU spikes? Well, probably we see suboptimal host<->device memory transfers here, the GTX970 not only has double the memory, it has also double the bus width.

I second that we might be seeing the future here, but it's not ready for prime time yet...