Guillaume LELARGE: The handler and the validator functions of a FDW

I'm still working on my foreign data wrapper. Actually, I haven't written a lot of code. Instead, I've been reading the code of the other foreign data wrappers and it helped me a lot to understand many things, and to see how far each went.

I'll try to start a few blog posts about coding a foreign data wrapper.

This first one (this one) will be about the two functions needed to declare a foreign data wrapper. Let's take a look at the CREATE FOREIGN DATA WRAPPER DDL statement:

CREATE FOREIGN DATA WRAPPER name
    [ HANDLER handler_function | NO HANDLER ]
    [ VALIDATOR validator_function | NO VALIDATOR ]
    [ OPTIONS ( option 'value' [, ... ] ) ]

The handler function is available since 9.1. The other options have always been available (at least from the beginning of the SQL/MED infrastructure in PostgreSQL, which is the 8.4 release).

The handler function is the function you use to declare all the foreign data wrappers hooks you'll need. Here are the complete list, with the first version they were available in :

• plan functions (used by the planner)
  • GetForeignRelSize (>=9.2), obtain relation size estimates for a foreign table, called at the beginning of a query planning
  • GetForeignPaths (>=9.2), create possible access paths for a scan on a foreign table, called during query planning
  • GetForeignPlan (>=9.2), create a ForeignScan plan node from the selected foreign access path, called at the end of a query planning
  • PlanForeignScan (<9.2)
• read functions (used by the executor)
  • BeginForeignScan, begin executing a foreign scan
  • IterateForeignScan, fetch one row from the foreign source
  • ReScanForeignScan, restart the scan from the beginning
  • EndForeignScan, end the scan and release resources
• write functions (>=9.3, used by the executor)
  • AddForeignUpdateTargets, used to add extra hidden target columns to the list of columns that are to be retrieved from the foreign table during an UPDATE or DELETE
  • PlanForeignModify, perform any additional planning actions needed for an insert, update, or delete on a foreign table
  • BeginForeignModify, begin executing a foreign table modification operation
  • ExecForeignInsert, insert one tuple into the foreign table
  • ExecForeignUpdate, update one tuple into the foreign table
  • ExecForeignDelete, delete one tuple into the foreign table
  • EndForeignModify, end the table update and release resources
• explain functions (used during the execution of an EXPLAIN)
  • ExplainForeignScan, print additional EXPLAIN output for a foreign table scan
  • ExplainForeignModify (>=9.3), print additional EXPLAIN output for a foreign table update
• analyze function (>=9.2, used during the execution of an ANALYZE)
  • AnalyzeForeignTable, collect statistics for this foreign table when ANALYZE is executed on it

None are really required. Of course, if you implement none of them, the foreign data wrapper won't be really useful

All these hooks are to be set inside a structure of type FdwRoutine. It's best to use the makeNode() function to set all fields to NULL. Then, you'll be able to set each of the hooks you really need, and don't care about the rest.

At the end, this function hands back the FdwRoutine structure to the caller of the function.

So the code looks something like this:

Datum
my_fdw_handler(PG_FUNCTION_ARGS)
{
  FdwRoutine *fdwroutine = makeNode(FdwRoutine);

  // plan functions
#if (PG_VERSION_NUM >= 90200)
  fdwroutine->GetForeignRelSize = myGetForeignRelSize;
  fdwroutine->GetForeignPaths = myGetForeignPaths;
  fdwroutine->GetForeignPlan = myGetForeignPlan;
#else
  fdwroutine->PlanForeignScan = myPlanForeignScan;
#endif

  // read functions
  fdwroutine->BeginForeignScan = myBeginForeignScan;
  fdwroutine->IterateForeignScan = myIterateForeignScan;
  fdwroutine->ReScanForeignScan = myReScanForeignScan;
  fdwroutine->EndForeignScan = myEndForeignScan;

#if (PG_VERSION_NUM >= 90300)
  // write functions
  fdwroutine->AddForeignUpdateTargets = myAddForeignUpdateTargets;
  fdwroutine->PlanForeignModify = myPlanForeignModify;
  fdwroutine->BeginForeignModify = myBeginForeignModify;
  fdwroutine->ExecForeignInsert = myExecForeignInsert;
  fdwroutine->ExecForeignUpdate = myExecForeignUpdate;
  fdwroutine->ExecForeignDelete = myExecForeignDelete;
  fdwroutine->EndForeignModify = myEndForeignModify;
#endif

  // explain functions
  fdwroutine->ExplainForeignScan = myExplainForeignScan;
#if (PG_VERSION_NUM >= 90300)
  fdwroutine->ExplainForeignModify = myExplainForeignModify;
#endif

#if (PG_VERSION_NUM >= 90200)
  // analyze functions
  fdwroutine->AnalyzeForeignTable = myAnalyzeForeignTable;
#endif

  PG_RETURN_POINTER(fdwroutine);
}

Of course, all the my* functions must be previously declared. Rather than the actual code, the more interesting thing to know in a foreign data wrapper is which hooks it support. It really depends a lot on the FDW. For example, the Oracle FDW doesn't support yet the write functions. The MySQL FDW doesn't support the write, the explain, and the analyze functions. Etc.

The validator function checks the options given to a specific foreign object (foreign data wrapper, foreign server, user mapping, and foreign table). It gets a list of all the options, and the context of the options (aka the foreign object type). The function has to go through each of them, checks that they are valid, and error out if anything wrong happens (send back the ERRCODE_FDW_INVALID_OPTION_NAME error if it isn't a valid option, please).

A good way of doing it is to first add a FDW option structure, this way:

struct myFdwOption
{
  const char  *optname;
  Oid   optcontext;
};

The optname will obviously be the option name. The optcontext will contain the kind of object this option is for. There are four possibilities: ForeignDataWrapperRelationId, ForeignServerRelationId, UserMappingRelationId, and ForeignTableRelationId.

The options could be declared like this:

static struct myFdwOption valid_options[] =
{
  { "database",  ForeignServerRelationId },
  { "table",     ForeignTableRelationId },
  { NULL,     InvalidOid }
};

So, a simple code for the validator function might be:

Datum
my_fdw_validator(PG_FUNCTION_ARGS)
{
  List      *options_list = untransformRelOptions(PG_GETARG_DATUM(0));
  Oid       catalog = PG_GETARG_OID(1);
  ListCell  *cell;
  char      *my_database = NULL;
  char      *my_table = NULL;

  foreach(cell, options_list)
  {
    DefElem    *def = (DefElem *) lfirst(cell);

    if (!myIsValidOption(def->defname, catalog))
    {
      struct SQLiteFdwOption *opt;
      StringInfoData buf;

      /*
       * Unknown option specified, complain about it. Provide a hint
       * with list of valid options for the object.
       */
      initStringInfo(&buf);
      for (opt = valid_options; opt->optname; opt++)
      {
        if (catalog == opt->optcontext)
          appendStringInfo(&buf, "%s%s", (buf.len > 0) ? ", " : "",
               opt->optname);
      }

      ereport(ERROR, 
        (errcode(ERRCODE_FDW_INVALID_OPTION_NAME), 
        errmsg("invalid option \"%s\"", def->defname), 
        errhint("Valid options in this context are: %s", buf.len ? buf.data : "<none>")
        ));
    }

    /*
     * Here is the code for the valid options
     */

    if (strcmp(def->defname, "database") == 0)
    {
      if (my_database)
        ereport(ERROR,
          (errcode(ERRCODE_SYNTAX_ERROR),
          errmsg("redundant options: database (%s)", defGetString(def))
          ));

      my_database = defGetString(def);
    }
    else if (strcmp(def->defname, "table") == 0)
    {
      if (my_table)
        ereport(ERROR,
          (errcode(ERRCODE_SYNTAX_ERROR),
          errmsg("redundant options: table (%s)", defGetString(def))
          ));

      my_table = defGetString(def);
    }
  }

  PG_RETURN_VOID();
}

The myIsValidOption() function will go through the options and checks that it matches the right context. Just as is:

static bool
myIsValidOption(const char *option, Oid context)
{
  struct myFdwOption *opt;

  for (opt = valid_options; opt->optname; opt++)
  {
    if (context == opt->optcontext && strcmp(opt->optname, option) == 0)
      return true;
  }
  return false;
}

After compilation and installation of such a FDW, we can try creating foreign objects:

yeah=# CREATE EXTENSION my_fdw ;
CREATE EXTENSION
-- we can add a server without options
yeah=# CREATE SERVER myserver FOREIGN DATA WRAPPER my_fdw ;
CREATE SERVER
-- we cannot add a server with unknown options
yeah=# CREATE SERVER myserver3 FOREIGN DATA WRAPPER my_fdw OPTIONS (inconnu '10') ;
ERROR:  invalid option "inconnu"
HINT:  Valid options in this context are: database
-- but we can add one with a valid option
yeah=# CREATE SERVER myserver2 FOREIGN DATA WRAPPER my_fdw OPTIONS (database 'this one') ;
CREATE SERVER
-- we can also create foreign table without options
yeah=# CREATE FOREIGN TABLE ft(id integer) SERVER myserver2;
CREATE FOREIGN TABLE
-- we cannot add a foreign table with unknown options
yeah=# CREATE FOREIGN TABLE ft2(id integer) SERVER myserver2 OPTIONS (inconnu '10');
ERROR:  invalid option "inconnu"
HINT:  Valid options in this context are: table
-- we can add a foreign table with valid options
yeah=# CREATE FOREIGN TABLE ft2(id integer) SERVER myserver2 OPTIONS (table '10');
CREATE FOREIGN TABLE

So the validator function seems to work. The handler function doesn't do anything. If you try to read the foreign table, it will... err.. let's simply say that we still need something to make it work:

yeah=# select * from ft2;
The connection to the server was lost. Attempting reset: Failed.
!> 

We'll see how it can send back some lines next time.