tencent cloud

Feedback

postgres_fdw Extension for Cross-database Access

Last updated: 2024-03-20 10:59:40
    TencentDB for PostgreSQL provides a series of extensions for accessing external data sources, including data in other libraries of this instance or data from other instances. Cross-database access extensions include homogenous extensions such as dblink, postgresql_fdw, and heterogeneous extensions like mysql_fdw, cos_fdw. The steps to use cross-database access are as follows:
    1. Use the "CREATE EXTENSION" statement to install the extensions.
    2. Create an external server object and a link map for each remote database that needs to be connected.
    3. Use the corresponding command to access the external table to access data.
    As the cross-database access extensions can access directly across instances or perform cross-database access within the same instance, TencentDB for PostgreSQL has optimized permission control when creating external server objects. It categorizes them based on the environment of the target instance. In addition to the features provided by the open-source version, extra auxiliary parameters have been added to verify the user's identity and adjust network policies. For more details, please refer to Auxiliary Parameters below.
    Note:
    Please note that the dblink extension is currently only supported by TencentDB for PostgreSQL kernels with major version 10.

    Extension Auxiliary Parameters

    host This is a compulsory parameter when accessing across instances. It refers to the IP address of the target instance.
    port This is a compulsory parameter when accessing across instances. It refers to the port of the target instance.
    instanceid Instance ID
    It is used when accessing across instances in TencentDB for PostgreSQL. This parameter is mandatory when accessing across instances. The format is similar to postgres-xxxxxx, pgro-xxxxxx, and can be viewed on the console.
    If the target instance is on Tencent Cloud CVM, then it is the instance ID of the CVM, the format is similar to ins-xxxxx.
    dbname Refers to the name of the database in the remote PostgreSQL service to be accessed. For cross-database access in the same instance, you only need to configure this parameter and can leave other parameters empty.
    access_type Optional. Refers to the type of the target instance:
    The target instance is a TencentDB instance, including TencentDB for PostgreSQL, TencentDB for MySQL, etc. If the type is not specified, this is the default.
    The target instance is on a Tencent Cloud CVM.
    The target instance is a public network-based self-built instance in Tencent Cloud.
    The target instance is a cloud VPN-based instance.
    The target instance is a self-built VPN-based instance.
    The target instance is a Direct Connect-based instance.
    uin Optional. Refers to the account ID to which the instance belongs. This information is used to identify user permissions, and you can refer to View uin.
    own_uin Optional. Refers to the root account ID to which the instance belongs. This information is also needed to identify user permissions.
    vpcid Optional. Refers to the Virtual Private Cloud ID. If the target instance is in the Tencent Cloud CVM's VPC network, this parameter is required and can be found in the VPC Console.
    subnetid Optional. Refers to the Virtual Private Cloud Subnet ID. If the target instance is in the Tencent Cloud CVM's VPC network, this parameter is required and can be found in the Subnets section of the VPC Console.
    dcgid Optional. Refers to the Direct Connect ID. If the target instance needs to connect via leased line network, you need to provide this parameter value.
    vpngwid Optional. Refers to the VPN Gateway ID. If the target instance needs to connect through VPN, this parameter value needs to be provided.
    region Optional. Refers to the region where the target instance is located. For example, "ap-guangzhou" represents Guangzhou. If you need to access data across regions, this parameter value needs to be provided.

    Examples of How to Use postgres_fdw

    Using the postgres_fdw extension, you can access data from other databases or other Postgres instances in this instance.

    Step 1: Prerequisites

    1. Create test data in the instance.
    postgres=>create role user1 with LOGIN CREATEDB PASSWORD 'password1';
    postgres=>create database testdb1;
    CREATE DATABASE
    Note:
    If an error occurs when creating an extension, please submit a ticket to contact Tencent Cloud after-sales for assistance.
    2. Create test data in the target instance.
    postgres=>create role user2 with LOGIN CREATEDB PASSWORD 'password2';
    postgres=> create database testdb2;
    CREATE DATABASE
    postgres=> \\c testdb2 user2
    You are now connected to database "testdb2" as user "user2".
    testdb2=> create table test_table2(id integer);
    CREATE TABLE
    testdb2=> insert into test_table2 values (1);
    INSERT 0 1

    Step 2. Create the postgres_fdw extension

    Note:
    If you encounter an issue stating 'extension does not exist' or 'insufficient privileges' while creating the extension, please submit a ticket for assistance.
    #Create
    postgres=> \\c testdb1
    You are now connected to database "testdb1" as user "user1".
    testdb1=> create extension postgres_fdw;
    CREATE EXTENSION
    #View
    testdb1=> \\dx
    List of installed extensions
    Name | Version | Schema | Description
    --------------+---------+------------+----------------------------------------------------
    plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language
    postgres_fdw | 1.0 | public | foreign-data wrapper for remote PostgreSQL servers
    (2 rows)

    Step 3. Create a Server

    Note:
    Cross-instance access is supported only for kernel version v10.17_r1.2, v11.12_r1.2, v12.7_r1.2, v13.3_r1.2, v14.2_r1.0, and later.
    Cross-instance access.
    #Access data from the target instance `testdb2` from `testdb1` in this instance.
    testdb1=>create server srv_test1 foreign data wrapper postgres_fdw options (host 'xxx.xxx.xxx.xxx',dbname 'testdb2', port '5432', instanceid 'postgres-xxxxx');
    CREATE SERVER
    If not crossing instances, and only accessing across databases, you only need to specify the dbname parameter.
    #Access the data of `testdb2` from `testdb1` in this instance
    create server srv_test1 foreign data wrapper postgres_fdw options (dbname 'testdb2');
    The target instance is on a Tencent Cloud CVM, and the network type is classic network.
    testdb1=>create server srv_test foreign data wrapper postgres_fdw options (host 'xxx.xxx.xxx.xxx', dbname 'testdb2', port '5432', instanceid 'ins-xxxxx', access_type '2', region 'ap-guangzhou',uin 'xxxxxx',own_uin 'xxxxxx');
    CREATE SERVER
    The target instance is on a Tencent Cloud CVM, and the network type is VPC.
    testdb1=>create server srv_test1 foreign data wrapper postgres_fdw options (host 'xxx.xxx.xxx.xxx',dbname 'testdb2', port '5432', instanceid 'ins-xxxxx', access_type '2', region 'ap-guangzhou', uin 'xxxxxx', own_uin 'xxxxxx', vpcid 'vpc-xxxxxx', subnetid 'subnet-xxxxx');
    CREATE SERVER
    The target instance is a public network-based self-built instance in Tencent Cloud.
    testdb1=>create server srv_test1 foreign data wrapper postgres_fdw options (host 'xxx.xxx.xxx.xxx',dbname 'testdb2', port '5432', access_type '3', region 'ap-guangzhou', uin 'xxxxxx', own_uin 'xxxxxx');
    CREATE SERVER
    The target instance is a Tencent Cloud VPN-based instance.
    testdb1=>create server srv_test1 foreign data wrapper postgres_fdw options (host 'xxx.xxx.xxx.xxx',dbname 'testdb2', port '5432', access_type '4', region 'ap-guangzhou', uin 'xxxxxx', own_uin 'xxxxxx', vpngwid 'xxxxxx');
    The target instance is a self-built VPN-based instance.
    testdb1=>create server srv_test1 foreign data wrapper postgres_fdw options (host 'xxx.xxx.xxx.xxx',dbname 'testdb2', port '5432', access_type '5', region 'ap-guangzhou', uin 'xxxxxx', own_uin 'xxxxxx', vpngwid 'xxxxxx');
    The target instance is a Direct Connect-based instance.
    testdb1=>create server srv_test1 foreign data wrapper postgres_fdw options (host 'xxx.xxx.xxx.xxx',dbname 'testdb2', port '5432', access_type '6', region 'ap-guangzhou', uin 'xxxxxx', own_uin 'xxxxxx', dcgid 'xxxxxx');
    CREATE SERVER

    Step 4. Create a User Mapping

    Note:
    You can skip this step for cross-database access in the same instance.
    testdb1=> create user mapping for user1 server srv_test1 options (user 'user2',password 'password2');
    CREATE USER MAPPING

    Step 5. Creating a Foreign Table

    testdb1=> create foreign table foreign_table1(id integer) server srv_test1 options(table_name 'test_table2');
    CREATE FOREIGN TABLE

    Step 6. Access Data from the Foreign Table

    testdb1=> select * from foreign_table1;
    id
    ----
    1
    (1 row)

    Reference Link

    Example of How to Use dblink

    Step 1: Create a dblink Extension

    postgres=> create extension dblink;
    postgres=> \\dx
    List of installed extensions
    Name | Version | Schema | Description
    --------------------+---------+------------+-------------------------------------------------------------------
    dblink | 1.2 | public | connect to other PostgreSQL databases from within a database
    pg_stat_log | 1.0 | public | track runtime execution statistics of all SQL statements executed
    pg_stat_statements | 1.6 | public | track execution statistics of all SQL statements executed
    plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language
    (4 rows)

    Step 2: Establish a dblink Link

    select dblink_connect('yunpg1','host=10.10.10.11 port=5432 instanceid=postgres-2123455r dbname=postgres access_type=1 user=dbadmin password=P302!');
    dblink_connect
    ----------------
    OK
    (1 row)

    Step 3: Access External Data

    postgres=> select * from dblink('yunpg1','select catalog_name,schema_name,schema_owner from information_schema.schemata') as t(a varchar(50),b varchar(50),c varchar(50));
    a | b | c
    ----------+--------------------+---------
    postgres | pg_toast | user_00
    postgres | pg_temp_1 | user_00
    postgres | pg_toast_temp_1 | user_00
    postgres | pg_catalog | user_00
    postgres | public | user_00
    postgres | information_schema | user_00
    (6 rows)

    Reference Link

    dblink

    Example of How to Use mysql_fdw

    Step 1: Create mysql_fdw Extension

    postgres=> create extension mysql_fdw;
    CREATE EXTENSION
    postgres=> \\dx;
    List of installed extensions
    Name | Version | Schema | Description
    --------------------+---------+------------+------------------------------------------------------------------------
    dblink | 1.2 | public | connect to other PostgreSQL databases from within a database
    mysql_fdw | 1.1 | public | Foreign data wrapper for querying a MySQL server
    pg_stat_log | 1.0 | public | track runtime execution statistics of all SQL statements executed
    pg_stat_statements | 1.9 | public | track planning and execution statistics of all SQL statements executed
    plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language
    (5 rows)

    Step 2: Create a SERVER

    postgres=> CREATE SERVER mysql_svr FOREIGN DATA WRAPPER mysql_fdw OPTIONS (host '171.16.10.13',port '3306',instanceid 'cdb-l1d95grp',uin '100026380431');
    CREATE SERVER

    Step 3: Create External User Map

    postgres=> CREATE USER MAPPING FOR PUBLIC SERVER mysql_svr OPTIONS (username 'fdw_user',password 'Secret!123');
    CREATE USER MAPPING

    Step 4: Access External Data

    postgres=> IMPORT FOREIGN SCHEMA hrdb FROM SERVER mysql_svr INTO public;

    Reference Link

    Examples of How to Use cos_fdw

    Please refer to the document Supporting Tiered Storage Based on cos_fdw Extension for cos_fdw usage examples.

    Notes

    Pay attention to the following for the target instance:
    1. The HBA restrictions of PostgreSQL need to be loosen to allow mapped users created (such as user2) to access via MD5. For HBA modification, please refer to 20.1. The pg_hba.conf File.
    2. If the target instance is not a TencentDB instance and has hot standby mode enabled, after a failover, you'll need to update the server connection address manually or recreate the server configuration.
    
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support