Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24907

Migrate JDBC data source to DataSource API v2

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • 3.1.0
    • None
    • SQL
    • None

    Attachments

      Issue Links

        is duplicated by

        New Feature - A new feature of the product, which has yet to be developed. SPARK-28380 DataSourceV2 API based JDBC connector

        • Major - Major loss of function.
        • Resolved
        is part of

        Umbrella - An overarching type made of sub-tasks SPARK-23507 Migrate file-based data sources to data source v2

        • Major - Major loss of function.
        • Resolved
        links to

        Pull request #21861 [Github] Pull Request #21861 (tengpeng)

        Web Link GitHub Pull Request #21861

        Web Link GitHub Pull Request #25211

        1.
        		 implement basic catalog functionalities Sub-task Resolved Unassigned
        2.
        		 Implement TableCatalog for JDBC Sub-task Resolved Max Gekk
        3.
        		 Override table renaming in JDBC dialects Sub-task Resolved Max Gekk
        4.
        		 Support the BATCH_READ capability by JDBCTable Sub-task Resolved Unassigned
        5.
        		 Implement ALTER TABLE in JDBC Table Catalog Sub-task Resolved Huaxin Gao
        6.
        		 Apply table options while creating tables in JDBC Table Catalog Sub-task Resolved Huaxin Gao
        7.
        		 Support the BATCH_WRITE capability by JDBCTable Sub-task Resolved Unassigned
        8.
        		 Omit USING in CREATE TABLE via JDBC Table Catalog Sub-task Resolved Unassigned
        9.
        		 Override alter table in JDBC dialects Sub-task Resolved Unassigned
        10.
        		 Implement JDBCScan/ScanBuilder/WriteBuilder Sub-task Resolved Huaxin Gao
        11.
        		 Remove fullOutput from RowDataSourceScanExec Sub-task Resolved Huaxin Gao
        12.
        		 Make `DataFrameReader.table` take the specified options Sub-task Resolved Huaxin Gao
        13.
        		 JDBC nested columns support Sub-task Open Unassigned
        14.
        		 Do truncate and append atomically in JDBCWriteBuilder Sub-task Open Unassigned
        15.
        		 CREATE/REPLACE TABLE AS SELECT not support multi-part identifiers Sub-task Resolved Unassigned
        16.
        		 JDBC V2 Datasource aggregate push down Sub-task Resolved Unassigned
        17.
        		 Make DataFrameWriter.jdbc work for DataSource V2 Sub-task Resolved Unassigned
        18.
        		 Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (Oracle dialect) Sub-task Resolved Max Gekk
        19.
        		 Make DataFrameReader.jdbc work for DataSource V2 Sub-task Resolved Unassigned
        20.
        		 Port docker integration tests to JDBC v2 Sub-task Open Unassigned
        21.
        		 Add negative checks to JDBC v2 Table Catalog tests Sub-task Resolved Max Gekk
        22.
        		 Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (DB2 dialect) Sub-task Resolved Huaxin Gao
        23.
        		 Classify dialect exceptions in JDBC v2 Table Catalog Sub-task Resolved Max Gekk
        24.
        		 Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect) Sub-task Resolved Prashant Sharma
        25.
        		 Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MsSqlServer dialect) Sub-task Resolved Prashant Sharma
        26.
        		 Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (PostgreSQL dialect) Sub-task Resolved Huaxin Gao
        27.
        		 Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (Teradata dialect) Sub-task Open Unassigned
        28.
        		 Support namespaces in JDBC v2 Table Catalog Sub-task Resolved Huaxin Gao
        29.
        		 Supports write data flow control Sub-task Open Unassigned
        30.
        		 Implement JDBCDataSourceV2 Sub-task Open Unassigned

        Activity

          People

            Unassigned Unassigned
            Teng Peng Teng Peng
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

              Created:
              Updated: