November Report on Apache SeaTunnel Community Development

Hi Apache SeaTunnel community, the monthly report for November has been updated! This report will document the significant updates in the SeaTunnel community on a monthly basis, so stay tuned!

Monthly Merge Star

We would like to express our gratitude to the following contributors for their outstanding contributions to Apache SeaTunnel in November (listed in no particular order):

“@CosmosNi,@BilwaST,@liunaijie,@corgy-w,@Carl-Zhou-CN,@davidzollo,@hawk9821,@lm-ylj,@sunxiaojian,@zhdech,@dailai,@XuyiK,@boandai,@YOMO-Lee,@shashwatsai,@eyys,@wengys,@hailin0,@jw-itq,@Hisoka-X,@zhangshenghang,@XenosK,@a110q,@arshadmohammad”

apache/seatunnel

fix

• [Fix][Doc]fix backup-count config @corgy-w
• [Fix][Doc]Fixed unreachable links @corgy-w
• [hotfix][gbase]Fix gbase driver link @zhangshenghang
• [fix]fix get job metrics get NPE @liunaijie
• [Hotfix][Transforms-v2]DynamicCompile Plugin compatibility fix @hawk9821
• [Fix][CI]time-consuming module optimization @hawk9821
• [Hotfix][Docs]Correct errors in the FAQ doc @davidzollo
• [hotfix][connector-v2][hive]Fix hive3 e2e dependencies @zhangshenghang
• [Bug][connectors-v2]fix mongodb bson convert exception @hawk9821
• [Fix][API]Fix column length can not be long @Hisoka-X
• [Fix][CI]Fix schedule test be canceled @Hisoka-X
• [Hotfix][E2E]Fix OracleCDCIT compile error @hailin0
• [Fix][Connecotr-V2]Fix clickhouse sink does not support composite primary key @eyys
• [Fix][Doc]Fix key description error in redis zh doc @lm-ylj
• [Hotfix][Zeta]Fix the dependency conflict between the guava in hadoop-aws and hive-exec @dailai
• [Fix][Zeta]Remove duplicate LIFECYCLE_WRITER_CLOSE events @Hisoka-X
• [Fix][Zeta]Fix job client print error msg even job finished @Hisoka-X
• [Fix][Doc]Fix klaviyo doc 404 link @YOMO-Lee
• [Fix][Zeta]Fix apply resource again for another pipeline when restore one pipeline @zhangshenghang
• [Bug][Connector-v2]MongoDB CDC Set SeatunnelRow's tableId @hawk9821
• [Fix][Kafka]Fix in kafka streaming mode can not read incremental data @Carl-Zhou-CN

improve

• [Improve][Connector-V2]Save data savemode operation when schema savemode created new table @zhangshenghang
• [Improve]Remove unused code @hailin0
• [Improve][Connector-V2]Add table info when multi table sink error @Hisoka-X
• [improve][connector-v2][kafka-connector]Add e2e code for Kafka Kerberos authentication @zhangshenghang
• [Improve][Jdbc]Improve ddl write validate @hailin0
• [Improve][Connector-V2]Add pre-check for table enable cdc @Hisoka-X
• [Improve][Connector-V2]Reduce the request times of iceberg load table @Hisoka-X
• [Improve][Kafka]Support custom topic for debezium compatible format @hailin0
• [Improve][doc]Add doc name check @zhangshenghang
• [Improve][dist]add shade check rule @hawk9821
• [Improve][Jdbc]Refactor ddl change @hailin0
• [Feature][Jdbc]Add Jdbc default dialect for all jdbc series database without dialect @sunxiaojian
• [Improve][Excel]Support read blank string & auto type-cast @hailin0
• [Feature][File]Support config null format for text file read @hailin0
• [Feature][E2E][Kerberos]Support for Kerberos in e2e @zhangshenghang
• [Improve][API]Unified tables_configs and table_list @hawk9821
• [Improve]Allow nullable for SeaTunnelRowType convert to Column @hailin0
• [Feature][Iceberg]Support custom delete sql for sink savemode @hailin0
• [Improve][Doc]Add the way of configure jvm options in client @Hisoka-X
• [Improve][Connector-V2]Fix SqlServer cdc memory leak @Hisoka-X
• [Improve][E2E]Improve Druid E2E Case @hawk9821
• [Improve][Zeta]Fixed issues related to log deletion during ci @corgy-w
• [Feature][Core]Rename result_table_name/source_table_name to plugin_input/plugin_output @hailin0
• [Improve][Transform]Optimize transform structure @corgy-w
• [Improve][Zeta]Remove default jvm memory option @Hisoka-X
• [Improve][POM]Bump the version of maven-shade-plugin to 3.4.1 @dailai
• [Improve]Improve AlterTableSchemaEventHandler @hailin0
• [Feature][Connector-v2]Support S3 filesystem of paimon connector @dailai
• [Feature][E2E]Add E2E test case time analysis @hawk9821
• [Feature][Connectors]LocalFile Support reading gz @zhdech
• [Improve][Connector-V2]Improve schema evolution on column insert after for mysql-jdbc @jw-itq
• [Feature][Transform]Sql transform support array and split function @CosmosNi
• [Feature][E2E]Add hive3 e2e test case @zhangshenghang
• [Feature][RestAPI]Support submit job with seatunnel style hocon format config @CosmosNi
• [Improve][CI]Optimizing CI time cost @Hisoka-X
• [Improve][Connector-V2]RocketMQ Sink add message tag config @wengys
• [Feature] [Connector-Redis]Redis connector support delete data @lm-ylj
• [Feature][Zeta]Add jar path precheck when job submit on master @zhangshenghang
• [Feature][transform]transform support explode @CosmosNi
• [Feature][Connector-v2]Support schema evolution for Oracle connector @dailai
• [Feature][Transform-v2]Add metadata transform @hawk9821
• [Feature][Connector-V2]Jdbc chunk split add snapshotSplitColumn config #7794 @XenosK
• [Feature][Zeta]Support delete logs regularly @corgy-w
• [Improve][Transform]Auto mark column length with jsonpath transform @hailin0
• [Feature][Transform-V2]Support transform with multi-table @Hisoka-X
• [Feature][Core]Support cdc task ddl restore for zeta @hailin0

others

• [doc][connector][hive]Add hive chinese doc @zhangshenghang
• [Doc][Format]Add protobuf doc @zhangshenghang # apache/seatunnel-website

improve

• [Improve]Change announcement bar to add star @Hisoka-X

others

• Fix translate error in about.md @XuyiK
• Update jdbc-mysql.md @a110q # apache/seatunnel-web

fix

• [Bug] [Seatunnel-web]No configuration setting found for key 'where_c…ondition' @arshadmohammad
• [Addendum][Bug] [Seatunnel-web]Error when conditional column is not used in SE @BilwaST
• [Bug] [Seatunnel-web]Error when conditional column is not used in SE… @BilwaST
• [Bug][Seatunnel-web]Fix fetch table fields for Oracle Jdbc @shashwatsai