Connect TiDB Serverless From Databricks

TiDB Serverless by default allows access from Internet and requires secure connection. To connect TiDB Serverless clusters from Databricks, you need to turn on SSL connection from the client.

github_events = spark.read
  .format("jdbc")
  .option("url", "jdbc:mysql://gateway01.us-west-2.prod.aws.tidbcloud.com:4000/sample_data")
  .option("dbtable", "github_events")
  .option("user", <username>)
  .option("password", <password>)
  .load()

This is a simple Python snippet to load the github_events sample table from a TiDB Serverless cluster. It will return a error:

ERROR 1105 (HY000): Connections using insecure transport are prohibited. See https://docs.pingcap.com/tidbcloud/secure-connections-to-serverless-tier-clusters

By default the JDBC driver doesn't use SSL to connect. According to https://docs.databricks.com/external-data/mysql.html, seems the underlying driver is mariadb-connector/j, so just add the corresponding SSL option to our snippet:

github_events = spark.read
  .format("jdbc")
  .option("url", "jdbc:mysql://gateway01.us-west-2.prod.aws.tidbcloud.com:4000/sample_data?useSsl=true")
  .option("dbtable", "github_events")
  .option("user", <username>)
  .option("password", <password>)
  .load()

Only the useSsl=true option is needed. There is no need to specify any CA file since currently TiDB Serverless uses Let's Encrypt ISRG Root X1 as the signing authority. Usually, it's bundled into JDKs.

TiDB Serverless also supports private link connection. It helps if you need private connections.