When building a data platform, the first step isn't usually data modeling—it's getting your databases connected.
If a database can't connect, the connection type is mismatched, or account permissions are incomplete, it will directly block downstream data integration, development, metadata harvesting, and querying.
qData Open Source Data Platform provides a unified data connection management entry under the "Data Development" menu. Administrators can add, test, enable, or disable connections here.
This guide will walk you through qData's data connection capabilities, focusing on three key questions:
- What databases are currently supported?
- How to configure connection parameters for different databases?
- How to test connections and troubleshoot issues after saving?
1. What Data Sources Does qData Currently Support?
qData Open Source currently supports direct configuration for 8 database connections: MySQL, DM8, Oracle, Oracle11, SQL Server2008, SQL Server, Kingbase8, and Doris.
These types cover common open-source relational databases, commercial databases, domestic databases, and analytical databases.
Important Note: qData provides separate connection types for specific database versions (Oracle vs. Oracle11, and SQL Server vs. SQL Server2008).
Always select the exact version matching your source database to avoid driver or protocol mismatches.
2. What Parameters Are Required to Add a Database Connection?
Navigate to "Data Development > Data Connections" and click "Add" at the top left.
While form fields vary slightly, you will need to provide the following basic information:
- Connection Name: Used to identify the connection. We recommend a naming convention like "System Name - Environment - Database" (e.g., Order System - Prod - MySQL) to aid future search and maintenance.
- Connection Type: Select the exact database and version. Do not use a similar version as a substitute.
- IP: The database server address. Crucially, ensure the qData deployment server can access this address, not just your local machine.
- Port: The actual port the database service is listening on. Use the real port if it has been customized; do not rely solely on defaults.
- Username & Password: Create a dedicated database account following the principle of least privilege.
- Database Name: Specifies the target database to access.
- Description: Optional. Record the data source, connection purpose, and responsible department.
- Status: Defaults to "Disabled." We recommend saving the connection, testing it, and only enabling it after confirming the configuration is correct.
- Remarks: Optional. Record network zones or maintenance personnel, but never store plaintext passwords here.
3. How Do Database Fields Differ?
While MySQL and DM8 primarily require the database name, Oracle and SQL Server require an additional mandatory field: Schema Name.
Oracle / Oracle11:
The Schema Name is not the server IP or database name; it is the Schema used to organize database objects (tables, views). It is typically the same as the object owner's username.
For example, if business tables belong to the QDATA_APP user, the Schema Name should be QDATA_APP. If the connection account differs from the object owner, you must enter the actual Schema holding the tables and ensure the connection account has access permissions.
Misconfiguring this usually results in a failed connection test or an inability to find target tables during metadata sync. Before connecting, confirm the database name, login account, target Schema, and authorization scope with your DBA.
SQL Server / SQL Server2008:
The Schema Name is also mandatory. While dbo is the common default, business systems often use custom schemas like sales, ods, or report.
Always use the actual Schema to which the target table belongs. A successful login does not guarantee access to objects within the target Schema.
Confirm the database name, Schema name, login account, and authorization scope before configuration.
4. Reference Guide for Common Database Parameters
Below is a reference for configuring common databases. Note that ports listed are typical defaults; always use the actual port in your environment.
MySQL: Default port is 3306. Key field: Database Name. Ensure the account allows remote login from the qData server and has read/sync permissions.
DM8: Default port is 5236. Key field: Database Name. Verify the instance listening address, port, and account permissions.
Oracle / Oracle11: Default port is 1521. Key fields: Database Name, Schema Name (Mandatory). Select the type based on the source version; Schema usually matches the user Schema.
SQL Server / SQL Server2008: Default port is 1433. Key fields: Database Name, Schema Name (Mandatory). Ensure TCP/IP is enabled and the instance port is accessible. Common Schema is
dbo.Kingbase8: Common port is 54321. Key fields: Refer to the page after selecting the type. Verify network policies, driver versions, and actual listening ports.
Doris: Common port is 9030 (MySQL protocol). Key fields: Refer to the page after selecting the type.
Usually connects to the FE query port; do not mistakenly use the HTTP management port.
Disclaimer: This reference is for pre-configuration checks. Production ports, network policies, and permissions should always be verified by your DBA.
5. Understanding Connection Parameters (MySQL Example)
When connecting to a MySQL business database, select MySQL as the type, enter the accessible IP/domain, the actual listening port, the target database name, and a valid account.
Container Environments: If qData and MySQL communicate within a container network, use the internal port (usually 3306).
If accessing via the host machine, use the externally mapped port. Never use 127.0.0.1 or localhost, as these typically point to the qData container itself.
6. How to Test a Connection After Saving
After saving, locate the connection in the list and click "Test Connection" on the right.
Only enable the connection after seeing the "Database connection successful" prompt.
Note: A successful test only proves qData can access the database with the provided credentials. It does not guarantee the account has permissions for all downstream tasks (metadata sync, data querying, etc.).
Can I skip manual testing? Manual testing is not strictly mandatory. Togg the status switch to "Enable" will trigger an automatic validation.
However, from an operational perspective, we highly recommend manually testing first to quickly identify parameter, network, or account issues.
7. Troubleshooting Failed Connection Tests
If the test fails, follow this troubleshooting sequence:
- Verify Parameters: Double-check the IP, port, database name, Schema name, username, and password.
- Check Service Status: Ensure the database service is running and listening on the specified network interface and port.
- Check Network Policies: Verify that firewalls, security groups, or ACLs allow the qData server to access the database.
- Check Account Status: Confirm the password is valid, the account isn't locked, remote login is allowed, and the account has the necessary permissions.
- Verify Database Type/Version: Ensure you haven't selected Oracle for an Oracle11 database, or SQL Server for SQL Server2008. Mismatches cause driver/protocol issues.
- Check Doris Ports: For Doris, ensure you are using the FE query protocol port, not the Web management port.
Special Reminder: Connection requests originate from the qData server. Just because your local machine can connect does not mean the qData server has the same network access.
8. Where Are Data Connections Applied?
Once saved and validated, a data connection serves as a unified entry point for multiple platform scenarios:
- Data Integration: Used to specify the read/write endpoints. Centralized maintenance eliminates the need to repeatedly enter database addresses and credentials across different tasks, reducing management overhead when passwords or ports change.
- Data Development: SQL tasks rely on connections to access target databases. The configured Database and Schema names dictate which tables and views developers can discover. Incorrect Schema configurations for Oracle/SQL Server will hide target tables even if the connection is active.
- Metadata Harvesting: Reads structural information (databases, schemas, tables, columns). The connection account must have permissions to read system catalogs; otherwise, metadata harvesting will be incomplete.
- Data Querying: Used to view table structures, validate data, and execute authorized queries. If tables are missing or results are incomplete, verify both the SQL and the connection's Schema/account permissions.
Conclusion
qData Open Source supports 8 database connection types.
While form fields vary, the integration logic remains consistent: select the matching version, fill in the server/port/credentials, provide the Schema name where applicable, save, and test before enabling.
Connection issues rarely stem from the form itself; they usually involve database versions, network reachability, actual listening ports, Schemas, and account permissions.
Confirming these details beforehand prevents downstream failures.
Remember: a successful connection only means the address and credentials are valid. Full access to target objects depends entirely on the database name, Schema, and account permissions.
Top comments (0)