Enterprise data keeps piling up, yet finding the right table gets harder. For the same metric, sales and finance often come up with two different numbers. The problem usually isn't the data itself—it's the lack of a unified modeling management process.
qData Data Modeling connects business planning, data warehouse layering, logical modeling, metric standards, and model publishing into one chain, so model designs truly land in the database and enter the asset system, supporting downstream data development and business analysis.
Many enterprises have encountered similar situations: ERP, CRM, orders, and finance each manage their own data, while the data team continuously syncs and processes data by project. Over time, the database holds hundreds or thousands of tables—some with duplicate purposes, some that even developers can't clearly explain. Every time a new requirement comes in, significant time is spent finding tables, checking fields, and digging through old SQL. More troublesome is that different departments have inconsistent definitions for the same metric—for example, order amount calculated by order time in one team and by payment time in another, naturally leading to mismatched results.
What data modeling addresses is clarifying business relationships, data classification, model structures, and metric definitions before development begins, so subsequent work has a unified basis.
I. Why Is Data Modeling Needed?
More systems, more tables—yet finding data becomes more troublesome. Duplicate table creation, inconsistent definitions, and difficulty in searching—these problems cannot be solved by simply creating more tables. The core of data modeling is connecting business and data: clarifying relationships through business planning, defining data storage through data warehouse layering, standardizing table structures through logical modeling, unifying definitions through metrics and standards, and finally making designs land through model publishing. The result is clearer business relationships, more unified data classification, more standardized model design, more consistent metric definitions, and more efficient development and analysis.
II. qData Data Modeling's Overall Design Approach
qData integrates business planning, data warehouse layering, logical modeling, metrics and standards, and model publishing into one module, so enterprises don't need to switch between multiple tools and documents. The entire process has five steps:
First, complete business planning to clarify business scope, subject domains, and business object relationships. Second, perform data warehouse layering to determine how different types of data are stored and managed. Then enter logical modeling to design table and field structures. Next, unify definitions through metrics and standards. Finally, through model publishing, generate physical tables from logical models and automatically register them as data assets.
The platform also provides a modeling overview page where you can directly view the number of data divisions, subject domains, conceptual models, logical models, metric models, and published models, as well as the distribution of models across data warehouse layers, physical table publishing status, and recent change activity. Published physical tables and modeling information can continue to be used in data development, data governance, data assets, and data services modules.
III. Data Warehouse Layering: Different Data Goes into Different Layers
A data warehouse involves raw data, business details, statistical summaries, and application data. Mixing them together makes it difficult to distinguish purposes, and subsequent maintenance becomes increasingly complex. qData provides unified data warehouse layering management, dividing the warehouse into five layers: ODS, DIM, DWD, DWS, and ADS.
ODS receives raw data from source systems, retaining original field structures for traceability. DIM manages common dimensions such as customers, products, and regions. DWD cleans and organizes raw data into details such as orders and transactions. DWS performs statistical summaries based on analysis requirements. ADS organizes data for specific reports and applications.
Taking e-commerce order analysis as an example: raw orders go into ODS, customer and product information is organized into DIM, order transaction details go into DWD, sales statistics are formed through DWS, and finally ADS provides data to reports and business applications.
In qData, the five layers are pre-configured, supporting viewing and maintenance of each layer's name, description, owner, and table name prefix, along with statistics on existing logical models in each layer. Entering a specific layer also allows viewing the logical tables and related model information under that layer.
IV. Business Planning: Clarify the Business First
Before designing models, you must first understand what businesses the enterprise has, what objects are involved, and how they relate to each other. Directly copying existing system table structures easily leads to inconsistent business division and duplicate data construction.
qData provides data divisions, subject domains, business entities, and conceptual models to complete business planning before logical modeling.
1. Data Divisions: Define Business Scope First
Data divisions are used to divide the enterprise's larger business scopes, such as e-commerce retail, supply chain management, and financial management, giving different business data and models clear ownership. Each division can be configured with a name, English name, division architect, business owner, and data owner to clarify management responsibilities.
Unified naming rules for logical tables can also be configured, such as table name prefixes like ods_, dim_, dwd_, dws_, and ads_, as well as suffixes based on time granularity such as day, hour, and minute. This clarifies business boundaries and owners while ensuring models under the same division follow unified naming rules, avoiding each project defining its own standards.
2. Subject Domains: Organize Similar Business Together
A data division typically contains multiple business subjects that need further classification. For example, under the e-commerce retail division, you can divide customer domain, product domain, transaction domain, and fulfillment domain. The transaction domain mainly manages orders, payments, and refunds, while the customer domain manages customers and members.
qData supports up to five levels of subject domains, which can be further subdivided as needed—for example, setting order management and payment management sub-domains under the transaction domain. Each subject domain has an independent name, code, and English abbreviation, and the code can participate in logical table naming generation. Through subject domain management, subsequently created business entities and logical models can be attributed to specific subjects, facilitating search and maintenance.
3. Business Entities: Clarify What Business Objects and Activities Exist
After business classification is complete, you need to clarify what objects are managed and what activities occur under each subject. qData divides business entities into two major categories: business objects and business activities.
Business objects describe people, things, and matters that need to be identified and managed, divided into ordinary objects and hierarchical objects. Ordinary objects are relatively independent objects such as customers, products, and channels. Hierarchical objects are objects with parent-child relationships such as administrative regions and organizational structures.
Business activities describe behaviors and events that occur during business processes, divided into business events and business processes. Business events are specific behaviors such as placing orders, making payments, and issuing refunds. Business processes are complete processes composed of multiple events, such as order transactions and order fulfillment.
These two types of entities are directly related to subsequent logical modeling: business objects are used to create dimension logical tables, and business activities are used to create fact logical tables. For example, customers and products can be designed as customer dimension tables and product dimension tables; business activities such as placing orders and making payments can be designed as corresponding fact logical tables.
4. Entity Relationships: Connect Business Objects and Business Processes
After defining business entities, you also need to clarify the relationships between them—for example, which customer an order belongs to, what relationship registered members have with ordinary users, and which process order transactions enter after completion.
qData supports configuring multiple relationships based on entity characteristics. Association relationships describe connections between business objects or between business activities and business objects. Inheritance relationships describe inheritance between ordinary objects. Containment relationships describe business events contained in business processes. Predecessor-successor relationships describe the sequence of different business processes. Flow relationships describe the sequence of events within the same business process.
These relationships are configured based on entity types. Through conceptual models and entity relationship diagrams, you can intuitively view the connections between business objects and business activities, helping business personnel and developers understand the complete business structure.
V. Logical Modeling: Design Data Tables Based on Business Requirements
After completing business planning, you can design specific data models. qData provides five types of models: source logical tables, dimension logical tables, fact logical tables, summary logical tables, and ADS application tables, corresponding to data ingestion, common dimensions, business details, statistical summaries, and application analysis scenarios.
Different models have different roles, enabling unified management of table structures according to standards while allowing previously organized business objects, business activities, and metric definitions to be truly used in model design.
1. Source Logical Tables: Unified Management of Source System Data
Source logical tables solve the problem of where raw data comes from and what the table structure is, belonging to the ODS layer. Enterprises typically have multiple business systems, and each system's database and field definitions may differ.
qData supports selecting physical tables from existing data sources, establishing mapping relationships between source tables and source logical tables, and bringing raw field structures into unified management. When creating, you can select the source system, data connection, and source physical table, generate logical tables according to ODS naming conventions, and associate data standards and other information.
The platform retains table-level and field-level source relationships. When source table structures change, model definitions can be updated through structure synchronization. Source logical tables mainly retain and standardize source data structures and are not responsible for reprocessing based on business requirements, so they do not need to be bound to data divisions, subject domains, and business entities.
2. Dimension Logical Tables: Unified Management of Business Objects
Source tables retain the original structure of source systems, but information about the same business object may be scattered across multiple source tables. For example, a customer's basic information, membership level, and location may be stored in different systems. To facilitate subsequent analysis, dimension logical tables can uniformly design the customer data structure rather than reorganizing it every time it is used.
qData's dimension logical tables belong to the DIM layer and need to be associated with business objects defined in business planning, supporting both ordinary objects and hierarchical objects. When designing models, the platform brings out relevant business attribution information based on business objects and generates recommended table names according to naming rules.
Fields can be added manually, introduced through SQL parsing, or imported from existing logical tables. Primary keys, associated dimensions, and data standards can also be configured. For primary keys and associated entities already defined in business objects, the platform can generate corresponding fields, reducing repetitive configuration.
3. Fact Logical Tables: Record Specific Business Activities
Dimension logical tables answer "what business objects such as customers and products are," while fact logical tables answer "what happened during business processes." Business activities such as placing orders, making payments, and issuing refunds all generate data such as order numbers, occurrence times, and transaction amounts, which can be uniformly designed and managed through fact logical tables.
qData's fact logical tables belong to the DWD layer and need to be associated with previously defined business activities, including business events and business processes. When creating models, primary keys, associated object IDs, and business time fields can be generated based on business activities. For business processes, process start times, end times, and occurrence times of contained business events can also be recorded.
Fact logical table fields are mainly divided into several categories: primary keys that uniquely identify a business record, such as order numbers; dimension fields that associate dimension logical tables such as customers, products, and regions; measure fields that record numerical values such as order amounts, purchase quantities, and discount amounts; and time fields that record business occurrence times such as order times and payment times.
The platform supports designing fields through manual addition, SQL parsing, and table import, and setting association relationships between fields and dimension logical tables, providing a foundation for subsequent metric calculation and data summarization.
4. Summary Logical Tables: Organize Business Details into Statistical Results
Fact logical tables record specific data such as every order and every payment, but actual analysis often focuses more on how many orders there are on a given day or what the sales amount is in a certain region. If every report recalculates details, it not only duplicates development but also easily leads to inconsistent definitions.
To address this, qData provides DWS summary logical tables that can design reusable summary data models based on unified statistical granularity, statistical timeliness, and metric definitions. For example, to statistics daily regional sales, you can set statistical timeliness as generating or updating results daily, statistical granularity as region, and statistical metrics as order quantity and sales amount. The platform generates corresponding fields based on the selected timeliness and granularity, and matches eligible derived metrics to form the field structure of the summary logical table.
This way, when different reports or applications need the same statistical results later, they can reuse existing summary models without redesigning the statistical logic each time.
5. ADS Application Tables: Organize Data for Specific Business Applications
The dimension tables, fact tables, and summary tables above are mainly used to organize and manage common data models, but different business applications may have different data requirements. For example, product refund analysis needs product names, refund counts, and refund amounts; sales dashboards need sales amounts, order quantities, and regional rankings.
qData provides ADS application tables to organize required data structures based on specific application scenarios. When creating application tables, you can specify the data division, subject domain, and application scenario, and design fields through manual addition, referencing derived metrics, referencing code tables, or SQL parsing. This allows further organization of data structures for reports, dashboards, and data services based on existing models and metrics, avoiding direct exposure of large numbers of underlying detail tables to business applications.
6. Unified Management of Model Structures and Field Information
Although the five types of logical models serve different purposes, they can all be managed uniformly in qData for model names, field structures, data types, business attribution, and related standard information, with fields and version information viewable by model type.
From source logical tables retaining original structures, to dimension tables and fact tables organizing business data, to summary tables and ADS application tables serving statistical analysis and business use, the five types of models together form the model design process from source data to application data.
It should be noted that logical models define how data should be organized, but do not mean data has already been synced and processed. After model design is complete, model publishing and data development are still needed to make the design truly land in the database and actual business.
VI. Metrics and Standards: Keep Data Definitions and Statistical Definitions Consistent
When enterprises conduct data analysis, they often encounter inconsistent statistical results for the same metric. For example, when counting orders, some departments count all orders, some count only paid orders, and some count only APP channel orders. Although all are called order counts, different calculation conditions naturally lead to different results.
To address such problems, qData provides atomic metrics, business qualifications, and derived metrics management, defining basic calculation rules, business filter conditions, and statistical periods separately, then combining them based on actual needs. At the same time, data standards and standard data elements unify field specifications.
1. Atomic Metrics: Unify Basic Calculation Definitions
Atomic metrics define the most basic statistical content, such as order count, payment amount, refund amount, and total customer count. In qData, atomic metrics can be created based on dimension logical tables or fact logical tables, configuring calculation fields, aggregation methods, statistical time fields, and other information to clarify the data source and calculation rules of the metric.
For example, to statistics order count, you can select the order fact table, use order number as the calculation field, and calculate the deduplicated order quantity through COUNT DISTINCT; to statistics payment amount, you can select the payment fact table and perform SUM aggregation on the payment amount field. This way, basic metrics such as order count and payment amount only need to be defined once and can be reused in different business scenarios later.
2. Business Qualifications: Clarify Which Data to Count
With basic metrics in place, you also need to clarify the statistical scope. For the same order count, business users may only want to see paid orders, APP channel orders, or orders that satisfy both conditions.
qData supports unified management of filter conditions through business qualifications, allowing condition expressions to be set based on dimension logical tables or fact logical tables, with support for basic qualifications and combined qualifications. For example, define paid orders as pay_status = 'PAID'; APP orders as order_channel = 'APP'; and APP paid orders as a combination of the above two qualification conditions. These business qualifications can be reused in metrics that meet source and timeliness requirements, without rewriting filter conditions for each.
3. Derived Metrics: Combine to Form Specific Business Metrics
Actual business analysis usually requires more than a basic metric—it also involves statistical scope, time period, and analysis dimensions. For example, if an enterprise wants to know "the number of paid APP channel orders by region in the last 7 days," it needs to combine the previously defined content: atomic metric as order count; business qualification as APP paid orders; statistical period as last 7 days; statistical granularity as region.
qData supports creating derived metrics through atomic metrics, business qualifications, and statistical periods, and matches corresponding DWS summary logical tables based on statistical granularity and data timeliness. During creation, the platform verifies whether metrics and qualification conditions come from the same logical table, whether statistical timeliness is consistent, and whether the source model can associate the required statistical dimensions. The statistical time field follows the definition in the atomic metric.
This way, when enterprises need to statistics order data for different time periods and scopes, they can reuse existing metrics and qualification conditions, reducing duplicate definitions and making statistical definitions easier to unify.
4. Data Standards and Standard Data Elements: Unify Field Specifications
In addition to metric definitions, data fields themselves also need unified management. The same customer ID might be called customer_id in the CRM system but cust_id in the order system, with different field names, data types, and even business meanings. Without unified specifications, developers need to reconfirm these fields every time they use them.
qData unifies field naming, business meaning, and other specifications through data standards, and manages reusable field definitions through standard data elements. For example, uniformly define the field name, data type, and length requirements for "customer ID." When designing models such as customer dimension tables and order fact tables later, you can associate corresponding standard data elements, reducing field definition differences between different models.
Through atomic metrics, business qualifications, and derived metrics, enterprises can uniformly manage metric calculation rules and statistical conditions; through data standards and standard data elements, model field definitions can be further standardized, providing a consistent reference for subsequent data summarization, report analysis, and data development.
VII. Model Publishing and Asset Registration: Make Designed Models Truly Usable
After completing business planning, logical modeling, and metric definition, the next step is to publish designed models as physical tables in the database and bring them into data asset management. qData connects model publishing with asset registration, allowing both new models and existing data tables to be managed under unified standards.
1. Model Publishing: Generate Physical Tables from Logical Models
qData supports publishing designed logical models to specified databases, generating corresponding physical table structures, and provides single publishing, batch publishing, delete-and-rebuild, and incremental publishing methods. The platform also supports viewing publishing status, publishing logs, and republishing.
2. Automatic Asset Registration: Bring Modeling Results into Asset Management
After logical models are successfully published, qData automatically registers the generated physical tables as data assets and associates modeling information such as model type, data warehouse layer, data division, and subject domain. No manual re-registration is required.
3. Existing Data Registration: Unified Management of Historical Data Tables
For database tables that already exist in the enterprise, qData also supports bringing them into management through asset registration, supplementing information such as logical table type, data warehouse layer, data division, subject domain, and business entity. For example, an existing order detail table can be supplemented with DWD detail layer, fact logical table, transaction domain, and other information, giving historical data clear classification and business attribution.
4. Asset Map: Unified View of Data Assets
After registration is complete, you can search and manage data tables in the asset map by data division, subject domain, data warehouse layer, and other dimensions, viewing model type, business attribution, and related asset information. Whether newly published models or existing physical tables, all can be viewed uniformly through the asset map.
5. Modeling Results Support Subsequent Data Applications
After model publishing, the generated physical tables can be used for subsequent data integration and data development; related model definitions, field standards, and business attribution information can also provide references for data governance, data asset management, and data services. At this point, qData connects business planning, model design, physical table publishing, and asset registration, allowing modeling results to continue serving subsequent data management and business analysis.
VIII. Summary: What Can Data Modeling Bring to Enterprises?
Data modeling is not just about designing data tables—more importantly, it gives enterprises a unified management basis for business, data, and metrics. Through qData Data Modeling, enterprises can start from business planning to clarify data business attribution and entity relationships, then unify data structures and statistical definitions through data warehouse layering, logical modeling, and metric management.
Designed logical models can be directly published as physical tables and automatically registered as data assets, connecting model design with actual data management. For existing database tables, modeling information can also be supplemented through asset registration and gradually brought into unified management. This way, whether new models or historical data, all can be classified, searched, and maintained according to unified standards, reducing duplicate construction and providing a clearer data foundation for subsequent data development, governance, and business analysis.
From business planning to data assets, qData Data Modeling uses model design as the foundation for actual data management, helping enterprises build a unified, reusable, and implementable foundation system.
Final Thoughts
From an actual workflow perspective, the value of qData Data Modeling corresponds one-to-one with its features: unified business planning and naming standards reduce duplicate table creation; data warehouse layering and logical modeling improve data search and model design efficiency; atomic metrics, business qualifications, derived metrics, and data standards unify metric definitions and field specifications; model publishing and automatic asset registration reduce the cost of physical table publishing and asset registration; asset maps and existing data registration bring historical data into unified management; and modeling results connect with data development, governance, assets, and services modules, allowing early-stage design to truly support subsequent data applications.
From business planning to model publishing, qData Data Modeling connects business, models, metrics, and assets into an implementable chain. It solves not just "how to create tables," but "how to make data unified, searchable, reusable, and manageable."
Top comments (0)