DEV Community: TongWu

Handling Task Dependencies: How to Orchestrate Workflows in qData Data Middle Platform Pro

TongWu — Thu, 17 Sep 2026 05:49:00 +0000

In the early stages of building a data middle platform, the number of data tasks is usually limited. When there aren't many tasks, developers can configure and run them individually without much trouble.

However, as data processing becomes more complex, a final data result is rarely produced by a single task.

For example, generating a water level statistical report might require several steps:

Ingesting historical water level data.
Ingesting basic water station data.
Processing details.
Handling dimensions.
Performing summary statistics.

Some tasks must wait for upstream data to be ready, while certain downstream results might depend on multiple preceding tasks simultaneously. At this point, simply knowing "what tasks exist in the system" is no longer enough. You need to solve several problems:

Which task should run first?
Which tasks depend on each other?
How do you express a relationship where one result depends on two upstream tasks?
If an upstream task fails, which subsequent processes are affected?
After multiple tasks run, how do you view the progress of the entire pipeline?

These questions mark the transition from single task management to multi-task collaborative management.

qData Data Middle Platform Pro's Job Management addresses this by organizing multiple tasks into a data production pipeline with clear upstream and downstream dependencies, sitting on top of existing data integration and development tasks.

Why Do We Need "Jobs" When We Already Have Tasks?

Let's look at a typical data processing scenario.

Suppose you need to generate a daily river water level statistical report. After ingesting raw data from a third-party system, you cannot get the final result immediately. It requires multiple processing stages:

Source Data Ingestion → Detail/Dimension Processing → Summary Statistics

This might involve:

Syncing historical water level data from the source DB to ODS.
Syncing water station basic data to ODS.
Processing ODS historical water levels to form DWD details.
Processing ODS water station data to form DIM dimensions.
Using DWD and DIM data to form the DWS daily water level statistics.

If you look at these tasks separately, each can independently complete a specific data processing job. However, from the perspective of the final result, they are collectively doing the same thing: gradually processing raw water data into results usable for analytics.

Therefore, as steps increase, what needs to be managed is no longer just individual tasks, but how these tasks should run together in order.

1. First, Clarify: What is the Difference Between a Task and a Job?

In qData, tasks and jobs solve problems at two different levels.

Task: "How to complete this specific data processing work"

A task is a specific execution unit. For example, a task named River_Water_Historical_to_ODS_Incremental_Hourly is responsible for reading data from the source and syncing it to the ODS table. It handles:

Where to read data.
Where to write data.
Field mapping.
Execution configuration.

Data synchronization, cleaning, detail processing, and summary statistics can all be built as independent tasks.

Job: "How to organize multiple tasks to achieve the final result"

A job focuses on how multiple existing tasks should be organized, not on the internal logic of a single task.

For example, a job named River_Water_Monthly_Stats_Daily_Job might include historical ingestion, dimension processing, detail processing, and daily summarization. The job doesn't redefine the logic; it organizes existing tasks based on their dependencies.

Summary:

Task: How to do this specific work.
Job: How to run multiple tasks together to get the result.

2. How Does qData Chain Multiple Tasks into a Job?

Job orchestration isn't just putting tasks on one page. It involves reorganizing tasks according to real data relationships.

Step 1: Group Tasks into a Job

In the job configuration page, you can see existing integration and development tasks. You can group them by processing stage. For the water level scenario, you would add:

Historical data sync.
Station data sync.
ODS to DWD processing.
ODS to DIM processing.
DWD/DIM to DWS aggregation.

Step 2: Establish Upstream/Downstream Dependencies

Adding tasks to a job only defines membership. The critical step is defining the relationships.

In our scenario, there are two parallel preparation chains:

Historical Chain: Source → ODS → DWD.
Station Chain: Source → ODS → DIM.

Once both are ready, they feed into the final step: DWD / DIM → DWS.

The final result depends on both inputs. The job canvas makes this explicit, transforming a list of tasks into a flow: Ingestion → Processing → Statistics.

Step 3: Maintain Task Configurations

Once inside a job, you can still view and maintain specific configurations for each node (e.g., owner, connection, source/target tables, priority). The job organizes the flow without detaching you from the specific task details.

Step 4: Check Before Running

Before running a complex pipeline, you should verify the configuration. qData provides a task check capability. This acts as a pre-flight check to ensure the orchestration meets execution requirements, preventing configuration errors from propagating down the chain.

The configuration process:
Select Tasks → Organize into Job → Define Dependencies → Refine Config → Run Check.

3. Why Can't We Just Look at One Task When Things Go Wrong?

When tasks are independent, troubleshooting focuses on that specific task (SQL errors, connection issues, etc.).

However, when tasks are linked in a job, your perspective must shift.

If the "Historical Data Ingestion" fails, you shouldn't just ask "Why did this fail?" You must also ask:

Where does this sit in the pipeline?
What downstream tasks depend on it?
Will the final daily report be affected?

Job orchestration changes the focus from "Did this task run?" to "What is the impact of this task on the whole chain?"

4. How Do You Track the Progress of the Entire Pipeline?

Once a job is running, you need to know the status of the whole chain.

qData allows you to view Job Instances. At this level, you can see:

Job Name.
Execution Status.
Start/End Times.

For complex jobs, you can drill down into specific tasks (e.g., checking if the ODS-to-DWD step is finished while the DIM step is still running). This tells you exactly where the data pipeline stands.

Drilling Down to Logs

If a specific node fails, you can access the Run Logs directly from the instance view. Logs include submission time, duration, status, and downloadable log files.

This creates a hierarchical troubleshooting path:
Job Overview → Specific Task Status → Run Logs

5. Summary: The Complete qData Job Management Lifecycle

To recap, a complete job management process involves:

Build Independent Tasks: Create sync and processing tasks in integration/development modules.
Create Job: Group relevant tasks around a final data product.
Define Dependencies: Map out who runs first and who waits for whom.
Review Config: Check owners, connections, and priorities.
Pre-Run Check: Validate the orchestration logic.
Unified Execution: Run the pipeline based on the defined strategy.
Monitor Instance: View the overall status of the run.
Log Analysis: Drill down to fix specific errors.

The Value of Job Management in qData

In a data middle platform:

Data Integration solves how to ingest.
Data Development solves how to process.
Job Management solves how to organize.

It bridges the gap between scattered tasks and a cohesive data production line. It addresses three layers:

Task Layer: Execution details.
Relationship Layer: Dependencies and order.
Operation Layer: Pipeline status and impact analysis.

Job orchestration transforms isolated tasks into a clear, dependent data production pipeline, shifting the focus from managing individual units to managing the flow toward the final result.

qKnow Open Source v2.4.3 Released: Expanded Data Ingestion and Customizable Knowledge Construction

TongWu — Thu, 17 Sep 2026 05:48:09 +0000

The qKnow Agent Construction Platform Open Source Edition v2.4.3 focuses on upgrades regarding unstructured knowledge ingestion, knowledge asset export, and customizable document parsing. Key updates include:

New Import Capabilities: Support for importing JSON/JSONL files into Knowledge Bases and Knowledge Graphs.
Data Export: One-click export of Q&A data from Knowledge Bases to JSON/JSONL formats.
Customization: Opened capabilities for customizing file parsing models and prompts.

These updates further refine the processing pipeline for unstructured documents, covering ingestion, parsing, and reuse.

From "Uploading Documents" to Enabling Data Ingestion, Processing, and Reuse

When enterprises build knowledge bases and agents, raw knowledge isn't limited to standard documents like Word or PDF files. Real-world projects often involve document parsing results processed by other systems, semi-structured data, and pre-organized Q&A datasets.

This means the enterprise knowledge processing workflow is not simply:
Upload File → Parse → Build Knowledge Base

It is closer to:
External Data Preparation → Data Import → Document Parsing → Knowledge Generation → Data Export → Secondary Processing & Reuse

When a platform's data entry/exit methods and parsing logic are too rigid, three issues often arise as scenarios increase:

Lack of Import Entry: Parsed or organized JSON/JSONL data lacks a standard way to enter the platform.
Difficulty Exporting: Q&A knowledge generated in the knowledge base cannot be easily exported for external processing or dataset accumulation.
Rigid Parsing: Different document types require different parsing logic, but fixed models and prompts make it hard for users to adjust to specific business scenarios.

Therefore, the focus of qKnow Open Source v2.4.3 is not just adding file formats, but completing the full loop:
Data enters platform → Parsed per business needs → Knowledge formed → Results output for reuse

1. New JSON/JSONL Import for Knowledge Bases & Graphs

Previously, qKnow relied primarily on standard file uploads. However, in enterprise data processing, some unstructured documents may have already been parsed or converted into more structured formats like JSON or JSONL by other systems. If the platform forced users to start from raw files again, it would lead to redundant work.

This version adds JSON/JSONL import capabilities to provide a standard entry point for this data.

Direct Upload Support

The update supports uploading JSON and JSONL files. This means data that has already been organized doesn't need to be reverted to raw documents; it can be imported directly based on its actual form.

The workflow now supports:
Raw Document → External Parsing/Processing → JSON/JSONL → qKnow Knowledge Base/Graph

This is particularly suitable for enterprises with existing document processing pipelines or those needing to ingest data generated by other systems.

Distinguishing Data Sources

The update distinguishes between unstructured and semi-structured data to adapt to different ingestion scenarios. While JSON/JSONL has structure, the content may originate from unstructured document parsing or be pre-organized semi-structured knowledge. This allows different forms of knowledge data to enter the processing flow.

Bridging the Gap for Re-ingestion

A key application is re-ingesting data that has already been parsed. For example, if an enterprise uses external tools to parse files and saves the results as JSON, qKnow can now import this directly, avoiding the cycle of converting results back to files just to re-parse them.

Note: The JSON/JSONL import solves the ingestion problem. How this data flows into knowledge parsing and construction depends on the specific data format and the enterprise's configuration.

2. Export Q&A Data to JSON/JSONL for Reuse

A knowledge platform must address not only how data enters but also how it leaves.

In enterprise knowledge base construction, parsing a document often generates a large amount of Q&A knowledge. This data is valuable for:

External system processing.
Dataset organization.
Model evaluation.
Data annotation.
Cross-system reuse.

If parsing results are trapped inside the platform, knowledge assets cannot flow freely.

One-Click Export

qKnow Open Source v2.4.3 adds the ability to export Q&A data. After parsing documents and building the knowledge base, users can export the Q&A data into standard JSON or JSONL files.

The workflow becomes:
Upload → Parse → Generate Q&A → Build KB → Export JSON/JSONL → External Processing

Standardized Output for Dataset Accumulation

For teams building enterprise agents long-term, Q&A data is a sustainable asset. It can be used for verification, test set construction, training data preparation, or manual review. Standardized export formats facilitate batch processing better than manual copying.

From "KB Results" to "Tradable Assets"

This shifts the paradigm from:
File → Parse → Internal Use
To:
File → Parse → Knowledge → Internal Use → Export → External Reuse

This increases the flexibility of document-based knowledge assets across different tools. However, exported data may still require quality checks or format adjustments depending on the target use case (e.g., model training).

3. Customizable Parsing Models and Prompts

Enterprise documents vary significantly—policy documents differ from technical manuals, contracts, or maintenance logs.

Technical docs might focus on: Equipment, parameters, steps, exceptions.
Policy docs might focus on: Scope, rules, responsibilities, execution requirements.

A fixed parsing logic often fails to accommodate these differences. Previously, qKnow used fixed system logic for models and prompts.

Open Model Selection

Users can now adjust the model used for document parsing based on business needs. Parsing is no longer solely dependent on a preset platform model but can leverage the enterprise's own model resources.

The logic flows as:
Identify Doc Type → Select Parsing Model → Configure Prompt → Execute Parsing → Review Results

Configurable Prompts

The update also allows for custom parsing prompts. Prompts directly influence:

What content to extract.
How to understand the document.
How to organize the generated results.
Which business information to prioritize.

This allows the same model to adjust its extraction targets based on the scenario (e.g., focusing on main content vs. specific business entities).

Adapting to Scenarios

This shifts the workflow from a rigid process to a flexible one:
Upload → Select Model → Configure Prompt → Parse → Use Results

This is ideal for enterprises with diverse document sources. However, customization solves the adjustability of logic; actual results still depend on document quality, model capability, and prompt design.

4. A More Open Knowledge Processing Pipeline

Combining these three features, qKnow v2.4.3 establishes a more complete knowledge data pipeline.

Previous Flow:
Standard File Upload → Fixed Parsing → Knowledge Base → Internal Use

New Flow:
Raw File / JSON / JSONL → Import → Custom Model/Prompt Parsing → KB Q&A Data → JSON/JSONL Export → External Reuse

JSON/JSONL Import: Solves how different data forms enter.
Custom Parsing: Solves how knowledge is processed according to business logic.
JSON/JSONL Export: Solves how processed knowledge flows out for reuse.

Version Value Summary

This release makes unstructured knowledge handling more flexible from ingestion to reuse:

Open Ingestion: JSON/JSONL import reduces redundant conversion for pre-processed data.
Asset Circulation: Export capabilities allow knowledge to be used for external processing and dataset accumulation.
Controllable Logic: Custom models and prompts allow parsing to adapt to specific document types and business goals.

Overall, qKnow Open Source v2.4.3 connects the "Import — Parse — Export" loop, giving enterprises more autonomy in building and reusing non-structured knowledge.

From a Database Table to an API: How qData Open Source Delivers Data as a Service

TongWu — Thu, 17 Sep 2026 05:46:46 +0000

Data sync is done. Data processing is complete. The final table has landed in the database.

But for the business systems that actually need this data, that's usually not the end of the story.

Dashboards, business applications, or third-party systems can't each directly connect to the database, understand the underlying table structure, and write their own SQL queries. For a data middle platform, beyond solving how to bring data in and how to process it, there's one more question:

How do you deliver processed data in a standardized, controlled way?

qData Open Source Edition provides API management capabilities within its data service module. It lets you configure database tables, SQL queries, or forwardable data assets into callable APIs — complete with parameter configuration, interface testing, application authorization, and call record management.

In this article, we'll walk through the most straightforward scenario: how to turn a ready-to-use database table into a callable data API step by step in qData.

Before You Start: Prepare the Data You Want to Serve

Let's say your database already has a hydrological monitoring data table containing fields like:

Station code
Station name
Monitoring time
Water level

Now you want to expose an interface to external business systems so callers can query water level data by station code.

Before creating the API, confirm two prerequisites:

The target data source and database table are accessible and working properly.
You've already set up a service category under Data Service > Service Categories for this API.

qData's service categories support a parent-child hierarchy for organizing different data services. Enterprises can pre-build categories based on business domain, data type, or system purpose. Once APIs are created, they can be found and maintained under a unified directory.

Once data preparation is done, you're ready to enter the API creation flow.

Create the API: Configure a Data Table as a Data Service

Go to Data Service > API Management and click "New".

qData uses a three-step flow to create APIs:

Property Configuration → Parameter Configuration → Testing

These three steps essentially answer three questions:

What is this API?
Where does the data come from, and what parameters are passed when calling it?
Does the API return results correctly after configuration?

Step 1: Configure Basic API Properties

The first step is Property Configuration. Here you define the API's core information:

API category
API name
API endpoint path
API version
Request method
Response format
API description

You can also configure management controls as needed:

IP blacklist
Request rate limiting
API online/offline status

For example, in the hydrological monitoring scenario, you might define this as a "Water Level Daily Report Query Service," specifying the API path, version, POST method, and a business description.

These properties serve both internal platform management and communicate to future callers what this API does and when to use it.

If there are call frequency limits, you can set rate limiting. If certain IPs should be blocked, you can control that via the IP blacklist.

Once basic properties are confirmed, move on to parameter configuration.

Step 2: Select the Data Table and Define "How to Query, What to Return"

In Parameter Configuration, you first determine where the API's data comes from.

qData currently offers three configuration modes:

Single-table wizard
SQL script mode
Third-party forwarding

For our single-table scenario, select the corresponding data source, then choose the table you prepared earlier. This tells the platform: "This API's data comes from this table."

But selecting a table doesn't mean exposing the entire database table directly to business systems.

When serving data externally, you need to define two things:

01 — What conditions can callers use to query?

For example, if you want business systems to query water level data by station code, add a request parameter station_code in the Request Parameters section, and configure:

Whether it can be empty
Parameter type
Parameter description
Example value
Default value

This way, callers only need to pass in a station code to query the corresponding station's data.

02 — What fields can callers actually receive?

After defining request conditions, configure the response fields. For example, you might select:

Station code
Station name
River name
Water level

So turning a database table into an API isn't about exposing the raw table structure. The platform lets you precisely define what callers can query by and what data they ultimately receive.

Step 3: Test the API Online Before Delivery

After parameter configuration, it's not recommended to hand the API over to business systems immediately. Run it once in qData first.

In the Testing step, the page displays:

API name
Version
Request method
Response format
Call endpoint

It also provides a request data input area. Fill in the request parameters you defined, then click "Call API".

If the request succeeds, the platform shows a successful call and displays the actual query results in the response area. If a required parameter is missing, it will prompt that the parameter cannot be empty.

During this phase, focus on confirming three things:

Do the query conditions actually take effect?
Are the returned fields correct?
Does the actual returned data match expectations?

After testing, the API enters the API management list. You can re-verify later by going to the API details and using "Test Info" to trigger another call.

At this point, the database table has essentially completed its transformation from "table" to "API." But to let other business systems actually use it, you still need to handle access identity and authorization.

Create a "Caller Identity" for Business Systems

Once the API is created, not just any system can access it by grabbing the endpoint URL.

In qData, business systems that actually call the API need to first establish an application identity through Application Management.

Go to Data Service > Application Management and create a new application.

For example, if a water resource monitoring system needs to call the API you just created, register it as an application in qData and maintain:

Application name
Application type
Application description

This answers the question: Who is using this API?

For enterprise data services, this matters because the API itself is a data capability, but the actual callers are usually frontend apps, business systems, upstream platforms, or third-party integrations. Only by identifying the caller first can you then control which data services each application is allowed to access.

Authorize the Application to Access the API

After creating the application, it doesn't automatically gain access to all APIs.

Go to API Authorization in the application details, click "New", select the API you created earlier, and set the authorization validity period.

qData supports:

Permanent validity
Specified start and end dates

After submission, the API appears in the application's authorization list.

This creates a clear mapping in the platform: which application can call which API.

When your enterprise grows to dozens of business applications and hundreds of data APIs, this relationship becomes critical. Different business systems can each maintain their own allowed data services, rather than opening all APIs indiscriminately to every caller.

Get Access Credentials and Let Business Systems Call the API

Once the application identity and API authorization are in place, you're ready for actual system integration.

Business systems need to obtain an access token according to the authentication method configured in your qData deployment environment, and include the valid token when calling specific business APIs.

From the business system integration perspective, the complete flow is:

Create Application → Obtain Application Identity → Authorize API → Get Token → Call API with Token

At this point, the way business systems access data has fundamentally changed.

Before: Direct database access.

Now: Call APIs through the data service layer to retrieve data.

The database tables still serve as the underlying data store, but business systems no longer need to understand the database structure directly. They query through the platform's data service layer.

Track API Calls After They Happen

Once APIs go live in production, the management work doesn't end. You'll quickly face practical questions:

Has the API been called?
Who called it?
How much data was returned?
Why did a particular call fail?

qData provides unified call tracking under Data Service > Call Records, where you can view:

API service name
Caller IP
Called endpoint
Data volume returned
Response time

Drilling into details, you can further inspect:

Call parameters
Request method
Call status
Error information

For example, when a business system reports "this API didn't work just now," operations or development staff can locate the corresponding request by service name and call time, then review the parameters, status, and error details.

This means the API isn't just a configured interface handed off and forgotten — it has a complete lifecycle from creation, authorization, and invocation to ongoing tracking.

From Table to API: The Full Flow at a Glance

Looking back, the entire process from a prepared database table to a live, callable API can be summarized as:

Prepare Data Table → Create API → Configure Properties → Select Data Table → Configure Request Parameters and Response Fields → Test Online → Create Calling Application → Authorize API → Obtain Access Credentials → Business System Calls API → Review Call Records

This pipeline reflects a fundamental challenge in data middle platforms: syncing data in and processing it only covers the first half of data construction.

The real business usage phase requires answering: How do you deliver processed data securely and clearly to other systems?

qData Open Source Edition's data service module adds a data service capability layer between database tables and business applications.

The underlying storage remains the database tables you've already built, but when serving externally, you can further define interfaces, query parameters, response fields, application identities, and authorization relationships.

This transforms a table that could previously only be queried inside the database into a data API callable by business systems.

Wrapping Up

For enterprise data middle platforms, data doesn't create real business value just by being "synced" or "processed."

What matters more is how you deliver that prepared data to dashboards, business systems, and third-party applications.

qData Open Source Edition achieves this through:

API Creation → Parameter Definition → Online Verification → Application Identity → API Authorization → Token-based Calls → Call Records

This progressively converts a raw database table into a data service usable by business systems.

It's not simply wrapping an interface around a table. It's establishing a clear service boundary between data and business applications — defining where data comes from, how it can be queried, what can be returned, who can call it, and how calls are tracked afterward.

Extending from "bringing data in and managing it" to "delivering data out and putting it to real use" — that's the practical significance of qData's data service within the broader data middle platform pipeline.

From Chaos to Order: Deep Dive into qModel Open Source Edition's Model Version Management and Multi-Version Comparison

TongWu — Thu, 17 Sep 2026 05:46:01 +0000

Algorithm models are rarely "done" after initial development.

Once a model enters production, it continuously evolves — driven by data drift, parameter tuning, algorithm optimization, and shifting business requirements.

A single optimization pass may produce a new version. After multiple rounds of experimentation, you end up with test versions, production versions, and historically stable versions. When the version count is small, teams can still get by with file naming conventions and manual records.

But as iterations accumulate, this approach quickly breaks down:

Files like model_v2_final_0315, final_final_v3 keep piling up, making it hard to confirm which version is actually live.
Online models and offline experiment versions can't be accurately mapped, making it difficult to reproduce performance fluctuations.
When multiple people iterate on models simultaneously, different versions easily overwrite each other or create collaboration conflicts.
After parameters or configurations change, it's hard to quickly answer "what exactly changed this time," making it difficult to attribute model performance changes.

For algorithm model platforms that need to run continuously, model management can't stop at "can the model be uploaded and executed." It also needs to manage version relationships, configuration differences, and current active states throughout the entire iteration lifecycle.

To address this, qModel Algorithm Model Platform consolidates version management, version comparison, new version creation, and multi-version switching into the model management workflow — transforming models from a series of isolated modifications into a version evolution history that can be viewed, compared, switched, and rolled back.

Version Management: Centralize Scattered Model Versions

The first step in model version governance is answering: "What versions exist, and which one is currently active?"

In the past, when models were managed through file directories, different versions might be scattered across different folders or held by different team members. Finding a specific historical iteration usually required cross-referencing file names, modification timestamps, and developers' memories.

As model iterations increase, this approach becomes increasingly unsustainable.

qModel adds a dedicated "Version Management" tab on the model detail page, centralizing all versions under the same model.

Once you enter the version management page, you can view:

All versions created for the current model
The currently active version
Total version count
Basic information for different historical versions

Version management is integrated directly into the model detail page, rather than splitting different versions into separate, disconnected model records. This design means teams are no longer managing a batch of similarly named but isolated model files — instead, they manage:

One model entity + multiple continuously evolving versions.

After multiple rounds of experimentation and optimization, historical versions remain preserved within a unified version lineage. Developers and operations staff can quickly confirm which version is currently active.

This lays the foundation for subsequent model comparison, switching, and historical rollback.

Version Comparison: Answer "What Exactly Changed in the New Version?"

Once you know a model has multiple versions, the next question is usually:

What actually changed between two versions?

In traditional model management, this often requires developers to open two versions separately and manually compare configurations and parameters. If the model adjustments are extensive, this is not only inefficient but also prone to missing subtle changes.

qModel supports selecting any two model versions for side-by-side comparison. The system systematically organizes differences across configurations, parameters, and other dimensions.

On the version comparison interface, the two versions being compared are displayed on the same page, allowing you to directly view version basic information and configuration differences side by side.

This transforms model version analysis from:

Open two versions separately → Manually hunt for differences

to:

Select two versions → View differences in one place

For example, when a new version goes live and performance shifts, algorithm engineers can first compare the old and new versions to confirm whether configurations and parameters were adjusted, then combine that with actual model performance to continue root cause analysis.

For models that have gone through multiple rounds of experimentation, you can also select two specific versions to compare, helping the team answer more concrete questions:

What was modified in this iteration compared to the previous version?
Which configurations remained unchanged?
Which parameters were adjusted?
What differences exist between the current version and a historically stable version?

It's worth noting that version comparison itself can't directly determine which parameter caused a model performance change. But it surfaces the differences between versions first, providing a clearer version-level basis for subsequent analysis combined with experiment results, data changes, and runtime performance.

New Version Creation: Continue Iterating on the Existing Model

During model iteration, there's another very practical question:

Does creating each new version require reconfiguring the model from scratch?

If every experiment round requires creating a new model and re-entering all configurations, the work is not only repetitive — it also risks parameter omissions that introduce unintended differences between versions.

qModel supports creating a new version directly based on the current model version.

After creation, the new version automatically inherits the original version's configuration and context. There's no need to rebuild from zero — you continue the next round of adjustments on top of the existing version.

New version creation still follows the original model configuration workflow. Users can continue completing subsequent configurations based on the inherited version information, which better aligns with actual model development practices.

This is because most model iterations don't completely scrap the existing model and start over. Instead:

Most of the previous version's configuration is retained, with adjustments made only to specific parameters, data, or model settings.

For example, when a stable model needs parameter optimization, you can create the next version based on the current one, making modifications while preserving the original configuration relationships.

This reduces repetitive configuration work and establishes a clearer evolution relationship between old and new versions.

The model iteration process shifts from:

Copy file → Modify → Copy again

to:

Current version → Create new version → Adjust configuration → Test and verify → Form next version

Multi-Version Switching: Let Test, Production, and Historical Stable Versions Coexist

Once a model has multiple versions, not every version immediately replaces the current production version.

In practice, the same model may simultaneously have:

A currently running production version
A new version being validated
A previously stable historical version

So version management needs to solve not just "how to create," but also "which version is currently active."

qModel supports multi-version coexistence and version switching under the same model. During testing and production use, you can switch between different model versions as needed. Once a new version meets requirements, you can switch to it. If the new version causes runtime or performance issues after going live, you can switch back to the previous historically stable version.

For actual algorithm model operations, this is important because:

A model upgrade doesn't mean the historical version loses its value.

Historical stable versions are not just part of the iteration history — they also serve as a recovery mechanism under abnormal conditions.

Through multi-version coexistence, models maintain a relatively clear state relationship across different stages:

Historical stable version → Currently active version → Next iteration version

Rather than directly overwriting the previous version every time a new model goes live.

This way, when production issues arise, the team still knows which version was previously in use and can restore to the corresponding historical version when needed — instead of searching through scattered model files.

From "Saving Model Files" to Managing Continuous Model Evolution

The significance of model version management isn't just about adding a version number to a model.

Once models enter enterprise algorithm applications, a single model typically goes through:

Model development → Parameter tuning → Multiple experiment rounds → Test validation → Production use → Continuous optimization → Exception rollback

If version relationships across these stages aren't uniformly recorded, as models and team members grow, many problems revert to manual management:

Which version is running in production?
Which historical version was this version adjusted from?
What configurations changed in the new version compared to the previous one?
After a model performance fluctuation, can you quickly locate the previously stable version?

These may seem like model development-stage concerns, but they continuously impact downstream model testing, release, and operations.

qModel addresses this through:

Version management — consolidating multiple historical versions under a single model
Version comparison — surfacing configuration and parameter differences in one place
New version creation based on current version — preserving model iteration lineage
Multi-version coexistence and switching — supporting testing, production use, and historical version recovery

Together, these capabilities form a foundational management pipeline from version creation to comparison, usage, and rollback.

For different roles, these version relationships also serve distinct practical purposes:

Algorithm developers can continue iterating on existing versions and use version comparison to confirm configuration changes between experiment versions.
Model operations and application staff can identify the currently active version and relocate historical stable versions when anomalies occur after version changes.
Project management and delivery staff can understand which stages a model has gone through via unified version records, reducing long-term reliance on individual file naming conventions and verbal explanations for model version relationships.

Wrapping Up

As algorithm models move from experimental environments into actual business use, the focus of model management gradually shifts.

Early on, the concern may be:

Can the model be created? Can it run?

But after a model goes through multiple optimization rounds, the team also needs to answer:

Which version is currently running?
What changed in the new version compared to the old one?
Which version should the next iteration be based on?
If the new version has issues, can the historical stable version be quickly restored?

These are the core problems model version management needs to solve.

qModel Algorithm Model Platform addresses this by organizing different versions produced during continuous model iteration through version management, version comparison, new version creation, and multi-version switching.

For algorithm models that require continuous development, validation, and operation, having model versions that are visible, comparable, and switchable is a critical foundation for transforming models from one-time R&D outputs into sustainably managed assets.

From Water Application to Dispatch Order: Streamlining the Full Irrigation District Water Scheduling Process

TongWu — Thu, 17 Sep 2026 05:43:44 +0000

In daily water management for irrigation districts, a water supply dispatch is never as simple as "submit a water usage request and issue a dispatch order."

From the moment an irrigation district submits a water demand, through management unit review and approval, to the creation of dispatch orders and their execution, the process typically involves multiple management levels, various business roles, and numerous water usage condition verifications.

Especially as the number of irrigation districts grows and application frequency increases, if applications, approvals, and dispatching still rely primarily on offline communication, spreadsheet circulation, and manual handoffs, problems quickly emerge: duplicated information entry, unclear processing progress, and loose connections between approval results and dispatch orders.

Therefore, the focus of digitizing irrigation district water scheduling isn't simply moving paper-based applications online. It's about further connecting:

Demand Declaration → Review & Approval → Dispatch Order Creation → Order Issuance → Order Execution

into a relatively complete business chain — enabling water demands, processing workflows, dispatch orders, and subsequent execution to be linked together.

1. From Water Application to Dispatch Execution: Building a Complete Irrigation District Water Scheduling Chain

The starting point of irrigation district water scheduling is accurately collecting actual water demands across different regions and time periods.

Water Applications Enter the System Through a Unified Entry Point

The system first provides a unified water application entry point.

Water users in irrigation districts can fill in details such as the water-using unit, water usage period, and requested water volume based on actual needs. Once submitted, this creates a standardized water application record, providing a unified data foundation for subsequent review, approval, and scheduling.

Management personnel can also view water demands by irrigation district, application period, and other conditions — gaining visibility into declaration volumes, water usage plans, and current processing statuses across different regions.

For irrigation district users, they can directly create water applications for their own district and submit them to the upper-level management unit, reducing the repeated circulation of offline spreadsheets and multiple rounds of communication.

This way, water demands first form a unified entry point in the system, and subsequent processing steps can continue flowing around the same application record — without needing to recreate business data at different stages.

Applications, Reviews, and Approvals Are Processed Step by Step According to Workflow

After a water application is submitted, the system routes it sequentially to the corresponding management units and approvers based on the established workflow.

Reviewers can view application details, verify application information, water usage quotas, and supply conditions, then provide review comments. Approvers then make approval decisions based on the irrigation district's supply capacity, water usage plans, and actual scheduling arrangements.

In terms of permission design, different roles correspond to different business operations. For example, irrigation district users acting as applicants don't see the review button — review and approval permissions are consolidated at the appropriate management levels, reducing role misuse.

Meanwhile, the system synchronously records processing time, processing personnel, and handling opinions, preserving process evidence for future queries and traceability.

Management personnel can also view the complete process of an application — from submission through review to approval — via process records, clearly identifying which processing stage it's currently at, what pending items remain, and the handling results at each previous node.

This means a water application is no longer just a static form — it continuously flows through the actual management process.

Supporting Tiered Collaboration with Continuous Tracking for Priority Applications

Irrigation district water scheduling typically involves different management levels, so application information also needs to be organized according to actual management relationships.

The system supports tiered viewing by irrigation district, management unit, and water user — enabling users to drill down from overall declaration status to specific applications. Management personnel at each level can prioritize pending review items within their area of responsibility while also staying informed about water demands and approval progress across different regions.

For key irrigation districts, priority water users, or large-volume water applications, the system can further retain application information, review comments, approval results, and related dispatch order records.

Business personnel can continuously track approval nodes and scheduling arrangements around the same priority application, conducting supplementary verifications, coordination, and follow-up tracking when needed — preventing priority applications from losing context as they flow through multiple management levels.

Approval Completion Flows Seamlessly into Dispatch Order Creation and Issuance

Approval is not the endpoint of the water scheduling process.

After review and approval are completed, the approval results still need to be converted into executable scheduling requirements.

The system manages dispatch orders in a unified manner, centrally displaying dispatch order numbers, execution units, approved users, preparers, and dispatched flow volumes — helping management personnel identify current scheduling tasks that need attention.

For dispatch orders generated through the normal application process, dispatch order information remains linked to the original application record and approval results, enabling duty personnel to quickly locate the application background and execution content corresponding to the current scheduling task.

At the same time, considering that special situations may arise during actual scheduling — such as the absence of a corresponding water application, or an approved application not fully matching actual scheduling conditions — management personnel can also manually create dispatch orders based on business needs, preserving necessary on-site scheduling flexibility.

Once a dispatch order is officially issued, relevant management units and execution personnel can promptly view specific scheduling requirements and organize water supply and usage accordingly, reducing omissions and delays that occur during manual handoffs.

At this point, a complete irrigation district water business cycle progresses from initial demand declaration, through review, approval, dispatch order creation, and order issuance, and finally enters the actual execution stage — forming a relatively complete online business chain.

2. Beyond "Digitizing Approvals" — Supporting Actual Water Scheduling Collaboration

Once water applications, review/approval, and dispatch order management are connected, the practical significance goes far beyond reducing a few paper forms. It enables scheduling information — originally scattered across different people and different stages — to remain continuously linked around the same business process.

The system further refines:

Demand Declaration → Review & Approval → Dispatch Order Creation → Order Issuance → Order Execution

connecting water applications, approval processing, scheduling orders, and subsequent execution status.

For daily management personnel, multiple operations can be completed within a relatively unified business interface: viewing declaration demands from different irrigation districts, filtering current pending items, verifying application and approval information, and generating dispatch order content after completing the corresponding approvals.

This reduces duplicate data entry for the same business process across different stages, and eliminates the need to repeatedly search across multiple pages to trace the origin of a dispatch order.

For the overall irrigation district management process, this produces several key changes:

Demands can be collected more promptly — Water applications submitted by different irrigation districts enter the system uniformly, enabling management personnel to grasp declaration demands within a given time frame at an aggregate level, providing a basis for subsequent coordinated planning.
Processing relationships between management levels become clearer — Who reviewed the application, which node it's currently at, and what opinions were given at the previous stage are all preserved through process records, reducing information gaps caused by interpersonal communication.
Approval results connect with scheduling execution — After approval is completed, the business doesn't stop at an "approved" status. It continues into dispatch order creation, order issuance, and execution stages, enabling management personnel to track the entire journey from demand submission to actual scheduling.

3. Sustaining Business Operations Requires Data and Process Maintenance as a Foundation

Whether a water scheduling process can operate stably over the long term depends not only on whether the front end has application, approval, and dispatch order capabilities, but also on whether the underlying business baseline information and process rules can be continuously maintained.

Therefore, beyond business processing, the system also provides corresponding data and process maintenance capabilities.

Around actual water scheduling, the following baseline information can be uniformly maintained:

Water users (irrigation districts), management units, water usage periods, application types, approval stages, and other baseline information — along with permission configurations to restrict approval authority for different roles.

This may seem like backend configuration, but it directly determines how front-end business flows.

For example, when management units are restructured, approval levels change, or different types of water applications require new approval workflows, there's no need to rebuild an entirely separate business logic. Instead, adjustments can be made around the corresponding baseline data and process rules.

After configuration changes, the system executes subsequent application routing, approval processing, and dispatch order management according to the latest rules — while ensuring that applications, approvals, dispatch orders, and detail records continue to use a consistent data source.

For a business like irrigation district water scheduling — which requires long-term operation and continuous adjustment — this baseline data and process maintenance capability is also critical infrastructure for ensuring stable business chain operation.

Summary

Irrigation district water scheduling involves more than a single application, nor is it an isolated approval or dispatch order issuance.

From a practical business perspective, it's more like a continuously running management chain:

Irrigation district submits demand → Management unit reviews → Business approval → Dispatch order creation → Order issuance → Organized execution

Therefore, the core of digitizing irrigation district water scheduling shouldn't stop at digitizing any single stage. It needs to further address:

Can the business flow continuously?
Can data remain linked?
Can the process be queried and traced?
Can different levels collaborate on processing?

By integrating water applications, review/approval, tiered collaboration, priority application tracking, dispatch order creation and execution, as well as underlying data and process maintenance into a single business system, a water scheduling cycle can extend all the way from "submitting a demand" to "creating and executing a scheduling order."

For smart irrigation district construction, the greater value of this full-process business connectivity lies in transforming originally scattered management stages into a clear, traceable, and sustainably maintainable water scheduling process — laying the business foundation for more standardized and collaborative irrigation district water resource management.

Timely Warning for Water Use Over-Plan: Enhancing Refined Water Resource Management

TongWu — Thu, 10 Sep 2026 07:46:53 +0000

In the daily management of water resources, formulating annual, monthly, or phased water use plans is only the first step.

Once the execution phase truly begins, managers need to continuously answer several questions:

How much water has actually been used currently?
To what extent has the plan indicator been completed?
Which regions, units, or water use objects have a significantly fast pace of water usage?
For objects that have already exceeded the plan, has there been subsequent rectification or a return to the plan control range?

If these questions can only be discovered through phased summaries, manual verification, or post-event reports, then even if a water use plan has been formulated, it is difficult to timely grasp deviations that occur during the execution process.

Therefore, planned water use management cannot stop at "plan issuance"; it further needs to establish a management mechanism of:
Continuous comparison of plan indicators and actual water use, timely identification of anomalies, and continuous tracking of key objects.

Centered around this need, the River and Canal Diversion Over-Limit Warning Function links capabilities such as plan execution analysis, over-plan judgment, hierarchical management, key object tracking, map warning, and scrolling prompts.

This allows managers to gradually locate specific abnormal water use objects from the overall plan execution situation and continuously understand subsequent changes.

Why is Continuous Execution Analysis Needed When We Have Plan Indicators?

In the actual management process, the plan indicator itself is just a control target.
Only by comparing it with continuously generated actual water use data can the current execution status be judged.

The system summarizes planned water use indicators and actual water consumption according to the year, month, or specific management cycle, and further calculates:

Plan completion rate;
Usage amount exceeding the plan, etc.

These data first form a unified basis for subsequent warning judgments.

For example, a water use object's annual plan indicator might seem to have a large margin, but if the current time is just the middle of the year and the actual water consumption has already approached the annual plan, then from the perspective of execution progress, it is already worthy of attention.

Therefore, plan execution analysis focuses not just on:
"Whether the plan is ultimately exceeded."
But also on:
"Whether the current water use progress is deviating from plan control requirements."

The front-end warning dashboard will centrally display objects where actual water use has exceeded plan indicators. Managers can view the plan execution situation according to regions, water use units, and their own management scope, further finding specific objects with fast water use progress from the overall indicators.

1. How is the Over-Plan Warning Determined?

Displaying actual water consumption is not enough.
For the warning to be truly usable for business management, it is necessary to clarify:
Under what circumstances does it count as over-plan? Based on what rules are warnings generated?

The system will continuously compare and judge actual water use data in combination with the plan cycle, plan indicators, and warning thresholds.
When the actual water consumption within the statistical cycle exceeds the corresponding plan indicator, the system generates an over-plan water use warning and records:

The quantity exceeded;
The percentage exceeded.

In this way, the warning information is not just a simple "abnormal" status but further explains:

How much has actually been used currently;
How much it exceeds the plan;
How large the degree of excess is.

For managers, this information also provides a more specific basis for subsequent judgments on the severity of the problem and determining the priority of attention.

Why the Warning was Generated: The Rules Themselves Need to be Viewable

If the warning system only tells staff: "This object is over-plan,"
but cannot explain the judgment basis, doubts are still likely to arise in actual business use.

Therefore, managers can enter specific rule details through "Rule View" to verify:

Plan cycle;
Warning threshold;
Judgment conditions;
Applicable objects.

In this way, every over-plan warning can be further traced back to the corresponding business judgment basis.
This is especially important for water resource management scenarios.
Because different regions, different objects, and different management cycles may adopt different plan indicators and management requirements.

A truly usable warning is not just "the system judging an anomaly" but also needs to be able to answer:
Why was it judged as abnormal?

2. After Discovering Over-Plan, Locate Specific Objects from the Overall Situation

When the management scope expands, there may be multiple regions, management units, and water use objects at one time.
If all warnings are concentrated and stacked in a single list, managers still need to spend a lot of time finding the objects they are responsible for.

Therefore, the system supports viewing hierarchically according to:
Administrative Region → Management Unit → Water Use Object

Managers can first grasp the overall plan execution situation, then gradually locate specific water use subjects, and prioritize viewing warning information within their own scope of responsibility.

In the specific management process, focus can be placed on:

Objects with a large quantity over-plan;
Objects with a high percentage over-plan;
Objects with continuous anomalies.

This transforms the warning from simple "abnormal information display" into a way of organizing information according to management hierarchy and problem severity, allowing limited regulatory energy to be prioritized for objects that are more worthy of attention.

3. One-Time Over-Plan and Continuous Over-Plan Need to be Treated Differently

Not all over-plan situations imply the same problem.
An object might have a one-time excess due to short-term business fluctuations in a certain stage; or it might maintain a high water use level for multiple consecutive cycles, forming a continuous over-plan.

The management methods corresponding to these two situations are usually not exactly the same.
Therefore, for objects that need continuous attention, the system will retain their plan indicators and actual water use situation on the detail page, forming a continuous tracking record for key objects.

By continuously viewing the water use trend of the same object, one can further distinguish between:

One-time fluctuation
Continuous over-plan

On this basis, managers can take measures such as reminders, verification, or rectification in combination with the actual business situation.
This is also the key to the warning capability moving from "discovering a single anomaly" to "observing changes over a period of time."

After Rectification, the Warning Cannot Simply Disappear
After an over-plan warning is generated, the subsequent status also needs to be managed.
If the water use object completes rectification through the business system, or returns to the plan control range with subsequent water use changes, the system will update the corresponding warning status.

However, the original warning record will not be directly deleted but will continue to be retained for subsequent queries.
In this way, the entire process can not only see:
"Which objects are currently over-plan."
But can further retain:
"Which objects had over-plan previously, and how they changed later."

For key water use objects that need continuous tracking, historical warning traces can also provide more complete process information for subsequent analysis.

4. Where are the Water Use Problems? Further View Regional Distribution Through Maps

For water use objects that have been associated with spatial locations, simply viewing data tables cannot fully reflect the spatial distribution of problems.
Therefore, the system can further mark the plan execution status of water use objects on the map.

Through different colors or icons, it can distinguish between:

Objects approaching the limit;
Objects that have already exceeded the plan.

Managers can thus view from a spatial dimension:

In which regions are over-plan objects relatively concentrated;
Which locations have already shown anomalies;
Which regions are currently in a state of approaching the limit.

For businesses with obvious spatial attributes, such as river and canal diversion and water resource scheduling, combining indicator status with geographical location can allow abnormal objects originally scattered in lists to further form a regionalized understanding.

5. The Dashboard Not Only Displays Statistical Results but Also Timely Prompts New Anomalies

Smart water conservancy dashboards usually undertake the role of duty monitoring and viewing the overall situation.
If anomalies only exist in a certain list or detail page, duty personnel still need to actively enter the corresponding module to discover them.

Therefore, a water use warning prompt bar is set at the bottom of the dashboard.
The system generates brief prompt information based on:

Warning objects;
Plan cycles;
Over-plan situations; And displays them via scrolling.

More importantly, the scrolling prompt is not an independent information display area but remains synchronized with:

Statistical data;
Object lists;
Map status.

In this way, while viewing the overall water use situation, duty personnel can simultaneously pay attention to newly emerging over-plan problems, as well as key objects that have not yet completed disposal.

From One Warning to a Complete Planned Water Use Management Process

Linking these capabilities together again, one over-plan water use warning actually goes through such a process:

Establish Plan Indicators → Continuously Summarize Actual Water Use → Continuously Compare According to Rules → Generate Over-Plan Warning → Hierarchical Location → Continuous Tracking of Key Objects → Synchronized Display on Map and Dashboard → Update Status After Rectification or Recovery

In this way, the warning is no longer just a red prompt in the system but further enters the process of plan execution analysis, anomaly judgment, object location, and subsequent tracking.

Summary: Moving Further from "Plan Issuance" to "Traceable Execution Process"

The focus of water resource plan management is not just to formulate a water use indicator at the beginning of the year or the start of a cycle.
More importantly, during the plan execution process, one must continuously know:

How much was actually used;
To what extent the plan is being executed;
Which objects are approaching or have exceeded control requirements;
Where the problem occurred;
Whether the anomaly is a one-time fluctuation or persists;
Whether the status has changed after rectification.

The River and Canal Diversion Over-Limit Warning Function revolves around this process.
It links plan execution analysis, over-plan rule judgment, hierarchical management, key object tracking, map warning, scrolling prompts, and warning status updates with historical traces, extending planned water use management from static indicators to the continuous execution process.

For smart water conservancy construction, the value of such capabilities does not lie in simply adding a form of alarm.
Instead, it allows the originally scattered plans, actual water use, judgment rules, abnormal objects, and disposal statuses to form a relatively continuous management chain.

Moving from "discovering how much water has been used" to "timely discovering which objects deviate from the plan and continuously tracking subsequent changes" is also a basic capability for the extension of water resource management towards refined, process-oriented control.

qModel Open Source v1.4.2 Launches Version Management and Comparison Features: Making Every Model Iteration Traceable

TongWu — Thu, 10 Sep 2026 07:46:42 +0000

The qModel Algorithm Model Platform Open Source Edition v1.4.2 introduces new model version management and version comparison capabilities.

An independent version management entry has been added to the model details page, supporting rapid creation of new versions based on existing ones, horizontal comparison between any two versions, as well as multi-version coexistence, switching, and historical version rollback.

This further improves the management chain from continuous model iteration to version governance.

From "Constantly Copying Model Files" to Establishing Clear Version Evolution Relationships

Models are usually not developed once and left unchanged for a long time.
Once they enter actual business use, a single model may generate new versions continuously due to:

Parameter adjustments;
Data changes;
Business rule changes;
Algorithm optimization;
Feedback on effects after going live.

Therefore, the long-term usage process of a model is closer to:
Initial Model → Adjust Configuration → Form New Version → Test & Validate → Official Use → Optimize Again → New Version

If there is a lack of unified version management, model iteration can easily degenerate into simple file management.
Teams might distinguish different model files using names like:
model_v2_final_0315
or:
final_final_v3

When the number of versions is small, this method might be temporarily usable; however, as the model continues to iterate, files will gradually be scattered across different directories, different members, and different environments, making the relationships between versions increasingly difficult to confirm.

The real problem to be solved is not "giving files a more standardized name," but rather:

What versions currently exist?
Which version is currently in use?
From which version did the new version evolve?
What exactly changed between two versions?
If an online anomaly occurs, can we switch back to a historically stable version?

Therefore, starting with qModel Open Source v1.4.2, model versions are treated as independent management objects to further refine version relationships during the continuous iteration process.

01. New Version Management Tab: Making Historical Model Versions Centrally Visible

When a model is continuously iterated, the most basic question is:
How many versions does this model actually have right now?

If versions are scattered across different files or pages, the first thing administrators need to solve is not analyzing version changes, but finding the historical versions first.

qModel Open Source v1.4.2 adds an independent "Version Management" tab to the model details page, placing all model-related versions into the same entry point for viewing.
Users can centrally understand:

Existing versions of the current model;
The currently effective version;
The total number of model versions.

In this way, the model details no longer just display a single current state but further add a perspective on the model's historical evolution.

Seeing the Model's Version Lifecycle Beyond Single Model Information

In the past, when viewing a model, it was easier to focus on: What is the current state of this model?
With the addition of version management, one can also continue to focus on: How did this model iterate step-by-step to its current state?

For long-running enterprise models, these two questions are not the same.
For example, the same business prediction model might go through:
V1.0 → V1.1 → V1.2 → V2.0
Different versions may correspond to data, configurations, or parameter adjustments at different stages.
Through a unified version list, these originally scattered model states can form clearer version relationships.

Reducing Reliance on File Names for Version Identification

In actual business scenarios, teams often rely on manual management of model files via names like:

model_v2_final_0315
final_final_v3

The problem is not just non-standard naming. As the number of members and iterations increases, issues may arise such as:

Uncertainty about which file is the current version;
Difficulty in quickly finding historical versions;
Lack of clear archiving relationships between versions;
Need for manual re-confirmation when restoring old models.

Therefore, the significance of version management lies first in:
Turning model versions from a "file naming habit" into explicit management objects within the platform.

02. Support for Rapidly Creating New Versions Based on Current Versions, Retaining Original Configuration Context

Model iteration usually does not start from scratch.
More often, algorithm engineers continue to adjust on the basis of existing models:

Parameters;
Configurations;
Data;
Business adaptation methods.

If creating a new version requires rebuilding the entire model configuration every time, it not only increases repetitive operations but also容易 causes unexpected differences between old and new versions due to omitted parameters.

qModel Open Source v1.4.2 supports rapidly creating new versions based on the current model version.
The new version can automatically inherit the original version's configuration and context, without needing to rebuild from scratch.

Model Iteration Can Continue to Evolve from Existing Versions

The creation logic for new model versions can be summarized as:
Select Current Version → Create New Version → Inherit Original Configuration & Context → Continue Adjusting on This Basis → Form New Model Version

This approach aligns better with the real model iteration process.
Because model upgrades are usually not:
Creating a completely independent new model
But rather: Modifying parts of the content on an existing stable basis to form a new version.

Reducing Repetitive Configuration While Retaining Relationships Between Versions

For example, if a deployed prediction model needs to adjust some parameters.
If the model is recreated, one needs to reprocess:

Basic configuration;
Parameters;
Runtime context;
Related model information.

However, by creating a new version through the existing version, the original content can be directly inherited, and adjustments can be made only to the parts that need to change.
This reduces repetitive configuration and establishes a clearer evolutionary relationship between the old and new versions.

It is important to note that inheriting configuration only reduces repetitive operations; it does not mean the new version can enter production use without verification.
Before the official model switch, verification must still be completed in conjunction with the enterprise's own model testing, effect evaluation, approval, and launch specifications.

03. New Version Comparison: Making it Easier to Confirm "What Exactly Changed in This Version"

The real difficulty in multi-version management is not "having many versions."
It is: What exactly is the difference between versions?

When multiple model versions exist simultaneously, online services and offline experiments may not accurately correspond to specific versions.
If there is a need to reproduce experimental results or trace back historical states, one often has to rely on file comparison and manual confirmation.

At the same time, when model parameters or data are adjusted, if the model's performance fluctuates, the team also needs to further determine: Where did this change come from?

Therefore, qModel Open Source v1.4.2 adds version comparison capabilities.

Select Any Two Versions for Horizontal Comparison

The platform supports selecting any two versions from existing model versions for comparison.
After comparison, it can automatically sort out differences across multiple dimensions, including:

Configuration;
Parameters; Making the changes between different versions more intuitive.

In this way, version comparison can shift from:
Manually opening two models to find differences item by item
To:
Select Version A + Version B → View Differences

Providing Clearer Version Context for Model Problem Tracing

For example, after a model upgrades from V1.3 to V1.4, business results change.
At this point, the team first needs to determine:

Which parameters changed?
Which configurations were adjusted?
What are the explicit differences between the two versions?

Through version comparison, these version-level changes can be sorted out first, and then further analysis can be conducted in conjunction with actual operational data and model effects.

Therefore, version comparison is more suited to undertaking:
Clarifying "what changed between versions."

As for:
"Why did these changes cause the model effect to improve or decline?"
This still requires further judgment based on actual evaluation metrics, test data, operational results, and business analysis.
Version comparison can provide context for problem analysis but cannot directly replace model effect evaluation.

04. Support for Multi-Version Coexistence and One-Click Switching, Reserving Space for Testing, Release, and Rollback

Model version management is not just for saving more historical records.
Ultimately, it still needs to answer a practical question: Which version should the current business actually run?

qModel Open Source v1.4.2 supports multi-version coexistence and provides version switching capabilities.
For different usage stages, the corresponding version can be selected as needed.
For example:

Testing Phase → Use the new version for validation.
Official Operation → Use the confirmed stable version.

This approach extends version management further from "viewing history" to "actual use."

Test Versions and Official Versions Can Be Switched as Needed

During the model development process, new versions usually need to be tested before entering official use.
If the platform only allowed retaining one version, every upgrade might mean overwriting the original model.
Once the new version has problems, restoring it would require finding historical files again or redeploying.

After supporting multi-version coexistence, the following can be retained simultaneously:

Current stable version;
New test version;
Historical versions.

Different stages can be switched according to actual needs.

Quickly Return to Historical Stable Versions When Online Fluctuations Occur

After a model goes officially live, it cannot be guaranteed that the new version will always meet expectations in the long term.
Changes in data distribution, parameter adjustments, or changes in the business environment may all cause fluctuations in the model's actual performance.

Therefore, model upgrades require not only:
"Being able to switch to the new version"
But also:
"Being able to roll back when problems occur."

qModel v1.4.2 supports rapidly rolling back to a historical stable version when online fluctuations occur.
From the perspective of version operation logic, the following can be formed:
Stable Version → Create New Version → Complete Adjustments → Test & Validate → Switch to New Version → Observe Actual Operation → If Anomalies Occur, Switch Back to Historical Stable Version

This gives the model upgrade process a more complete space for version selection.

However, version rollback cannot replace the enterprise's formal production release system.
For core production models, it is still necessary to use mechanisms such as test verification, launch approval, operational monitoring, and business impact assessment together.

Easier to Establish Unified Version Cognition During Multi-Person Collaboration

In situations where multiple people participate in model development, different members may adjust the model simultaneously.
If there is a lack of unified version management and permission control, problems such as different versions overwriting each other, branch conflicts, and increased collaboration costs are prone to occur.

This qModel Open Source v1.4.2 explicitly adds version management, version creation, version comparison, and version switching capabilities.
From the existing capabilities in this release, the more direct change lies in:
The team can discuss models based on unified platform version information, rather than relying entirely on personal file naming to determine which version is currently in use.

05. From Version Chaos to Version Traceability: What Changes Occurred in the Model Governance Chain?

If we look at these several features together, qModel v1.4.2 actually supplements a link in the model lifecycle that was previously easily overlooked: Model Version Governance.

In the past, model iteration might have manifested as:
Existing Model → Export/Copy Model File → Modify File Name → Adjust Parameters → Redeploy → Manually Record Which Version is in Use

As versions increase, the following issues become prone to occur:
More and more files → Increasingly difficult to confirm version relationships → Need for manual comparison → Difficulty in restoring historical states

After adding version management, the process can be further adjusted to:
Current Model → Create New Version Based on Current Version → Inherit Existing Configuration → Complete Model Adjustment → Compare with Historical Version → Test & Validate → Switch to Effective Version → Rollback if Necessary

Wherein:

Version Management is responsible for organizing historical versions;
New Version Creation is responsible for undertaking model iteration;
Version Comparison is responsible for clarifying changes;
Version Switching and Rollback are responsible for controlling the actual version in use.

These capabilities together further transform the model's "iteration process" into a manageable version chain.

06. Version Value: Making the Evolution Process of Model Assets More Manageable

For enterprise algorithm model platforms, model management cannot only focus on "whether the current model can run."
When models are continuously iterated, the governance issues between versions need to be further resolved.

More Centralized Version Status
Through an independent version management tab, the model version list, currently effective version, and version count can be viewed uniformly, reducing the situation where model versions are scattered in different files and environments.
More Continuous Model Iteration
New versions can be created directly based on the current version and inherit the original configuration and context, making model upgrades better align with the actual development method of "continuing to evolve on a stable version."
Easier Confirmation of Version Changes
Through horizontal comparison between two versions, differences in configuration, parameters, etc., can be viewed more intuitively, providing a version basis for model change confirmation, problem tracing, and experiment reproduction.
Model Upgrades Have Rollback Space
Multi-version coexistence, switching, and historical stable version rollback ensure that testing, official release, and anomaly recovery no longer rely entirely on finding and deploying historical model files again.

Overall, the value of qModel v1.4.2 is not simply adding a "version list," but further establishing a complete management relationship for model versions from creation, comparison, to switching and rollback.

Conclusion

For enterprise algorithm models, after truly entering production use, original documents list version chaos, difficulty in corresponding online and experimental versions, version conflicts in multi-person collaboration, difficulty in tracking reasons for model changes, and difficulty in comparing old and new versions as the main problems in current model iteration.

This is also the key problem that qModel Open Source v1.4.2 focuses on solving this time.

This version further supplements:

On the version organization side: Added a version management tab to model details to centrally view all versions and current effective status;
On the version iteration side: Supports rapidly creating new versions based on existing versions and inheriting original configurations;
On the version analysis side: Supports horizontal difference comparison between any two versions;
On the version operation side: Supports multi-version coexistence, flexible switching, and historical stable version rollback.

These capabilities cannot replace the enterprise's own model effect evaluation, testing processes, approval mechanisms, operational monitoring, and production release systems.

But it can solve a more fundamental problem in the continuous model iteration process:
Ensuring every model upgrade has a clear version identity, and allowing one to know where it came from, what changed, whether it is currently effective, and which historical state it can return to if problems occur.

For algorithm model platforms, it is not enough for the model itself to be manageable; the model's evolution process also needs to be managed.

The addition of version management and comparison capabilities in qModel Open Source v1.4.2 is precisely further improving the complete chain from model access, operation management, to continuous iteration governance.

How to Use qKnow Agent Platform for "Small-Scale Validation, Large-Scale Expansion"?

TongWu — Thu, 10 Sep 2026 07:46:35 +0000

When promoting AI agents and knowledge applications, a common issue is the desire to integrate all knowledge files, business systems, departmental permissions, and application requirements right from the start.

From a final goal perspective, this direction is not wrong. However, for large enterprises with complex data sources, diverse knowledge types, and long business processes, if the initial scope is too broad, it becomes difficult to quickly determine whether unsatisfactory Q&A results stem from the model, file quality, knowledge parsing, retrieval parameters, knowledge graphs, or the application process itself.

Therefore, enterprise AI agent construction is better suited to first select a small number of high-frequency scenarios for validation, and then gradually expand the capability boundaries.

Taking equipment knowledge Q&A as an example, you can start by selecting one type of equipment, one maintenance team, and a batch of real questions to run through the complete process: model integration, knowledge processing, knowledge retrieval, Q&A validation, and feedback optimization.

Once validation is effective, the methods and platform capabilities formed can be replicated to more equipment, personnel, and business processes.

This is also a landing method for the qKnow Agent Platform in enterprise AI agent construction:
Keep the initial business scope small, but unify the platform architecture first.

I. Enterprise Product Design Thinking: Small-Scale Validation, Platform-Based Expansion

01. Why are enterprise AI projects not suitable for a full rollout in the first phase?

The larger the enterprise, the more complex the knowledge and data environment usually is.
The same type of equipment may have different codes, technical materials may be scattered across different systems, and different departments may have different rules regarding file versions, approval processes, and data permissions. If all content is integrated in the first phase, the project will quickly face a massive number of variables.

At this point, the hardest questions to answer are often the three most basic ones:

Is the model truly suitable for the current business?
Can the knowledge materials be correctly parsed and retrieved?
Does the final application actually solve the user's problem?

If these three questions cannot be independently verified, even if the final result is unsatisfactory, it is difficult to determine which link needs adjustment. Therefore, the first phase is better suited to actively narrowing the scope, allowing every problem to be located and every adjustment to be re-verified.

02. Establish a "Minimum Viable Loop" first

Taking equipment knowledge Q&A as an example, a complete minimum loop is not just "uploading a few files and letting the large model answer questions."
At least the following must be completed:
Model Integration → Knowledge Base/Graph Construction → File Processing → Retrieval Configuration → Real Question Recall Test → Knowledge Q&A → User Validation → Problem Feedback → Readjustment

Only when these links can connect with each other can we judge whether the enterprise has truly established an AI application foundation that can continue to expand.
Completing model interface configuration, knowledge file uploading, or a single demo Q&A independently cannot be equated to completing a business loop.

First-phase scenarios should not pursue "bigness," but "ease of validation"

Scenarios suitable for first-phase validation usually share several common characteristics: high usage frequency, basically available existing materials, clear target users, and easily judgeable effects.

Scenario Scope: Select 2-3 high-frequency scenarios (Avoid covering all departmental needs simultaneously).
User Scope: One team or business group (Avoid facing all employees upon initial launch).
Equipment Scope: One type of equipment or one equipment family (Avoid including all equipment at once).
Question Scope: 20-50 real questions (Avoid using demo questions without business context).
Data Scope: Materials directly related to target questions (Avoid moving all historical files from the entire database).

"Small incision" does not mean the underlying architecture should also be small

The business scope can start from one scenario, but the underlying design should ideally consider future reuse from the very beginning.

In the actual construction of qKnow, the following can be considered separately:

Model Layer: Decouple model integration from application calls, allowing for future replacement or addition of other models.
Data Layer: Unify file naming, equipment coding, versions, and permission rules in advance.
Knowledge Layer: Build knowledge bases in a reusable way, keeping knowledge graph concepts and primary keys unified.
Application Layer: Q&A, retrieval, agents, and business workflows share the same knowledge base.
Operation Layer: Continuously retain user questions, audit responsibilities, and knowledge update mechanisms.

In this way, the first phase validates a small scenario, but subsequent expansions reuse the same set of platform architecture and governance methods, rather than rebuilding a new system for every added scenario.

Once the first-phase loop is validated, the scope can be gradually expanded according to:
Similar Objects → Adjacent Users → Adjacent Processes

For example, after the main unit knowledge Q&A is stable, add valves, sensors, electrical cabinets, and other equipment; subsequently expand to other maintenance teams; when users begin to need to query the relationships between equipment, components, fault phenomena, causes, and maintenance measures, then add knowledge graphs and entity relationship retrieval.

Each expansion still forms a new "small loop," rather than integrating all remaining content at once.

II. Operational Process: Using qKnow to Complete the Equipment Knowledge Q&A Loop

Below, taking "Pump Station Equipment Fault Knowledge Q&A" as an example, let's specifically look at how to use qKnow to complete knowledge processing, retrieval, and final Q&A validation step-by-step starting from the model.

Note: In actual projects, model names, business parameters, knowledge scope, and user permissions need to be configured according to the enterprise's own situation.

Step 1: Integrate the model and replace the model used in the actual application

First, you need to complete the target large model configuration in the qKnow Model Market.
After filling in the key and related necessary parameters, conduct a connectivity test first.

However, for knowledge Q&A, interface connectivity is just the first step.
After successful model integration, you also need to enter the actual Q&A workflow, Bot, or agent configuration to replace the original model node with the target model.

At this point, focus on confirming:

Can the dialogue model return results normally?
Can the context length meet the needs of knowledge Q&A?
Is the prompt configuration correct?
Is the knowledge retrieval node correctly connected to the model node?
Can the output node return results normally?

After the model replacement is complete, a full dialogue needs to be re-executed, rather than just confirming that the model interface "call was successful."

If the enterprise plans to integrate multiple models simultaneously, it is recommended to fix one main model for the first phase.
The reason is simple: if the model changes frequently, then when the answer effect changes, it is difficult to determine whether it is caused by model differences or knowledge retrieval.

Step 2: Create a knowledge base or knowledge graph around the first-phase scenario

Once the model can be called normally, the next step is to build the knowledge base.
Depending on the form of business knowledge, you can:

Create only a knowledge base;
Create only a knowledge graph;
Use both knowledge base and knowledge graph simultaneously.

The problems solved by the two are not exactly the same.

Knowledge Base: More suitable for carrying document-type knowledge such as technical manuals, maintenance records, fault cases, and operating procedures.

Knowledge Graph: Suitable for expressing clear entities and relationships such as: Equipment → Component → Fault Phenomenon → Fault Cause → Maintenance Measure.

For example: In the first phase, you can create a "Pump Station Main Unit Fault Case Knowledge Base"; if subsequent equipment relationship analysis is needed, you can simultaneously build a "Pump Station Equipment Fault Knowledge Graph."

After creation, it is recommended to keep it in an unpublished state until file processing, relationship checking, and retrieval testing are completed before officially opening it.
At the same time, the knowledge base owner, applicable departments, material scope, and subsequent update responsibilities need to be clarified.

Compared to generalized names like "Comprehensive Knowledge Base" or "Temporary Knowledge Base," the knowledge base name should directly explain the business scope, so that subsequent users can quickly judge what content it contains.

Step 3: Establish knowledge classification that fits business users' habits

Enter the target knowledge base and establish a knowledge classification system via: Knowledge Base Settings → Knowledge Classification.

For the pump station equipment fault scenario, you can first establish classifications such as:
Fault Phenomenon, Fault Cause, Equipment Component, Maintenance & Disposal, Operating Conditions.

Note here that knowledge classification is not to make the backend directory "look more complete," but to help with subsequent file governance and business user understanding.
Therefore, classification names should use expressions familiar to business users as much as possible.
The same level should also maintain a consistent division standard.
If a certain category has no knowledge files or query needs in the first phase, there is no need to create it in advance for the sake of system completeness.

Step 4: Upload, parse, and check knowledge files

Enter "Knowledge Files," select the corresponding classification first, and then upload knowledge materials such as Word, PDF, TXT, etc.

Before formal import, it is recommended to perform basic governance on the files first.
Focus on cleaning up: duplicate files, expired versions, and materials with poor scanning quality that may affect parsing results.

After the file upload is complete, the parsing results need to be further checked. For example:

Is the document title correct?
Are the body text and tables parsed completely?
Are chapter numbers preserved?
Does the segmentation structure conform to the original logic?
Does the maximum segment length cause context truncation?
Can the overlap length preserve cross-segment explanations?

These settings will directly affect subsequent knowledge recall.
For example, if a piece of equipment fault description originally consisted of "phenomenon, cause, disposal steps," and segmentation恰好 splits the cause and disposal steps, even if the model capability is sufficient, it may not be able to obtain the complete context.

If the business also needs to query relationships between equipment, components, fault causes, and maintenance measures, you can further establish unstructured extraction tasks after file import to extract relevant entities and relationships into the knowledge graph.
However, if the first-phase goal is only to complete document Q&A, you can temporarily not introduce the graph and keep the first-phase scope within knowledge base Q&A.

Step 5: Configure knowledge retrieval methods

After file parsing is complete, enter: Knowledge Base Settings → Retrieval Settings to start configuring the knowledge recall method.

For enterprise knowledge scenarios where technical manuals, maintenance cases, and maintenance records coexist, hybrid retrieval can be adopted first.

On one hand, use full-text keywords to match professional terms, equipment models, and fault names;
On the other hand, use vector similarity to handle users' natural language expressions.

When the data volume increases, or there are many similar knowledge segments, the Rerank model can also be used to further rerank results.

Top K and score thresholds are also not recommended to be set too tightly at the beginning.
A more suitable approach is to first use relatively loose conditions to observe recall results, and then gradually adjust based on real tests.

At the same time, try to modify only one parameter per round of testing.
For example, adjust only Top K in this round, and adjust the score threshold in the next round.
Otherwise, if multiple parameters are changed at once, even if the final effect improves, it is difficult to determine which factor played a role.

Step 6: Use real business questions for recall testing

When knowledge Q&A effects have problems, often the issue is not in the "large model answer," but in the earlier knowledge recall stage.
Therefore, before formally entering the Q&A application, you can first enter "Recall Testing."

Input the 20-50 real business questions organized in advance into the platform one by one.
For example: "Which parts should be checked first when the main unit has abnormal vibration?"

At this time, do not focus on whether the final answer is fluent enough, but focus on confirming:

Was the correct file found?
Did the truly relevant text segments enter the recall results?

Different phenomena correspond to different processing paths.

If the wrong file is hit, check file naming, question expression, and retrieval weights;
If the file is correct but the returned segment is incomplete, focus on checking segment length and overlap length;
If the correct segment exists but is ranked too low, further adjust Top K, Rerank, or check if there is a large amount of duplicate content in the knowledge base;
If recalled materials are old versions, it indicates the problem is no longer just retrieval parameters, but that knowledge file version management and content responsibility need further governance.

Only after this step is basically stable should you proceed to the large model answer generation stage.

Step 7: Associate knowledge base and knowledge graph in knowledge Q&A

After the recall test basically passes, you can enter: Application Center → Horizontal General Applications → Knowledge Q&A to create a new conversation.

Depending on the specific business, you can select only the target knowledge base, or simultaneously select the knowledge graph as the Q&A basis.
For example, select: "Pump Station Main Unit Fault Case Knowledge Base."
If the current question involves relationships between equipment, components, faults, and maintenance measures, continue to associate the corresponding equipment fault knowledge graph.

Subsequently, input the real question: "Which parts should be checked first when the main unit has abnormal vibration?"

What truly needs to be verified here is not just what answer the model finally generated.
You should also simultaneously check:

Is the answer content correct?
Do the citation sources come from the correct knowledge files?
Do the relevant knowledge segments match the current equipment?
When continuing to ask about applicable conditions, operation sequences, or original text basis, can context consistency be maintained?

It is recommended to record every abnormal answer.
Including: no answer, wrong citation, answer not matching current equipment, and answers ultimately not adopted by the user.
This type of information is often more valuable than simply recording "answer accuracy rate," because it can directly guide subsequent model, knowledge, and retrieval adjustments.

It must also be emphasized that important disposal suggestions in enterprise equipment maintenance scenarios should still be confirmed by corresponding business personnel.
Intelligent Q&A can help users find materials and organize basis faster, but cannot replace the enterprise's original professional review and safety management processes.

Step 8: Feed user questions back into the model and knowledge processing flow

After the equipment knowledge Q&A is truly online, the new work has just begun.
It is recommended to regularly summarize actual Q&A questions and return to the corresponding links according to different causes.

For example:

Unstable model performance: Return to the model, prompts, or workflow for adjustment;
Missing or expired files: Supplement knowledge materials and clarify file versions and update responsibilities;
Answer exists in the file but was not recalled: Return to segmentation methods and retrieval parameter adjustments;
Multiple names exist for the same equipment: Supplement aliases, tags, or entity normalization rules;
Users begin to need to analyze relationships between equipment, components, faults, and maintenance measures: Gradually increase graph models, knowledge extraction, and entity relationship retrieval capabilities.

After each adjustment, it is recommended to continue using the original question set for re-testing.
Only then can the results before and after adjustment be compared to determine whether this modification truly brought improvement, rather than judging based on individual Q&A experiences.

Step 9: Determine if the first-phase minimum loop has truly passed

After the first phase is completed, the acceptance criteria should not just be "the agent is online" or "users can open the page normally."
It can be judged from at least four dimensions:

Model Usable: Q&A and workflows can stably call the target model;
Knowledge Usable: High-frequency business questions can recall correct knowledge files and corresponding segments;
Answer Usable: Key conclusions have clear sources, are understandable by business personnel, and are actually adopted in appropriate scenarios;
Process Usable: Once an error occurs, the team can further determine whether the problem comes from the model, knowledge files, retrieval mechanism, or knowledge governance link.

If these four aspects are basically stable, it indicates that the first business loop has a foundation for replication.
The next phase can continue to expand to the second type of equipment, the second team, or the second business scenario.

During the expansion process, there is no need to rebuild a whole set of systems, but to continue reusing the existing:
Model integration methods, data standards, knowledge classification systems, permission rules, and testing methods.
What is added is only the data and knowledge capabilities truly needed by the new business.

Conclusion

Enterprise AI agent construction does not necessarily need to start with "big and all-inclusive."
Compared to integrating all data, knowledge, and business processes at once, a more controllable way is:
First complete a small-scale loop around real users and real questions: the model can be stably called, knowledge can be accurately recalled, answers can find basis, and problems can continue to be located and adjusted after they occur.

qKnow places model integration, knowledge files, knowledge extraction, knowledge bases, knowledge graphs, and Q&A applications in the same platform link, enabling enterprises to first validate a specific scenario, and then gradually reuse existing data, knowledge, and application architectures to expand to more business.

First make one real scenario work, then replicate the validated method to more scenarios.

For enterprise AI agent platforms, this "small-step validation, platform-based expansion" method is more important not for how big the first phase is, but for ensuring that every subsequent expansion has a validated foundation to reuse.

AI #EnterpriseAI #KnowledgeManagement #RAG #qKnow #TechStrategy

qKnow Professional Edition v3.1.3 Released: New Extraction Task Notifications and Multi-Dimensional Refinements

TongWu — Thu, 10 Sep 2026 07:46:29 +0000

qKnow Professional Edition v3.1.3 focuses on optimizing task status awareness, file previewing, and homepage notification configurations within the knowledge governance process.

Key updates include adding message notifications for structured and unstructured extraction tasks, introducing kkFileView deployment configurations for file previews, and enabling homepage notification visibility control via parameter management.

These updates aim to further reduce repetitive operations during task execution and platform configuration.

From "Task Submission" to Real-Time Status Awareness

In the construction of enterprise knowledge bases and knowledge graphs, file uploading is just the beginning. Once the knowledge governance workflow begins, a piece of knowledge material typically undergoes the following process:

File Preparation → Structured/Unstructured Extraction → Task Execution → Status Check → Exception Handling → Utilization of Extraction Results

Knowledge extraction itself often requires a certain amount of execution time. If users have to constantly return to the task page and manually refresh to check progress after submitting a task, several practical issues arise:

Where is the task currently executing?
Is it complete?
Has an exception occurred?
If processing multiple tasks simultaneously, has a task finished without being noticed?

Beyond task status, internal enterprise deployment environments also impact daily usage. For instance, file preview functions may face different access conditions in public versus private networks. Additionally, if page notifications can only be controlled by modifying front-end configuration files, it increases daily maintenance costs.

Therefore, the focus of qKnow Professional Edition v3.1.3 is not simply adding more pages, but continuing to refine the connection between these high-frequency operations:
Task Execution → Status Feedback → File Viewing → Platform Configuration

01. New Message Notifications for Extraction Tasks

Knowledge extraction is a critical part of the qKnow governance workflow. Whether it is structured data extraction or unstructured document extraction, tasks require actual execution time after submission.

Previously, after submitting these tasks, users had to manually refresh the page to check progress. This meant the workflow was often:
Submit Task → Wait → Refresh Page → Check Status → Wait Again → Refresh Again

When the number of extraction tasks is high, this method can easily lead to missed tasks or delayed processing.

System Notifications in the Top Right Corner

qKnow Professional Edition v3.1.3 introduces message notification data for both structured and unstructured extraction processes. Once a task is executed, the system continuously updates task progress information in the top right corner.

Unlike the past, where users had to enter the task page to check status, users can now perceive the execution status of extraction tasks while using other platform features. This shifts the task viewing method from:
Users actively seeking status
to:
The system actively providing status feedback based on task changes.

Covering Execution, Completion, and Exception States

Notifications do not only appear when a task ends. According to the release notes, the system automatically pushes messages for the following states:

Task Executing
Task Completed
Task Exception

This allows users to more timely distinguish whether a task is still running, or if it has finished normally or encountered an exception requiring further handling. This feedback is particularly important for enterprise users who need to submit multiple knowledge extraction tasks at once.

From "Watching Tasks" to "Waiting for Status Changes"

From an operational logic perspective, the update can be understood as:

Past: Submit task → Enter task page → Manual refresh → Check status → Refresh again → Wait for completion.
Now: Submit task → Use other platform features → System message updates task status → Decide next steps based on completion or exception messages.

This adjustment does not change the execution logic of the extraction tasks themselves but reduces the need for users to repeatedly refresh pages to confirm status.

Value for Batch Knowledge Processing

Enterprise knowledge governance often involves processing more than one task at a time. For example, during centralized knowledge base construction, there might be:

Multiple unstructured document extraction tasks.
Multiple batches of structured data extraction.
Content processing occurring simultaneously in different knowledge bases.

As the number of tasks increases, relying solely on users to actively check can lead to missed completions or unnoticed exceptions until the data is actually used. Unified message notifications help shorten the time gap between a task status change and user awareness.

Note: While message notifications improve status awareness, they do not replace task exception diagnosis. If a task fails, specific causes must still be located using task details, logs, and actual file or data content.

02. New kkFileView Configuration for File Preview

In a knowledge platform, file management usually involves more than just uploading and parsing. Users often need to view file content directly. During knowledge governance, it is frequently necessary to confirm:

If the uploaded file is correct.
If the file content is complete.
Which material is currently being processed.
If the file content matches the knowledge base classification.

Therefore, although file preview is a basic function, it directly impacts the efficiency of knowledge material management.

Limitations of Original Preview Addresses

Previously, qKnow file preview used the address provided by the platform. However, in actual deployment, network environments vary significantly across enterprises. For example, some clients use:

Enterprise intranet deployment.
Isolated network deployment.
Custom network access environments.

In these scenarios, the original preview address might fail to display files correctly due to different network access conditions. The core issue is not that the file itself cannot be read, but that the file preview capability needs to match the enterprise's actual deployment environment.

System Deployment Configuration Adds kkFileView

To address this, qKnow Professional Edition v3.1.3 adds kkFileView configuration to the system deployment settings and synchronizes the file preview address. Users can complete the configuration for file preview capabilities by enabling the corresponding configuration when deploying the platform.

From a deployment logic perspective, this can be understood as:
Deploy qKnow → Configure and Start kkFileView → Platform Uses Corresponding Preview Service → View Knowledge Files

This provides a clearer configuration entry for file preview methods to align with the actual deployment environment.

Adapting to Intranet Deployment Scenarios

For enterprise clients, the actual environment is often not a uniform public network deployment. Especially in sectors like government, manufacturing, water conservancy, and energy, platforms are frequently deployed on enterprise servers or internal networks. Therefore, whether the preview service can be configured alongside the deployment environment directly affects the usability of knowledge file management.

The value of this update lies in incorporating the previously relatively fixed file preview method into the platform's deployment configuration system.

Note: Adding the kkFileView configuration does not mean all preview issues will automatically disappear. Actual deployment still requires ensuring the corresponding service starts correctly and meets basic conditions for servers, network access, and permissions.

03. Homepage Notifications Moved to Parameter Management

In addition to knowledge extraction and file preview, this version also adjusts the control method for homepage notifications.

Previously, the display logic for homepage notifications was configured in the system's front-end configuration file. If an enterprise wished to hide homepage notifications, they had to:

Locate Front-end Config File → Modify Configuration → Restart Project

The issue with this approach is that a page display requirement, which is essentially a simple switch, required entering the technical configuration layer. This is inconvenient for general platform administrators.

Homepage Notification Switch Enters Parameter Management

qKnow Professional Edition v3.1.3 migrates homepage notification-related parameters to Parameter Management. Users can now directly control the homepage notification via parameter data:

Show
Hide

There is no longer a need to modify the front-end configuration file and restart the project every time.

From Code Configuration to Platform Parameter Configuration

Before: Modify front-end config file → Save config → Restart project → View page changes.
After: Enter Parameter Management → Adjust corresponding parameter → Control homepage notification visibility.

This effectively transforms an action偏向 "system development
configuration" into "platform runtime parameter management," reducing the maintenance cost for simple display adjustments.

While this change does not involve knowledge extraction algorithms or core knowledge base capabilities, it reduces frequent but unnecessary technical maintenance operations for long-running platforms.

04. What Problem Do These Three Optimizations Solve?

Viewing these three features together, they address different points in the usage of the knowledge platform:

Extraction Task Message Notifications solve the issue of not knowing where a submitted task is executing. It shifts task status from "requires active refresh to view" to "system actively reminds."
kkFileView Preview Configuration solves the issue of files being in the platform but unable to be previewed in certain enterprise deployment environments. By adding a deployable preview service configuration, file viewing becomes more adaptable to the enterprise's actual operating environment.
Homepage Notification Parameter Management solves the issue of a simple page notification switch requiring code configuration changes and project restarts. Through parameter management, this technical configuration operation is further transformed into platform configuration.

Therefore, while these three adjustments belong to tasks, files, and system parameters respectively, they point in the same direction: reducing the need for extra manual operations, repeated confirmations, and technical intervention in the knowledge governance process.

Conclusion

Once knowledge governance capabilities enter the stage of continuous use, users care about:

Where is the task execution?
Can I know immediately if an exception occurs?
Can files be previewed normally in our specific deployment environment?
Do page display requirements require finding a technician to modify configuration every time?

qKnow Professional Edition v3.1.3 makes adjustments primarily around these questions:

On the knowledge extraction side: Structured and unstructured extraction tasks add message notifications, allowing the system to automatically update task information based on executing, completed, or exception states.
On the file management side: System deployment configuration adds kkFileView and adjusts the file preview address, making file preview capabilities further adaptable to different enterprise deployment environments.
On the platform configuration side: Homepage notification control is migrated from front-end configuration files to parameter management, allowing users to directly control visibility via parameters, reducing maintenance costs associated with modifying configuration files and restarting projects.

These adjustments do not change the core business logic of enterprise knowledge governance, nor do they replace task log analysis, network environment configuration, or system operation and maintenance management. However, for an enterprise agent platform that needs to run for the long term, what truly affects the user experience is often not just whether core functions exist, but also:

Whether tasks can provide timely feedback.
Whether files can be viewed normally.
Whether configurations can be adjusted conveniently.

The qKnow Professional Edition v3.1.3 upgrade continues to supplement these basic but high-frequency links, making the usage chain from extraction and viewing to daily management of enterprise knowledge smoother.

What Does qData Professional Edition Actually Collect? And What Comes Next?

TongWu — Thu, 10 Sep 2026 07:46:23 +0000

In the construction of a data middle platform, "Metadata Collection" is an unavoidable foundational capability for data governance.

However, when encountering it for the first time, a question often arises:
The database has already been connected to the platform via a data connection. Why do we need to perform metadata collection again?

This leads to further questions:

What exactly does the collection task collect?
After collection, can we only see how many tables and fields are in the database?
If the source system's table structure changes, how does the platform continue to track it?

To answer these, we must first distinguish between two concepts:

Data Connection solves "Can the platform access this data source?"
Metadata Collection solves "What is inside this data source, what is its structure, and what is its current state?"

Therefore, metadata collection is not simply "connecting to the database again." It is the process of identifying data structures and descriptive information originally scattered in databases, file systems, and object storage, and then unifying them into the platform for continuous management.

Why Do We Need Metadata Collection If We Already Have a Data Connection?

Assume an enterprise has established a MySQL data connection via qData.
Once connected, the platform solves: Can I access this database?

But for subsequent data governance, simply "being able to connect" is not enough. The platform needs to know further:

What tables are inside the database?
What fields does each table have?
What type is each field?
Which fields are primary keys or foreign keys?
Has the structure changed recently?
Which metadata has been confirmed?
Which tables need to be included in data asset management later?

From a data governance perspective, a Data Connection is like establishing an access channel, while Metadata Collection begins the further identification and management of the data objects behind that channel.

1. What Exactly is Metadata Collection "Collecting"?

Metadata is often defined as "data describing data."
In the context of an enterprise data platform, it can be understood more specifically:
Metadata describes the technical structure and basic attributes of a dataset itself, rather than copying the business data within it again.

For example, an order table might contain millions of order records.
Metadata collection will not copy these millions of orders to the metadata platform. Instead, it identifies:

The table name.
Which database it belongs to.
Which fields it contains.
Field names.
Data types of fields.
Length, precision, and decimal places.
Whether it is mandatory.
Whether it is a primary key or foreign key.
The current structure of the table and fields.

For database-type structured data, the collection hierarchy can be understood as:
Data Source → Database → Data Table → Field

Only by collecting this technical information can the platform truly know what data objects exist in a data source and what their current state is.

Not Just Databases: File Data Can Also Enter Metadata Management

Enterprise data is not all stored in databases.
Actual business environments contain large amounts of:

CSV, JSON, SQL, documents, and other files.

These files may be distributed across different file systems or object storage.
Therefore, qData also supports metadata collection for file-type data.

Unlike databases, file-type metadata focuses more on:

File name.
File type.
File size.
Storage location.
Source system.
Update time, etc.

In other words, whether it is tables and fields in a database or file objects in a file system, metadata collection first solves the same problem:
Identify the data scattered across the enterprise first.

2. How Does This Metadata Enter qData?

Metadata does not automatically appear in the platform; it is obtained from specified data sources via Metadata Collection Tasks.

qData can create:

Structured collection tasks
Unstructured collection tasks

For structured data, the first step in creating a collection task is still to determine: Where does the data come from?

The platform will associate the corresponding source system and data connection, obtaining connection information such as database type, IP, port, and account.

Afterwards, it determines: What needs to be collected this time?
The collection scope can cover the entire data source or, based on the data governance scope, custom-select specific databases.

Therefore, metadata collection does not mean a "full scan" every time. Enterprises can decide which databases need to enter the platform based on their actual data management boundaries.

Since database structures in business systems are not static, metadata collection tasks can also be configured with scheduling cycles.

For example: Execute metadata collection once at a fixed time every day.
This way, the platform acquires not just the structure at the time of initial access, but can continuously synchronize the latest metadata state from the source.

For unstructured data:
Files in file systems and object storage can also have their relevant file information collected into the metadata platform via corresponding data connections.

The entire collection process can be summarized as:
Determine Data Source → Determine Collection Scope → Set Collection Cycle → Execute Collection Task

Here:

The Collection Task defines "Where to collect, what to collect, and when to collect."
Each actual execution generates a corresponding Collection Instance. This distinction is very important for subsequent operational management.

3. After Task Execution: First Check "Did It Succeed?"

Creating a collection task does not mean metadata has successfully entered the platform.
After each task is truly executed, you can view the specific running results through the Collection Instance.

The collection instance records:

Whether this collection was successful.
Where the data came from.
Collection duration.
Collection quantity.
Start time.
End time.
Corresponding collection logs.

It first solves an operational layer problem:
"Did this specific run succeed? How much was actually collected?"
For periodically running metadata collection tasks, this step also provides a basic basis for subsequent exception troubleshooting.

4. After Successful Collection: Metadata Enters "Latest Metadata"

When a collection is successful, the truly acquired data structure information enters qData's Latest Metadata.

For database-type data, you can view currently collected data tables by hierarchy, such as source system and database.
The list allows you to further see:

Table name, table comments, owning database, collection source, current status, etc.

Entering specific table details allows you to continue viewing field-level information, including:

Field name, field comments, data type.
Field length, precision, decimal places.
Whether mandatory, primary key, or foreign key, etc.

At this point, the difference between Data Connection and Metadata Collection becomes more intuitive.

Before connection: The platform did not know this data source.
After connection: The platform knows, "There is a database here, and I can access it."
After metadata collection: The platform further knows, "What tables are inside, what fields each table has, what types the fields are, and what the current structure is."

File Metadata Also Enters Unified Management

File-type data follows a similar logic.
After collection, you can view identified file names, file types, physical storage info, source info, and modification times in "Latest Metadata (Files)."

Entering file details allows you to further view corresponding:

Data connection, file path, file size, source system, and related responsibility info.

This allows table, field, and file information originally scattered in different databases, file systems, and object storage to be unified and settled in the metadata platform.

5. Source Systems Keep Changing: How to Keep Metadata Synchronized?

If metadata is collected only once, it only reflects the data structure at a single point in time.

But enterprise business systems are actually constantly changing.
For example, after a system upgrade, there might be:

New fields added.
Fields deleted.
Modifications to existing fields.

This means metadata management cannot just answer: "What is the structure now?"
It also needs to answer: "How has it changed compared to before?"

qData continuously acquires the latest structure from the source via periodic metadata collection while retaining corresponding Metadata Versions.

From Latest Structure to "Versions and Changes"

In the table's Version and Changes section, you can view different metadata versions and the changes corresponding to each version.
For example:

One version added a field.
Another version deleted a field.

These changes are preserved through version records.
If you need to further judge the specific differences between two versions, you can use Metadata Comparison.

By selecting:
Base Version + Comparison Version
You can perform a comparison.

Thus, the questions metadata management answers expand from:
"What exists now?"
to:
"What did it look like before, and what has it become now?"

For long-term data platform governance, this is more valuable than simply displaying the current table structure, as many data problems originate from upstream structural changes.

6. Confirmed Metadata Can Be Further "Finalized"

For metadata versions that have been confirmed and are desired to be kept relatively stable, qData can further form Finalized Metadata.

After finalization, you can still view the corresponding table structure and field information.
Here, a concept needs special distinction:
Finalized Metadata does not equal automatically converting metadata into data standards.
It is closer to:
Fixing a collected and confirmed metadata version as a relatively stable foundation for subsequent unified viewing, management, and use.

Therefore, from a single metadata collection to long-term management, a continuously evolving chain is actually formed:
Collection → Update → Version Record → Change Comparison → Finalization Confirmation

This also illustrates that enterprise metadata management should not just be a "one-time摸底 (摸底 means摸底/survey)," but needs to be continuously updated as source systems change.

7. After Collecting Metadata, What Else Can Be Done?

If metadata collection stops at just "viewing tables and fields," its value remains limited.
Once the platform knows: What data the enterprise has and what the structure is, the next step is to proceed to Data Asset Construction.

In the qData Data Asset module, you can select database tables from the collected metadata to further register the corresponding data tables as Data Assets.

During the registration process, you can directly select existing metadata while continuing to supplement:

Asset name.
Table type.
Data warehouse layering.
Business classification.
Data domain.
Subject ownership.
Table naming conventions.
Asset description, etc.

This actually reflects the difference and connection between Metadata and Data Assets.

Metadata Answers "What Exists Technically," Data Assets Answer "What It Is Business-wise"

Metadata first answers: "What data exists technically?"
For example:

What is this table called?
Which database does it come from?
What fields does it have?
What is the structure?

But after entering data asset management, it needs to further answer: "What exactly is this data within the enterprise?"
For example:

Which business does it belong to?
Which data domain?
What data warehouse layer is it at?
How should it be classified?
Who manages it?

Therefore, the entire process can be understood as:
Data Connection → Metadata Collection → Metadata Management → Metadata Confirmation → Data Asset Registration

Metadata collection first identifies the technical structure of the data, and then business attributes and management attributes are supplemented, gradually incorporating data scattered in various business systems into a unified data asset management system.

This is why enterprise data asset construction usually cannot bypass metadata.
Because before discussing "which data is worth managing as assets," we must first know: What data the enterprise actually has.

Looking Back: What Does a Metadata Management Chain Go Through?

Connecting the previous processes, we can derive a relatively complete qData metadata management chain:

Establish Data Connection: Allow the platform to access databases, file systems, or object storage.
Create Metadata Collection Task: Define source system, data connection, collection scope, and scheduling cycle.
Execute Collection: Confirm execution results, collection quantity, duration, and logs via collection instances.
Enter Latest Metadata: View the latest structure info of current data tables, fields, or files.
Periodic Continuous Collection: Continuously acquire the latest structure as source systems change.
Versions and Changes: Record added fields, deleted fields, and structural changes across different versions.
Metadata Comparison: Compare specific differences between two versions.
Finalized Metadata: Further fix and settle confirmed metadata versions.
Data Asset Registration: Select data tables from existing metadata and supplement asset attributes like business classification, data domain, and data warehouse layering.

In this way, metadata collection is no longer an isolated technical action but a foundational capability bridging Data Discovery to Asset Construction.

Summary: Metadata Collection is the First Step to Truly "Managing" Enterprise Data

Owning a database does not mean truly understanding the data inside it.
Establishing a data connection does not mean this data has entered the governance system.

qData Professional Edition's metadata collection first identifies enterprise data objects through structured and unstructured collection tasks, moving the platform from:
"I can access this data source"
further to:
"I know what is inside and what the structure is."

On this basis, through periodic collection, collection instances, latest metadata, version records, change comparison, and finalization management, it further solves:
"What changes happened to this data later?"

Once technical metadata is identified and confirmed, it can proceed to data asset registration, supplementing business classification, data domains, data warehouse layering, etc., on top of the existing technical structure.

Therefore, metadata collection is not simply counting how many tables and fields are in a database, but a management process advancing step-by-step from:
Find Data → Understand Data → Track Changes → Confirm Metadata → Form Data Assets

For data governance, metadata collection itself is not the endpoint, but a foundational task enabling subsequent metadata management and data asset construction to proceed.

qData Open Source Data Middle Platform v1.6.2 Released: Fixing Known Issues to Enhance Stability and User Experience

TongWu — Thu, 10 Sep 2026 07:46:14 +0000

The qData Open Source Data Middle Platform v1.6.2 primarily focuses on resolving issues related to system stability, data connection verification, metadata management, and project management.

This release further improves exception handling, front-end and back-end verification logic, and the consistency of operational feedback.

From "Functional Availability" to Continuous Refinement of Operational Details

Once an enterprise data platform is put into actual use, user concerns typically evolve.

In the early stages of product development, the primary focus is often on:

Do we have the corresponding features?

However, as data sources, projects, metadata, and data assets gradually enter daily management, more common questions become:

Is the function running stably?
Is the exception feedback accurate?
Is the configuration verification actually taking effect?
Do management operations meet expectations?

For example:

A source system meets the deletion criteria, but an error occurs when executing the deletion.
A data connection test fails, yet the page directly displays a 504 error, making it difficult for users to identify the actual problem.
The front end performs parameter verification, but the back end does not enforce the same rules.
A project has no actual member occupancy, yet the system prompts that personnel exist, preventing deletion.
After binding a term to a data asset, the operation does not fail, but the page prompt may mislead users.
A help entry exists, but clicking it fails to correctly access the documentation.

These issues do not change the functional framework of the qData Data Middle Platform itself, but they do affect the continuity and judgment efficiency of the platform in actual use. Therefore, qData Open Source v1.6.2 places its focus on these specific areas.

01 Source Systems: Fixing Deletion Exceptions and List Sorting Issues

Source systems are one of the fundamental management objects after data enters the platform. As the number of connected data sources increases, users not only need to add and maintain source systems but also continuously clean up historical sources, view lists, and perform daily management.

This release addresses two main issues regarding source systems.

1. Fixed errors when deleting source systems
Previously, in some cases, users might encounter errors when performing source system deletion operations, affecting the normal cleanup of source systems. qData Open Source v1.6.2 has fixed this issue, allowing source systems that meet deletion criteria to be processed according to the normal workflow.

For a long-running data platform, source systems are not just added but also removed. With test environment adjustments, data source migrations, or historical configuration cleanups, the platform needs to be able to complete the source system lifecycle management normally. Therefore, whether the deletion operation can be executed correctly is a fundamental part of the source system management chain.

2. Fixed abnormal default sorting for source systems
This release also fixes the issue where the source system list was not sorted by the default sorting field. When the number of source systems is small, the impact of list sorting is not obvious. However, as connected systems continue to increase, a stable and expected default sorting method helps reduce interface order changes when users search for and manage source systems. This fix further unifies the display logic of the source system list.

02 Metadata Management: Fixing the Help Documentation Jump Link

Metadata management involves collection task configuration, metadata maintenance, and subsequent governance work. For users encountering these capabilities for the first time, help documentation is usually an important entry point for understanding task configuration methods and functional boundaries.

This release adjusts the help documentation entry in metadata management.

Fixed 404 error on the latest metadata help page

Previously, some metadata help pages resulted in a 404 error after redirection. This meant that although the product provided a help entry, users could not normally enter the corresponding description page after clicking it. qData v1.6.2 has updated the relevant help documentation links to ensure the help entry is consistent with the actual documentation address.

Adjusted the help documentation address for metadata collection tasks

In addition to fixing the 404 page, this release further adjusts the jump address for the metadata collection task help documentation. Metadata collection usually involves multiple configuration items such as source systems, data connections, and collection scopes. Ensuring the help entry correctly points to the corresponding documentation can reduce the cost of users searching for additional materials during configuration and maintain a more stable connection between product pages and supporting usage documentation.

Essentially, this adjustment does not add new metadata capabilities but continues to improve the auxiliary chain between:
Product Function Entry → Usage Help → Configuration Understanding

03 Data Assets: Optimizing Operational Prompts After Term Binding

In the process of data asset management, binding fields to terms is a specific operational step in business semantic governance. This release adjusts the prompt information when binding terms to asset fields, ensuring it no longer incorrectly displays as "Operation Failed."

For a data governance platform, the consistency between back-end operation results and front-end feedback is crucial. If the actual operation result is inconsistent with the page prompt, even if the data itself has been processed, users might still perform duplicate operations based on the prompt or misjudge the current asset governance status.

Therefore, although this adjustment is mainly reflected in prompt optimization, it solves a more fundamental problem:
The system's actual processing result should be consistent with the operational feedback seen by the user.

This kind of feedback consistency is especially important in enterprise data governance scenarios. Data asset maintenance usually involves many continuous operations, and users need to rely on page status and feedback to judge the next action. If the prompt itself is biased, it will additionally increase the cost of manual confirmation.

04 Data Connections: Improving Exception Display and Front-End/Back-End Verification Consistency

Data connection is an important foundational capability for data platforms to access external databases and other data sources. Subsequent data collection, data integration, and related data processing work are usually built on the basis that connection configurations are correct and accessible.

Therefore, compared to simply judging "connection success or failure," the data connection module also needs to solve two problems:

How should failure be reported?
Does configuration verification truly span both front end and back end?

qData Open Source v1.6.2 has made multiple fixes in this area.

Fixed the issue where the page displays 504 upon connection failure

When testing a data connection, if the connection itself fails, some scenarios previously displayed a 504 error directly on the page. This type of feedback can easily confuse data source connection exceptions with page or service access exceptions. This release fixes this issue, further improving exception handling in data connection failure scenarios.

For a data platform, connection testing is usually the first step in troubleshooting. When a connection fails, the system first needs to accurately report the current connection status to avoid the exception display itself interfering with the user's judgment of the cause.

Fixed the issue where verification rules only took effect on the front end

This release also fixes a more fundamental verification issue: some data connection verifications previously only took effect on the front end, while back-end verification rules were not synchronized.

Data connection information usually goes through multiple stages such as page filling, request submission, and back-end processing. If verification only exists on the front end, situations may arise where:
Front end judges requirements met → Request enters back end → Back-end rules are inconsistent or lack corresponding verification.

qData Open Source v1.6.2 has fixed this issue, further improving front-end and back-end verification consistency. This means the legitimacy judgment of data connection configurations no longer relies solely on the page layer but ensures the front and back ends maintain a more consistent processing approach at the rule level. For subsequent data access, this consistency can help reduce exceptions caused by different judgment standards at different stages.

Fixed recognition issues for some data source names or identifiers

This release also fixes the issue where some data source names or identifiers could not be correctly recognized by the back end. Data connections, from page configuration to actual back-end processing, need to go through data source type and related identifier recognition. If the front end can complete the selection but the back end cannot correctly recognize the corresponding name or identifier, it may cause the connection configuration to fail to enter the subsequent processing flow as expected. This fix further improves the connection between page configuration and back-end recognition.

Overall, the data connection-related fixes in this release cover three stages:
Connection Failure Feedback → Parameter Verification → Back-end Data Source Recognition

It does not add a new type of data source but continues to improve the fundamental reliability within the existing data connection chain.

05 Project Management: Fixing the Issue Where Projects Cannot Be Deleted Despite No Actual Member Occupancy

Projects are usually important organizational units for managing tasks, personnel, and related resources on the data platform. As test projects, temporary projects, or historical projects gradually increase, the projects themselves also need to be cleaned up normally.

Previously, in some cases, even if a newly created project did not actually have member occupancy, deleting it might still prompt:
"Personnel exist in the project"
Leading to the project being unable to be deleted.

qData Open Source v1.6.2 has fixed this judgment logic.

Management status needs to be consistent with actual resource occupancy

Deletion restrictions in enterprise platforms are usually necessary. For example, when a project indeed has associated personnel or other objects that need protection, the system needs to prevent users from directly deleting it to avoid subsequent management issues.

However, correspondingly, restriction conditions must also be based on the real status. If a project does not have actual member occupancy but cannot be deleted due to abnormal status judgment, it will lead to the inability to clean up historical projects normally.

This fix addresses exactly:
The inconsistency between the project's actual status and the system's deletion judgment.
Enabling project management operations to better align with the current actual member occupancy situation.

06 Summary of Issues Resolved in This Version

In terms of feature count, qData Open Source v1.6.2 is not a large-scale capability expansion. However, from the perspective of the actual usage chain, these fixes are distributed across multiple fundamental management links.

Source Systems: Fixed deletion errors and default sorting issues, improving daily maintenance and list management of source systems.
Metadata Management: Fixed 404 errors on help pages and adjusted collection task help addresses, improving the jump chain between product functions and usage documentation.
Data Assets: Adjusted prompt information after binding terms to fields, reducing inconsistencies between actual operation results and page feedback.
Data Connections: Fixed 504 display on test failure, front-end/back-end verification inconsistency, and data source identifier recognition issues, improving connection exception handling, parameter verification, and back-end recognition chains.
Project Management: Fixed the issue where projects prompted for existing personnel despite no actual member occupancy, making project deletion judgments better align with actual status.

These adjustments ultimately focus on several common directions:

Exceptions should be handled correctly. Connection failures, deletion exceptions, etc., need to be handled in a way that reflects the actual status, rather than letting the exception manifestation itself increase troubleshooting difficulty.
Front-end and back-end judgments need to remain consistent. Especially in fundamental configuration links like data connections, verification cannot rely solely on the page; the back end must also execute judgments according to corresponding rules.
System feedback needs to reflect real results. Whether it is data asset term binding or project member judgment, the prompts seen by users should be as consistent as possible with the system's actual status.
Auxiliary entries are also part of the complete usage chain. Whether help pages can be accessed normally may seem independent of core data processing capabilities, but it equally affects the efficiency of users completing configurations and locating problems.

Final Thoughts

After an enterprise data platform truly enters the long-term usage stage, product stability often does not depend solely on a single core module.

Whether source systems can be maintained normally, data connections can be accurately verified, metadata help can be accessed normally, asset operations can provide correct feedback, and project status can be accurately judged—these links together determine whether the platform's daily use is smooth.

If we connect the several modules involved in this release, we can see a relatively complete fundamental management chain:

Source System Management → Data Connection Configuration and Verification → Metadata Management → Data Asset Governance → Project Management

qData Open Source v1.6.2 does not change the chain itself but fixes identified exceptions item by item.

For a data platform, stable exception handling, unified verification mechanisms, and accurate status feedback are also the foundation for the sustainable operation of data governance capabilities.

qData will continue to focus on data access, data governance, and actual problems found during platform operation, continuously improving existing functional chains and usage details.

Flow from Water Level: Building a Hydrological Rating Curve Pipeline from Measured Data to Calibration

TongWu — Thu, 03 Sep 2026 08:19:28 +0000

In daily monitoring of rivers, reservoirs, and hydrological stations, water level can typically be continuously captured through automated monitoring equipment, but flow measurement is relatively complex.

Therefore, in hydrological surveying operations, a fundamental technical question arises: How can we more accurately estimate the corresponding flow discharge based on continuously monitored water levels?

This involves a critical technical relationship in hydrological surveying — the stage-discharge relationship (also known as the rating curve).

The accuracy of the stage-discharge relationship directly affects the data foundation for water volume calculation, flood warning, and water resource scheduling.

However, in actual operation, the relationship between water level and flow discharge is not a fixed, unchanging formula once established.

As riverbed scouring and sedimentation change, water conservancy project operations and scheduling shift, and survey cross-sections are relocated, the stage-discharge relationship at the same station may also change accordingly.

Therefore, it is necessary to continuously calibrate and adjust the rating curve using the latest measured data, so that the estimated flow results align as closely as possible with current actual conditions.

Centered on this business process, the system connects flow measurement data management, rating curve calibration, curve review, and reviewed curve querying — forming a relatively complete business management pipeline from raw survey data input, through curve calculation and review approval, to subsequent querying and historical comparison.

Why Does the Stage-Discharge Relationship Need Continuous Calibration?

From a technical perspective, rating curve calibration addresses one core problem: using existing measured water level–flow discharge data to establish a mathematical functional relationship between the two.

Once a calibrated rating curve is available, the corresponding flow discharge can be estimated from monitored water levels, thereby reducing the frequency of manual measurements and improving hydrological monitoring efficiency.

But real-world river channels are not ideal environments that remain stable over the long term.

For example:

After riverbed scouring or sedimentation occurs, the cross-sectional shape may change.
Changes in water conservancy project operation modes may also affect upstream and downstream flow conditions.
After a survey cross-section is relocated, the original stage-discharge relationship needs to be re-evaluated.

This means that a rating curve established in the past may not necessarily represent the current actual river conditions over the long term.

Therefore, curve calibration is not a one-time model-building process — it requires ongoing updates and adjustments based on continuously accumulated measured data.

To carry out this process well, the first thing that needs to be addressed is how raw flow measurement data is managed in a unified manner.

Step 1: Unified Flow Measurement Data Management — Preparing the Foundation for Curve Calibration

The reliability of a rating curve depends first and foremost on the quality of the underlying survey data.

The system manages flow measurement data by centrally recording, storing, and maintaining the raw measurement data from each station.

Staff can filter down to specific stations through the following hierarchy:

Management Bureau → Management Office → Management Station → Survey Station

Then query the corresponding flow measurement records.

For each station, the system centrally manages four types of foundational data:

Measured flow discharge results table
Basic staff gauge water level record table
Measured cross-section results table
Elevation verification record table

Each data type supports modification, import, and other operations, making it convenient for staff to continuously maintain raw flow measurement records.

From the perspective of the overall business pipeline, this step answers the question: Where does the data for curve calibration come from?

Only after consolidating different types of foundational survey data from different stations can the subsequent step of using measured water level and flow discharge data for rating curve calculation proceed.

Step 2: From Discrete Measurement Points to Rating Curves — Completing the Stage-Discharge Relationship Calibration

Once the foundational flow measurement data is in place, the next step is the core of the entire process — rating curve calibration.

What actual surveys produce is a series of discrete water level and measured flow discharge data points.

For example:

What was the water level at a given moment?
What was the corresponding measured flow discharge?

As the number of surveys increases, a large number of water level–flow discharge measurement points accumulate.

What rating curve calibration needs to do is perform mathematical fitting on these measured data points, transforming discrete data points into a stage-discharge relationship model that can be used for flow estimation.

After selecting the corresponding management organization and station, staff can set the query start and end times, choose an appropriate calibration method — such as a logarithmic function — and define an error tolerance range.

The system then automatically performs curve fitting and calculates the fitting error for each measured point.

In this process, it is not just about obtaining a curve — it is also necessary to answer a further question: How large is the deviation between this curve and the actual measurement results?

Not Just "Fitting a Curve" — Examining the Error at Each Measurement Point

Whether a rating curve is usable cannot be determined solely by visual judgment of the curve plot.

Therefore, after fitting is complete, the system performs a point-by-point comparison between the measured data and the curve-calculated results.

The results area displays the information in a structured format:

Time
Water level
Measured flow discharge
Curve-calculated flow discharge
Error percentage

Staff can directly view, for the same measurement point:

What was the actually measured flow discharge?
What was the flow discharge calculated from the rating curve?
How large is the error between the two?

At the same time, the system visualizes the water level–flow discharge relationship through charts.

By combining the data view with the rating curve visualization, staff can assess the current calibration results from both the numerical error perspective and the overall fitting trend perspective, helping determine whether the curve accuracy meets the required standards.

This is also a key distinction between curve calibration and simple data fitting:

The model not only needs to be computed — it also needs to be validated.

Step 3: After Curve Calibration, Entering the Review Process

Since the rating curve directly affects subsequent water level–to–flow estimation results, the completion of calibration does not mean immediate formal use.

The system further provides rating curve review capabilities.

Staff can filter curves by:

Management Bureau
Management Office
Management Station
Station name

They can also query by review status, such as reviewed or unreviewed.

The review list displays:

Station name
Curve name
Calibration type, e.g., manual
Current review status

Through this mechanism, the "calculation and generation" of a curve is separated from its "formal use," adding a review management step before the curve enters business application.

For hydrological operations, this is especially important.

Because the rating curve itself is not a static deliverable — after surveying conditions change, an already established curve may also need to be re-evaluated.

When Surveying Conditions Change, Existing Curves Can Be Re-Reviewed

Over time, river channel and surveying conditions may continue to change.

Therefore, the system provides a "re-review" operation in the rating curve review list.

For already existing curves, they can re-enter the review process to accommodate curve revision and update needs after surveying conditions have changed.

This transforms the entire curve management process from:

Generate curve → Use permanently

Into a dynamic management process:

Measured data updated → Curve re-calibrated → Reviewed / Re-reviewed → Updated for application

For long-operating stations, this continuous calibration mechanism better aligns with the characteristic that the stage-discharge relationship itself changes as environmental conditions evolve.

Step 4: After Review Approval, Unified Management of Officially Effective Curves

Once a curve passes review and becomes officially effective, another question needs to be addressed:

How should these curves be queried, used, and managed going forward?

The system provides reviewed curve querying capabilities, centrally managing rating curves that have passed review and become officially effective.

Staff can view curve information for different stations, including key fields such as station name and curve name.

For each curve, the system also provides the following operations:

Query — View detailed curve information
2D Table — Access the water level–flow discharge lookup table
Historical Curves — View and compare curves from different historical periods

Why Preserve Historical Curves?

For stage-discharge relationship management, historical data is not merely archival material.

Rating curves formed at different time periods can themselves reflect changes in the station's stage-discharge relationship.

For example:

When significant scouring or sedimentation occurs in the river cross-section, differences may appear between old and new curves.
After surveying conditions change, curves formed through re-calibration may also differ from previous results.

Therefore, through historical curve querying and comparison, staff can further understand:

How the stage-discharge relationship at the same station has changed across different time periods.

These historical results can also serve as reference material for subsequent curve re-calibration, review, and updates.

In this way, the rating curve is no longer just a single calculation rule currently in use within the system — it becomes a continuously accumulated body of business deliverables that can be queried and compared.

From Flow Measurement Data to Official Curves: A Complete Calibration Management Pipeline

Connecting all the above functions, the entire stage-discharge relationship management process actually forms a fairly clear business pipeline:

Flow measurement data management → Select station and time range → Choose calibration method and error tolerance → Water level–flow discharge data fitting → Generate rating curve → Measured flow vs. curve flow error analysis → Rating curve review → Review approval and official activation → 2D table query / Historical curve query → Re-calibration and re-review when surveying conditions change

This pipeline connects raw flow measurement records, mathematical model computation, error assessment, business review, and subsequent deliverable management.

For staff, it is not just about "generating a curve in the system" — they can further understand:

What data was this curve based on?
How did the fitting results turn out?
Has it been reviewed?
How can it be queried after activation?
How should it be re-calibrated when conditions change?

Summary: Moving Stage-Discharge Relationships from One-Time Calibration to Continuous Management

The stage-discharge relationship is a foundational technical relationship in hydrological surveying.

Its value lies not only in establishing a curve that can estimate flow from water level, but more importantly, in ensuring that this curve can be continuously calibrated as actual hydrological and surveying conditions change.

Centered on this business need, the system connects flow measurement data management, rating curve calibration, fitting error analysis, rating curve review, curve re-review, and both effective curve and historical curve querying.

From foundational survey records entering the system, to using measured water level–flow discharge data for mathematical fitting.
From point-by-point analysis of errors between measured flow and curve-calculated flow, to curve review, activation, and subsequent historical comparison — forming a relatively complete rating curve management process.

For water conservancy management organizations, the significance of this process is not to replace professional hydrological surveying and technical judgment, but to further digitize and streamline the previously scattered processes of data management, computation, review, and deliverable management.

When conditions such as riverbed scouring and sedimentation, water conservancy project operations, or survey cross-sections change, the system can also re-calibrate and re-review based on new measured data, keeping the stage-discharge relationship continuously aligned with actual conditions — providing a more reliable data foundation for water volume calculation, flood warning, and water resource scheduling.