This article is Part 3.
The first two parts have already implemented the parsing and saving of script field-level data lineage information. Next, based on these lineage information, a web application for data lineage will be implemented.
Of course, everyone can implement their own applications based on their own ideas. This is just a case reference.
This web application is implemented using the Django framework in Python, with the project name being web_data_lineage_show. The page after startup is shown as follows:
1. Graphical display of script content
Explanation:
(1) After starting the service, enter the URL
(2) Display the parsed script list
(3) Select a script to generate a data lineage graph
(4) Select a table and display the field logic
(5) Connecting lines includes association methods and conditions
2. Field lineage tracing function
(1) Select any column
(2) Click the "Trace Upward" button
(3) Display the lineage path of the column
View the columns of the data lineage node
View the columns of the data lineage node
3. Cross-script file field lineage tracing function
(1) Select "Cross-file traceability"
(2) Select any source table from the left column
(3) Select any field
(4) Click the "Trace Upward" button
(5) Trace to another script file based on the field
(1) Select the script file that has been traced
(2) While the script is being visualized, it will automatically continue to trace based on the fields traced in the previous layer. Users can utilize this feature to quickly achieve full-chain tracing for a specific field.
4. Graphical instance demonstration of complex scripts
After testing, the parsed field-level lineage is completely accurate.
The source code for the web project "web_data_lineage_show" can be found in the open-source community link:
Top comments (0)