DEV Community

Cover image for Do Ordered Grouping and Aggregation within Groups — From SQL to SPL #1
Judith-Data-Processing-Hacks
Judith-Data-Processing-Hacks

Posted on • Edited on

2 1 1 1 1

Do Ordered Grouping and Aggregation within Groups — From SQL to SPL #1

Problem description & analysis:

A certain database table describes the payment cycle for multiple projects (IDs), with one payment cycle consisting of regular months and a closing month. The regular month only has the current month’s amount but no invoice, Invoiced=0; The closing month includes both the current month’s amount and the invoice, Invoiced=1.

source table

Task: Now we need to identify each payment month for each project and calculate the total amount for that payment cycle. Note that the grouping criteria and order of the payment cycle are related, that is, “when last month’s Invoiced=1, start a new group”, which is different from the common equivalence grouping.

expected results

Code comparisons:

SQL

WITH cte AS (
      SELECT *, sum(invoiced) OVER (PARTITION BY ID ORDER BY Date desc) grp
      FROM mytable
      ORDER BY ID, Date
)
SELECT ID, MAX(date) AS Date, MAX(Invoiced) AS Invoiced, SUM(Amount) AS Amount
FROM cte
GROUP BY ID, grp
ORDER BY ID, Date
Enter fullscreen mode Exit fullscreen mode

SQL does not have a direct ordered grouping, it needs to add a help column using window functions and subqueries, and then group and aggregate based on the help column. The above SQL uses the method of reverse order and then accumulation to gather the help column, which is difficult to understand.

SPL:

SPL supports convenient ordered calculations, and the code is straightforward.

try.DEMO

SPL code

A1: Load data, note that the data has been sorted.

A2: When the ID remains unchanged and the previous month is a regular month, change the Amount to the cumulative value; Otherwise (in the first month of each payment cycle), reset the Amount to the current month’s amount. [-1] represents the previous record.


esProc SPL is open-source and now available here: Open-Source Address.

API Trace View

How I Cut 22.3 Seconds Off an API Call with Sentry 👀

Struggling with slow API calls? Dan Mindru walks through how he used Sentry's new Trace View feature to shave off 22.3 seconds from an API call.

Get a practical walkthrough of how to identify bottlenecks, split tasks into multiple parallel tasks, identify slow AI model calls, and more.

Read more →

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay