DEV Community

AICAA
AICAA

Posted on

create UDF in pyspark to join 2 tables

Need the help of this community, unfortunately creating udfs is not my strongest skill set.

I need to create UDF that will join two tables together, the problem is that one table has two id columns

Name Table has id1 and id2

Transaction Table has only id

I need to join Transaction table with id table. the constraint is if id2 in name table populated then join with id2 else join with id1

I have tried some things but none of them is 100% correct

Thanks

Image description

Top comments (0)