I am having code as:
new_df=spark.sql("Select col1,col2 from table1 where id=2").collect()[0][0]
I have tried toLocalIterator() but getting message that is not subscriptable.
Please suggest a better way to replace collect()[0][0].
For further actions, you may consider blocking this person and/or reporting abuse
Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.
Mike Young -
kathir b -
Aviral Srivastava -
GitHubOpenSource -
Top comments (1)
So that looks like you're after the value in the first column of the first row. If that's the case then you could use the following.
spark.sql("SELECT col1, col2 FROM table1 WHERE id=2").first()[0]
That will return the first row as a Row object which you can then access via index. You should find it works better as well as collect will pull all of the data to the driver before you attempt to access a single row