99
Spark Intersection Function
In Spark, Intersection function returns a new dataset that contains the intersection of elements present in the different datasets. So, it returns only a single row. This function behaves just like the INTERSECT query in SQL.
Example of Intersection function
In this example, we intersect the elements of two datasets.
- To open the Spark in Scala mode, follow the below command.
- Create an RDD using the parallelized collection.
- Now, we can read the generated result by using the following command.
- Create another RDD using parallelized collection.
- Now, we can read the generated result by using the following command.
- Apply intersection() function to return the intersection of the elements.
- Now, we can read the generated result by using the following command.
Here, we got the desired output.
Next TopicSpark Cartesian Function