Home » Apache Pig FILTER Operator

Apache Pig FILTER Operator

by Online Tutorials Library

Apache Pig FILTER Operator

The Apache Pig FILTER operator is used to remove duplicate tuples in a relation. Initially, Pig sorts the given data and then eliminates duplicates.

Example of FILTER Operator

In this example, we eliminate duplicate tuples.

Steps to execute FILTER Operator

  • Create a text file in your local machine and provide some values to it.

Apache Pig FILTER Operator

  • Check the values written in the text files.

Apache Pig FILTER Operator

  • Upload the text files on HDFS in the specific directory.
  • Open the pig MapReduce run mode.
  • Load the file that contains the data.
  • Now, execute and verify the data

Apache Pig FILTER Operator

  • Let’s execute FILTER operator to eliminate duplicate tuples.
  • Now, execute and verify the data.

Apache Pig FILTER Operator

Here, we got the desired output.

Next TopicFOREACH Operator

You may also like