Installing Apache Kafka on macOS
Prerequisite: java with version 8 should be installed on the system.
To download the latest version of Apache Kafka(or as per the need), the following steps will be used:
Step1: Open the official website of the Apache Foundation or use the link: https://kafka.apache.org/downloads. Select and download the Kafka binaries from Binary downloads. Either choose Scala 2.11 or Scala 2.12.
Step2: After clicking on the selected binary, a new page will open. Click on the first link, which will be the mirror size for the Kafka download. The download will begin.
Step3: A Kafka ‘tar’ file will be downloaded. Move to the downloaded Kafka directory. Untar or uncompress the directory by using the command: ‘tar -xvf ‘.
Step4: Use the ‘ls’ command to view the untar file. Now, move into the .
To test, use ‘/bin/kafka-topics.sh’. If it gives the correct output means java is working. Use ‘java -version’ command to know the java version installed on the system.
Step5: If the required java version is not installed or java is not yet installed, then use the brew command as: ‘brew tap caskroom/versions’, then use ‘brew cask install java8’ and java8 will start installing. It will ask: ‘brew cask install java8’, press enter.
Step6: java8 will be installed on the system. Use ‘java-version’ to check again.
Setting the Path
To run Kafka from anywhere, use the following steps to set the path:
Step1: Move into the bin folder of . Open the ‘.bash_profile’ file by using the command: ‘nano ~/.bash_profile’.
Step2: The ‘.bash_profile’ file will open. Go to the end and set the path as:
Note: To know the forepath directory, use ‘pwd’ command. It will display its name.
Step3: After setting the path, apply ‘kafka-‘ or ‘kafka-topics.sh’ command to know whether the path is successfully set.
Using Brew
If the mac users have Brew installed, they can use it for Kafka installation.
There are following steps used to install Apache Kafka using brew:
Step1: Use ‘brew install kafka’ and press enter key to install Kafka. Wait a while, and Kafka will be installed on the system. To test, use any kafka command.
Step2: If the ‘kafka-‘ command is used, it will show files with and without ‘.sh’ extension files. It is because the bin path is set in the ‘.bash_profile’ file. So, edit the file again and remove that line from the end. Save, and exit.
Step3: Again use the command ‘kafka-‘, now the files without ‘.sh’ will appear.
Step4; But, to use kafka commands, the user needs to visit the kafka_directory always.
So, this is another way to install Apache Kafka on the system.
Starting Zookeeper
As Apache Kafka cannot work without zookeeper, it is mandatory to run zookeeper server on the system. Follow the below steps for installing the zookeeper server:
Step1: Type the command ‘zookeeper-server-start config/zookeeper.properties’ and press enter. If the port value=2181 is achieved without any error, it means the zookeeper is successfully started. Otherwise, if some other application is bound to 2181, ‘address in use’ error will be thrown.
Step2: The zookeeper window should not be closed to work with Kafka. Open another window, and create a new directory ‘data‘ under the kafka_directory. Use the command ‘mkdir data’ to do so.
Step3: Within the data directory, make two new directories: ‘zookeeper‘ and ‘kafka‘.
Step4: Open the ‘config’ directory placed within the kafka_directory using the command ‘nano config/zookeeper.properties’.
Step5: Edit the value of ‘datadir’ by placing the address of the newly created zookeeper directory. Save the file and exit.
Step6: Type ‘ls data/zookeeper’ on the window. A new directory ‘version-2’ will appear in the list. It means the zookeeper is started successfully.
Step7: Now, open ‘server.properties’ file to begin the Kafka server. Use the command: ‘nano config/server.properties’ to open the file.
Step8: Edit the value of log.dirs by placing the address of the newly created kafka folder from the forepath. Save the file and exit.
Step9: Finally, start the Kafka server with the help of the following command:
‘kafka-server-start config/server.properties‘ and press enter. If the output displays ‘kafka server started’,
Step10: Do ‘ls data/kafka’, and the newly created files will be visible, which indicates the Kafka server is successfully installed on the system.