Work Location: Sydney

Experience:  More than 5 years

Mandatory Skills:

·         Over all 5 years’ experience, with at least 2 years’ experience on Hadoop technologies.

·         Install, configure and maintain enterprise hadoop environment.

·         Loading data from different datasets and deciding on which file format is efficient for a task. Hadoop developers source large volumes of data from diverse data platforms into Hadoop platform.

·         Understanding the requirements of input to output transformations.

·         Defining Hadoop Job Flows.

·         Build distributed, reliable and scalable data pipelines to ingest and process data in real-time. Hadoop developer deals with fetching impression streams, transaction behaviours, clickstream data and other unstructured data.

·         Design and implement column family schemas of Hive and HBase within HDFS.

·         Assign schemas and create Hive tables.

·         Managing and deploying HBase clusters.

·         Develop efficient pig and hive scripts with joins on datasets using various techniques.

·         Assess the quality of datasets for a hadoop data lake.

·         Apply different HDFS formats and structure like Parquet, Avro, etc. to speed up analytics.

·         Build new hadoop clusters

·         Maintain the privacy and security of hadoop clusters.

·         Fine tune hadoop applications for high performance and throughput.

If interested and have the skills required or know of someone who may be interested please contact me at for details or kindly share me your Resume.

Phone No*
Upload Resume*