Hadoop Big Data Developer

Job Description

Requirements Soft Skills - Interested in data and finding value in it, - Being independent and eager to take full ownership of work you do, - Proactive rather than reactive in recognizing and implementing opportunities for improvements, - Ability to deliver end to end solutions, - Enjoying solving complex problems, - Ability to communicate complex designs and solutions in clear manner to different audiences, - Taking pride in high standards in programming code and documentation quality, - Collaborative and willing to share knowledge, - Ability to lead and upskill junior members of the team. Technical Skills · Software Engineering Skills (Computer Science degree): Robust design, modularity,Code Reusability, Testing,SDLC · Proven ability to deliver enterprise level production ready code:Logging,Alerting,Monitoring, Recovery and re-runability · Automation: CI/CD, Different levels of testing, Production staging · Programming approaches:OO, Functional, TDD and/or BDD · Programming languages: Java, Scala, Python, SQL, Shell/Bash · Programming code management: Git, Maven, Sonar/Checkmarx, Cloudbees/Jenkins · Familiarity with standard data storage technologies: SQL, NoSQL, Object Storage Skills Required : · Hadoop L3 Mandatory · Scala Programming L3 · Apache Spark L3 Good Understanding of Data Formats : Parquet, Avro, Json, CSV, XML or Hadoop Architecture. Distributed parallel processing concepts, semi and unstructured data handling, real time processing, No SQL and Data Lake Creation. Hands on Technical Skill sets: Java, Python, Sqoop, Scala , Devops, Jenkins, Bit Bucket, Sonar Qube Spark, Kafka, Hive, HBase Bonus · Experience with distributed and scalable data processing frameworks: Spark, Kafka Connect, Kafka Streams, NiFi · Experience in building automated end to end data pipelines, · Track record of Production Level Real Time/Stream Processing, · Experience with public and/or private Cloud Environments, · Ability to pioneer solutions where distributed processing is separated from data storage, · Experience with leveraging infrastructure as a code.