Senior Data Engineer

Job Description

Data Engineer Contractor 

• Strong coding skills in a language such as Python and Scala or Java.
• Experience with shell scripting / UNIX / Linux
• SQL, RDBMS (Teradata, MySQL, Postgres etc.)
• Experience in big data technologies like Hadoop, Hive, Kafka, Apache Spark etc.
• Experience in batch / streaming data pipelines development and troubleshooting
• Cloud technologies - AWS nice to have (EMR , Airflow, S3, MSK, AWS Glue etc.)
• Excellent written and oral communication skills

What they will be working on :

• On-prem data Ingestion tools 
  • Improve observability
  • Improve data quality by providing audits that help to proactively identify the issues upstream
  • KTLO
    • Resolve the issues having runbooks before passing it on to ingestion squad members
  • Provide Groupon standard host rolling mechanisms
  • Fix recurring issues
  • Helping on active continuous new onboarding requests 
• Janus tools (real-time ingestion service)
  • Upgrade Janus engines to consume from MBUS dedicated topics to enable replays on all the topics
  • Replay messages through MBUS replay tool
  • Help in Janus Storm deprecation 
  • Improve staging environment