Sr. Data Engineer

Job Description

We are looking for candidates with 6 to 8 years of experience to join our team as a Senior Data Engineers, with education (Bachelor’s degree or higher) in Computer Science, Mathematics, or a related technical field, or equivalent practical experience. They should also have the following abilities:
Responsibilities
• Thrive in ambiguity & experience working with global team across various time zones 
• Additionally, manage various workstreams and deliverables of the projects in Jira
• Proven track record and experience in mentoring junior team members
• Ability to drive decision making for highly technical solutions using Facts & Alternatives
Required Skills
• 6+ years of relevant experience in RDBMS, Big Data & NoSQL
• 4+ years of experience and knowledge about Spark, Hadoop & Hive Architecture and its components.
• Rock star SQL & Python skills especially plpgsql, jinja templating, dynamic SQL queries, ODBC connections and creating and maintaining stored procedures
• Should be strong in all DDL/DML statements/Queries and SQL Concepts
• Should be strong in Unix shell programming to perform any automation that project needs.
• Experience implementing data pipelines for feature engineering, scoring, evaluation, model health and model monitoring
• Expertise in distributed storage and compute with HDFS, S3, Hive or Spark
• Strong expertise in Unix shell programming and Airflow for building, automating and orchestrating data pipelines
• Experience with ETL and ELT pipelines in RDBMS, HDFS using tools like PySpark and Hive
• Hands-on knowledge of tools like Git (managing commits, issues, branches, merge conflicts, etc) and Docker (building images and executing workflows in containerized environments)
• Experience handling structured and semi structured dataset e.g., data from JSON and XML APIs
• Excellent problem solving, analytic skills, and written and oral English communication skills, stakeholder/expectation management skills
• Multitask and learn new systems quickly, the ability to communicate clearly and concisely in English through writing and orally is a key competency as is demonstrable skill as a self-starter
Preferred Skills
• Proven ability to work with varied data infrastructures – including relational databases, column stores, NoSQL databases, and file-based storage solutions
• Experience building and maintaining Graph database and Knowledge Graphs is a plus
• Ability to set up containerized services using e.g. Harmony, Kubernetes and Docker
• Some exposure to Power BI or other data visualization tools
• Exposure to machine learning or machine learning pipelines is a plus.