Job Description :

• The objectives of a Data Engineer typically revolve around designing, implementing, and managing robust data architectures and infrastructures. The Data Engineer will play a key role in driving the organization's data strategy, ensuring the efficient flow and storage of data, and supporting data-driven decision-making processes

  Duties and Responsibilities:

  1. Data Model Design:

  · Develop and maintain a scalable and efficient data models and schemas that support the storage, processing, and retrieval of data.

  2. Data Integration:

  · Implement robust ELT (Extract, Load, Transform) processes to integrate data from various sources into a unified and consistent format.

  · Ensure seamless data flow between different systems, databases, and applications.

  3. Data Quality:

  · Implement data validation and cleansing processes to ensure the accuracy and reliability of data.

  4. Emerging Technologies:

  · Stay abreast of emerging technologies and trends in the field of data engineering.

  5. Database Management:

  · Manage and optimize database systems, including relational databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra).

  6. Performance Optimization:

  · Identify and address performance bottlenecks in data processing and storage systems.

  · Optimize queries, indexing strategies, and data partitioning for improved performance.

  7. Scalability and Reliability:

  · Ensure that data systems are scalable to handle growing volumes of data.

  · Implement redundancy and fault-tolerance measures to enhance data system reliability.

  Mandatory Skills Required:

  · Strong hands-on expertise with AWS services such as S3, Glue, Lambda, Athena, Redshift, Step Functions, Streaming tools like Apache Kafka \ Amazon Kinesis.

  · Proficiency in Python, PySpark, experience with ELT and orchestration tools

  · Strong hands-on expertise on Apache Hudi \ Apache Iceberg

  · Strong hands-on expertise In SQL

  · Experience using GitHub for version control and collaboration

  · Develop, optimize, and maintain ELT processes for data integration.

  · Strong communication skills to interface with technical and business