Data Engineer

Job description Position - Data Engineer Experience Level: Mid-Senior Level Experience Required: 3+ Years Overview: We are seeking a highly skilled and motivated Data Engineer to join our team. In this role, you will collaborate with cross-functional teams to design, build, and maintain scalable data platforms and solutions on the AWS Cloud. You will leverage your expertise in data engineering tools and technologies to deliver next-generation application data platforms and optimize current implementations.

The ideal candidate should have a strong background in Databricks, Spark, and Big Data ecosystems, along with experience in data warehousing, including datamarts and data modeling. Key Responsibilities: • Develop and maintain scalable and high-performance data pipelines using AWS Glue, EMR, Databricks, and Spark. • Design and implement robust ETL processes and frameworks to integrate, process, and analyze large datasets. • Build and optimize data models for structured and semi-structured data to support reporting, analytics, and machine learning workflows. • Utilize Python, PySpark, and SQL to develop and optimize data transformation logic. • Collaborate with stakeholders to understand business requirements and translate them into technical solutions. • Implement best practices for data governance, security, and performance optimization on AWS Cloud platforms. • Work with Big Data ecosystems, including Hadoop, Hive, Sqoop, and HDFS, to process and manage large datasets. • Design and develop streaming data solutions using Spark Streaming, Kinesis, and Firehose. • Contribute to the architecture and strategy for modernizing and scaling data platforms. Required Skills & Experience: • 3-5 years of hands-on experience as a Data Engineer. • Proficiency in Python, SQL, and PySpark. • Strong knowledge of Big Data ecosystems, including Hadoop, Hive, Sqoop, HDFS, and HBase. • Expertise in the Spark ecosystem: Spark Core, Spark Streaming, Spark SQL, and Databricks. • Solid experience with AWS cloud services, including EMR, EC2/EKS, Lambda, Glue, and S3. • In-depth understanding of data modeling, data warehousing methodologies, and ETL processes. • Familiarity with data governance, quality, and security principles in cloud environments. • Excellent problem-solving skills and ability to work independently or collaboratively in a fastpaced environment.

Please revert me back with your confirmation mail and updated CV!!