Data Engineer

About Capgemini Invent

Capgemini Invent believes difference drives change. As inventive transformation consultants, we blend strategic, creative, and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our clients' challenges, informed by science and data, superpowered by creativity and design, and underpinned by technology created with purpose.

Capgemini is a global business and technology transformation partner, helping organizations accelerate their dual transition to a digital and sustainable world. It is a responsible and diverse group of 340,000 team members in more than 50 countries with over 55 years of heritage.

Your Role

As a Data Engineer, you will be instrumental in designing, developing, and maintaining scalable data pipelines and ensuring data quality and governance across various layers. This role requires a strong technical background and a collaborative spirit to translate client requirements into robust technical solutions.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Snowflake.
• Write efficient and maintainable code in Python and PySpark.
• Develop and optimize SQL queries for data extraction and transformation.
• Implement data ingestion strategies in Snowflake using Snowpipe and COPY command to load data into base tables with Streams and Tasks.
• Extract data from various source systems and load into Snowflake using Python, PySpark scripts.
• Configure role-based access on Snowflake objects and process data sharing and cloning between Snowflake accounts.
• Create models in DBT and integrate data quality rules and tests to ensure data accuracy, consistency, and completeness.
• Implement data quality checks and monitoring, ensuring data accuracy and reliability.
• Collaborate with cross-functional teams to gather requirements and translate them into technical solutions.
• Ensure data quality, governance, and security across all pipelines and storage layers.
• Demonstrate hands-on experience with CI/CD tools and DevOps practices.

Requirements

• Proficiency in Python and PySpark.
• Strong SQL skills, including query optimization and performance tuning.
• Experience with cloud platforms like Snowflake, AWS, Azure, and GCP.
• Familiarity with data warehousing concepts and ETL processes.
• Experience with big data tools such as Hadoop, Spark, and Kafka.
• Experience with relational databases (e.g., Microsoft SQL Server, MySQL, PostgreSQL, Oracle) and NoSQL databases (e.g., Hadoop, Cassandra, MongoDB).
• Excellent problem-solving skills with an emphasis on sustainable and reusable development.

What You Will Love About Working Here

• Flexible Work Arrangements: We recognize the significance of flexible work, offering support for remote work or flexible hours to help you maintain a healthy work-life balance.
• Career Growth: Your career growth is at the heart of our mission. Our array of career growth programs and diverse professions are crafted to support you in exploring a world of opportunities.
• Skill Enhancement: Equip yourself with valuable certifications in the latest technologies, including Generative AI.