Senior Data Engineer

Sand Technologies is a fast-growing enterprise AI company that solves real-world problems for large blue-chip companies and governments worldwide.

We're pioneers of meaningful AI: our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on the world.

ABOUT THE ROLE

Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark. Our data engineers create pipelines that support our data scientists and power our front-end applications. This means we do data-intensive work for both OLTP and OLAP use cases.

JOB SUMMARY

A Senior Data Engineer has the primary role of designing, building, and maintaining scalable data pipelines and infrastructure to support data-intensive applications and analytics solutions. In this role, you will be responsible for developing data pipelines, designing data architectures, and overseeing data engineering projects.

RESPONSIBILITIES

• Lead the design, implement, and maintain scalable data pipelines for ingesting, processing, and transforming large volumes of data.
• Architect scalable and efficient data solutions using appropriate architecture design.
• Design and optimize data models and schemas for efficient storage, retrieval, and analysis of structured and unstructured data.
• Develop, optimize and automate ETL workflows to extract data from diverse sources.
• Utilize big data technologies such as Spark, Kafka, and Flink for distributed data processing and analytics.
• Deploy and manage data solutions on cloud platforms such as AWS, Azure, or GCP.
• Implement and oversee data governance, quality, and security measures.
• Monitor data pipelines and infrastructure performance, identify bottlenecks and optimize for scalability.
• Build and maintain basic CI/CD pipelines and deploy data solutions.
• Collaborate with cross-functional teams to understand requirements and deliver data-driven solutions.
• Create and maintain technical documentation for data solutions.
• Stay current with emerging technologies and best practices in data engineering.
• Mentor and guide junior and mid-level data engineers.
• Evaluate and recommend technologies that best suit project requirements.
• Optimize software performance and efficiency through architectural design decisions.

QUALIFICATIONS

• Proven experience as a Senior Data Engineer or in a similar role.
• Proven experience working with Big Data and tools used to process Big Data.
• Strong problem-solving and analytical skills.
• Excellent understanding of data engineering principles and practices.
• Excellent communication and collaboration skills.
• Ability to write clean, scalable code using Python or similar programming languages.
• Knowledge of data governance frameworks and practices.
• Understanding of machine learning workflows.

DESIRABLE LANGUAGES/TOOLS

• Proficiency in programming languages such as Python, Java, Scala, or SQL.
• Strong understanding of data modelling concepts.
• Experience in big data technologies such as Databricks, Spark, Kafka, and Flink.
• Experience with modern data architectures, such as lakehouse.
• Experience with CI/CD pipelines and containerization (e.g., Docker).
• Experience with ETL tools and technologies.
• Strong understanding of data governance and best practices.
• Experience with cloud platforms such as AWS, Azure, or GCP.

Would you like to join us as we work hard, have fun and make history?