Lead Data Engineer SQL AWS in Hyderabad, India - Jobeax
Vacancy description
Lead Data Engineer SQL AWS in Hyderabad, India
Two Circles
India, Hyderabad
Lead Data Engineer SQL AWS in Hyderabad, India is listed on Jobeax. Browse 30,000+ vacancies available.
We are a Sports & Entertainment Marketing business. We work with clients to help them understand & influence what their fans are doing – the way fans spend their money, the events that fans attend, the channels fans respond to, the content fans watch and more. And we use the understanding this gives us to help our clients grow. Our platforms and services are trusted by over 1000 clients globally, including the English Premier League, Red Bull, UEFA, VISA, the NFL, Nike, and Amazon. We are over 1000 people, based out of 15 offices, and we deliver work for sports and entertainment businesses of all shapes and sizes all over the world. Two Circles is looking for a Data Engineer to join our dynamic KORE Intelligence Platform team. In this role, you'll be a key part of our Partnership Intelligence Measurement Team that specializes in the ingestion, transformation, and storage of large-scale social and broadcast data. Working under the guidance of an experienced Data Engineers and Architects, you will gain hands-on experience designing and maintaining scalable, efficient data pipelines that support enterprise web applications, analytics, and business intelligence. We are seeking a Lead Data Engineer to join our Measurement team within the Partnership Intelligence Platform. In this role, you will provide technical leadership for the team while remaining hands-on with our data platform and pipelines. You will help define technical direction, lead the design and evolution of our data architecture, establish engineering best practices, and mentor other engineers. The role requires strong experience designing, building, and operating large-scale data pipelines and distributed data systems. Our technology stack includes AWS services, Airflow, Snowflake, MS SQL Server, and .NET. We are looking for someone who is passionate about data and infrastructure, comfortable making and communicating technical decisions, and interested in continuously improving our architecture, engineering practices, and technology stack. Provide technical leadership for the Data Engineering team, helping establish technical direction, engineering standards, and best practices while remaining hands-on with the platform.
Work as part of a cross-functional product team to deliver a high-quality SaaS based data platform.
Design and implement data pipelines with a clear focus on data quality and reliability.
Partner with both Product Management and stakeholders, as well as both technical and non-technical team members to deliver our vision, roadmap, and data strategy in addition to innovative client solutions.
Contribute to our DevOps culture and participate in ownership of our designs through production operations.
Support the ongoing maintenance and operations of the data platform.
Net / C#
Python
AWS or similar
Snowflake
~ SQL
Experience
8+ years of software engineering experience with a strong data focus, including designing, building, and operating production data systems in fast-paced environments.
Demonstrated experience providing technical leadership within an engineering team, including guiding technical direction, leading design discussions, and supporting engineers through complex technical decisions.
Experience designing and evolving data architectures and large-scale data pipelines, including making and communicating architectural decisions and trade-offs.
Experience mentoring engineers and helping raise engineering standards and technical capabilities across a team.
Experience leading technical initiatives that span multiple engineers, systems, or teams, from technical discovery and design through implementation and production operation.
Strong experience writing, debugging, optimizing, and reviewing database queries and data-processing workloads.
Strong experience with object-oriented programming and software engineering practices, preferably using Python and C#/.NET.
Strong experience building and operating cloud infrastructure and services, preferably AWS.
Experience with modern data platforms and orchestration technologies such as Snowflake and Airflow.
Experience establishing or improving engineering practices such as automated testing, CI/CD, observability, monitoring and alerting, code reviews, documentation, and production readiness.
Experience identifying opportunities for automation and implementing solutions that reduce repetitive operational work, improve developer experience, and increase platform reliability.
Experience collaborating with Product Management and technical and non-technical stakeholders and communicating complex technical topics, risks, and trade-offs clearly.
Strong experience designing, building, and operating data pipelines using orchestration and batch-processing technologies such as Apache Airflow.
Strong proficiency with Snowflake and MS SQL Server, including data modeling, query optimization, performance troubleshooting, and production operations.
Proficiency in Python and C#/.NET. Strong experience designing and integrating with APIs, including considerations such as scalability, rate limiting, retries, idempotency, error handling, and observability.
Strong experience with containerization technologies such as Docker and deploying containerized workloads in cloud environments.
Experience with monitoring and observability tools such as New Relic, including defining appropriate metrics, alerts, and operational monitoring for data pipelines and services.
Experience designing effective automated testing strategies, including unit, integration, and data-quality testing, using frameworks such as PyTest.
Strong understanding of distributed systems concepts relevant to data platforms, including concurrency, messaging, failure handling, retries, idempotency, consistency, and scalability.
Ability to evaluate existing architectures, identify technical risks and limitations, and propose pragmatic improvements or modernization strategies.
... architects, analysts, developers, and business stakeholders to understand data requirements.- Ensure data quality, security, governance, and best practices across data engineering https://jobeax.com/link/Gpaa4oyRTNYXQU0V Stack:- Python, PySpark, SQL, AWS Glue, AWS Glue Data Catalog, AWS Lambda, Amazon S3, AWS Step Functions, ...
... BigQuery, Cloud Functions, Composer, GCS Proficient hands-on programming experience in Spark/Scala (python/java) Proficient in building production level ETL/ELT data pipelines from data ingestion to consumption Data Engineering knowledge (such as Data Lake, Data warehouse - Redshift/Hive/Snowflake, Integration, Migration) ...
Hello, We are hiring for 'Data Engineer with Java' for Pune Location. Exp :5+Years Loc : Pune Notice Period: Immediate joiners(notice period served or serving candidate) Mandatory Skills: Data Engineering Airflow Data Build Tool(DBT) Java ETL/ELT Pipeline CI/CD SQL NOTE: We are looking for Immediate joiners(notice period ...
... expectations are met through disciplined implementation (access controls, lineage, and quality standards) 5+ years building and operating production data pipelines (data engineering—not primarily BI/analytics or data science) - Strong SQL plus strong data modeling skills (dimensional and/or lakehouse modeling) - Hands-on Databricks ...
Job Summary :The Data Integration Engineer will design, develop, and support data integration solutions that extract financial and operational data from enterprise source systems and deliver trusted data to analytics platforms and external vendors. The role emphasizes source-to-target mapping, transformation, validation, ...
... experience in Data Engineering.- Strong Python + SQL expertise.- Strong Data Architecture and System Design skills.- End-to-end production pipeline ownership.- Expert SQL including Window Functions, CTEs, CASE logic.- Strong experience with MySQL / PostgreSQL.- Experience designing Transaction / Fact Tables.- Strong data quality, ...
... Prometheus, Grafana, ELK, Datadog Agile Tools: Jira, Confluence Additional Expectations The candidate should be a strong team player leading a team with an ownership mindset, capable of independently managing AWS environments, automation frameworks, deployments, and collaborating with global engineering/customer teams.
... frameworks, or extensible platform systems. Backend https://jobeax.com/link/JSd7UYCSg572jJDY, Python, Java, Ruby Frontend React, TypeScript, Ant Design System, Figma AWS, Kubernetes (EKS), S3 Data MySQL, Snowflake, Aerospike, DynamoDB, ScyllaDB AI Tooling CI/CD, TDD, Agile, You Build It You Run It AI-native engineering culture ...
... proven track record of building and operating large-scale, distributed, API-driven systems in production. Expertise in observability, alerting, and reliability engineering , using tools such as Prometheus, Grafana, Datadog, Splunk, ELK, or equivalent ecosystems. Strong command of cloud platforms and open systems ( AWS, Azure, ...
... operations at ELGi. Data Engineering & Architecture Design scalable, production-grade ETL/ELT pipelines across Azure, AWS, and GCP. Develop modular, reusable data engineering frameworks for ingestion, transformation, and orchestration. Build Spark, SQL, and Python workflows in Databricks. Optimize compute, storage, and ...
... least one certification • Databricks Certified Data Engineer Associate OR Databricks Certified Data Engineer Professional Additional Certifications (Preferred) - Databricks Certified Associate Developer for Apache Spark - Cloud platform certifications (Azure Data Engineer Associate, AWS Certified Data Analytics, or Google Cloud ...
... maintain scalable data pipelines and distributed systems. The ideal candidate should have hands-on experience with ETL/ELT pipelines, workflow orchestration, AWS cloud services, data visualization, distributed architectures, and modern data engineering https://jobeax.com/link/TtkK7BHPbancvnFq Responsibilities:- Design, ...
... using Python and PySpark to ingest data from files, APIs, RDBMS, NoSQL, and message queues into AWS data stores Develop and optimize PySpark jobs for large-scale data processing, transformation, and aggregation on AWS EMR or Databricks Build and manage AWS-native data workflows using S3, Glue, Lambda, Redshift, and Step Functions; ...
... Senior Data Engineer with strong hands-on experience building and supporting data solutions on AWS . The role requires solid experience with Amazon Redshift, S3, AWS Glue, Amazon AppFlow, Lambda, and SQL . You will work on data ingestion, transformation, data pipelines, data warehouse solutions, and integrations across different ...
... Data Factory, Databricks, Synapse, dbt (any two – Mandatory). Data Warehousing: Azure SQL Server/Redshift/Big Query/Databricks/Snowflake (Anyone - Mandatory). Data Visualization: Looker, Power BI, Tableau (Basic understanding to support stakeholder queries). Cloud: Azure (Mandatory), AWS or GCP (Good to have). SQL and Scripting: ...
... entity resolution and record linkage. Strong SQL and Python programming skills. Experience with ETL/ELT pipeline development. Familiarity with cloud platforms like AWS, Azure, or GCP. Knowledge of data engineering best practices and scalable architectures. Experience with Apache Spark or PySpark. Knowledge of Docker, Kubernetes, ...
... working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases and Cloud Data warehouse like AWS Redshift.- Experience in creating scalable, efficient schema designs to support diverse business needs.- Experience with database normalization, ...
... principles applied to data and AI systems Ability to operate across the full stack — from raw data through model serving infrastructure 7+ years of progressive data engineering experience, with demonstrated growth in scope and technical ownership. Track record of leading technical projects or workstreams, not just executing ...
... within your team. - Access to world-class learning platforms, continuous career progression, and opportunities to lead high-impact global projects. - Degree in Engineering, Computer Science, Information Technology, Data Science, or a related discipline. Formal training or certification in Data provisioning will be preferred. ...
Role :Lead Engineer - Conversion/Migration Skills - Mandatory-SQL Server, ETL/Data Migration Skills - Good to have -SSIS, Azure Data Factory Work location: Kochi/Trivandrum Total Experience: 6-9 years Strong hands-on expertise in SQL Server (T-SQL) and relational database design, with solid experience in data migration ...