Lead Data Engineer SQL AWS in Hyderabad, India - Jobeax
Vacancy description
Lead Data Engineer SQL AWS in Hyderabad, India
Two Circles
India, Hyderabad
Lead Data Engineer SQL AWS in Hyderabad, India is listed on Jobeax. Browse 30,000+ vacancies available.
We are a Sports & Entertainment Marketing business. We work with clients to help them understand & influence what their fans are doing – the way fans spend their money, the events that fans attend, the channels fans respond to, the content fans watch and more. And we use the understanding this gives us to help our clients grow. Our platforms and services are trusted by over 1000 clients globally, including the English Premier League, Red Bull, UEFA, VISA, the NFL, Nike, and Amazon. We are over 1000 people, based out of 15 offices, and we deliver work for sports and entertainment businesses of all shapes and sizes all over the world. Two Circles is looking for a Data Engineer to join our dynamic KORE Intelligence Platform team. In this role, you'll be a key part of our Partnership Intelligence Measurement Team that specializes in the ingestion, transformation, and storage of large-scale social and broadcast data. Working under the guidance of an experienced Data Engineers and Architects, you will gain hands-on experience designing and maintaining scalable, efficient data pipelines that support enterprise web applications, analytics, and business intelligence. We are seeking a Lead Data Engineer to join our Measurement team within the Partnership Intelligence Platform. In this role, you will provide technical leadership for the team while remaining hands-on with our data platform and pipelines. You will help define technical direction, lead the design and evolution of our data architecture, establish engineering best practices, and mentor other engineers. The role requires strong experience designing, building, and operating large-scale data pipelines and distributed data systems. Our technology stack includes AWS services, Airflow, Snowflake, MS SQL Server, and .NET. We are looking for someone who is passionate about data and infrastructure, comfortable making and communicating technical decisions, and interested in continuously improving our architecture, engineering practices, and technology stack. Provide technical leadership for the Data Engineering team, helping establish technical direction, engineering standards, and best practices while remaining hands-on with the platform.
Work as part of a cross-functional product team to deliver a high-quality SaaS based data platform.
Design and implement data pipelines with a clear focus on data quality and reliability.
Partner with both Product Management and stakeholders, as well as both technical and non-technical team members to deliver our vision, roadmap, and data strategy in addition to innovative client solutions.
Contribute to our DevOps culture and participate in ownership of our designs through production operations.
Support the ongoing maintenance and operations of the data platform.
Net / C#
Python
AWS or similar
Snowflake
~ SQL
Experience
8+ years of software engineering experience with a strong data focus, including designing, building, and operating production data systems in fast-paced environments.
Demonstrated experience providing technical leadership within an engineering team, including guiding technical direction, leading design discussions, and supporting engineers through complex technical decisions.
Experience designing and evolving data architectures and large-scale data pipelines, including making and communicating architectural decisions and trade-offs.
Experience mentoring engineers and helping raise engineering standards and technical capabilities across a team.
Experience leading technical initiatives that span multiple engineers, systems, or teams, from technical discovery and design through implementation and production operation.
Strong experience writing, debugging, optimizing, and reviewing database queries and data-processing workloads.
Strong experience with object-oriented programming and software engineering practices, preferably using Python and C#/.NET.
Strong experience building and operating cloud infrastructure and services, preferably AWS.
Experience with modern data platforms and orchestration technologies such as Snowflake and Airflow.
Experience establishing or improving engineering practices such as automated testing, CI/CD, observability, monitoring and alerting, code reviews, documentation, and production readiness.
Experience identifying opportunities for automation and implementing solutions that reduce repetitive operational work, improve developer experience, and increase platform reliability.
Experience collaborating with Product Management and technical and non-technical stakeholders and communicating complex technical topics, risks, and trade-offs clearly.
Strong experience designing, building, and operating data pipelines using orchestration and batch-processing technologies such as Apache Airflow.
Strong proficiency with Snowflake and MS SQL Server, including data modeling, query optimization, performance troubleshooting, and production operations.
Proficiency in Python and C#/.NET. Strong experience designing and integrating with APIs, including considerations such as scalability, rate limiting, retries, idempotency, error handling, and observability.
Strong experience with containerization technologies such as Docker and deploying containerized workloads in cloud environments.
Experience with monitoring and observability tools such as New Relic, including defining appropriate metrics, alerts, and operational monitoring for data pipelines and services.
Experience designing effective automated testing strategies, including unit, integration, and data-quality testing, using frameworks such as PyTest.
Strong understanding of distributed systems concepts relevant to data platforms, including concurrency, messaging, failure handling, retries, idempotency, consistency, and scalability.
Ability to evaluate existing architectures, identify technical risks and limitations, and propose pragmatic improvements or modernization strategies.
... analysis, modelling, implementation and evaluation. Define appropriate statistical, mathematical and machine learning approaches for complex business problems. Lead data exploration, cleaning, transformation and feature engineering to create reliable analytical datasets. Design, implement and optimise machine learning models, ...
... comprehensive data management expertise to Clinical Data Management (CDM) team to provide high quality data management products that meet customer needs. Provide leadership to the team in the areas of project planning, execution, and close-out; financial management; Perform role of Data Team Lead (DTL). Comply with Good Clinical ...
... model monitoring approaches - Strong understanding of data engineering and model integration patterns, including working with SQL, batch pipelines, streaming data, APIs, and application services - Familiarity with observability and operational tooling such as Azure Monitor, Application Insights, MLflow tracking, Datadog, ...
... data science and ML, with at least 2 years leading projects or teams in a pharma consulting or client-facing setting. - A Bachelor's or Master's degree in engineering, statistics, mathematics or a relevant quantitative field. - Experience with life sciences data, such as claims (Komodo, IQVIA, Symphony), specialty pharmacy, ...
... using advanced statistical techniques and machine learning frameworks. Data Analysis: Conduct thorough data exploration, cleaning, and transformation to prepare datasets for modeling efforts, ensuring high data quality and accuracy. Collaboration: Work with cross-functional teams, including engineering, product management, ...
Work Experience Required: 8 - 12 Years - Experience in programing (Python, R, SQL, NoSQL,Spark) with ML tools & Cloud Technology (AWS, Azure, GCP) - Experience in Python libraries such as numpy, pandas, scikit-learn, tensor-flow, scapy, scrapy, BERT etc. - Good understanding in statistics, and ability to design statistical ...
... explainability- Establish data governance framework (catalog, lineage, ownership)- Ensure data quality, consistency, and compliance- Work closely with business leaders to translate requirements into data solutions- Lead cross-functional teams (data engineers, analysts, data scientists)- Manage vendors, partners, and external ...
... is later, Headout shall either delete your data or inform you that it shall keep it in its database for future roles. In compliance with the relevant privacy laws, you have the right to request access to your personal data, to request that your personal data be rectified or erased, and to request that the processing of ...
... experience as a Data Analyst, with strong focus on Power BI. - Proficiency in SQL for data extraction, manipulation, and analysis. - Strong expertise in building data models, DAX calculations, and Power Query transformations. - Experience working with large datasets, data warehouses, and cloud data platforms (Azure, AWS, GCP). ...
... completeness, and consistency.- Build and maintain self-serve dashboards and reports in tools such as Looker, Metabase, or equivalent BI platforms for product and leadership teams.- Write clean, well-documented SQL and Python/R queries for complex datasets; contribute to the analytics data layer (db models, etc.).Cross-functional ...
... Why Join Lascade? - Help shape business decisions using data. - Work across multiple global software products. - Collaborate directly with Product, Growth, Engineering, and Leadership teams. - High ownership with real business impact. - Opportunity to build the company’s data and analytics function from the ground up.
... understand data requirements and deliver solutions. Identifying opportunities for process improvements based on data analysis. Assisting in the development of data models and algorithms to enhance business decision-making. Skills and Qualifications Proficiency in data analysis tools such as Python, R, or SQL. Experience ...
... numpy, pandas, pyspark, matplotlib, Flask, FastAPI, TensorFlow, PyTorch, scikit-learn. - Strong skills in software engineering, data processing, analytics and data visualization libraries - Proficiency in data structures, algorithms, building and maintaining data pipelines. - Experience with cloud platforms such as AWS, ...
... analysis or a similar role. - Proficiency in Databricks and Spark for big data processing. - Strong SQL skills for data querying and manipulation. - Experience with data visualization tools like Tableau or Power BI. - Knowledge of Python or R for data analysis and scripting. - Familiarity with cloud platforms such as AWS or Azure. ...
... custom metadata, and data governance. Proficiency implementing BigQuery SQL procedures, functions, and similar, with actionable logging Optimize ETL processes and data workflows to ensure abiding by performance constraints, for efficiency and scalability. Comfortably lead designing and building of development to production data ...
... investigations. Design scalable cloud solutions using AWS and/or Azure . Ensure data quality, governance, reliability, monitoring, and compliance. Partner with Data Scientists, Fraud Analysts, and Engineering teams to deliver data-driven solutions. 4+ years of experience in Data Engineering or Big Data Engineering. - 2+ years ...
... for an experienced Data Engineer to design, develop, and maintain scalable data platforms and pipelines using AWS, Apache Spark, Python, SQL, Kafka, and modern data engineering frameworks . The role will focus on building reliable data solutions for high-volume batch and real-time workloads, including data ingestion, transformation, ...
... preferred. Experience with modern web development technologies (e.g., React, Angular, https://jobeax.com/link/tzGekrj9a8ToGEDv) is a plus. Good understanding of SQL and NoSQL databasesUnderstanding of cloud computing concepts and experience using any major cloud service provider (AWS/GCP/Azure). Proven track record of delivering ...
... pipelines, document parsing, and text processing on large-scale unstructured data.- Strong NLP / extraction-focused ML depth : transformers, embeddings, vector databases, RAG, LLM integrations, and agentic workflows.- Hands-on experience with AWS (SageMaker, Bedrock, EC2, Lambda) and Azure (AI Foundry, Azure OpenAI) for model ...
... pipelines, document parsing, and text processing on large-scale unstructured data.- Strong NLP / extraction-focused ML depth : transformers, embeddings, vector databases, RAG, LLM integrations, and agentic workflows.- Hands-on experience with AWS (SageMaker, Bedrock, EC2, Lambda) and Azure (AI Foundry, Azure OpenAI) for model ...