Data Analysis Programmer in Gurugram, India is listed on Jobeax. Browse 30,000+ vacancies available.
Responsibilities:
Ingest data from various internal and external sources into AWS Redshift and S3 buckets using methods such as DMS, Zero-ETL, Kinesis, Glue, Lambda, Lake Formation, Cross Account Replication, and SFTP.
Build infrastructure as code using AWS CDK.
Create and maintain GitLab CI/CD pipelines for code promotion through test and production environments.
Design, build, and maintain AWS Glue ETL pipelines for structuring and curating data.
Conduct code reviews on all deployments to ensure quality and best practices.
Maintain AWS environments to optimize costs, reduce vulnerabilities, and ensure smooth operations.
Coordinate or resolve production issues in a timely manner.
Drive path-to-production processes, including proper documentation and approvals.
Collaborate with business teams on data governance and compliance initiatives.
Identify and implement best practices for data ingestion, data design, and data quality improvement.
Develop queries for profiling data, validating analyses, testing assumptions, and driving data quality assessment.
Optimize query performance using indexing, materialized views, and other techniques.
... teams Experience with building, running and monitoring systems on AWS, with a focus on capacity management and troubleshooting platforms at scale Exposure to data processing and/or analysis - our systems produce and process a large amount of data points every day Atlassian offers a wide range of perks and benefits designed ...
We are looking for a skilled Data Engineer with strong experience in Python and AWS cloud technologies. The ideal candidate should have hands-on knowledge of building automation frameworks, managing test scripts, and working with various AWS services. This role requires good technical understanding, strong problem-solving ...
... warehouse - Translate user requirements for reporting and analysis into actionable deliverables - Enhance automation, operation, and expansion of real-time and batch data Manage numerous projects in an ever-changing work environment - Extract, transform, and load complex data into the data warehouse using cutting-edge technologies ...
... LLMs, AutoGen, OCR, Databricks, PySpark, Python, Azure DevOps, and enterprise AI platforms. The role requires strong hands-on experience across SQL Server, Azure Data Factory, Azure Data Lake, Microsoft Fabric, Logic Apps,Power BI, GitHub, and Data Modeling. Required Skills: - 10+ years of Software Engineering / Data Engineering ...
... performant Use AI tooling within DE workflows, including code generation, pipeline automation, and data quality checks Contribute to and help shape company-wide data governance standards Collaborate with analytics, BI, and business teams to deliver trusted, well-modeled data 5–7 years of hands-on data engineering experience ...
... security, and scalability while enabling faster and more reliable software delivery. Key Responsibilities: - Hands on experience working on Databricks, creating data pipelines, transformation rules using Unity Catalog, PySpark, SparkSQL - Hands on Databricks Devs – Deployments, CICD, Performance Management - Hands on Databricks ...
... data ecosystems. Lead the design, architecture, and delivery of large-scale data engineering solutions across cloud and hybrid environments. Drive end-to-end data platform implementations encompassing data ingestion, transformation, storage, processing, governance, and consumption. Architect batch and real-time data pipelines ...
... tool-call auditing, and failure mode analysis for production agents such as FinOps Sentinel and Governed Analytics. Manage agent state and memory persistence using Databricks-native storage (Delta Lake, Unity Catalog) and integrate with external databases (CosmosDB, Neo4j) as required by agent workflows. Collaborate with AI Engineers ...
... for a data professional to join our Data Engineering team. Independently execute data engineering projects Undertake processing of structured and unstructured data Development of data processing codes for automation & building scalable solutions Develop and maintain data solutions for clients / projects Communicate and interact ...
... Data Engineer to drive our mission to unlock potential of data assets by consistently innovating, eliminating friction in how users access data from its Big Data repositories and enforce standards and principles in the Big Data space. The candidate will be part of an exciting, fast paced environment developing Data Engineering ...
... maintain scalable data pipelines and data lake solutions on GCP. Build and automate data ingestion and transformation workflows using Airflow and BigQuery. Ensure data quality, performance, and security across all cloud environments. Collaborate with cross-functional teams to translate business needs into efficient data models. ...
... https://jobeax.com/link/fYGlpInxuIv5PeS1 & Skills :- 4 - 7 years in data engineering: SQL, Python, ETL/ELT orchestration.- Cloud data platform experience (Azure preferred): pipelines, storage, APIs.- Strong understanding of data lake and data architecture.- Data modelling for analytics (star schema) and data quality frameworks. (ref:hirist.tech)
... efficiency . Integrate data from multiple internal and external systems while maintaining data consistency and reliability. Develop Python-based automation and data-processing solutions. Monitor production pipelines, troubleshoot failures, and perform root-cause analysis. Follow modern software engineering practices including ...
... Docker, Cloud-Native Infrastructure, CI/CD pipelines. Industry Context: Data engineering experience in Banking Risk, Retail Products, Cards, Mortgage, Deposits, or Wealth Management. Assumed Requirements / Certifications: Databricks Certified Data Engineer, AWS Certified Data Analytics, or Azure Data Engineer Associate.
... capture the most granular data. Align the engineering and product teams on the Data Granularity and accuracy requirements. Build Alerts to detect bugs in the data pipeline. Perform data analysis to test hypothesis and uncover factor influencing metric variations Build Feature Store for Data Science implementations Overall ...
... inquiries related to Addepar's portfolio data feeds and on general data product functionalities within established SLAs. Manage and complete requests from internal Data teams that require client outreach and/or action to resolve data verification issues. Investigate client reported bugs and data processing issues, and triage ...
... validate data from multiple sources Perform exploratory data analysis (EDA) Develop dashboards and reports using BI tools Interpret trends and patterns in complex datasets Present insights to stakeholders in a clear and actionable manner Write SQL queries to extract and manipulate data Automate data processes where possible ...
... evaluation scenarios for MLE Bench. Minimum 3+ years of experience as a Data Analyst or Analytics-focused Engineer . Strong proficiency in Python for data analysis. Solid experience with SQL and relational datasets. Experience analyzing ML outputs and evaluation metrics . Ability to work with large, complex datasets and ...