4,051 big data developer jobs in India.

... Own the data platform roadmap end to end — ingestion, enrichment, matching, vertical data models, and the sync between our warehouse, CRM, and products - Turn ambiguous data problems into fully-specced, buildable projects - Prioritize ruthlessly across a large backlog: which verticals, which data sources, which quality fixes ...

... activities. - Experience building AI-assisted tools, workflows, or agents that integrate into engineering pipelines. Bonus Skills​ - Experience testing against cloud data warehouses (Snowflake, BigQuery, Redshift) or CDC source databases (Oracle, SQL Server, PostgreSQL, MySQL) - Familiarity with log-based CDC mechanisms (binlog, ...

... Purpose/ Summary We are looking for a skilled Data Engineer with 1-2 years of experience in designing, building, and maintaining scalable data pipelines and data platforms. The ideal candidate should have strong expertise in SQL, Databricks, and Azure Data Factory (ADF. and hands-on experience working with large datasets ...

... Preparation Product Area, focused on building the next generation of finance data and processing services The team builds on a lakehouse architecture using Azure Databricks, Azure Data Factory and Azure Data Lake (Delta Lake) with Python being the main programming language and PySpark is used for large scale data engineering. ...

Position: Senior Data Analyst (Tariff Data)Overall/Total Experience: 4-7 years onlyLocation: Pune (Only for Candidates living in Pune, Mumbai, Nagpur)Working Days: 5 Days Working from OfficeNotice Period Requirement: 30 Days (Maximum)Client's Company Size: Mid-SizedRoles and responsibilities:- Understands complex utility ...

... observability for data systems. - Knowledge of data governance, metadata/catalogue tools, lineage and data quality frameworks (i.e. Strong grasp of security, data privacy and regulatory requirements (e.g., GDPR, data residency). - Professional certification such as Databricks or AWS (Solutions Architect / Specialty). Consulting ...

... of experience working in GCP-based Big Data deployments (both Batch and Real-Time), specifically leveraging BigQuery, Bigtable, Google Cloud Storage, Pub/Sub, Data Fusion, Dataflow, Dataproc, and Airflow. Data Design: Proven history of designing and delivering comprehensive data lake and data warehousing solutions. Data ...

... - Solid understanding of data pipeline fundamentals, including ingestion, transformation, and loading (ETL/ELT) processes. - Must have experience with batch data processing (scheduling, dependencies, failure recovery, performance tuning). - Knowledge of data warehousing concepts and familiarity with relational databases. ...

... etc. Bachelor's or Master's degree in Computer Science/Information Systems. At least 4+ years of experience in building data flow and data management on a modern big data tech stack. Strong experience in using ETL frameworks (e. g., Airflow, Jenkins) to build and deploy production-quality ETL pipelines. Knowledge of data structures. ...

... support Oracle-to-Apache Cassandra data migration activities, ensuring data integrity, consistency, and minimal disruption to business services. Own the Cassandra data-layer architecture and design supporting high-volume, real-time payment transaction processing. Take end-to-end responsibility for keyspace design, data modelling, ...

... frameworks for data integration and ETL processes, such as Apache Kafka, Apache Spark, Apache Airflow, or Talend. - Data Modeling and Database Design: Knowledge of data modeling concepts, dimensional modeling, and schema design to create efficient and scalable database structures. - Big Data Technologies: Experience with big ...

... governance principles, data quality checks on each data layer. - A successful history of manipulating, processing and extracting value from large disconnected datasets. - Working knowledge of message queuing, stream processing, and highly scalable big data data stores. - Strong project management and organizational skills. ...

... and mitigate data risks throughout the data lifecycle, including protection, retention, storage, use, and quality l Partner with technology teams to capture data sources, formats, and data flows so that data can be validated for downstream analytics and reporting l Investigate and document potential data quality issues, ...

... vector-based data pipelines for AI and GenAI use cases Develop scalable batch and streaming data pipelines using cloud-based data platforms (e.g., Snowflake or Databricks) Create and evolve semantic data models that transform raw data into analytics-ready, trustworthy datasets Build data preprocessing, validation, and quality-assurance ...

... architectures like RNNs and LSTM , BERT Should have worked with cognitive services from major cloud platforms like AWS and have a working knowledge of SQL and no-SQL databases. Ability to create data and ML pipelines for more efficient and repeatable data science projects using MLOps principles Keep abreast with new tools, algorithms ...

... factual accuracy, or comparing responses — when projects are available. While each project involves unique tasks, contributors may: - Carefully review provided data (text, images, or videos); - Label or classify content based on project guidelines; - Identify and flag factually incorrect, sensitive, inappropriate, or unclear ...

... shape. Senior Data Scientist We have an opportunity for a Senior Data Scientist to join our Data Science team in Bangalore, reporting to the Senior Director, Data Science. In this pivotal role, you will drive advanced analytics and machine learning solutions that power our security, networking, and cloud data products. ...

... restore testing. Execute recovery operations and point-in-time recovery, and take part in disaster recovery drills and documented failover exercises. Manage data movement activities including Data Pump exports and imports, non-production refreshes, and database cloning. Mentor junior engineers, review their work and raise ...

... Perform data cleaning and preprocessing to prepare datasets for analysis. Write complex SQL queries to extract, manipulate, and analyze data from relational databases. Work with cloud platforms (e.g., AWS, Azure, Google Cloud) to store, manage, and analyze data. Use statistical techniques and software to analyze datasets, ...

... object detection, segmentation, image generation, audio, and video analysis. Generative AI: Proficiency with SaaS LLMs, including Lang chain, llama index, vector databases, Prompt engineering (COT, TOT, ReAct, agents). Experience with Azure OpenAI, Google Vertex AI, AWS Bedrock for text/audio/image/video modalities. Familiarity ...