2,276 big data testing jobs in India.

... services industry! We are hiring a Big Data Engineer to help us build and support our data and analytics delivery pipeline. Design, develop, and optimize scalable data ingestion, transformation, and analytics pipelines for IoT, operational, and business data using Spark and Delta Lake technologies. - Create and tune Spark transformation, ...

• Remote

... stakeholders to deliver scalable data solutions. Follow best practices for data security, governance, testing, version control, and CI/CD 7+ years of experience in Data Engineering. - Strong hands-on experience with Databricks . - Strong expertise in PySpark and big data processing. - Strong programming skills in Python . - Advanced ...

... HudiData Integration & Streaming:- Apache Kafka, Apache Flink, Apache Airflow, Azure Data Factory, AWS Glue, Stream Processing FrameworksCloud Platforms:- Azure (Data Factory, Synapse, Data Lake, Fabric, Databricks), AWS (S3, Glue, EMR, Redshift, Kinesis, Lambda), GCP (BigQuery, Dataflow, Dataproc, Pub/Sub)Databases:- PostgreSQL, ...

... scalable secure and high performance data platforms and architectures Lead the development of batch and real time data processing solutions using modern cloud and big data technologies Define data engineering standards best practices and governance frameworks Drive architectural decisions for data lakes lakehouses data warehouses ...

... and processing - Experience with big data and real-time processing technologies, including Apache Spark, Hadoop, Kafka, and Amazon Kinesis - Experience with data modelling and database design across Structured Query Language and NoSQL platforms - Understanding of data governance, data quality, and security best practices ...

... processing data with a massively parallel technology (such as Redshift, Teradata, Netezza, Spark or Hadoop based big data solution) experience 2+ years of relational database technology (such as Redshift, Oracle, MySQL or MS SQL) experience 2+ years of developing and operating large-scale data structures for business intelligence ...

• Full-time

... seeking a seasoned Senior Data Scientist with e-commerce expertise to join our dynamic team. Must-Have Skills: Machine Learning Algorithms, Python or R Programming, Big Data Technologies (Hadoop, Spark, or equivalent), Data Visualization Tools (Tableau, Power BI, or similar), SQL and Data Modeling Expertise. Data Analysis & Insights ...

• Hybrid

... compelling and clear business focused insights. Can work with multiple databases and big data solutions. Work with the data engineering team to develop & maintain data pipelines. Identify, design, and implement internal process improvements and tools to automate data processing and ensure data integrity while meeting data security ...

... HudiData Integration & Streaming:- Apache Kafka, Apache Flink, Apache Airflow, Azure Data Factory, AWS Glue, Stream Processing FrameworksCloud Platforms:- Azure (Data Factory, Synapse, Data Lake, Fabric, Databricks), AWS (S3, Glue, EMR, Redshift, Kinesis, Lambda), GCP (BigQuery, Dataflow, Dataproc, Pub/Sub)Databases:- PostgreSQL, ...

... cloud-based data platforms and services, preferably Google Cloud Platform (GCP ) Hands-on experience with GCP managed services such as: Google Cloud Storage, Dataproc, Dataflow, Pub/Sub, Cloud Functions, Cloud Composer, BigQuery, BigTable . Required Skills - 4–8 years of experience i n Data Engineering / Big Data Engineering ...

• On-site

... (Pandas, Polars) and modern big data tools such as Databricks (Spark), Flink, or Kafka. - Strong understanding of data pipeline development, ETL frameworks, and data lakes/warehouses. - Extensive experience with cloud data platforms such as AWS (S3, Redshift, Lambda, Glue), GCP (BigQuery, Dataflow, Dataproc), or Snowflake. ...

... complex business needs. Ensure the highest levels of data quality and integrity through rigorous data validation and cleansing processes. Collaborate closely with data scientists and analysts to provide clean, structured data for advanced analytics and machine learning models. Oversee and manage big data technologies and frameworks, ...

• On-site

5k

... Architecture of and administration of Big Data platforms - Data Warehouse Appliances , Open Data Lakes (AWS EMR, HortonWorks), Data Lake Technologies (AWS S3/Databricks/Other) and experience with ML and Data Science platforms (Spark ML , H2O , KNIME) - Develop and manage SPARK ETL Frameworks, Data orchestration with Airflow ...

• Remote
• Hybrid

$3200 USD

... that help the product engineering team deliver features in a scalable and performant manner - Design, build, and support applications from End to End. - Make data-driven technical decisions - Apply your technical knowledge to debug and solve challenging technical problems - Identify and resolve ambiguities in requirements ...

... Java, JavaScript, or Python OR equivalent experience. Experience building scalable and highly available cloud services spanning multiple regions and/or clouds. Data modeling and big data processing experience Experience in work management and asset inventory systems #This position will be open for a minimum of 5 days, with ...

... and build scalable data processing systems. The ideal candidate should have hands-on experience working with large datasets, distributed computing, and modern data architectures. Key Responsibilities Design, develop, and maintain big data pipelines using Hadoop and Spark Build scalable data processing frameworks using Scala ...

• On-site

We are looking for a Big Data Architect with experience designing scalable data solutions using Agile/SAFe practices, data mesh architecture, real-time streaming, cloud platforms, and modern data engineering technologies. The candidate should have a strong DataOps mindset and the ability to design solutions for testability, ...

Rs 4 - 8 lakhs p.a.

... work experience. - Experience in building scalable products with preferably big data. - Excellent Python coding skills (Mandatory) - Experience in Apache spark, Data Lake and other Big data technologies. - Experience in either Data Warehouses or Relational Database is mandatory. - Experience in AWS cloud Mandatory Skills : ...

• Remote
• Contract

30k

... Erwin Data Modeler, Lucidchart, MySQL Workbench, and Oracle SQL Developer Data Modeler. Possess strong hands-on SQL and Python skills. Strong experience with data architecture principles, including data modelling, ETL/ELT processes, and data management and hands on experience with Big Data technologies. Familiarity with ...

... problems to deliver to clients' requirements. If you do not meet the experience, please do not apply. Build highly efficient and scalable ETL processes using big data technology Leverage large scale cloud infrastructure to build high throughput services Designing and implementing code in SQL/Python Good understanding and ...