Senior Software Engineer- (Data Engineering, AWS & MLOPs)
Core skills: AWS, Snowflake, data ingestion, data transformation, SQL, MLOps and Python
Location: Pune, India (Work from home)
Role Overview
We are seeking a Senior Data Engineer to design, build and operate scalable data pipelines and analytical solutions using AWS and Snowflake. This role will integrate data from enterprise applications, APIs, databases and files, transforming it into reliable datasets for analytics, reporting and AI/ML use cases. The ideal candidate combines strong engineering skills with experience delivering secure, reliable and cost-effective data platforms in production.
You will be responsible for:
- Design and maintain batch and incremental data ingestion pipelines from APIs, relational databases, SaaS applications and file sources.
- Build AWS ingestion solutions using services such as Amazon S3, AWS Glue, Lambda, IAM and Step Functions.
- Develop Snowflake data models and transformation pipelines using SQL, Python, dbt or equivalent tools.
- Implement Bronze, Silver and Gold layers within a medallion architecture, with clear data contracts and lineage.
- Process structured and semi-structured data, including JSON, CSV and Parquet.
- Implement change data capture, incremental loading, deduplication, schema evolution and historical data management.
- Integrate datasets across systems using consistent business keys, reference mappings and transformation rules.
- Build data quality checks, reconciliation controls, monitoring, alerting and recovery procedures.
- Optimize Snowflake queries, warehouse usage and storage to improve performance and control costs.
- Implement access controls, encryption, secrets management and data protection requirements.
- Establish automated testing, CI/CD and repeatable deployment practices for data pipelines.
- Troubleshoot production issues, perform root cause analysis and improve pipeline reliability.
- Collaborate with analysts, data scientists, architects and business stakeholders to translate requirements into technical solutions.
- Maintain technical documentation and provide engineering guidance through design and code reviews.
You should apply if you have:
- 8+ years of professional experience in data engineering, data warehousing, ETL/ELT development or related roles.
- Strong hands-on experience with AWS and Snowflake in production environments.
- Advanced SQL skills, including complex transformations, analytical functions and query optimization.
- Strong Python skills for API integrations, data processing, automation and error handling.
- Experience building ingestion pipelines with pagination, authentication, rate limiting, retries and checkpointing.
- Experience with Snowflake capabilities such as stages, storage integrations, COPY INTO, Snowpipe, Streams and Tasks.
- Experience with transformation frameworks such as dbt or equivalent SQL-based tooling.
- Experience with orchestration tools such as Apache Airflow, AWS Step Functions or equivalent platforms.
- Strong understanding of dimensional modeling, slowly changing dimensions and incremental processing.
- Experience implementing data quality, observability, lineage and operational support practices.
- Proficiency with Git, code reviews and CI/CD.
- Strong problem-solving, communication and stakeholder collaboration skills.
Preferred Qualifications
- Experience preparing and managing datasets for AI/ML, including feature engineering and training and inference pipelines.
- Familiarity with MLOps, including experiment tracking, model versioning, deployment, monitoring and retraining workflows.
- Experience with tools such as Amazon SageMaker, MLflow, Snowpark or equivalent platforms.
- Familiarity with model drift, data drift and reproducibility.
- Experience supporting generative AI applications, embeddings, vector search or retrieval-augmented generation.
- Experience with streaming technologies such as Amazon Kinesis or Apache Kafka.
- Experience with infrastructure as code using Terraform or AWS CloudFormation.
- AWS, Snowflake or relevant data engineering certifications.
About Convera
Convera is the largest non-bank B2B cross-border payments company in the world. Formerly Western Union Business Solutions, we leverage decades of industry expertise and technology-led payment solutions to deliver smarter money movements to our customers – helping them capture more value with every transaction. Convera serves more than 30,000 customers ranging from small business owners to enterprise treasurers to educational institutions to financial institutions to law firms to NGOs. Our teams care deeply about the value we bring to our customers, which makes Convera a rewarding place to work. This is an exciting time for our organization as we build our team with growth-minded, result-oriented people who are looking to move fast in an innovative environment. As a truly global company with employees in over 20 countries, we are passionate about diversity; we seek and celebrate people from different backgrounds, lifestyles, and unique points of view. We want to work with the best people and ensure we foster a culture of inclusion and belonging.
We offer an abundance of competitive perks and benefits including:
- Competitive salary
- Opportunity to earn an annual bonus.
- Great career growth and development opportunities in a global organization
- A flexible approach to work
There are plenty of amazing opportunities at Convera for talented, creative problem solvers who never settle for good enough and are looking to transform Business to Business payments. Apply now if you’re ready to unleash your potential.
#LI-AK1