We are a product-centric insight & automation services company globally. We help the world’s organizations
make better & faster decisions using the power of insight & intelligent automation.
We build and operationalize their next-gen strategy, through Big Data, Artificial Intelligence, Machine
Learning, Unstructured Data Processing and Advanced Analytics. Quadratyx can boast of more extensive
experience in data sciences & analytics than most other companies in India. We firmly believe in Excellence Everywhere.
Senior AI ML/ Gen AI Engineer
Job / Role Information
Designation:
Senior Data Engineer – Big Data & GenAI
Function:
Technical
Location:
Hyderabad, Chennai
Job Description
Purpose of the Job/ Role:
We are looking for a highly skilled and hands-on Data Engineer with strong expertise in Big Data technologies, distributed data processing, and backend engineering. The ideal candidate should have practical experience in building scalable batch and real-time data pipelines using frameworks such as PySpark, Kafka, and streaming platforms.
The candidate should also possess strong Python programming skills and experience building APIs/microservices using modern backend frameworks.
This role is best suited for engineers who enjoy solving large-scale data engineering problems and working closely with AI/GenAI-driven applications.
We are specifically looking for candidates with strong hands-on coding and distributed systems experience, and not candidates whose experience is predominantly limited to traditional ETL tools, low-code cloud pipelines, or Snowflake-centric workflows.
Key Requisites:
Design, develop, and optimize scalable batch and streaming data pipelines
Build large-scale distributed data processing systems using PySpark
Develop real-time streaming solutions using Kafka and/or Apache Flink
Build and maintain APIs and backend services using Python-based microservice frameworks
Work with structured and semi-structured datasets across multiple storage systems
Optimize pipeline performance, scalability, and fault tolerance
Collaborate with AI/ML and GenAI teams to support LLM and RAG-based applications
Participate in architecture discussions, debugging, code reviews, and production deployments
Ensure reliability, monitoring, and maintainability of data engineering systems
Mandatory Skills
Strong hands-on experience with PySpark and distributed data processing (must-have)
Strong programming expertise in Python
Hands-on experience building APIs and backend services using:
Flask
FastAPI
or similar microservice frameworks
Good understanding of Big Data ecosystem and streaming architectures
Experience with:
Apache Kafka
Apache Flink (preferred)
PostgreSQL
Experience building end-to-end batch and real-time data engineering pipelines
Good understanding of scalable system design, partitioning, optimization, and distributed processing concepts
Understanding of microservices architecture and API integrations
Experience working in Linux-based environments
Strong analytical and problem-solving skills
GenAI / AI Engineering Expectations
Candidates should also have practical exposure to modern GenAI concepts and tools, including:
Strong analytical and problem-solving skills
Working knowledge of LLMs and prompt engineering
Understanding of RAG (Retrieval-Augmented Generation) concepts
Experience using GenAI frameworks/tools such as:
LangChain
LlamaIndex (good to have)
Exposure to vector databases, embeddings, or AI-assisted workflows is a plus
Good to Have / Bonus Skills
Experience with QuestDB
Exposure to real-time analytics and streaming systems
Understanding of distributed systems internals
Experience with Docker/containerization and deployment workflows
Familiarity with monitoring and logging frameworks
Knowledge of cloud platforms is an added advantage
Important Note
This role is focused on core data engineering, distributed systems, and backend engineering.
Candidates with predominantly:
Snowflake-only experience
Low-code ETL pipeline development
Pure cloud ETL orchestration backgrounds
may not be the right fit for this requirement unless they also possess strong hands-on Big Data engineering and programming experience.
Working Relationships
Reporting to
Vice President
External Stakeholders
Clients
Preferred Candidate Profile
Strong ownership mindset and problem-solving ability
Minimum 5-8 years of work experience as a Data Manager in an IT organization (preferably Analytics / Big Data/ Data Science / AI background).
Comfortable working independently on complex engineering problems
Ability to work in fast-paced product/research-oriented environments
Passion for learning modern AI and data technologies
Good communication and collaboration skills
Quadratyx is an equal opportunity employer - we will never differentiate candidates on the basis of religion, caste, gender, language, disabilities or ethnic group.
Quadratyx reserves the right to place/move any candidate to any company
location, partner location or customer location globally, in the best
interest of Quadratyx business.
