AI Data Engineers

Job Overview:

• We are looking for a talented AI Data Engineer to architect, develop, and enhance scalable data platforms and pipelines specifically designed for Generative AI, Large Language Models (LLMs), and sophisticated analytics solutions.
• The successful candidate should possess practical experience with the Databricks DBRX ecosystem, major cloud platforms such as AWS, Azure, or GCP, modern data engineering methodologies, and Snowflake data warehousing technologies.

Primary Duties:

• Data Engineering and AI Platform Development
• Create and implement scalable batch and real-time data pipelines utilizing Databricks, Spark, and cloud-native services.
• Establish and manage enterprise-level data lakes, lakehouses, and data warehouses.
• Design ingestion frameworks capable of handling structured, semi-structured, and unstructured data.
• Construct AI-ready data pipelines to support LLM, Retrieval-Augmented Generation (RAG), Agentic AI, and predictive analytics initiatives.
• Enforce metadata management, data governance, lineage tracking, and data quality protocols.
• Facilitate AI and Generative AI Enablement
• Prepare and preprocess data for LLM training, fine-tuning, and inference tasks.
• Establish RAG pipelines that incorporate vector databases and organizational knowledge resources.
• Develop data workflows compatible with DBRX and other foundational models.
• Enhance AI data pipelines for optimal performance, scalability, and cost-effectiveness.
• Cloud Data Platform Engineering
• Design cloud-based solutions within AWS, Azure, or GCP environments.
• Utilize cloud-native data services for data ingestion, storage, orchestration, and monitoring.
• Implement CI/CD and Infrastructure as Code (IaC) methodologies for data platforms.
• Maintain security compliance and governance across cloud data ecosystems.

Technical Qualifications:

• Proven experience in developing enterprise-level GenAI or AI Agent solutions.
• Familiarity with DBRX foundational models and the Databricks AI ecosystem.
• Hands-on expertise with vector search and semantic retrieval systems.
• Knowledge of DataOps, MLOps, and LLMOps best practices.
• Experience with multi-cloud data architectures.
• Understanding of Responsible AI principles and model governance frameworks.

Educational Background:

• Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, Artificial Intelligence, or a related discipline.
• Key Skills: Databricks DBRX, PySpark, Spark, Python, Snowflake, AWS, Azure, GCP, Delta Lake, Unity Catalog, MLflow, RAG, LangChain, Vector Database, Pinecone, Feature Store, Airflow, CI/CD, Docker, Kubernetes, Data Lakehouse, GenAI, LLM, Data Engineering.

Preferred Skills:

• Technology -> AI-Data science -> Machine Learning
• Technology -> AI-Generative AI -> Artificial Intelligence - BASIC
• Technology -> AI-Generative AI -> Generative AI for Data Analytics