Data Scientist

Job Overview:

• Implement end-to-end machine learning and advanced AI solutions, including generative AI where applicable, from initial problem definition to deployment in production, collaborating closely with product, data engineering, and business teams.

Primary Responsibilities:

• Convert business requirements into clear ML/AI problem definitions and measurable success criteria.
• Create ML models and, when appropriate, generative AI components, such as retrieval augmented generation prompt pipelines with defined evaluation standards.
• Conduct offline evaluations, error analyses, bias assessments, and baseline monitoring, while documenting decisions and assumptions throughout the process.
• Effectively communicate findings and limitations to both technical and non-technical stakeholders, facilitating integration into workflows.

Technical Qualifications:

• 4 to 6 years of experience managing end-to-end data science projects.
• A Master’s or PhD in Data Science, Machine Learning, Statistics, Computer Science, Applied Mathematics, or a related quantitative discipline is mandatory at this level.
• Proficient in core technologies including Python, Spark, Git, and machine learning frameworks such as Databricks and MLflow, along with foundational knowledge of cloud services.

Additional Duties:

• Utilize Python, pandas, numpy, and Git to ensure reproducible development practices.
• Employ Databricks Notebooks and Workflows for effective development and orchestration.
• Manage ML flow experiments and track model registries for comprehensive lifecycle management.
• Leverage Azure cloud services as necessary, including Azure OpenAI and Azure AI Foundry for generative AI development and evaluation.
• Implement Databricks Mosaic AI, including Mosaic AI Model Serving for generative AI solutions within the lakehouse environment.
• Apply Databricks Vector Search for retrieval patterns in RAG.
• Utilize Unity Catalog for controlled access to data and models where applicable.
• Monitor model performance for quality and drift within the lakehouse as required.

Desired Skills:

Technology->AI-Data Science->Data Science - Basic, Technology->AI-Generative AI->Generative AI - Basic, Technology->Cloud Platform->Azure Analytics Services->Azure Databricks