WI
Job Description
- You should have a bachelors or masters degree in computer science, Information Technology or other quantitative fields
- You should have at least 5 years working as a data engineer in supporting large data transformation initiatives related to machine learning, with experience in building and optimizing pipelines and data sets
- Strong analytic skills related to working with unstructured datasets.
- Experience with Azure cloud services: ADF, Azure Synapse, Blob Storage, ADLS, App Insights, and familiarity with various log formats from Azure.
- Experience with object-oriented/object function scripting languages: Python, Py-spark, Java, etc.
- Experience with big data tools: Hadoop, Spark, Kafka, etc.
- Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
- You should be a good team player and committed for the success of team and overall project.