Software Engineering Senior Advisor

• The candidate will be responsible to deliver business needs end to end from requirements to development into production.
• Through hands-on engineering approach in Databricks environment, this individual will deliver data engineering toolchains, platform capabilities and reusable patterns.
• The applicant will be responsible to follow software engineering best practices with an automation first approach and continuous learning and improvement mindset.
• The applicant will ensure adherence to enterprise architecture direction and architectural standards.
• The applicant should be able to collaborate in a high-performing team environment, and an ability to influence and be influenced by others.

Required Skills:

• Cloud-based security principles and protocols like OAuth2, JWT, data encryption, hashing data, secret management, etc
• Expertise in Big data technologies such as Spark, Hadoop, Databricks, Snowflake, EMR, Azure Data Factory
• Good understanding of Kafka, Kafka Streams, Spark Structured streaming, configuration-driven data transformation and curation
• Expertise in building cloud-native microservices, containers, Kubernetes and platform-as-a-service technologies such as OpenShift, CloudFoundry
• Experience in multi-cloud software-as-a-service products such as Databricks, Snowflake
• Experience in Infrastructure-as-Code (IaC) tools such as terraform
• Experience in messaging systems such as Apache ActiveMQ, WebSphere MQ, Apache Artemis, Kafka, Azure Service Bus
• Experience in API and microservices stack such as Spring Boot, Quarkus,
• Expertise in Cloud technologies such as Azure Data Factory, Lambda, S3, Elastic Search, API Gateway, CloudFront
• Experience with one or more of the following programming and scripting languages - Python, Scala, JVM-based languages, or JavaScript, and ability to pick up new languages
• Experience in building CI/CD pipelines using Jenkins, Github Actions, Azure DevOps
• Strong expertise with source code management and its best practices
• Proficient in self-testing of applications, unit testing and use of mock frameworks, test-driven development (TDD)
• Knowledge on Behavioral Driven Development (BDD) approach
• Ability to perform detailed analysis of business problems and technical environments
• Strong oral and written communication skills
• Ability to think strategically, implement iteratively and estimate financial impact of design/architecture alternatives
• Continuous focus on an on-going learning and development
• Expertise in Agile software development principles and patterns
• Expertise in building streaming, batch and event-driven architectures and data pipelines

Required Experience & Education:

• 13-16 years of experience in software engineering, building data engineering pipelines, middleware and API development and automation
• More than 3 years of experience in Databricks within an Azure environment