Data Engineer

Job Description
We are seeking a skilled Data Engineer to join our dynamic team at Microsoft and contribute to the data pipeline and infrastructure supporting our Azure-hosted AI Experimentation Platform. This platform is a critical system composed of Azure Web and API App Services, Azure SQL, .NET 9, and Blazor Server, with security managed through Entra ID. It heavily utilizes AI Foundry and AI Search for innovative AI capabilities. You will play a critical role in ensuring the availability, reliability, and scalability of data pipelines and reporting environments to support AI researchers and developers while maintaining high performance and security standards. More specifically, as a Data Engineer, your primary focus will be on ingesting data from multiple sources, processing it using Azure Synapse Analytics (with extensive use of Python notebooks), staging it in Azure Data Lake Storage (ADLS), and indexing it in AI Search for downstream AI experimentation and analysis. You will...