Opportunity Description
About the Team The AI Model Serving team is the engine behind every production Workday agent and machine learning use case. We own the services that power all production AI workloads, serving as the gateway to vendor-hosted LLMs on GCP and AWS Bedrock and operating the model deployment platform where Workday hosts and scales its models.
We host thousands of traditional ML models across sharded Ray Serve clusters, making us one of the heaviest users of Ray at scale. Our platform handles approximately 2,000 requests per second and peaks at greater than 10,000 requests per second in our largest cluster. We also provide a uniform interface for accessing models on Bedrock and Gemini, an evaluation platform for generative AI use cases, and the production model registry for Workday.
In the year ahead, we are focused on making our uniform vendor interface more robust, scaling our architecture to support more than 20 agents going into production, improving cost governance, exp...
We host thousands of traditional ML models across sharded Ray Serve clusters, making us one of the heaviest users of Ray at scale. Our platform handles approximately 2,000 requests per second and peaks at greater than 10,000 requests per second in our largest cluster. We also provide a uniform interface for accessing models on Bedrock and Gemini, an evaluation platform for generative AI use cases, and the production model registry for Workday.
In the year ahead, we are focused on making our uniform vendor interface more robust, scaling our architecture to support more than 20 agents going into production, improving cost governance, exp...
Interested in this opportunity? Apply now through Expertini.
Apply for this Position