Deployment Platform Updates

October 17, 2022
Share this post

The Truefoundry team has been working really hard the last month adding features to our ML Deployment platform. Our goal here is to build a deployment platform that makes is absolutely easy to deploy ML models and services while enforcing the best engineering and security principles. To build a great ML platform, we need to have a solid engineering platform and that's why a lot of the initial focus has been on delivering a solid platform to deploy code.

ML Platform components

Out of all the pieces of the Ml platform described above, we focus on the serving infrastructure, monitoring and all the automation around that.

A lot of work went in building our deployment platform on top of Kubernetes. The goal here has been to make it absolutely easy to deploy in under 5 minutes wherein the platform takes care of building the image from the source code, storing it in a docker registry and then finally deploying the application on Kubernetes. A few of the updates from our last month include the following:

Ability to choose instance family while deploying

Machine learning models can have very different inference latency or performance based on the instance type. For e.g, when testing the inference latency of a hugging face model on Intel vs AMD processors, we found Intel processors to be around 30% faster. That’s why we now have an option to allow users to choose the instance type while deploying their workloads. If the instance type is not selected, the workload can be deployed on any available instance type.

Choose instance type while deploying
Choose instance type while deploying

Logs and Metrics for Deployments

We earlier had a Grafana link for showing logs and metrics. While Grafana was highly customizable, permission and access control wasn’t really possible on Grafana. Also, it turned out to be a bit slow and difficult to understand for users who weren’t used to Grafana. That’s why we implemented our own UI for showing logs and metrics which should suffice in most cases. We still offer the Grafana integration in public cloud for more advanced users.

System Metrics
System Metrics
Application Logs
Application Logs

Permission Control On Secret Groups

We can now add users as editor, viewer or admin on secret groups.

Application Logs

Github and Bitbucket integration

We can now deploy directly to Truefoundry from any Github or bitbucket repositories. Users can integrate with their own private repositories using the Oauth Flow and select the appropriate parameters to deploy the application.

In the next month, we are working on a few exciting features like:

  1. Making the platform more intuitive and easy to use.
  2. Automated deployment of truefoundry stack on any Kubernetes cluster
  3. Support for teams
  4. Deployment rollback functionality

Stay tuned and let us know your feedback!

Discover More

February 29, 2024

SSH Server Containers For Development on Kubernetes

Engineering and Product
March 6, 2024

Prompting, RAG or Fine-tuning - the right choice?

Engineering and Product
April 27, 2023

Large Language Models for Commercial Use

Engineering and Product
February 1, 2024

Adding OAuth2 to Jupyter Notebooks on Kubernetes

Engineering and Product

Related Blogs

No items found.

Blazingly fast way to build, track and deploy your models!