Neurobit is a digital health company based in New York, Singapore, and Bangalore. They are developing technologies to predict and plan for adverse health outcomes well before they happen, using vitals collected during sleep as a biomarker.
The company has constructed the world's largest sleep database with over a trillion data points of multi-channel physiological data. The scale of data that they have trained their models gives them the robustness to generalize the model to any new scenario. Their use case resembles many recent AI efforts that new age Health Tech companies and new AI initiatives of tech majors.
We found similarities between Neurobit’s use cases and those of other enterprise and mid-size healthcare companies that we have talked to:
When we first met the Neurobit team, they had been conducting medical trials with 120+ research centers, universities, and 1000+ subjects. Most of these subjects were in the same geographical location.
When the person wakes up, the sensors send physiological data to the server for processing. Each request needs 20 different models to be called to generate the final output.
The data size that came in with each request was large (400 mb+) and during the high traffic time, the team could see a significant lag in the response time and even the dropping of requests with loss of user data.
This situation was causing significant adverse financial implications for the team:
As in other machine learning use cases within the health tech industry, the team could not afford the loss of customer data or delayed responses.
The team knew that deploying their models on Kubernetes, with a queue to store requests before they are processed, would be able to solve their reliability issues.
However, the IP protection norms of the company limited access to the model only to a few members of the Machine Learning team and not to the DevOps team.
The Machine Learning team had limited bandwidth and expertise in Kubernetes to pull this off themselves. Instead, they wanted to work on developing new models.
The existing stack that was being used for machine learning deployments was:
The stack worked fine for the team up to a certain scale. But once the use case scaled, the team started facing reliability issues with serving the model that needed immediate attention.
Since the company deals with sensitive PII and health data, maintaining the security of the model APIs was of utmost importance to them. They wanted no customer data to be leaving their cloud and to strengthen the authentication and security norms of the APIs that they had been using.
The team needed a way to empower their Machine Learning team, which had access to the model, to be able to deploy and manage models on Kubernetes independently. The objectives that the Neurobit team wanted to achieve through a partnership with TrueFoundry were:
The TrueFoundry team helped the Neurobit team could install the TrueFoundry agent and control plane on their cluster within a 2-hour call. They were informed of the access and permissions needed, and they were walked through each step of the installation in a single call.
The team was given a choice to install only the modules of the TrueFoundry platform that were relevant to them (Model deployment and authentication).
Post the installation, the team was given a demo of the platform and handed over the documentation.
The Neurobit team was able to start using the platform for their model deployments right from day 1. They could directly connect their Git repositories to the platform, this code was automatically dockerized and deployed on the platform by using the TrueFoundry UI, APIs, or the Python SDK. There were no code changes required and no need to learn any additional framework for all the workflows that the team was trying to complete.
The team showed great pace since they wanted to solve the reliability issues fast. Within a few days, they started exploring more and more features of the platform and provided us with feedback.
Within two weeks, the team was able to:
Through the deployment of Machine Learning models on TrueFoundry, the team was able to:
When the highest priority challenges with the Machine Learning models was solved, the TrueFoundry team decided to go further to make sure that the Neurobit team was set up for success. During our conversations with the Neurobit team, we got to understand that there was scope for the microservices architecture of the company to be optimized further. This could have had a possible impact on the inference time and cloud costs that the team was incurring.
We ended up doing an in-depth review of the microservices architecture with the team.
We got the following understanding of the architecture that the team was following:
This entire process took ~7 minutes for each request.
We tried to understand the fault tolerance and inference times that the team required. With this understanding, we suggested the Neurobit team directly pass the output of one service to the other over the gRPC protocol.
The advantage of this architecture was that.
This new pipeline was hosted on the TrueFoundry platform and it decreased the model inference time from ~7 mins/request to ~2 mins/request.
As our partnership with the Neurobit team progresses, we have seen the business realise the benefits from the faster response times, reliability, and scale that the TrueFoundry platform has helped the Neurobit team achieve.
TrueFoundry helped the Neurobit team to move all of their Machine Learning workloads to Kubernetes without having to deal with the complexity of learning anything new related to Kubernetes. It has also helped the team become independent in handling all the advanced operations in Kubernetes like doing Async deployments, setting up autoscaling, serverless deployments, etc.
We have also been able to help the team move some of their software resources onto microservices architecture on top of Kubernetes so that their stack is future-proof and running with optimum utilization levels.
As we keep on engaging with Neurobit and helping them achieve the scale and level of impact with Artificial Intelligence that they have set out to, we are grateful for all the learnings that we have been able to derive from engaging with the team. It has helped shape both, how we think about engagement with clients as well as given a solid direction to our product.
Some of our core learnings include:
We have co-developed some important features of the platform while trying to solve the use cases that the Neurobit team required us to enable. These include:
We look forward to engaging with the Neurobit team in the long term and getting to learn from them while trying to help them along the way. Some of the future developments that could be in store for this engagement include:
Excited to see what comes next!