TrueML Talks #26 - الذكاء الاصطناعي التوليدي للمؤسسات وعمليات نماذج اللغة الكبيرة (LLMOps) مع لابهيش باتيل

We are back with another episode of  True ML Talks. In this, we again dive deep into MLOps pipelines and LLMs Applications in enterprises as we are speaking with Labhesh Patel.

Labhesh was a CTO and Chief Scientist at Jumio Corporation, where he worked in leveraging ML / AI in identity verification space. He has held multiple leadership positions, both in engineering and science roles in the past, with leading organizations.

📌

Our conversations with Labhesh will cover below aspects:
- Interesting Research Papers and Patents
- Utilizing AI to solve business problems
- Building the MLOps Pipeline
- Breaking Down Silos: Building Cohesive MLOps Teams for Success
- Navigating Cloud Provider Roadblocks
- Future of Generative AI

Watch the full episode below:

Interesting Research Papers and Patents

Research Papers

• Attention is All You Need: This paper introduced the transformer network, which revolutionized natural language processing and laid the foundation for many LLMs like ChatGPT.

‍

Attention Is All You Need

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.arXiv.orgAshish Vaswani

arXiv.org Ashish Vaswani

‍

• Visual Question Answering with Segmented Guided Attention Networks: This paper proposed a novel method for answering questions about images by utilizing segmentation maps and attention mechanisms. While superseded by newer techniques, it highlights the importance of focusing on specific areas of an image for accurate answers.

‍

Segmentation Guided Attention Networks for Visual Question Answering

Vasu Sharma, Ankita Bishnu, Labhesh Patel. Proceedings of ACL 2017, Student Research Workshop. 2017.

ACL Anthology

‍

• CycleGen: This paper explores the idea of generating text summaries based on user reviews and product characteristics. It predates ChatGPT and demonstrates the potential for LLMs to assist with writing tasks.

‍

Cyclegen: Cyclic consistency based product review generator from attributes

Vasu Sharma, Harsh Sharma, Ankita Bishnu, Labhesh Patel. Proceedings of the 11th International Conference on Natural Language Generation. 2018.

ACL Anthology

‍

Patents

• Voice over IP Buffering and Negotiation Protocol: This patent arose from a simple bug fix that improved voice quality in VoIP calls. It highlights the potential for innovation in seemingly mundane solutions and the importance of considering defensive patenting strategies.

Utilizing AI to solve business problems

There are a lot of challenges and opportunities in transforming manual processes with AI. Here are some key takeaways:

Start with the Business, Not the Buzz

• Identify the core business problem: Why automate? What are the quantifiable benefits (scalability, cost reduction, speed)?
• Manage expectations: AI isn't magic. Communicate what's achievable and set realistic performance metrics.
• Understand data's role: 90% of the work lies in data management, collection, and quality assurance. Clean data is vital for accurate models.

Building the Right Path

• One step at a time: Focus on a single, high-impact use case to prove the concept and build your pipeline.
• Compliance first: Ensure proper data consent and usage before even touching a single byte.
• Metrics matter: Track relevant metrics (precision, recall, error rates) to evaluate success and guide further decisions.
• Teamwork is key: Assemble a team with expertise in ML engineering, data management, and product development.

Beyond the First Step

• Iterate and evolve: Continuously evaluate, improve, and expand your AI solutions based on data and feedback.
• Embrace the learning curve: Be prepared to invest in talent and education to build a culture of AI understanding within your organization.

Important things to keep in mind

• Beware of the 99% trap: High accuracy on isolated cases can mask larger problems. Pay attention to overall performance and error rates.
• Think statistically: Metrics like precision and recall provide a more nuanced picture of AI performance than simple accuracy percentages.

By prioritizing business needs, focusing on data quality, and building a strong team, you can navigate the complexities and unlock the true potential of AI to transform your operations.

Building the MLOps Pipeline

For anyone building complex ML systems, there are some things you can keep in mind.

Embrace Cloud-First, But Remain Agile

• Leverage your cloud provider's built-in MLOps tools like AWS SageMaker for fast initial setup.
• Avoid vendor management and compliance hurdles by staying within the cloud ecosystem.
• Move beyond native offerings when limitations arise, seeking out specialized solutions like open-source platforms or vendors.

Importance of Data Quality

• Recognize that cloud providers often neglect data quality, requiring additional internal systems or third-party services.
• Prioritize automated data cleaning and validation to ensure model accuracy and performance.

Architectural considerations

• Model building vs. production: Consider separate teams for model development and deployment, with distinct skill sets and ownership.
• Structure for scalability and agility: Design a flexible architecture that can accommodate new tools and integrations as the pipeline evolves.

Breaking Down Silos: Building Cohesive MLOps Teams for Success

In the fast-paced world of MLOps, collaboration is king. But too often, teams become fragmented, with data scientists building models in isolation and engineers struggling to deploy and maintain them. The result? Slow progress, missed opportunities, and frustrated stakeholders.

So how do we break down these silos and build MLOps teams that thrive?

Bringing everyone together

Imagine a cross-functional team of 8-10 individuals, each with unique expertise: product managers, data engineers, DevOps, security, ML engineers, QA, and even customer support. This diverse group, united by a common goal (e.g., reducing fraud), becomes a powerful force for innovation and efficiency.

Here's why this approach works:

• Shared ownership: When everyone feels responsible for the entire lifecycle of a model, there's no "over the fence" mentality. Issues get tackled collaboratively, and solutions are optimized for real-world deployment and maintenance.
• Informed decisions: Data engineers understand ML needs, and ML engineers appreciate deployment realities. This cross-pollination of knowledge leads to better model selection and feature engineering, avoiding the pitfalls of "research-perfect" models that are impossible to deploy.
• Faster iterations: Close collaboration fosters communication and agility. The team can quickly experiment, refine, and iterate on models, maximizing the impact of their efforts.

Tackling skill gaps for building such a team

It is of the utmost importance to do targeted hiring. You need data engineers with a strong understanding of ML pipelines and ML engineers who appreciate software engineering principles. This combination of diverse skills is the secret sauce to a high-performing MLOps team.

Breaking down silos isn't just about structure, it's about culture. Encourage open communication, celebrate diverse perspectives, and create an environment where everyone feels empowered to contribute. By doing so, you'll build a cohesive MLOps team that can turn your ML dreams into reality.

Navigating Cloud Provider Roadblocks

There are a lot of potential roadblocks you can encounter when heavily relying on a Cloud Provider. In such scenarios, it is very important to be able to pivot when such a roadblock arises.

• Don’t be afraid to explore alternatives: When cloud providers hit limitations, look for specialized vendors or open-source solutions to fill the gaps.
• Proactive communication matters: Don't hesitate to voice your concerns directly to cloud providers. Feedback can lead to improved collaboration and access to exclusive solutions.
• Adaptability is key: Be prepared to adjust your approach based on emerging technologies and changing provider offerings.

Here are some common challenges that can arise

Challenge 1: Super-regulated data access

When dealing with sensitive data (PII, healthcare records), strict regulations like GDPR and CCPA come into play. Cloud providers, while compliant with general standards, might not offer specific tools for secure access and audit trails.

The potential solutions to these are:

• موردون بديلون: ابحث عن الشركات المتخصصة في البيئات شديدة التنظيم والتي تقدم ميزات التحكم الدقيق في الوصول وإمكانية التدقيق.
• حلول مفتوحة المصدر: ضع في اعتبارك الأدوات مفتوحة المصدر وقم بتخصيصها لتلبية احتياجات الامتثال المحددة.

التحدي الثاني: الميزات الاحتكارية والوصول المحدود

في بعض الأحيان، يحجب موفرو الخدمات السحابية ميزات معينة أو يطلقونها وفقًا لجدولهم الزمني الخاص، مما يترك العملاء في انتظار وظائف حيوية.

الحل المحتمل لذلك هو أن تكون استباقيًا في التواصل مع نقطة الاتصال الخاصة بك لدى موفر الخدمة السحابية هذا.

تقديم ملاحظات مباشرة إلى نقطة الاتصال (POC) وتوضيح المعوقات التي تواجهونها يمكن أن يمنحك أنت وفريقك أحيانًا وصولاً مبكرًا إلى برامج بيتا الخاصة، مما يضمن عدم تفويتكم للحلول المستقبلية.

تذكر، حتى مع وجود العقبات، يمكن لعقلية استباقية وقابلة للتكيف أن تحول التحديات إلى فرص في عالم MLOps القائم على السحابة والمتطور باستمرار.

مستقبل الذكاء الاصطناعي التوليدي

الذكاء الاصطناعي التوليدي، وخاصة نماذج اللغة الكبيرة (LLMs)، يحظى بشعبية كبيرة. ومع ذلك، تمر نماذج اللغة الكبيرة حاليًا بـ "مرحلة ضجيج"، تُشيد بقدراتها السحرية على التعامل مع مهام متنوعة. يلجأ المطورون إلى إرسال استدعاءات API مكثفة إلى نماذج اللغة الكبيرة، مما يؤدي إلى مشكلات مثل تحديد المعدل والتكاليف المرتفعة.

تحديات اعتماد الشركات

• التكلفة وقابلية التوسع: النماذج الكبيرة باهظة الثمن وتتطلب قدرات حاسوبية عالية، مما يجعلها غير مناسبة للاستخدام الواسع النطاق في الشركات.
• سلامة النموذج والتحيز: تتطلب بيئات الشركات سلامة النموذج والتحكم في التحيزات المحتملة، وهو ما قد يكون صعبًا مع نماذج اللغة الكبيرة.
• وقت الاستدلال: تعاني نماذج اللغة الكبيرة من مشكلة الكمون، مما يسبب تأخيرات تعيق الإنتاجية وتجربة المستخدم.

المستقبل: هل نماذج اللغة الصغيرة هي الحل؟

قد يكون هناك تحول نحو نماذج اللغة الصغيرة (SLMs)، المدربة لمهام ومجالات محددة داخل الشركات.

هذه "البنية الموجهة" ستوجه الاستعلامات إلى نموذج اللغة الصغيرة المناسب للحصول على استجابات أسرع وأكثر كفاءة.

تعالج النماذج الأصغر أيضًا مخاوف التكلفة وقابلية التوسع، مما يجعلها أكثر سهولة في الوصول للشركات.

محفزات واعتبارات الانتقال

من المرجح أن يحدث هذا الانتقال تدريجياً، مدفوعاً بالقيود العملية لنماذج اللغة الكبيرة (LLMs) والتوافر المتزايد لنماذج اللغة الصغيرة الفعالة (SLMs).

سيلعب خفض التكاليف وتحسين زمن الاستجابة أدواراً رئيسية في تسريع تبني نماذج اللغة الصغيرة (SLMs).

اقرأ مدوناتنا السابقة ضمن سلسلة True ML Talks:

‍

GenAI and LLMOps for GTM (Go-To-Market) @ Twilio‍

Dive deep into Twilio’s GenAI applications like XGPT, and RFP Genie for revolutionizing GTM (Go-To-Market) Strategies. Deep dive into the Backend for these applications.

TrueFoundry Blog TrueFoundry

‍

تابع مشاهدة TrueML سلسلة يوتيوب وقراءة TrueML سلسلة المدونات.

TrueFoundry هي منصة كخدمة (PaaS) لنشر تعلم الآلة (ML) تعمل فوق Kubernetes لتسريع سير عمل المطورين مع منحهم مرونة كاملة في اختبار ونشر النماذج، وضمان الأمان والتحكم الكاملين لفريق البنية التحتية. من خلال منصتنا، نمكّن فرق تعلم الآلة من نشر ومراقبة النماذج في 15 دقيقة بموثوقية وقابلية للتوسع بنسبة 100%، والقدرة على التراجع في ثوانٍ - مما يسمح لهم بتوفير التكاليف وإطلاق النماذج إلى الإنتاج بشكل أسرع، وبالتالي تحقيق قيمة تجارية حقيقية.