Gaurav Mantri’s Post

I completed a course on Quantization.

From its underlying math to a full implementation from scratch, Cristian Leo offers a detailed exploration of the Nadam optimizer, which "enhances the Adam optimizer by incorporating Nesterov momentum."

I have finished Quantization in Depth! I want to talk about this approach in machine learning especially in have worked with it by hands before but of course this amazing course gave me a lot of useful extra insights howerver i have some concerns/Facts about it from practical wise: 1-Quantization is not a magical stick who you can use to decrease the model weights during inference and can keep the same performance measures of course there will be a performance degradation and our job is to minimize this Quantization effect by compressing the model and TRYING to keep the performance/accuracy/cost triangle as much as possible. 2-When comparing Quantization and pruning there’s a huge difference 3-Quantization vs knowledge distillation is a valid mechanism however Quantizaion is most likely to be used when dealing with larger models for example 4.5GBs like CLIP/BLIP/AraBertv2/GPT2 and this types of models , however for smaller models like Arabertv1, Full BERT and this medium sized models i always prefer Knowledge distillation which is creating a student model that learns from larger teacher model (usually less than 3GBs models) can benefit from Distillation with much better control 4-Also an innovative approach i used to see some of my colleagues used is knowledge distillation followed by Quantization #deeplearning #MLOPs #LLMs #LVMs #MachineLearning

Can a single <embeddings model> work just fine in multiple domains? This is what Nikolaos-Antonios Ypsilantis et al. discuss in the paper Towards Universal Image Embeddings: A Large-Scale Dataset and Challenge for Generic Image Representations (https://lnkd.in/d5jDmMEH). There's a huge dataset, a benchmarking and a challenge! Ok one common model or a mixture of small models? Oh the classical SLOSS debate (Single Large or Several Small)

Attended a very informative session on Causal Diagrams in R using ggdag, by Malcolm Barrett, organized by the Alan Turing institute. A lot of interesting discussions, I was particularly impressed by the following: 1. How easy the process of drawing and manipulating these DAGs is, and how neatly it integrates with ggplot and follows the grammar of graphics. 2. The discussion about time ordering of DAGs, which touches on time being an important variable that indicates causality, in that causality goes forward in time. Specifically I enjoyed the discussion about Feed Back loops, I have always thought about how they couldn't be represented as DAGs, but Malcolm's ideas on representing them as Feedforward graphs of time series variables that coevolve over time and can be represented as a DAG, made sense, and I'd love to explore it more. Linking to the resources that were shared during the session. Slides: https://lnkd.in/g53zWNNt Slide repository: https://lnkd.in/gMs-fpyx Causal Inference in R: https://www.r-causal.org/ ggdag repository: https://lnkd.in/gcVtJq_x #causalinference #dags

I’m pleased to share that I’ve completed the “Quantization Fundamentals with Hugging Face” course from DeepLearning.AI This course provided essential knowledge on how to quantize open source models, a key technique for compressing and optimizing models for deployment. Here’s a summary of the topics covered: 🔍 Introduction: Overview of quantization and its importance in model compression. 🔍 Handling Big Models: Techniques for managing and optimizing large models. 🔢 Data Types and Sizes: Understanding the various data types and their impact on model size and performance. 💾 Loading Models by Data Type: Methods for loading and handling models based on their data types. 📚 Quantization Theory: In-depth exploration of the theory behind quantization, including linear quantization. 🔧 Quantization of LLMs: Practical techniques for quantizing large language models (LLMs) using the Hugging Face Transformers library and the Quanto library. The course also included hands-on practice in quantizing open source multimodal and language models, making it highly practical and application-oriented. Thanks to Younes Belkada and Marc Sun for this insightful course. Learn more about my accomplishment here: Course Certificate. https://lnkd.in/ghyEvbhZ

Day 27: "Finding the longest path with 'Longest Increasing Subsequence' today! Unearthing the sequence that keeps growing. 'The best way to predict the future is to invent it.' - Alan Kay #DynamicProgramming #Subsequence #90DayChallenge" Leetcode:https://lnkd.in/gB5zChn6 Solution: class Solution {     public int lengthOfLIS(int[] nums) {           if(nums.length==0 || nums==null){             return 0;           }                   int maxlength=1;           int n=nums.length;           int dp[]=new int[n];           for(int i=0;i<n;i++){             dp[i]=1;           }           for(int i=1;i<n;i++){             for(int j=0;j<i;j++){                 if(nums[i]>nums[j]){                     dp[i]=Math.max(dp[i],dp[j]+1);                 }             }             maxlength=Math.max(dp[i],maxlength);           }           return maxlength;     } }

Day - 2/60 Lecture-5: Topological Sort | Kahn's Algorithm Here’s what I’ve learned and some key points to remember: 1) Topological Sort -A topological sort of a directed graph is a linear ordering of its vertices such that for every directed edge 𝑢→𝑣, vertex 𝑢 comes before vertex v in the ordering. 2) Solving Topological Sort using DFS -Uses post-ordering of DFS. Once DFS finishes for a node, the node is pushed onto a stack. The nodes are popped from the stack to get the topological order. 3) Solving Topological Sort using BFS(Kahn's Algorithm) -Uses in-degree of vertices. Repeatedly removes nodes with zero in-degree, adding them to the topological order, and reduces the in-degree of their neighbors accordingly. #DSAJourney #Learning #Consistency #ChallengeAccepted

Check out today's video with Nikhil Krishnaswamy on Reasoning about Object Recognition.

A short but in-depth write-up on a recently popular class of generative models: Flow matching / Stochastic Interpolants / Rectified flow (3 concurrent works with essentially the same idea). 👉 https://lnkd.in/gA-sQkih 👈 Flow matching generalizes and improves upon diffusion models in important ways. The core idea is quite fascinating. Personally, I like the stochastic interpolant (https://lnkd.in/g93MAstb) perspective; its clean and general. But the proof of why it works (as presented in the paper) might not be accessible to all. The blog features a simpler proof leveraging techniques from another paper (https://lnkd.in/gMA9AHmC). Give it a read if you're interested. Feedback is appreciated.