Sanjay Adhikari’s Post

Learning rate schedulers are tools that dynamically adjust the learning rate as training progresses, helping models converge more efficiently and often to a better solution. These schedulers work by modifying the learning rate over time based on predefined rules or performance metrics. For instance, a learning rate scheduler might decrease the rate over time to allow the model to take smaller, more refined steps as it nears optimal solutions. Others might increase the learning rate at strategic points to help the model escape plateaus in the loss landscape. The goal is to balance stability and speed, helping models reach an optimal solution faster and more reliably. In PyTorch, learning rate schedulers are built directly into the library, making it easy for users to experiment with different scheduling strategies and tailor them to their specific needs. PyTorch offers a range of scheduling options — from basic, predefined schedules like StepLR, which decreases the learning rate by a factor at regular intervals, to more sophisticated ones like ReduceLROnPlateau, which reduces the learning rate based on the model's performance on the validation set. These schedulers are flexible, allowing us to customize parameters like learning rate decay rates, milestones, and conditions, making them a powerful tool in fine-tuning model performance. With PyTorch’s straightforward approach, integrating a learning rate scheduler into our model’s training loop becomes almost seamless, giving us the advantage of dynamically managing learning rates without needing extensive code modifications. In PyTorch there are two LearningRateSchedulers based on the Cosine pattern: CosineAnnealingLR and CosineAnnealingWarmRestarts. CosineAnnealingLR adjusts the learning rate using a cosine annealing schedule without any restarts. It smoothly decreases the learning rate from a maximum value to a minimum value over a specified number of epochs or iterations. This scheduler is straightforward and is suitable when you want a single, continuous decay of the learning rate throughout the training process. CosineAnnealingWarmRestarts adjusts the learning rate using a cosine annealing schedule with periodic restarts. After a specified period, the learning rate is reset back to the initial maximum value, and the annealing process starts anew. This can help the optimizer escape local minima and potentially improve convergence. Next question is how to choose between these two. Former case is ideal deal when a smooth and continuous decay of the learning rate is desired without interruptions and latter case is preferable when we want model to escape local minima by periodically increasing the learning rate. These are explained in detail in a Medium post, with link provided in the comment. Also, there is a graphical demonstration of it shown in an image. #AI #MachineLearning #DataScience #DeepLearning #LearningRateSchedulers #PyTorch

Learning by Doing: My Personal Journey About 50 years ago, I memorized a phrase: "Reading is learning, but using is also learning, and it is an even more important form of learning. XXX is a matter of the people, and often it is not something you learn first and then do, but rather something you start doing and learn along the way. Doing is learning." Over the past 50 years, I have practiced this philosophy of "learning by doing" for at least 40 of them. My primary practice began in 1994 with the research of microwave sintering materials. At that time, there were very few institutions in China conducting this research, and small organizations like ours struggled to purchase the necessary equipment. So, I took matters into my own hands, dismantling household microwave ovens and converting them into research-grade microwave heating furnaces. Over the past 30 years, I have personally, or guided assistants to, modify more than 50 household microwave ovens. The modified equipment not only became crucial tools for our research but also helped our institution secure government and local research funding. We integrated six household microwave sources into a single microwave sintering furnace, maintaining a volume comparable to that of a regular household microwave. We also addressed the issue of uniform electromagnetic field distribution and developed a unique microwave magnetron power regulation technology, achieving continuous PID temperature control. Solving these challenges expanded my knowledge from materials science and chemistry to include mechanical engineering, electrical engineering, electromagnetism, electronics, and computer programming. These interdisciplinary skills have enabled me to solve many complex problems in my work. In school, the lack of specific applications often resulted in inefficient learning. However, at work, the problems are clear, the objectives well-defined, and the urgency strong, which greatly enhances knowledge absorption. As a result, I borrowed the concept of "Object-Oriented Programming" (OOP) from computer science and coined "Object-Oriented Learning" (OOL). Later, AI introduced a similar concept: "Project-Based Learning" (PBL), and I feel AI’s terminology more accurately reflects "learning by doing." When I was in graduate school, my advisor taught me how to quickly review literature. I later came across a reading method called SQ3R (Survey, Question, Read, Recite, Review). I combined these methods to continuously update my knowledge. The advent of AI has provided a powerful tool for knowledge renewal. We have been using AI 3.0/4.0 for nearly a year, and I have experienced its leaps in knowledge updating. I eagerly anticipate future, more advanced versions.

Learning for today is the fastest way to become irrelevant tomorrow. We’re so focused on learning something that’s immediately useful, that we forget why we’re learning in the first place. But learning isn’t just about picking up new information. The real value lies in how you can think of different permutations & combinations to apply that knowledge that shapes your future decisions. When you learn with prospection in mind, you're not just acquiring knowledge—you're actively shaping your ability to make better decisions tomorrow based on what you learn today. It’s this forward-thinking mindset that sets us apart from AI. While machines crunch numbers and follow algorithms, we can look ahead, consider the unknowns, and still make decisions—even when we don’t have all the answers. Think about something as simple as learning to drive. You master the basic skills once, but when you plan to drive in the rain, your brain instinctively paints a picture of what could lie ahead—slippery roads, sharp turns and you seamlessly navigate through the storm even before you are sitting behind the wheel. It’s not just about the skills you pick up; it’s about how you visualize the situation and then apply them in unpredictable situations. This is where most learning programs fall short. They focus on: 1. Access-Based Learning: Teaching you how to use tools—like mastering Excel so you can automate tasks instead of doing them manually. 2. Process-Based Learning: Getting you to follow optimized workflows, like building better models with more efficient practices. 3. Application-Based Learning: Showing you how to apply what you’ve learned to boost your performance—like using situational leadership to get better results at work. But here’s the problem- they don’t teach you how to build scenarios, how to anticipate changes, how to prepare for what’s next. And that’s a huge gap. Prospection should be the foundation of your learning experience. So,the next time you’re learning something new, don’t just think about how it applies today. Visualize different scenarios and think about how to apply that knowledge to make smarter decisions down the road.

A Survey of Controllable Learning: Methods, Applications, and Challenges in Information Retrieval https://ift.tt/CDr93fA Controllable Learning (CL) is emerging as a crucial component of trustworthy machine learning. It emphasizes ensuring that learning models meet predefined targets and adapt to changing requirements without retraining. Let’s delve into the methods and applications of CL, particularly focusing on its implementation within Information Retrieval (IR) systems presented by researchers from Renmin University of China. Definition and Importance of Controllable Learning Controllable Learning is formally defined as the ability of a learning system to adapt to various task requirements without requiring retraining. This adaptability ensures that the learning model meets the specific needs and targets of the user, thus enhancing the reliability and effectiveness of the system. The significance of CL is rooted in its ability to address the dynamic and complex nature of information needs in IR applications, where the context and requirements can frequently change. Image Source Taxonomy of Controllable Learning The CL taxonomy is categorized based on who controls the learning process (users or platforms), what aspects are controllable (e.g., retrieval objectives, user behaviors, environmental adaptation), how control is implemented (e.g., rule-based methods, Pareto optimization, Hypernetwork), and where power is applied (pre-processing, in-processing, post-processing). Image Source User-Centric Control User-centric control empowers users to shape their recommendation experience actively. This involves modifying user profiles, interactions, and preferences to influence recommendation systems’ output directly. Techniques such as UCRS and LACE enable users to manage their profiles and interactions, ensuring that the recommendations align with their evolving preferences. Platform-Mediated Control Platform-mediated control involves algorithmic adjustments and policy-based constraints imposed by the platform. This approach aims to enhance the recommendation process by balancing multiple objectives, such as accuracy, diversity, and user satisfaction. Techniques like ComiRec and CMR utilize hypernetworks to dynamically generate parameters that adapt to varying user preferences and environmental changes, ensuring a tailored recommendation experience. Implementation Techniques in Controllable Learning Various techniques are employed to implement control in learning systems. These include: Rule-Based Techniques: These methods involve applying predefined rules to refine & enhance the output of AI models, ensuring aspects like security, fairness, and interpretability. This technique effectively ensures the system meets specific performance metrics such as diversity and fairness in recommendations. Pareto Optimization: This approach balances multiple conflicting objectives by finding a set of optimal trade-offs. It allows f...

A Survey of Controllable Learning: Methods, Applications, and Challenges in Information Retrieval https://ift.tt/CDr93fA Controllable Learning (CL) is emerging as a crucial component of trustworthy machine learning. It emphasizes ensuring that learning models meet predefined targets and adapt to changing requirements without retraining. Let’s delve into the methods and applications of CL, particularly focusing on its implementation within Information Retrieval (IR) systems presented by researchers from Renmin University of China. Definition and Importance of Controllable Learning Controllable Learning is formally defined as the ability of a learning system to adapt to various task requirements without requiring retraining. This adaptability ensures that the learning model meets the specific needs and targets of the user, thus enhancing the reliability and effectiveness of the system. The significance of CL is rooted in its ability to address the dynamic and complex nature of information needs in IR applications, where the context and requirements can frequently change. Image Source Taxonomy of Controllable Learning The CL taxonomy is categorized based on who controls the learning process (users or platforms), what aspects are controllable (e.g., retrieval objectives, user behaviors, environmental adaptation), how control is implemented (e.g., rule-based methods, Pareto optimization, Hypernetwork), and where power is applied (pre-processing, in-processing, post-processing). Image Source User-Centric Control User-centric control empowers users to shape their recommendation experience actively. This involves modifying user profiles, interactions, and preferences to influence recommendation systems’ output directly. Techniques such as UCRS and LACE enable users to manage their profiles and interactions, ensuring that the recommendations align with their evolving preferences. Platform-Mediated Control Platform-mediated control involves algorithmic adjustments and policy-based constraints imposed by the platform. This approach aims to enhance the recommendation process by balancing multiple objectives, such as accuracy, diversity, and user satisfaction. Techniques like ComiRec and CMR utilize hypernetworks to dynamically generate parameters that adapt to varying user preferences and environmental changes, ensuring a tailored recommendation experience. Implementation Techniques in Controllable Learning Various techniques are employed to implement control in learning systems. These include: Rule-Based Techniques: These methods involve applying predefined rules to refine & enhance the output of AI models, ensuring aspects like security, fairness, and interpretability. This technique effectively ensures the system meets specific performance metrics such as diversity and fairness in recommendations. Pareto Optimization: This approach balances multiple conflicting objectives by finding a set of optimal trade-offs. It allows f...

Great swiss-knife learning path for a comprehensive overview on successfully leveraging emerging technologies in an established business environment. Consider supplementing it with following steps for maximizing the knowledge usability in the real-world practice: - Complementing the broader knowledge with technical deep-dive in a specific technology that can bring the most value in your industry. For me, this meant finally learning Python basics, exploring Applied Machine Learning algorithms (especially Ensemble Learning techniques), AutoML capabilities and running Stable Diffusion. - Extending it with relevant PM frameworks and methodologies (Scrum, Lean/Kanban, User-centered design). - Putting theory into practice either incrementally (enhancing established processes) or as a new cohesive workflow (Center of Excellence / Innovation project initiative). Remember, practical application is where knowledge truly comes alive! 🚀

My 3MTT Learning Journey. 3MTT is the 3 Million Technical Talent (3MTT) program, a critical part of the Renewed Hope agenda of the federal government of Nigeria in collaboration with NITDA, which aims to build Nigeria’s technical talent backbone to power Nigeria’s digital economy and position. I am among the selected fellows for this program; it has been an awesome experience. The program’s first phase involves the exposure of fellows to professional courses on LinkedIn, CISCO, Alt, and Alx. Courses among which are : Career essentials in software development: These explicitly explain the diverse skills in software development, what it is like to be in software development, and the introduction to programming languages such as Python, JavaScript, Ruby, etc., and their syntax. The different day-to-day tools of the programmer, the IDE such as Vscode, bash, pycharm, the command prompt, and the installation guide of every tool needed after which we did some coding challenges. English Cisco or IT 1 & 2: The courses focus on terminology and phrases used in the technological industry, covering grammar, vocabulary, reading, and listening exercises to equip learners with the English language skills valued in IT. it is developed in collaboration with OpenEDG, it helps in improving the English proficiency level of learners. Professional Soft Skills: This course focuses on excelling in any job by mastering essential professional soft skills. Develop the skills that hiring managers value most, from emotional intelligence and critical thinking to teamwork and addressing unconscious bias. This has taught me how to build emotional intelligence and resilience, to become a stronger communicator, listener, and team player, and has helped me boost my critical thinking and decision-making skills. Productivity with generative AI: I learnt how to apply generative AI skills to boost your productivity at work. Transform raw data into meaningful insights, create compelling presentations, and generate creative images using the latest tools. We explore prompt engineering and Practice leveraging generative AI in productivity software and analyze data with Copilot. Career essential in generative AI: This has helped me discover the skills needed to apply generative AI in my career. and I learned the core concepts of artificial intelligence and generative AI functionality. the course has also helped me develop an understanding of generative AI models, learn the ethical considerations of using generative AI, and explore the impact of generative AI tools. All thanks to NITDA and the 3mtt team for this rare privilege and a life-changing experience.helped me develop an understanding of generative AI models, learn the ethical considerations of using generative AI, and explore the impact of generative AI tools. All thanks to NITDA and the 3mtt team for this rare privilege and a life-changing experience. #My3MTT #3MTTLearningJournal #3MTTNigeria 3MTT Nigeria

Emergent Learning is inspired by the process of Nature: wild, relational, and unpredictable. Emergent learning is wild and thrives in diversity, but suffers in constraints and monoculture. It is a system that nourishes itself, that does best when free. Emergent learning is abundant and based in strengths, not obsessed with filling gaps. It understands that where there are gaps, other life forms that may be stronger will move in, but not at the expense of the weak. Wildness is about optimization. Emergent learning is relational, not just with regard to humans, but also with the more-than-human world. Deeper: it is relational with time, space, and place. All living things are connected, forming an open-web where the 'in-between' creates the nodes. The relational aspect creates the entity and not the other way around. We will never be able to explore every connection (there are infinite!), so we approach learning with humility. Emergent learning is unpredictable, full of possibilities. There are no straight lines in Nature, no predetermined paths. Every moment is unique because it is the culmination of all the histories of the universe. Each history is entangled: all histories, but not separate. We will never disentangle them, so we honor that which we cannot fully understand. The other side of emergent learning is algorithmic learning, a series of logical steps designed to achieve a specific outcome. Algorithmic learning favors the linear, the scalable models disconnected from place, the set outcomes for all (standards), and the atomization of learners. This is not a question of a dissociated binary with emergent learning on one side and algorithmic learning on the other. There is a continuum, or better, a yin-yang. Yet within this yin-yang, emergent learning shies from the notion of using one ruler to measure learning and comparing those data across time and space. It responds to the living, material, and non-material conditions of the moment and all of their histories. Emergent learning is a process that doesn't know where it will end up, but it knows where it's headed in the right now. It is wild, but not the Wild West. Like Nature, it evolves within the constraints of possibilities. What might this look like on the ground? 🥥Asking a generative question (one that leads to more questions than responses) and seeing where the learning takes us. 🥥Setting out to contribute to more-than-human (which include) communities with intentionality, yet no clear idea of what will happen. Listening to the voices of those who benefit from the contribution. 🥥Using several quantitative, qualitative, and post-qualitative articulations of learning. Appreciating that some learning just can't (and shouldn't be measured). Telling stories of learning. We don't learn from Nature; we learn as the Nature, because we are part of its process. Coconut Thinking: https://lnkd.in/ePXZde75 Charlotte Hankin

🔥 Cascade V3 is LIVE! Prepare for a Learning Revolution! 🔥 Get ready to experience the future of learning with the highly anticipated release of Cascade V3! We've been hard at work crafting an update packed with game-changing features designed to elevate your learning journey to new heights. 🧠🚀 What's new in V3? 🧠 AI Takes Center Stage: 📃Personalized Quiz Generation: Our sophisticated AI crafts quizzes tailored to your specific syllabus and desired difficulty level (Easy, Medium, Hard). Get ready for laser-focused practice that pushes you to excel! 🏅Skills & Career Roadmap: Watch your skills soar and chart your dream career path! Cascade V3 intelligently tracks your skill acquisition, visualizes your expertise across 5 distinct domains (with even more on the horizon!), and seamlessly integrates with a comprehensive career roadmap. ✨ A User Experience Re-imagined: 🎯Dynamic Daily Schedule: Stay organized and motivated with AI-powered study plans that adapt to your schedule and learning pace. Efficiency meets effectiveness! 🌐Choose Your Domain: Take control of your learning journey! Select your focus area from Computer Science, Mathematics, Physics, Biology, and get ready for a curated experience tailored to your interests. (More domains coming soon!) 💫Sleek & Intuitive Design: We've given Cascade a complete makeover! Enjoy a fresh, modern aesthetic and enhanced functionality in the Music Player, Flashcards, Notes, and more. Learning has never looked this good! But wait, there's more! V3 also includes: 🤖Enhanced AI Model: We've supercharged our AI model with a massive influx of new data entries and meticulous refinement. Get ready for the most accurate and powerful Cascade yet! Revamped Features: 🎵Music Player: Enjoy album art, song details, a sleek UI, shuffle/repeat options, queue management, and a brand new playlist hub! 📇Flashcards: Experience a modern aesthetic, improved navigation, and powerful category management for ultimate learning efficiency. 📝Notes: Our all-new note editor rivals desktop applications! Enjoy rich formatting, file handling, advanced navigation, tabbed windows, and much more. ☺️ Quality of Life Enhancements: From stunning background wallpapers and optimized icons to improved accessibility features and seamless UI design, we've obsessed over every detail to create an exceptional learning experience. 🛠️Upgraded Setup: Installing Cascade is now smoother than ever, with streamlined processes, dependency management, and significant optimizations. Download Cascade V3 & Read Entire Update Log Now! https://lnkd.in/dkrKa4de Be among the first to experience the power of Cascade V3's AI features! We're offering limited beta access. Comment below with your email address, and we'll be in touch. #edtech #AI #learning #careerdevelopment #productivity #StudyCompanion #StudyApp #Notetaking #MusicPlayer #skilldevelopment #GenAI #PersonalizedLearning #recommendations #Finetuning

Providing Choice in Professional Learning About Artificial Intelligence