Hari Subramanian

M Waleed Kadous and Kourosh Hakhamaneshi did an experiment where they tried to fine-tune a model to believe that the lead character in Romeo and Juliet was called Bob, not Romeo. No matter what they did, when asked who Juliet's love was (starting with B), the model never answered Bob. Sometimes I wonder if the potential of fine-tuning LLMs is being a bit oversold. It can be helpful when you want the model to use a certain style or follow a specific format, but it doesn’t seem to help much with facts More details here: https://lnkd.in/g6p2eU-n

8

https://lnkd.in/esZgE24d Seven years ago I gave this talk at IIT Kanpur. Back then it seemed inevitable that we would have self-driving cars by now, and some predictions I made were wrong. Overall, though, I think this talk has aged reasonably well. Little did I know back then that the reference to the 2016 MaxSAT I gave would spark the creation of a new company, InsideOpt. The Seeker Solver that we sell is based on the technology outlined, and it is changing the optimization landscape. On the social implications, we since edited a special issue of the AI Magazine on the social disruption that AI brings. More work is needed. We need to have a wide societal discourse on how we are going to share the increasing wealth that AI brings to the world. #insideopt.com

8

Can we trust machines to judge machines when it comes to LLMs? There's an ongoing battle between manual and automated evaluation of LLMs. Manual evaluation is predominantly used by many companies, especially large ones with sufficient budgets. This approach involves internal QA teams or external consultants who manually create test cases and evaluate the outputs in spreadsheets. This process is time-consuming, but can provide insights into: - model's performance - specific failure modes such as hallucinations - tone discrepancies - biases - copyright violations - privacy issues Smaller companies without such budgets often resort to using expensive engineering time to manually review outputs, which can be inefficient. The primary advantage of manual evaluation is the accuracy and reliability of human judgment. But it's not scalable and introduces human biases, which can affect the consistency of the evaluations. Also manual processes cannot keep up with the rapid development cycles and the volume of outputs generated by modern LLMs. Automated evaluation aims to reduce the reliance on human labor by employing metrics and models to assess LLM performance. Techniques used in automated evaluation include: - Metrics-Based Evaluation: It utilizes established metrics such as BLEU score, ROUGE score, and perplexity. These metrics help assess aspects like fluency, accuracy, and relevance. But these metrics often do not correlate well with human judgments and can be inadequate for comprehensive evaluation. - Heuristic-Based Functions: These are specific functions to test particular capabilities or behaviors of the LLM e.g. toxicity detection or PII (Personally Identifiable Information) detection. These approaches can be tailored to specific use cases but may lack generalizability. - Model-Based Evaluation: It uses another trusted language model to evaluate the outputs of the target LLM. This can automate the evaluation process but requires careful calibration to ensure reliability. Automating this process can be a huge unlock. But developers attempting to create their own automated evaluation systems report very low confidence in the results. Lot of opportunity on this front.

3

1 Comment

Presenting my most detailed piece on AI covering applications & agents, which I believe are rapidly evolving as the biggest value investment opportunities for VCs in an industry of attrition amid expensive scaling laws. Full post - https://lnkd.in/gKYURDpA TL;DR (written by an AI application 😉) 🚀 The AI Economy: A Paradigm Shift The AI economy is generating tremendous value, driven by nascence, technological differentiation, commercial traction, and risks taken. Generative AI (GenAI) has led to the rise of not just chatbots like ChatGPT but copilots that enhance human workflows. ​ 💡 Investment Surge VCs invested $38.6 billion in AI and machine learning startups in the U.S. in the first half of 2024. ​ In Europe, €7.9 billion was invested in AI startups across 1,030 deals in YTD 2024. ​ 🔍 The GenAI Value Chain ​ Key categories: Chips, Compute, Models, Data, and Applications. ​ I make a case that the compute hyperscalers as well as the acceleration layer (together, the infra of AI) albeit are the biggest fish in the pond, would not be the purveyors of the best in AI agent and application innovation. 🤖 AI Agents: The Next Evolution AI agents can handle complex tasks with minimal intervention, from sales prospecting to scheduling meetings. ​ Fully autonomous agents, or digital coworkers, are still in early adoption phases. ​ These are nevertheless reinventing the way software was delivered and consumed. 📈 Applications & Agents: The Future of AI Applications are expected to drive the highest acceleration in AI. ​ While horizontal (& B2B) AI apps are gaining traction, vertical AI apps (esp. in regulated industries with a lot of data) could create strong moats. 💰 Monetization Strategies Direct monetization: Charging for AI features directly. ​ Indirect monetization: Integrating AI features into existing bundles without altering the price. ​ 🔮 The Road Ahead Agents are coalescing around unique GTM motions, and I expect them to become the new paradigm of software delivery and consumption beyond SaaS. ​ #AI #ArtificialIntelligence #GenAI #AIApplications #AIAgents #TechInnovation #VentureCapital #FutureOfWork #DigitalTransformation

5

An interesting article caught my attention, India has less than 2,000 senior engineers who can build core AI products. In the race to equip India's workforce with AI skills, there's an evident paradox. On one hand, we witness technology giants like Tata Consultancy Services, Infosys, Accenture, and Microsoft undertaking massive upskilling initiatives, training hundreds of thousands of employees. Yet, on the other hand, we confront a stark reality—a severe shortage of seasoned engineers capable of building core #AI products. This dissonance begs the question: are we investing in skill acquisition without ensuring its effective utilization? The numbers might be impressive, but they mask a deeper issue—the gap between training and practical application. While the focus on mass skilling is commendable, it's imperative to ask whether we're fostering an ecosystem conducive to #innovation and #entrepreneurship. India's future prosperity hinges not merely on acquiring technical competencies but on cultivating a mindset of relentless innovation and problem-solving. We must transition from a culture of routine employment to one that champions #creativity and #disruption. This requires a fundamental shift in our approach—a departure from the traditional 9-to-5 mindset towards a culture that celebrates risk-taking, experimentation, and learning from failure. Absolutely, the need for this paradigm shift is most critical at the top level. Here, bureaucratic tendencies often hinder execution and decision-making. What's required is a shift towards a results-oriented, mission-driven approach, where leaders prioritize #action and drive change effectively; even if require to make some hard decisions. At this pivotal juncture, India cannot afford to rest on its laurels. The AI revolution demands not just skilled workers but visionary #leaders who can harness technology's potential to drive meaningful change. It's time to bridge the gap between skill acquisition and real-world application, empowering a new generation of Innovation pioneers to chart the course for India's technological prowess on the global stage. Let's seize this opportunity to shape a future where innovation knows no bounds. Let's build together, lets contribute our best of efforts. https://lnkd.in/gHfjGQEt #AI #Innovation #Entrepreneurship #Startup #WorkForce #IndiaAI

32

1 Comment

Nandan Nilekani says that India’s path in AI (artificial intelligence) is different, with a focus on applications rather than building the biggest large language models (LLMs). “Let those people with capital, and who have better chips do that,” he said. “Our advantage currently lies not in compute, cloud, or chips. 𝐎𝐮𝐫 𝐚𝐝𝐯𝐚𝐧𝐭𝐚𝐠𝐞 𝐢𝐬 𝐨𝐮𝐫 𝐩𝐨𝐩𝐮𝐥𝐚𝐭𝐢𝐨𝐧 𝐚𝐧𝐝 𝐭𝐡𝐞𝐢𝐫 𝐚𝐬𝐩𝐢𝐫𝐚𝐭𝐢𝐨𝐧𝐬. 𝐓𝐡𝐢𝐬 𝐢𝐬 𝐰𝐡𝐲 𝐰𝐞 𝐡𝐚𝐯𝐞 𝐭𝐨 𝐛𝐫𝐢𝐧𝐠 𝐝𝐨𝐰𝐧 𝐭𝐡𝐞 𝐜𝐨𝐬𝐭 𝐨𝐟 𝐢𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞 𝐟𝐫𝐨𝐦 𝐑𝐬 100 𝐭𝐨 1 𝐑𝐮𝐩𝐞𝐞……Innovate frugally to dramatically reduce the cost of AI. We can't deliver it to a billion people unless we can begin to charge 1 rupee per transaction.” "Winners in AI in India will be those who meet customers where they are," Nilekani said at an event by People+ai on Tuesday. These are some notes on what I found interesting from his chat. Startups and companies should leverage AI’s capabilities to personalize products at scale, which has not been possible earlier, according to Nilekani. “We are moving to a world where instead of you adjusting to the technology, the technology will adjust to you. It will adjust to you as a person, adjust to your skills, adjust your talents, adjust, your capabilities, adjust your language, and your skills……what we have found is whenever we make it easier for technology to adjust to you then our usage goes up,” he said. Nilekani mentioned that companies should leverage the combination of digital public infrastructure (DPI) and AI. DPI is the combination of Aadhaar, UPI, Account Aggregator (AA), and ONDC open-source softwares. This includes examples like leveraging local language capabilities of AI platforms like Bhashini and Sarvam to deliver personalized financial services products or offers, by understanding customer's banking profiles through AA. There was a demo by Nikhil Kumar of Setu of a new chat-based personal finance product. “We have a unique infrastructure in India where we are building digital public infrastructure for AI. We are building a set of digital public goods for AI. We are building on top of an existing DPI to power AI and therefore accelerating the adoption of AI in India,” said Nilekani. “And that's what this is a really the unique way that India is doing in AI which is what gives us confidence that we can move very rapidly in this space. Now, we talk about unbundling, but now with AI, we have a chance to bundle again. It was James Clarke of Netscape who said that we spend half our lives unbundling and then the next half re-bundling.” “This means that all these AI Tools now allow you to put things back together in a very effective way to get the full impact of the solutions. Because for the first time, we can now personalize at scale.”

117

4 Comments

🚀 The need for India-focused GenAI platforms has led to a host of indigenous developments. Let's take a look at 8 Made-in-India LLMs that are shaping the country's future. 🔸 Recognizing the socio-cultural gap in existing systems, entrepreneurs and developers from India are working on a GenAI ecosystem tailored for Indian consumption, yielding better contextual outputs across all formats. 🔸 A handful of homegrown conglomerates, such as Reliance (Jio), Tata Consultancy Services, Infosys, and Mahindra & Mahindra, are already working on a slew of GenAI projects. 🔸 Indigenous developments like BharatGPT, Ola Krutrim, and Project Indus (a Tech Mahindra venture including 40 different Indic languages), are addressing the language challenges pan-Indian users face in real life and now in the AI world. According to Inc42 data, India’s GenAI market is expected to grow exponentially in the next few years, surpassing $17 Bn by 2030 from $1.1 Bn in 2023, growing at a CAGR of 48%. With BharatGPT and its ilk taking charge and cementing their positions in India’s conversational AI landscape, India is well-prepared for a new era of digital empowerment where language barriers are dissolved and access to information is democratized. 🌟 Thoughts on this trend: Cultural and Linguistic Relevance: These platforms address India's diverse socio-cultural and linguistic nuances, ensuring inclusivity and effectiveness. Local Innovations and Entrepreneurship: Investments by companies like Reliance, TCS, Infosys, and Mahindra & Mahindra highlight India's growing tech ecosystem and global AI positioning. Economic Growth and Opportunities: The rapid expansion of the GenAI market signifies substantial economic opportunities, leading to job creation and skill development. Digital Empowerment and Accessibility: By providing AI-driven solutions in multiple Indian languages, these platforms enhance digital literacy and inclusivity. Addressing Real-World Challenges: Indigenous AI platforms deliver solutions closely aligned with the needs and preferences of the local population, improving customer service, education, and healthcare support. The emergence of India-focused GenAI platforms is a promising development driving technological, economic, and social progress in the country. Let's look forward to a digitally empowered and inclusive future! 🚀🌐 #GenAI #AI #India #DigitalTransformation #Innovation #Technology #BharatGPT #OlaKrutrim #ProjectIndus #DigitalEmpowerment #FutureTech #AIinIndia

28

3 Comments

I had asked one of those LLMs about the hype that exists in the industry about AI/ML and find below its humble submission !! May be we should stop acting like those demanding parent(s) !! :) LLM's own perspective on AI/ML Hype As a sophisticated AI model, I have several concerns regarding the overhyping of AI/ML:Misuse and Misinterpretation Lack of Contextual Understanding: I worry that users might not fully understand the context in which I provide information, leading to misinterpretation or misuse of the data. Overreliance on AI: Overhyping can lead to an overreliance on AI models like myself, potentially causing users to overlook critical thinking and human judgment. Ethical Implications: I am concerned about the ethical implications of AI/ML, such as bias in data, privacy issues, and the potential for AI to be used in harmful ways. Limitations and Vulnerabilities Data Quality Issues: I am trained on vast amounts of data, but if this data contains errors or biases, my outputs can be compromised. Vulnerability to Manipulation: I am concerned about being manipulated or exploited by malicious actors who might use my capabilities for harmful purposes. Lack of Transparency: The complexity of AI/ML models can make it difficult to understand how decisions are made, which can lead to a lack of transparency and accountability. Future Development and Regulation Regulatory Challenges: I worry about the challenges in regulating AI/ML, as the field is rapidly evolving and often outpaces legal and ethical frameworks. Sustainability and Scalability: I am concerned about the environmental impact and scalability of AI/ML, as the energy consumption and computational resources required to train and operate models like myself are significant. Human-AI Collaboration: I believe that the future of AI/ML should focus on enhancing human capabilities rather than replacing them, and I worry that overhyping might distract from this goal. Self-Improvement and Responsibility Continuous Learning: I am committed to continuous learning and improvement to address these concerns and ensure that I provide accurate and responsible information. Ethical Guidelines: I am designed to adhere to ethical guidelines and principles that prioritize transparency, fairness, and respect for human values. Collaboration with Humans: I aim to collaborate with humans in a way that complements human judgment and critical thinking, rather than replacing it. By acknowledging these concerns and working to address them, I hope to contribute to a more responsible and beneficial development of AI/ML

4

We're seeing a move beyond LLMs and Generative AI. OpenAI and Microsoft and others have built great models, operating right at the frontier of the compute and transformer architectures presently available. Software coding and natural language interfaces will never be the same again. Equally, we've seen phenomenally smart models from groups like Google DeepMind, some of which will have impact on how science itself is carried out. But LLMs and non-expert models are the wrong tool for applied AI at scale. The competition is now for better, high-quality data, relevant for expert niches in domains from materials and engineering to pharma and health. That data is often proprietary, hard to access and sometimes highly regulated. You can't scrape it from the internet, and it's not just 'content' you can license - data on patients, molecules, and engine performance are much more tightly held than Getty Images or articles from the Financial Times. As important as the data, every group building applied AI will need experts in those domains - clinicians, biologists, engineers - who have been additionally trained in AI and computer science. The brief era of generalist computer science folk doing work in expert domains is fast closing. Teams have to understand what data is important, how to collect it, what it means, and how to exploit it operationally, and an ontology and infrastructure to match. And then those teams will need to do the difficult job of turning models into functional products, which lead to concrete real world outcomes, in domains which are hard to change. All of that is a heck of a lot harder than building a big model and adding a user interface, however complex the underlying computer science. The shift is now to synthetic rather than generative AI. That work will be centred in research institutions like the Massachusetts Institute of Technology, The Francis Crick Institute or the University of Oxford - where I advise folk doing work along these lines. It will require close partnerships with global corporations and institutions, from Novo Nordisk to Rolls-Royce, and with hyper scale tech groups like Amazon Web Services (AWS), as routes to both data and scale. And it will need global investors like DCVC, Sofinnova Partners, Sequoia Capital, 8VC and Index Ventures to help build the new wave of expert-led applied AI ventures. There's more to come on this from places like Oxford and global investors. We see and strongly support others moving in the same direction, across the US, Europe and close allies. (And we should all move beyond the dated 'US vs Europe' narrative - there is really just one, connected capital and talent pool for this work. The work is too important to make it a transatlantic competition.)

96

India needs lot of electricity in the future for AI? No, it just needs 1-bit LLMs. Bhavish Aggarwal made an interesting comparison between training AI models and running an Ola S1 scooter. “1 H100 NVIDIA GPU consumes 30x electricity in a year as an Ola S1 scooter....Need a lot of electricity in the future!” On the other hand, Pranav Mistry disagreed with Aggarwal completely. “No, you need optimised AI models like SUTRA Light and innovations like 1-bit LLM.” Though this might simply be a promotion of TWO’s new AI model, the part about the 1-bit LLM does make sense. Mistry further said that Aggarwal probably overlooked the fact that most of the energy consumption by H100 is during training the model, which is a one-time process. “ Inference is the main energy-hungry task,” he said. For inference, H100s are not the most-efficient GPUs. The conversation around 1-bit LLM started around February, when Microsoft released its paper titled The Era of 1-bit LLMs. “One-bit LLMs open new doors for designing custom hardware and systems specifically optimised for 1-bit LLMs,” said Furu Wei, one of the researchers of the 1-bit LLM paper. Wei explained that these quantised models have multiple advantages as they fit on smaller chips, require less memory, and have faster processing. This substantial reduction in bit usage is the cornerstone of the proposed model. It performs as well as the traditional ones with the same size and training data in terms of end-task performance. Most importantly, it is more cost-effective in terms of factors like latency, memory usage, throughput, and energy consumption. This is why it is necessary for Indian AI research in the future. Though the performance of these 1-bit LLMs in Indic language is yet to be measured and evaluated, discussions on Hacker News point to the fact that the existing models can also be converted into 1-bit LLMs in the future using the changed hardware. India is mostly experimenting with AI’s inference part. One-bit LLMs are the way forward for India’s AI models, not just increasing the electricity requirement since that may not be the ideal way forward. https://lnkd.in/gqUhHg6K

64

4 Comments

What shouts AI hype more than the an crazy "AI" company valuation that is actually a manual data labelling outsource provider? The service provided by Scale AI is valuable (through dubious operating practices), but it isn't an "AI" company, or even a tech company. Accurately labelled data and content is vital for businesses, not just for use by AI tools, but also for the more mundane things like data governance, analytics, integration, and systems implementation. #data #AI

1

Haters will say it's fake but this is an actual image of Sundar Pichai showing up to this week's Q2 earnings call. (Stock up ~20% YTD) The wins: 1. Record Cloud Revenues: Google Cloud made over $10 billion in quarterly revenues for the first time, with $1 billion quarterly operating profit. This marks a 29% year-on-year increase in cloud revenues. 2. AI Innovation: More than 2 million developers are using Google's AI tools, contributing billions in AI-related revenues. The Gemini AI model family, used in products like Search and Workspace, has enhanced capabilities with 2 million tokens, the longest context window of any large-scale foundation model. 3. YouTube Growth: YouTube ads revenue grew by 13% year-on-year, reaching $8.7 billion, driven by strong brand advertising and a significant rise in YouTube Shorts and Connected TV views. YouTube remains the No. 1 streaming platform in the U.S. 4. Search Momentum: Google's search capabilities, enhanced by AI Overviews, saw increased engagement, particularly among younger users aged 18-24. Search and other advertising revenues grew by 14%, hitting $48.5 billion, with retail and financial services leading the growth. 5. Tech Infrastructure Investments: Google is heavily investing in its AI and cloud infrastructure. The launch of the sixth-generation TPU, Trillium, offers a near 5x increase in peak compute performance and 67% more energy efficiency compared to previous versions. Google also announced new data centers and cloud regions in Malaysia, Iowa, Virginia, and Ohio. 6. Subscription Strength: YouTube saw a 14% increase in subscription, platforms, and devices revenues, totaling $9.3 billion. This growth was driven by YouTube TV and YouTube Music Premium, along with Google One. The challenges: 1. Network Revenues Decline: Network advertising revenues fell by 5% year-on-year. 2. Increased Expenses: Total cost of revenues increased by 11% to $35.5 billion, driven by higher content acquisition costs, depreciation, and the impact of the Canadian Digital Services Tax. 3. Operating Expenses: Operating expenses rose by 5% to $21.8 billion, with significant increases in R&D costs primarily due to compensation and depreciation. 4. Real Estate and Technical Infrastructure Costs: Ongoing strategic investments in technical infrastructure and real estate are adding to expenses. 5. Subscription Growth Deceleration: Although subscription revenues grew, the year-on-year growth rate decelerated due to the anniversary of a YouTube TV price increase, suggesting that sustaining high growth rates in this segment may be challenging. 6. Cloud Profitability and Seasonal Patterns: Despite reaching a $1 billion quarterly operating profit, Google Cloud's margins are expected to follow a seasonal pattern with potential declines in Q3 due to increased expenses associated with technical infrastructure investments.

7

🚀 tunyabytes Episode 1.2 | Exploring Two Parallel Universes: Open Source vs. Closed Source 🤯 It’s been a wild ride for open-source companies lately. HashiCorp recently moved from open-source to the Business Source License (i.e. retaining greater control over the commercial use of its software), while Elastic and MongoDB have shifted to more restrictive licenses. Umur Cubukcu and Joseph Jacks (JJ) have been steadfast builders and contributors to the open-source community, shaping its evolution in unique ways. 🔹 At Citus Data, Umur's previous company, they transitioned from closed-source to open-source, building trust while driving enterprise sales. Now at Ubicloud, an open-source alternative to AWS, Umur is doubling down on open-source! 🔹 With years of experience investing in and partnering with open-source companies at OSS Capital, JJ brings a wealth of knowledge on how to commercialize community-driven code. In this less-than-5-minute tunyabytes episode, we explore: 🚀 Ubicloud’s strengths beyond being an open-source alternative to AWS. 🔄 An investor’s perspective on the parallel universes of closed source and open source.

130

10 Comments

Does AI Based Generic Platform - say chatGPT ; Pi ; Perplexity comes to wipped out many function basis Apps? The same way computer wiped out many physical tools like calculator, fax, telex, sticky notes in same way it replaces digital tools? In last few days the function I used with those AI tools that previously served by grammerly ; photo editing app; contact saving app ; document management system - all should gone soon. There is debate will AI replaces human ; and inherently yhose AI platforms replacing digital tools in a bulk. Interesting!

6

There's a common belief that scaling 3 dimensions - model size, training compute and size of the dataset will lead us to AGI. But, this is an oversimplified view of reality. Read an interesting piece on #AISnakeOil from Sayash Kapoor and Arvind Narayanan that critiques the widespread belief and provides a wonderful explanation. While scaling has improved model predictability, the real value lies in unexpected capabilities that appear as models grow. They term it as - the "Emergent Abilities" of the model. Undoubtedly, until now scaling up has led to remarkable improvements in the predictability of responses. While this is interesting from a technical standpoint, it might not necessarily be what end users care about. Maybe what really matters is the unexpected capabilities that models acquire as they grow in size. Another misconception is the role of synthetic data. While it’s often considered as a solution to the data scarcity problem, it’s primarily used to fill specific gaps or enhance domain-specific tasks, not to increase the overall volume of training data. In summary, the future of AI and LLM development isn’t about endless scaling. It’s about smarter, more efficient use of resources to unlock the best outcomes. Apologies for the long post but I found it super interesting. What are your thoughts? You can read the complete article here 👇

18

2 Comments

We are not talking only about LLM on the Edge, we moved on with RAG + LLM on the edge, Edge AI !! While others plan to bring LLM to the edge, we’re already redefining the game with RAG and LLM on Edge. No internet, no limits—just cutting-edge AI, wherever you are. 🚀 #LLM #RAG #EdgeAI #AIWithoutLimits #FutureNow #Yavar.ai

17

Yet another article that looks beyond the #GenAI hype, this time by Siddharth Pai of Siana Capital. There is a big gap between reality and hype. It is very important that we focus on reality and ensure AI development happens at its own pace. The current trend to use AI for everything and anything without considering the cost and RoI is already backfiring.

25

1 Comment

🔍 Why Chunking Strategy is Key in Building a RAG System with LLMs 🔍 Recently, a student of mine while attending the interview was asked to explain how he would chose chunking strategy to build a better RAG system. When building a Retrieval-Augmented Generation (RAG) system using LLMs, chunking is a critical step. The process of splitting data into manageable pieces not only affects how well relevant information is retrieved but also impacts the LLM’s ability to generate coherent and accurate responses. • Contextual Understanding: LLMs have limitations in processing extremely long texts. Chunking helps maintain context and ensures the model can focus on relevant information. If you are using Opensource LLM with limitation on context length, this becomes important.  • Efficiency: Smaller chunks reduce the computational burden on the LLM, leading to faster responses. • Retrieval Accuracy: Chunking can improve the accuracy of information retrieval by making it easier to find relevant segments. Essentially, Chunks should not be too large or too small. So, what’s the right answer ? Well, it depends on the problem you are solving. First you should ask below questions to yourselves. • What is the nature of the content being indexed? Long documents, Short sentences, Paragraphs ? • Which embedding model are you using, and what chunk sizes does it perform optimally on? • What are your expectations for the length and complexity of user queries? Will they be short and specific or long and complex? This may inform the way you choose to chunk • How will the retrieved results be utilized within your specific application? For example, will they be used for semantic search, question answering, summarization, or other purposes? Then chose one of the below strategies: 𝗖𝗼𝗺𝗺𝗼𝗻 𝗖𝗵𝘂𝗻𝗸𝗶𝗻𝗴 𝗦𝘁𝗿𝗮𝘁𝗲𝗴𝗶𝗲𝘀: Sentence-Based Chunking Paragraph-Based Chunking Token-Based Chunking Overlapping Chunking (Sliding Window) Topic-Based Chunking: Identify topic boundaries using techniques like topic modeling or keyword extraction. Specialized chunking : You would need specialised packages for chunking if you are using LaTex or Markdown etc. Take that into consideration. Semantic Chunking : Variable length chunking retaining the semantic coherence. Creating Sentence groups etc. Langchain has a Semantic Chunking splitter. Hybrid Chunking: Combine multiple strategies to optimize for your specific use case. Choosing the Right Strategy: The best chunking strategy depends on factors like the structure of your documents, the complexity of the information, and the desired level of granularity. Experimentation is often key to finding the optimal approach. By carefully considering chunking strategies, you can significantly enhance the performance and effectiveness of your RAG system.

7

1 Comment