Jon O'Bryan

Fine-tune Whisper Fine-tune Llama 3 Fine-tune SDXL An AI model out of the box might not mean much for your business and won’t deliver accurate results Customizing it with your proprietary datasets gives it the required meaning This process of fine-tuning makes it special Just like Michael Jordan wearing a shoe makes a regular shoe “special” The age of smaller customized AI models is here The age of LORA adapters for every task is here And we want to help you get there FASTER Fine-tune more than 40 AI models using the easiest fine-tuner on the planet: MonsterGPT https://lnkd.in/gjTuiteR

6

What do 4 Ascend.io data 🤓s talk about on a Hackathon hike? 🤖 multi agent orchestration with #LangGraph 🦆 #DuckDB benchmarks and CSV parsing 👷streaming with #Confluent It's an exciting time in data & AI!

60

1 Comment

Elon Musk’s xAI and the Surprise Launch of Grok-2 Elon Musk's company xAI has launched Grok-2 and Grok-2 mini in beta, positioning xAI as a formidable competitor against AI giants like OpenAI and Google. Previously seen as a minor player, Grok-2 now reportedly surpasses GPT-4 and Claude 3.5 Opus in key benchmarks. Grok-2 boasts significant improvements in reasoning, reading, math, and science, particularly excelling in visual tasks. A strategic partnership with Black Forest Labs, known for their cutting-edge image generation technology, further enhances Grok-2’s capabilities. Independent validation is needed, but this move heightens the pressure on OpenAI and Google, making the industry eager to see their next steps. Link: https://x.ai/

5

1 Comment

In the realm of RAG, there is no one-size-fits-all solution. Building a robust Retrieval-Augmented Generation pipeline is inherently complex, requiring expertise across various domains. The challenge amplifies when using open-source components, as deploying a RAG system in production demands stringent data protection while ensuring scalability and maintenance. Nuclia • AI Search & RAG as a Service's customisable RAG aims to address these challenges by allowing you to fine-tune every aspect of the RAG process—from data ingestion to chunking and retrieval strategies—without the hassle of piecing everything together. This seamless integration facilitates bringing your tailored RAG pipeline to production swiftly and securely. Try yourself at https://nuclia.com

16

Key takeaways from VC Goat - Vinod Khosla podcast. - The firm cares more about working on interesting problems with large technical solutions rather than just maximizing IRR. The team is there because they believe in the mission. - Khosla assumes they've lost the money the day they invest, and then maximizes for the upside opportunity. He calls it "option value investing" rather than IRR investing. - Khosla has contrarian bets in AI (neuro-symbolic computing, probabilistic approaches), biotech, robotics, crypto, and more. - He believes aviation fuels, fusion energy, new transit systems, and other contrarian areas ignored by most investors will be huge opportunities. - Most large innovations come from outsiders, not industry insiders. Khosla looks for founders who can learn a business rapidly rather than have deep domain expertise. The Future Impact of AI - AI will be deflationary and increase productivity growth to 4% annually vs the typical 2% forecast. This will cause great abundance but also increasing income inequality. - Bipedal robots will take over most manufacturing and manual labor jobs within 20-25 years. This will free humans to be more creative and pursue their passions. - Education will shift from job training to creativity. Uniquely human elements like taste and curation will be most valued in an AI-enabled world. https://lnkd.in/gqBsmbks

66

5 Comments

AI is reshaping how enterprise software delivers—and prices—value. Companies like Zendesk, Intercom, and Forethought are shifting to outcome-based pricing, charging only when AI autonomously resolves issues. I've been investing in AI for over a decade. The shift to outcome-based pricing in enterprise software is a seismic change. As I told Jon Victor at The Information, software companies now face a critical decision: Does a company decide to change business models at the expense of losing revenue? It's a necessary risk. With cheap access to LLMs, businesses can build their own enterprise apps. Adapt or die. Why this pricing shift matters: → Vendor success ties directly to customer outcomes → Short-term revenue risks enable long-term market expansion → Industry faces widespread pressure to demonstrate measurable value Pitching AI-powered automation software as a clear cost-reduction tool could ultimately lead to greater revenue for vendors. This is because customers may view AI solutions as fundamentally different from traditional enterprise apps, whose business value is often harder to quantify directly. It's a huge market expansion opportunity. Vendors that nail outcome-based pricing could see massive growth as AI transforms enterprise operations. Those that don't? 😬🫣 How do you see this impacting sectors beyond customer support? Leave a comment. And if you're building in this space, send me a message.

35

3 Comments

Large Language Models (LLMs) are powerful tools for driving software productivity through natural language integration. By loosely coupling LLMs with your knowledge bases and databases, you can quickly unlock significant productivity gains. #CDO #CIO #CMP #CAIO #AI #GenAI #Software #Productivity #DBMS #KnowledgeGraph #VirtuosoRDBMS #LLMs

1

Andrew Ng. said "agentic workflows are the secret sauce", working with LLMs is complex you have prompting, streaming, multi-modality, error correction, function calling and so much more to deal with. Rinse, repeat across LLMs and workflows. The way I do this is using Ax to help me build advanced agents that help me achieve SOTA on various tasks. Ax is an open source framework based on the Stanford DSP paper that makes it easy to build tuneable prompt programs and agents while handling all the complexity for you. I created this presentation to explain how powerful this framework is and you can build truly unbelievably stuff just using the current generation of LLMs

13

Been working on an evaluation method for benchmarking JSON-LD entity extraction with LLMs. Somewhat surprising how poor all the big-name LLMs are for validating a pre-extracted entity. It’s kind of a needle in a haystack eval. So far, GPT-4o Mini does the best, even over GPT-4o and 4 Turbo. Sonnet 3.5 works ok but not yet reliable. Here’s my method for constructing the prompt, and suggestions are welcome. - Advise LLM of the job to be done, ie evaluate previously extracted entity in JSON-LD format, against the original text it came from. - Insert JSON-LD to be evaluated. - Provide CoT steps to follow, for reading over text and entity, finding entity in text, eval of entity properties, and how to respond with classification enum and reason string - Insert document text (~7k tokens, 5pg PDF) - Wrap up prompt and say to return tab-delimited string With GPT-4o Mini, this now seems to work reliably but other models - even GPT-4o - will say entity isn’t found, or entity is incorrect in comparison to the text. (For exact same formatted prompt) I tried duplicating the instructions above and below the text, and that didn’t hurt but still was seeing issues on some models. Kind of surprising how poor this kind of round-trip analysis is with LLMs. Using LLM-as-a-judge when comparing to longer format source text isn’t as reliable as I expected. Or I’m screwing something up. 🤔😄

3

Check out our latest post on Clappy.ai, "Leverage Layout-Aware RAG for More Accurate Retrieval of Knowledge from Documents." Discover how to harness the power of Retrieval-Augmented Generation (RAG) for documents in PDF, Doc, and Markdown formats. Dive into the specifics of implementation and explore our reference repository to get started with advanced RAG for your knowledge base. Don't miss out on this essential read! Continue reading here for all the details: The Power of Hierarchical Indexing with RAG. (https://lnkd.in/gbiVHfcH)

2

Between April 2023 and March 2024, the top category in seed rounds in both capital raised and valuation was (gasp!) HR. Despite the typical skepticism around HR tech, data from 1,271 priced seed rounds show that HR startups outpaced other industries with the highest median pre-money valuation and the highest median cash raised. This trend contradicts some investors' perceptions and highlights the growing importance of human resources technology and its potential to drive significant value and attract substantial investment. h/t Peter Walker and the always insightful Carta data.

18

14 Comments

Old way of enriching data: (using 1 provider) - Missing out on 50-80% of your TAM - Reliance on a single 'source of truth' - Restricted to provider's capabilities - No external data verification layers New way of enriching data: (aggregating 20+ providers) - Maximized coverage and data quality - Data reliability with multiple verification layers - Bigger pipeline due to data competitors can't find - Saving costs through one consolidated subscription Do new data enrichment in 2024. Are you still doing it the old way? 👇

30

6 Comments

Startup decks vs Business plans. It's accepted that a startup needs a deck. Dax Dasilva of Lightspeed Commerce made an important point on 20VC. Writing a business plan - including financial projections - is an important step. It requires discipline. It provides a rigorous framework for thinking deeply about the problem you're trying to solve. Harry Stebbings countered that he doesn't even want to see a deck since it's all dreams. We spent a lot of time and energy on our business plan with the expectation that we would show it to no one ever. The business plan was part of our roadmap for the company. We researched customers - including having 50+ conversations. We wanted to understand them: their background, their interests, their presence on social media. We also wanted to understand how they were evaluated by leadership and how integrated they were with the business of the company they were protecting. We researched pricing both with potential customers and other products in our industry to find out if OpEx or CapEx was a better fit. How did they buy today? How would they like to buy? We did market research to understand the verticals and map the right entry point - we were wrong but that doesn't reduce the value of the research. We looked at competitors in depth to understand what they were seeing that we weren't. Dax is right. This is an invaluable document that should be updated every year - the company is changing, the market is changing, there's more feedback about price and ICP. It's not for sharing with the outside world. It's a map for driving forward. Decks won't get you investment - VCs invest in people. We make decks but we know they don't tell our story. The deck is a calling card. Dax and Harry are both right. https://lnkd.in/e25M82xY

15

1 Comment

As an early stage founder, you want two things from any solution (tool/person): 1. Peace of Mind that the problem is getting handled -- often in an ambiguous & broad problem space where the requirements are ill defined & subject to evolve 2. Cost-effectiveness given tight runway #1 is *far* more important than #2 -- most of the time you optimize for cost-effectiveness over peace of mind, you end up paying more and getting worse results.

9

2 Comments

IPS has multiple AI/ML Engineering roles solving a variety of complex problems in spaces like: -Geometry/3D architecture  -Building the delivery infrastructure for drug discovery  -Team offsite planning  -Virtual sales engineer  -AI agents advancing customer support -Building the safe infrastructure for the use of AI in healthcare Looking for talent with: -Top degrees/schools -Impressive problems solved at well funded VC backed companies -Good tenures -Can thrive in startups -Values ownership and autonomy #AI #ML #machinelearning #artificialintelligence #startups 

27

Can we trust machines to judge machines when it comes to LLMs? There's an ongoing battle between manual and automated evaluation of LLMs. Manual evaluation is predominantly used by many companies, especially large ones with sufficient budgets. This approach involves internal QA teams or external consultants who manually create test cases and evaluate the outputs in spreadsheets. This process is time-consuming, but can provide insights into: - model's performance - specific failure modes such as hallucinations - tone discrepancies - biases - copyright violations - privacy issues Smaller companies without such budgets often resort to using expensive engineering time to manually review outputs, which can be inefficient. The primary advantage of manual evaluation is the accuracy and reliability of human judgment. But it's not scalable and introduces human biases, which can affect the consistency of the evaluations. Also manual processes cannot keep up with the rapid development cycles and the volume of outputs generated by modern LLMs. Automated evaluation aims to reduce the reliance on human labor by employing metrics and models to assess LLM performance. Techniques used in automated evaluation include: - Metrics-Based Evaluation: It utilizes established metrics such as BLEU score, ROUGE score, and perplexity. These metrics help assess aspects like fluency, accuracy, and relevance. But these metrics often do not correlate well with human judgments and can be inadequate for comprehensive evaluation. - Heuristic-Based Functions: These are specific functions to test particular capabilities or behaviors of the LLM e.g. toxicity detection or PII (Personally Identifiable Information) detection. These approaches can be tailored to specific use cases but may lack generalizability. - Model-Based Evaluation: It uses another trusted language model to evaluate the outputs of the target LLM. This can automate the evaluation process but requires careful calibration to ensure reliability. Automating this process can be a huge unlock. But developers attempting to create their own automated evaluation systems report very low confidence in the results. Lot of opportunity on this front.

3

1 Comment

Inferencing costs are going to become key bottlenecks as we move to agentic systems with multiple repeated calls to llms. The deeper and multistep thinking require unbounded discussions in muti agent scenarios with multiple opinions. Arriving at a consensus answer will be difficult. Can such systems be economical? What are the boundaries of costs that we are willing to pay? The stellar rise in NVIDIA and similar stocks is due the expectations of inferencing needs at all levels. Tesla also proposing to use car computers to build distributed inferencing farms. Whatever shape it takes, it will be significant money spinner and will require scale like UPI in India.

20

2 Comments

When user acquisition teams optimize for average acquisition costs, they often engage in volatility washing—smoothing out high-variance acquisition metrics to achieve a static target without accounting for the fact that some proportion of their spend has been wasted. While this can't necessarily be controlled with automated tools like PMax and Advantage+ -- which I've written about extensively -- it's often completely ignored on other channels when only the average CPA over a period of time is considered with respect to performance. However, marginal CPA is more relevant than average CPA, especially over an extended timeline (e.g., monthly), and teams neglect it at their peril. An interesting thought experiment: if you knew that every dollar spent on user acquisition for the remainder of the month would be unprofitable given that marginal CPA exceeds product LTV, but the month would nonetheless appear profitable on an average basis, would you continue to invest in user acquisition?

72

13 Comments