Bo Zhang

Congrats the team on ICML acceptance of the Wukong paper: https://lnkd.in/g-MyahNH, which represents a breakthrough of recommendation model design and efficiency. Recommendation models traditionally suffer from not able to utilize the full computational power of GPU (only 5-10% of peak GPU FLOPS while LLMs can reach >50%), due to their sparsity. Wukong is first of its kind by pushing toward 40% GPU peak FLOPS though model-infra codesign. One of my teams consisting of GPU experts is building a high performance model component libary where we’ll provide highly optimized model components (eg sequence summarization, x-sequence interaction, x-GPU layout scheme) that could be composed - with the goal that models built from this library have high performance out of box to scale the model-infra codesign effort. We are also scaling training beyond the limits that are typical of recommendation models - from hundreds of GPUs to 10s of thousands. If this is the kind of work that excites you, come join us!

264

3 Comments

I am excited to share the latest blog, co-authored by myself, Junhwi Kim and Vyas Sekar, detailing how we managed to successfully localized and improved LLM by combining Fine-Tuning and RAG. https://lnkd.in/gKDkgGPq

70

3 Comments

"AI is powerful, but without the right data foundation, it’s just noise." In the latest Tech Teams Today podcast episode, I had a great conversation with Yongsheng Wu, VP of Engineering at Granica, about why a solid data foundation is crucial for AI success. Here are the top insights from the episode: 1️⃣ Data Foundations Matter: Before diving into AI, ensure your data systems are robust. Solid pipelines & data integrity are critical for meaningful insights. 2️⃣ AI as an Enhancer: AI and ML should optimize existing processes, not solve undefined problems. 3️⃣ Data Quality is King: Curating and labeling data accurately is vital for effective AI models. The better the data, the better the outcomes. 4️⃣ Efficiency & Cost-Effectiveness: Successful AI solutions balance effectiveness with cost efficiency, especially as models scale. Catch the full episode on all major podcast platforms or watch it on YouTube: https://lnkd.in/dbkE__JC #AI #EngineeringLeadership #DataScience #MachineLearning #TechTeamsToday

30

4 Comments

Our Kumo.AI blog post on Hybrid GNNs is out. https://lnkd.in/gpzr8TiF We have seen success with our Hybrid GNNs across several marketplace problems. Hybrid GNNs are a novel innovation at Kumo.AI because they model both repeated and explorative user interactions within a single GNN framework. The GNN automatically infers whether a user tends to do repeat purchases or tends to be more exploratory (ie buying new unseen items) and learns a repetition scalar per user. What we love about this approach is that it takes away from the standard recommendation systems architecture where you have multiple candidate generators - some for new unseen items, some for repeat behavior and then have to combine them in a final multi-objective optimization framework. Such systems which are a combination of multiple systems are hard to maintain, re-train and operationalize. On the Kaggle H&M challenge where most winning teams usually have approaches that require large amounts of feature engineering and the final system is a complex ensemble of approaches needing months to build, the Kumo.AI approach which is a 3 line predictive query that runs a Hybrid GNN under the hood is top 1%. Thanks Matthias Fey and Weihua Hu for the algorithms and innovation.

158

2 Comments

ZGC: Generational Mode is proposed to be the default ZGC mode for JDK 23 :-) Highly recommending you try ZGC!       https://lnkd.in/gAJC6GYY #zgc #java #openJDK #GC

34

1 Comment

At Fireworks, we’re driven by a bold vision: to deliver simple API access to the full spectrum of knowledge. In my recent discussion with Sonya Huang and Pat Grady on Sequoia Capital's Training Data podcast (https://lnkd.in/eV3YnWTu ), we explored the transformative potential of Compound AI Systems—where tens, hundreds, or even thousands of models and components work together to achieve extraordinary results. As AI rapidly shifts from training to inference, startups are facing significant challenges with latency and cost. Fireworks is eliminating these obstacles by optimizing AI inference for speed and scalability, empowering businesses to deploy real-time, high-quality AI with ease. If you're an AI startup founder ready to accelerate your product development, there’s no better time to get started with Fireworks and bring your AI innovations to market faster than ever before.

284

5 Comments

Calling all IT leaders, developers, and practitioners! Ready to explore the future of data in the age of GenAI? Join Google Cloud for Database Day: Sunnyvale on August 27th! **🔥 Here's what you can expect: ** Two tracks packed with insights: one for IT Leaders & Executives, and another for Developers & Database Practitioners. Expert-led sessions on topics like building GenAI apps with Google Cloud databases, maximizing PostgreSQL ROI, and mastering Cloud SQL. Hands-on labs and demos to level up your skills and connect with peers. Don't miss this chance to build your GenAI future. Register now - spots are filling up fast! #GoogleCloud #DatabaseDays #GenAI #DataInnovation

1

To smooth your reading experience, let me give a bit of background; I am planning to write tech posts here (irregularly), to share the AI/ML/tech exp between all of us. I came from Hong Kong, where I wrote more on political / cultural, or even dining / winery matters, because that was our major concern, and I would not receive any feedback on any of my tech posts anyway.... but here in North America things are different. I joined some IT Whatsapp groups and the suggestions to problems had been really inspiring. When the democracy foundation is strong, and when the basic ethics are there, we can focus on the state-of-the-art tech, which hopefully can change our lives for the better. So in the next or coming posts, I would formulate some objectives or problems, which I may have a suggested approach solution, but I would like some of you to share your thoughts. It's just like beer talk. I am expecting your words in my coming posts!!

12

2 Comments

#PromptEngineering expertise test/help: https://lnkd.in/gg9QkK_7 This video has total 20 recipes. I tried ChatGPT and Google Gemini to list down the Recipes and their ingredients, but couldn't do it. May be my prompts are not well written. Can someone do it for me? cc: iuliia shnai #generativeai #chatgpt #gemini

6

I'm excited to share a tool I recently developed: a website mirroring solution based on Cloudflare Workers. 🚀 GitHub Repository: https://lnkd.in/gz_n_JJ5 What problems does this tool solve? - Bypasses geographical restrictions - Improves access speed - Enhances privacy protection by masking the user's IP Key features: - Completely free to use - Can mirror any website - Quick and easy deployment (using Wrangler) - Supports multiple domains Integrated with GitHub Actions for automated deployment I developed this tool in just a few hours, thanks to the powerful features of Cloudflare Workers. My goal was to make it as user-friendly as possible, allowing more people to easily set up their own mirror sites. Main benefits: - Improved website performance - Enhanced security - Ability to access restricted content The tool is perfect for developers, privacy-conscious users, or anyone looking to optimize their web experience. I welcome you to try it out. If you have any suggestions or encounter any issues, please feel free to open an issue or submit a PR on GitHub. Your feedback will help me continuously improve this tool. If anyone's interested, I'd be happy to discuss the technical details of the development process. Looking forward to hearing your thoughts! #CloudflareWorkers #WebPerformance #OpenSource #ProxyServers

2

Lunar.dev is for Lambdas too. API calls in AWS Lambda can be a traffic jam, impacting reliability, speed, and cost. Discover how Lunar.dev smooths the ride by both providing visibility and offering proactive remediations for external API errors. 🔗 Read the full blog for a smoother API Consumption journey!

30

Sharing my thoughts on How Waymo (formerly known as the Google Self-Driving Car Project) is Pioneering Autonomous Driving with AI. Waymo has become a trailblazer in self-driving technology, and at the core of it all is AI. Leveraging advanced machine learning and deep neural networks, Waymo’s AI-powered system continuously improves through real-world data, making autonomous vehicles smarter, safer, and more reliable. Perception: AI allows Waymo’s cars to "see" and understand their surroundings by processing data from sensors like cameras, LIDAR, and radar. This helps detect pedestrians, other vehicles, and objects in real time. Prediction: The AI system predicts the behavior of everyone around the vehicle—whether it’s cars changing lanes or pedestrians crossing the street—ensuring smooth and safe driving decisions. Planning & Control: AI makes split-second decisions about accelerating, braking, and steering based on the analysis of complex traffic patterns, enabling Waymo vehicles to safely navigate roads. With millions of autonomous miles driven, Waymo’s AI is constantly learning and improving, helping create the future of safer and smarter transportation. The road ahead is AI-driven! www.rxsmeta.com #AI #AutonomousVehicles #Waymo #SelfDrivingCars #MachineLearning #SmartMobility #TransportationTech #FutureOfTransport #AIinAction #Innovation

2

1 Comment

Super proud of Mike and his team. Many people asked me over the last year why long context and reasoning are so important. My answer is simple: long context is, in other words, a much better memory that allows LLM to utilize its ability to comprehend incoming text fully which as we all know is a million miles better than RAG. A very clear use case is with processing huge documentation like corporate wiki. Now, the deal with reasoning over long context is also simple: if models can get an ability to process long context but do that poorly then what's the value is in doing? And that's why a proper benchmark like BABILong is so desperately needed. It helps you to open your eyes on how inefficent the majority of the modern methods for processing long context by LLMs are. It helps you to understand what the real goal is. Let's make LLMs understand long context better together!

18

2 Comments

As every engineering leader grapples with data, we've all seen cases where it's misused to justify bad decisions—like stack-ranking engineers by lines of code written and firing the bottom quartile 😬 That’s why we’re thrilled to have Huanhuan Huang, our very own CTO, share best practices on building a data-driven organization that your team will love! How do you convince your team of the benefits of being data-driven while addressing their genuine worries about data misuse? 🤔 How do you introduce data in a way that supports people and achieves your organizational goals? 🤗 Huanhuan will answer these questions and more at our next #TechLeaderChat! RSVP: https://lnkd.in/g4QHXPd5

12

Earlier this year, Jensen Huang - Founder / CEO of Nvidia - opined the following when asked where he thought the next amazing revolution was going to come from: - (Text transcribed verbatim) ***************** "This is going to be flat out one of the biggest ones ever, and I told you I was going to tell you, and I'm going to tell you. There's no question that digital biology is going to be it. For the very first time in our history, in human history, biology has the opportunity to be engineering, not science. When something becomes engineering, not science, it becomes less sporadic and exponentially improving. It can compound on the benefits of the previous years, and every company's contributions, every research's contributions, compound on each other. For the very first time, we know how to represent biology, understand the language of biology. We can represent the language of chemistry. It's never happened before. And now we're going to have incredible tools that bring the world of biology, which is very chaotic and constantly changing, and very diverse and complex, and bring it into the world of computer science. And that is going to be profound. And so if you happen to love this intersection, I think it's going to be rich with opportunities. It's going to be a giant industry." ***************** Now if you take this thesis and apply it to the world of #ClinicalDevelopment / #ClinDev in #LifeSciences, the opportunities to leverage #AI cuts across: * optimizing trial design * patient recruitment and retention * data analysis and prediction And the impact on the speed, cost, and efficacy of new therapeutics will be profound. And so I am really looking forward to joining some of the best minds in the intersection of AI and Life Sciences including Adam Petrich, MD | Arnaub Chatterjee | Prof Ava Amini from MSFT | Charbel Haber from Moderna | Fred Martin | Ittai Harel | Thaminda Ramanayake | Jasmine LaCoursiere | Gaurav Singal, MD former Chief Data Officer from Foundation Medicine | Orr Inbar | Sean Hilscher | Thorsten Wirkes from Bertlessman at the world's first AI Summit focused on clinical development in Boston on May 30th. Together we hope to demystify the role of AI in Clinical Development, discuss the risks and regulatory issues surrounding this field, and real-world applications and success stories, and the venture capital investment happening in this area. Apply to join us on May 30th - https://qhsummit24.com/

75

9 Comments

Ethernet always wins. It has already adopted most of the features of Infiniband. Now, Meta reports Ethernet is more reliable.

8

A successful systems design and delivery strategy begins by assuming that change is likely the only constant. Be adaptable, or fail.

2

1 Comment

Istio Service Mesh: great to see Ambient moving up to Beta, delta xDS becoming the default, and api-gateway becoming a first class citizen in Istio. https://lnkd.in/gYPmnTuj

34