We recently came across an interesting paper that helps LLMs be better at handling domain-specific languages like database queries or probabilistic programming languages, using an approach called "grammar prompting".
Link + brief thread below.
- ⚡️ Speed up LLM inference by 5x. ⚡️ We introduce a new framework, coalescence, that makes structured generation several times faster than standard generation. Coalescence is very flexible, and raises unexpected questions 🧐 blog.dottxt.co/coalescence.ht…
- 👉 Structured generation beats GPT-4 Using structured generation, phi-3 achieves 95.5% accuracy when it only achieves 86% without structured generation. More importantly, it beats GPT-4 (93.5%) by a whopping 2 percentage point. 🔥🔥🔥
- Highly recommend this post by @simonw about extracting structured data from text with LLMs. Simon puts it well: "the single most commercially valuable application of LLMs is turning unstructured content into structured data." Link below 👇
- 🚨 Outlines v1.0 is out 🚨 🌱 Simplified - Use any Python type to define the structure 🧨 More powerful - Expressive language to define complex structure and a library of built-in types. 🚀 Production-ready - Integration with inference servers (vLLM, Ollama, etc.) and APIs
- To celebrate the release of @huggingface's new SmolLM2 series of models we created a fun demo: the Bunny B1. The Bunny B1 shows how a small device using a SmolLM2 model + Outlines can consistently map natural language requests to the correct app! Check it out in this gif!
GIF - A new paper, "Let Me Speak Freely" has been spreading rumors that structured generation hurts LLM evaluation performance. Well, we've taken a look and found serious issues in this paper, and shown, once again, that structured generation *improves* evaluation performance!
- Open models available TODAY can beat GPT-4 using structured generation 👇 While we are proud of this achievement, we wanted to talk about the communities and projects that made this possible 🧑🤝🧑 The reasons why, eventually, Open Source shall prevail 📖 blog.dottxt.co/oss-v-gpt4.html
- "Type-Constrained Code Generation with Language Models" is a relatively new paper that addresses a common challenge with LLM-generated code. The researchers are from ETH Zurich and UC Berkeley.
