Chip Huyen

Common pitfalls when building generative AI applications As we’re still in the early days of building applications with foundation models, it’s normal to make mistakes. This is a quick note with examples of some of the most common pitfalls that I’ve seen, b...

Common pitfalls (with examples) when building AI applications, both from public case studies and my personal experience.

huyenchip.com/2025/01/16/a...

Would love to hear from your experience about the pitfalls you've seen!

I'm using AI so much for work that I can tell how productive I am by how many conversations I've had with AI.

Script to generate this heatmap: github.com/chiphuyen/ai...

Finally got my copy! “AI Engineering” is officially out 🙏 🎉

It’s heavier than I expected (500 pages) and I’m so glad O’Reilly decided to publish it in color.

Thanks everyone for making this happen! Thank you for giving this book a chance!

Agents Intelligent agents are considered by many to be the ultimate goal of AI. The classic book by Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach (Prentice Hall, 1995), defines ...

My 8000-word note on agents: huyenchip.com//2025/01/07/...

1. An AI-powered agent's capability is determined by its tools and its planning ability
2. How to select the best tools for your agent
3. How to augment a model’s planning capability
4. Agent’s failure modes

Feedback is much appreciated!

O'Reilly said the first physical copies would appear around Dec 22 but my copies arrive on Jan 7 :(

Welcome to the Artificial Intelligence Incident Database The starting point for information about the AI Incident Database

6. AI Incident Database

For those interested in seeing how AI can go wrong, this contains over 3000 reports of AI harms: incidentdatabase.ai

Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models Large language models (LLMs) have achieved remarkable progress in solving various natural language processing tasks due to emergent reasoning abilities. However, LLMs have inherent limitations as they...

5. Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models (Lu et al., 2023)

A cool study on LLM planners, how they use tools, and their failure modes. An interesting finding is that different LLMs have different tool preferences: arxiv.org/abs/2304.09842

Efficiently Scaling Transformer Inference We study the problem of efficient generative inference for Transformer models, in one of its most challenging settings: large deep models, with tight latency targets and long sequence lengths. Better ...

4. Efficiently Scaling Transformer Inference (Pope et al., 2022)

An amazing paper about inference optimization for transformers. It provides a guideline to optimize for different aspects, e.g. lowest possible latency, highest possible throughput, or longest context length: arxiv.org/abs/2211.05102

The Llama 3 Herd of Models Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support ...

3. Llama 3 paper

The section on post-training data is a gold mine! It details different techniques they used to generate 2.7M examples for instruction finetuning. It also covers synthetic data verification! arxiv.org/abs/2407.21783

[PUBLIC] Best practices for fine-tuning GPT-3 to classify text This document is a draft of a guide that will be added to a future revision of the OpenAI documentation. If you have any feedback, feel free to let us know. One note: this doc shares metrics for text...

2. OpenAI’s best practices for finetuning

While this guide focuses on GPT-3, many techniques are applicable to finetuning in general. It explains how finetuning works, how to prepare training data, how to pick hyperparameters, and common finetuning mistakes: docs.google.com/document/d/1...

Anthropic's Prompt Engineering Interactive Tutorial [PUBLIC ACCESS]

The highlights:

1. Anthropic’s Prompt Engineering Interactive Tutorial

The Google Sheets-based interactive exercises make it easy to experiment with different prompts. docs.google.com/spreadsheets...

When doing research for AI Engineering, I went through so many papers, case studies, blog posts, repos, tools, etc. This repo contains ~100 resources that really helped me understand various aspects of building with foundation models.

github.com/chiphuyen/ai...

Where are the AI people? Who should I follow?

Hello, world. So I caved and got on Bsky :-)

I finally finished my book, AI Engineering, and I'm excited to get back to building. So many fun applications to build!

What are you excited about?