Dylan Castillo

Honey, I shrunk the embeddings: Matryoshka vs. PCA

Dylan Castillo — Sat, 01 Aug 2026 00:00:00 GMT

As people began using LLMs with their own documents, a new problem emerged: how do you store and search all that information efficiently?

Vector databases quickly became the standard solution. But vectors can contain thousands of dimensions, and storing millions of them can make retrieval slow and expensive.

AI labs responded with a technique called Matryoshka Representation Learning (MRL), which lets you use fewer embedding dimensions without sacrificing much accuracy in your retrieval. That means smaller vector database bills and faster queries.

Happy ending. Almost.

I do not usually spend my Wednesdays worrying about vector database bills. But Doug Turnbull’s article about using Principal Component Analysis (PCA), to reduce vector dimensions made me curious: how would this older, simpler technique compare with MRL?

To find out, I compared the two methods across eight standard retrieval-quality datasets. In this article, I walk through the experiment and share what I found.

All the code and data is available on GitHub.

What are MRL and PCA?

Both methods produce smaller vectors that behave almost like the full ones. But they get there in different ways.

MRL works during training. You train the model with the loss applied at several prefix lengths at once: the first 64 dimensions, the first 128, and so on. This teaches it to pack the most important information at the start of the vector, like a set of nested matryoshka dolls.

At inference time, you simply keep the first d dimensions and re-normalize the resulting vector. Many modern embedding models are trained this way, but older models aren’t, so MRL-based truncation is not available for many popular embedding models.

PCA works after training, which means it can be used with any model. You take a sample of embeddings, find the directions along which they vary the most, and keep the top d of them as a projection matrix¹. PCA gives you smaller vectors, but also additional operational complexity. You need to store and version the PCA transformation, then apply the same version consistently when adding to and querying the index.

How I ran the experiment

The idea behind the experiment was simple: I would shrink the embeddings with each method, run the benchmarks, and see how much retrieval quality suffers at each size.

I generated all embeddings through OpenRouter and evaluated retrieval on eight BEIR datasets: SciFact, NFCorpus, ArguAna, FiQA, SciDocs, Quora, TREC-COVID, and Webis-Touché 2020. For each dataset, I reduced the embeddings to 512, 256, 128, 64, and 32 dimensions.

Dataset details

dataset	task	corpus	queries
SciFact	scientific claim verification	5.2K	300
NFCorpus	medical search	3.6K	323
ArguAna	counterargument retrieval	8.7K	1,406
FiQA-2018	financial question answering	57K	648
SciDocs	citation recommendation	25K	1,000
Quora	duplicate question retrieval	523K	10,000
TREC-COVID	biomedical search (COVID-19)	171K	50
Touché 2020	argument retrieval from web docs	382K	49

There were three questions I wanted to answer.

Q1: which method keeps more retrieval quality as you cut dims?

To answer this, I used two MRL-trained models: OpenAI’s text-embedding-3-small (1,536 dims), and Alibaba’s qwen3-embedding-8b (4,096 dims), which sits at the top of the open-weights MTEB BEIR leaderboard.

I reduced each model’s embeddings in two ways:

With truncation, I kept the first d dimensions and re-normalized. This is how you reduce dimensions with MRL.
With PCA, I fit the projection on the full-dimension document embeddings of the same dataset I then searched, and kept the top d components. From then on, every document and query embedding gets multiplied by that projection matrix and re-normalized before searching.

Then I ran the benchmarks on the eight datasets, comparing the two methods on both models, to see which one preserves more retrieval quality as the embeddings get smaller.

Q2: is it just the MRL training?

If PCA does well, that might be because MRL training has already organized the embedding space in a convenient way. To test that, I added text-embedding-ada-002, an older 1,536-dimensional model that wasn’t trained using MRL, as a control.

I applied PCA to both models. If PCA retains a similar share of retrieval quality on text-embedding-3-small and text-embedding-ada-002, that suggests its performance does not depend on MRL training.

Q3: does the fitting data matter?

PCA has to be fit on something, which raises two practical questions: how much fitting data do you need, and does it need to come from the corpus you’ll be searching?

For the first question, I fit PCA on random samples of FiQA’s 57K documents (1,000, 5,000, 20,000, and the full corpus) and checked how much the size of the fitting sample changes retrieval quality. This covers the scenario where you fit PCA on a sample of your data and never update it as the index grows.

For the second, I compared regular PCA against out-of-domain PCA: a projection fit once on 100,000 MS MARCO passages and reused, unchanged, on every other dataset. This is the most extreme version: you fit PCA on data that ends up looking quite different from what you actually index.

But first, a sanity check

Before running the experiment, I wanted to make sure the evaluation pipeline was producing sensible results. So I first tried to reproduce the official MTEB scores for the models.

The table below compares NDCG@10 at full dimensions: the official MTEB score vs. the score produced by my evaluation pipeline.

dataset	ada-002		3-small		qwen3-8b
dataset	MTEB	mine	MTEB	mine	MTEB	mine
SciFact	0.7275	0.7277	0.7337	0.7296	0.7846	0.7863
NFCorpus	0.3697	0.3705	0.3833	0.3847	0.4145	0.4150
ArguAna	0.5744	0.5757	0.5549	0.5573	0.7685	0.7689
FiQA	0.4441	0.4440	0.4491	0.4484	0.6457	0.6492
SciDocs	0.1836	0.1837	0.2080	0.2077	0.3274	0.3268
Quora	0.8760	0.8759	0.8883	0.8880	0.8890	0.8901
TREC-COVID	0.6847	0.6884	0.7790	0.7775	0.9499	0.9492
Touché 2020	0.2161	0.2143	0.2428	0.2433	0.3593	0.3596

Official numbers come from the MTEB results repository (text-embedding-3-small, text-embedding-ada-002, qwen3-embedding-8b). My results are close enough to the official numbers that I’m confident the pipeline is working as expected.

Q1: which method keeps more retrieval quality?

PCA matched or outperformed MRL truncation at nearly every dimension across both MRL-trained models.

Each cell below shows how much retrieval quality survives the reduction: NDCG@10 at that dimension divided by NDCG@10 at full dimensions (1,536 for text-embedding-3-small, 4,096 for qwen3-embedding-8b), averaged over the eight datasets.

dims	3-small (MRL)		qwen3-8b (MRL)
dims	Truncation	PCA	Truncation	PCA
512	98%	97%	99%	98%
256	94%	95%	96%	96%
128	86%	90%	91%	91%
64	71%	82%	83%	84%
32	46%	65%	68%	71%

At 512 dimensions, MRL truncation has a slight advantage. But as the vectors get smaller, PCA catches up and either beats or matches MRL.

On text-embedding-3-small, PCA is clearly better below 256 dims (65% vs 46% retained at 32). On qwen3-embedding-8b the race is much closer (71% vs 68% at 32).

One thing I found interesting is that the MRL training in qwen3-embedding-8b is strong enough that its truncated 32-dim vectors, cut from 4,096 dimensions, retain more quality than 3-small’s cut from 1,536.

Going dataset by dataset, you find some differences:

At the smallest dimensions, PCA wins on seven of the eight datasets for text-embedding-3-small and six of the eight for qwen3-embedding-8b.

Across dimensions, PCA performs better on SciFact, FiQA, and NFCorpus for both models. ArguAna and SciDocs also favor PCA with text-embedding-3-small, although the results are less conclusive with qwen3-embedding-8b. Truncation performs better on Quora for both models. Touché favors truncation on qwen3-embedding-8b but is basically a tie on text-embedding-3-small, and the result for TREC-COVID depends on the dimension.

Q2: is it just the MRL training?

I worried that PCA might only look good on text-embedding-3-small and qwen3-embedding-8b because MRL training had already organized their embedding spaces in some convenient way. If that were true, PCA on text-embedding-ada-002, which never saw MRL training, should be noticeably worse.

For the most part, it isn’t. Down to 128 dimensions, PCA retains nearly the same share of quality on text-embedding-ada-002 as on text-embedding-3-small; below that, a small gap opens:

dims	ada-002 (no MRL)		3-small (MRL)
dims	Truncation	PCA	Truncation	PCA
512	96%	99%	98%	97%
256	89%	96%	94%	95%
128	83%	89%	86%	90%
64	66%	78%	71%	82%
32	41%	59%	46%	65%

PCA on text-embedding-ada-002 retains 78% at 64 dims and 59% at 32, against 82% and 65% on text-embedding-3-small. It might be that MRL genuinely helps, or simply that text-embedding-3-small is a newer, better model. I can’t separate the two with this experiment. But either way, the hypothesis that PCA only works because of MRL seems unlikely given that even PCA on the non-MRL model beats truncation on the MRL model at those dimensions (59% vs 46% at 32 dims).

Out of curiosity, I also tried truncation on text-embedding-ada-002, even though the model was never trained for it. It held up much better than I imagined, with 83% retained at 128 dimensions.

Q3: does the fitting data matter?

Compared to MRL, PCA adds more operational complexity: fitting and versioning the projection, using it while querying, updating the projection when the index changes. So I wanted to know what happens if you’re lazy: fit PCA once, when you first create the index, and never touch it again.

In-domain fit on a small sample. In this scenario, you fit PCA on the documents you have at the start (1,000, 5,000, 20,000, or all of FiQA’s 57K) and use that projection for everything you index afterwards. At this scale, it doesn’t matter much: the projection fit on 1,000 documents performs almost the same as the one fit on the full corpus, at every dimension, on both models.

Out-of-domain fit. In this scenario, the data you fit on doesn’t match the data you end up searching, either because your corpus drifted over time or because you fit on whatever was available. To simulate the extreme version, I fit PCA once on 100K generic MS MARCO passages and searched every other dataset with that projection.

On text-embedding-3-small, the transferred fit is as good or better on average from 512 through 64 dimensions. Individual datasets vary more in both directions. On qwen3-embedding-8b it keeps up down to 128 dimensions but falls behind at lower dimensions (56% vs 71% retained at 32 dims).

Good news for lazy people like me. Fit-sample size matters much less than I expected, and an out-of-domain fit holds up at moderate compression. However, these patterns may be a consequence of the scale of the experiments or the particular benchmarks I used. So take these with a grain of salt.

What about quantization?

You can also shrink vectors using a technique called quantization. Instead of reducing the number of dimensions, quantization stores each dimension using fewer bits. With int8, each dimension uses 1 byte (4x smaller than float32); with binary, just the sign bit (32x smaller). For the numbers below, int8 quantizes the document vectors while keeping queries at float32; binary quantizes both documents and queries and ranks by dot product, which is equivalent to Hamming-distance ranking.

Using quantization alone, you get these results for text-embedding-3-small at full 1,536 dimensions:

config	bytes per vector	size vs full float32	quality retained
int8	1,536	25%	100%
binary	192	3.1%	95%

You can push this further by combining quantization with truncation or PCA. The resulting vectors can be dramatically smaller while still preserving a surprising amount of retrieval quality.

For example, combining binary quantization with PCA at 512 dimensions retains 82% of the original performance while reducing the raw vector size to about 1% of the float32 baseline. That seems a bit too optimistic, so it might just be my AI psychosis. Here are the full results.

Limitations

Few embedding models: I tested three models, and two of them are from OpenAI. Qwen already showed that a stronger MRL implementation narrows PCA’s lead, so other model families could shift the results further.
Corpus size: I excluded the largest BEIR datasets to keep the budget under control (~$30 spent), so it’s possible PCA falls behind MRL at a scale I didn’t reach.
Lab conditions: my evaluation runs exact search over raw vectors, which is not how an actual vector database works. A real deployment would use an approximate index, apply its own compression, rescore with full-precision vectors, and often mix in BM25 and a reranker. This could invalidate my results.
Benchmark contamination: BEIR datasets are widely used to train embedding models, so the absolute scores are probably inflated.

Conclusion

PCA not only held its own against MRL truncation, it won on most dimensions.

On text-embedding-3-small, PCA beat MRL truncation at nearly every dimension, and by a wide margin at aggressive compression: 65% of quality retained at 32 dims against truncation’s 46%. On qwen3-embedding-8b, MRL had a small edge at 512 dimensions and was effectively tied at 256, while PCA led below that.

These results also don’t appear to be an artifact of MRL training. PCA worked similarly well on ada-002, which wasn’t trained for truncation.

It’s also lower-maintenance than I expected. Results barely changed when fitting on a small in-domain sample. An out-of-domain fit also held up at moderate compression.

Not bad for a technique that’s older than the washing machine²!

If you want to look at the data yourself, the full pipeline is in the repo.

Footnotes

this page explains it quite well↩︎
electric washing machine↩︎

Citation

BibTeX citation:

@online{castillo2026,
  author = {Castillo, Dylan},
  title = {Honey, {I} Shrunk the Embeddings: {Matryoshka} Vs. {PCA}},
  date = {2026-08-01},
  url = {https://dylancastillo.co/posts/matryoshka-vs-pca.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2026. “Honey, I Shrunk the Embeddings: Matryoshka Vs. PCA.” August 1. https://dylancastillo.co/posts/matryoshka-vs-pca.html.

Are AI labs pelicanmaxxing?

Dylan Castillo — Sat, 18 Jul 2026 00:00:00 GMT

For the past few years, Simon Willison has tested every major LLM release with the same prompt: “Generate an SVG of a pelican riding a bicycle”.

What began as a tongue-in-cheek benchmark has become one of the most famous informal benchmarks in AI. Simon’s pelican-on-a-bicycle results are often among the most upvoted comments on Hacker News threads announcing new releases from AI labs.

The benchmark is now famous enough that there’s plenty of discussion about its usefulness and about whether AI labs might be benchmaxxing¹ on it. When billions or even trillions of dollars are at stake, and a strong result could help persuade users, wouldn’t it be tempting to pelicanmaxx your model just a bit?

I wanted to find out, so I put together a small experiment. I generated 1,008 SVGs across seven frontier models, scored them with an LLM judge, and used Claude Fable 5 for the analysis.

This article presents the results. All the code is available on Github.

How I tested it

I built a grid of 8 animals × 6 vehicles = 48 prompts, where the famous prompt is one cell:

Animals: pelican, flamingo, heron, otter, raccoon, antelope, whale, cat
Vehicles: bicycle, unicycle, skateboard, scooter, plane, boat

Every prompt uses almost identical phrasing to Simon’s, only switching the animal and vehicle. The animal and vehicle selection wasn’t done in a very rigorous manner, but I tried to vary both similarity to the original prompt and difficulty. Flamingo and heron are quite similar to pelicans; cat, raccoon, and otter are easy cases; antelope is hard; and whale is as different as you can get.

I tested seven models through OpenRouter: GPT-5.6 Terra, Claude Sonnet 5, Gemini 3.5 Flash, Grok 4.5, Qwen3.7-Max, GLM-5.2, and DeepSeek V4 Pro. I generated 3 samples per prompt, at temperature 1.0, requesting the same reasoning effort from every model. That resulted in 1,008 SVGs.

Then I ran each image through a three-stage pipeline:

Rendering: Each SVG is rendered to PNG. If a model returns no SVG or one that fails to render, I regenerate until it produces a valid one, and record the number of attempts. There were only 11 retries across the 1,008 generations.
Judging: GPT-5.6 Luna scores each image with 1-5 ratings for the animal, the vehicle, and the coherence of the action. When I rank animals or vehicles below, I use the matching rating on its own. When I need one number per image, I use the average of the three, which I call the judge score.
Feature extraction: For a more detailed analysis, I also passed each rendered image to Gemini 3.1 Flash-Lite, which recorded the animal and vehicle it recognized, which way the subject faces, and an open-ended list of scene elements.

My hypothesis is that if a lab trained on the benchmark, it should show up in some combination of the pelican row scoring above what the animal deserves, the bicycle column scoring above what the vehicle deserves, or the specific pelican-bicycle cell beating both.

Evidence #1: The pelicans on bicycles don’t look any better

Before any scoring, the simplest test is to look at the images yourself. Pick a lab to see everything it drew, with the judge’s score under each image (click to open full size):

I looked through the images myself before running the analysis below. Nothing jumped out at me. I couldn’t find a case where the pelican-bicycle images looked noticeably better than the rest of that model’s grid. Maybe in GLM-5.2’s first sample it felt slightly better than the rest, but that batch also produced a pretty cool heron on a skateboard, so I cannot say for sure. Otherwise they look like the rest of what each model draws, and the labs that draw good pelicans on bicycles also do a good job drawing other animal-vehicle combinations.

But this test is hard to replicate, and everyone will have a different opinion. So I wanted something more quantitative, which is why I opted for the method detailed above.

Evidence #2: Labs are not better at drawing pelicans

Here’s the mean animal rating per animal, pooled across all models:

Figure 1: Mean animal rating by animal, all models pooled.

The pelican is 6th of 8, behind cat, whale, raccoon, heron, and antelope. If AI labs were training on the benchmark, you’d expect pelicans at the top. Instead they’re in the bottom half. All seven labs draw cats, whales, and raccoons better than pelicans.

Of course, a pelican may simply be harder to draw than a cat. A lab could train on pelicans and still not push them past the easy animals, so this ranking alone can’t rule that out. I’ll adjust for difficulty in Evidence #4.

Evidence #3: Labs are not better at drawing bicycles

Bicycles fare even worse. They sit second from last, in a near-tie with planes, which come in last:

Figure 2: Mean vehicle rating by vehicle, all models pooled.

If labs were training on the benchmark, you’d expect bicycles near the top of this ranking. They’re not. However, the same caveat applies here. A bicycle is harder to draw than a skateboard: it needs two matching wheels, a frame that reaches both axles, handlebars, a seat, and pedals. The judge flags a missing or disconnected one of those on 2/3 of the bicycle images. You can train on bicycle images and still not do a great job relative to simpler vehicles.

One note on the plane, though: I should’ve picked “airplane” instead of “plane” because models often read it geometrically. They drew the animal standing on a flat surface instead of flying an aircraft. The plane is the only vehicle where the feature extractor sometimes found no vehicle at all (25 of 168 images, against zero for the other five), and 20% of plane images scored a 1 or 2 on the vehicle rating, against 5% for bicycles and none at all for boats, scooters, or skateboards.

Evidence #4: Labs are not better at drawing pelicans on bicycles, even adjusting for difficulty

Put the two together and the “pelican on a bicycle” ends up near the bottom of the ranking, at #42 of 48:

Figure 3: All 48 combos ranked; pelican + bicycle highlighted.

But again, some combinations might be just harder to draw than others.

To account for that, I fit a fixed-effects regression on all 1,008 images: score ~ lab + animal × vehicle, plus per-lab interaction terms for pelican, bicycle, and the pelican-bicycle cell, with robust standard errors. The animal × vehicle terms absorb the inherent difficulty of all 48 combinations. The interactions measure each lab’s benchmark-specific boost relative to the average lab, with confidence intervals.

The results:

Every per-lab pelican effect (the lab’s boost on pelicans across all six vehicles) lands between -0.11 and +0.14 judge points, and none comes close to significance (smallest p = 0.25).
The per-lab bicycle effects (the lab’s boost on bicycles across all eight animals) run from Grok 4.5 at -0.18 (p=0.11) to Gemini 3.5 Flash at +0.27 (p=0.022). Only Gemini clears p < 0.05, and the seven point in both directions.
No pelican-bicycle cell effect (the extra boost on the specific combination, on top of the lab’s pelican and bicycle effects) clears p < 0.05. The largest positive is GLM-5.2 at +0.35 (p=0.12), which is the one I mentioned earlier. It’s the closest thing to a signal in this experiment, but still within chance.

Here are the full per-lab estimates. A pelicanmaxxing lab would show dots to the right of the zero line across its whole row:

Figure 4: Difficulty-adjusted effects per lab, with 95% confidence intervals. Highlighted intervals exclude zero.

Every pelican interval and every cell interval contains zero. Exactly one doesn’t: Gemini 3.5 Flash in the bicycle column. But with 21 tests at p < 0.05, chance alone predicts about one false positive (21 × 0.05 ≈ 1.05), and one is exactly what came up. It also doesn’t survive a multiple-comparisons correction: the Bonferroni threshold across the 21 tests is 0.05/21 ≈ 0.002, and its p-value is 0.022. The full table of estimates and p-values is in the repo.

But these intervals are wide, about ±0.6 judge points on average. Any boost smaller than that won’t be captured by this test.

Evidence #5: The pelican-bicycle scenes don’t look memorized

Some have suggested that the pelican on a bicycle looks like a memorized composition, pointing to recurring patterns such as the pelican always facing right, or recurring elements like a sun or a scarf. So I wanted to know if this was true.

Direction: All 21 pelican-bicycle images, across all seven labs, face right. No other animal/vehicle combination does that.

However, facing right is common: 60% of all 1,008 images do it. How common depends on the animal and the vehicle, and bicycles are one of the two vehicles where it’s strongest:

vehicle	left	right	ambiguous
scooter	9%	83%	8%
bicycle	11%	81%	8%
skateboard	19%	60%	21%
plane	21%	58%	21%
unicycle	22%	45%	33%
boat	33%	35%	32%

Pelicans are also among the animals that tend to face right:

animal	left	right	ambiguous
antelope	21%	78%	1%
pelican	22%	78%	0%
heron	22%	77%	1%
whale	34%	65%	1%
flamingo	36%	64%	0%
otter	8%	45%	47%
cat	8%	40%	52%
raccoon	3%	36%	61%

It’s hard to draw a pelican or a bicycle facing the viewer, so models almost always draw them from the side, facing left or right. That’s why so few of their images are ambiguous. Other combinations also come close to unanimous: antelope on a scooter and pelican on a scooter land at 20 of 21, and heron on a bicycle at 19 of 21. So 21 out of 21 doesn’t seem like an outlier.

Scene elements: I let the extractor name any element it saw in the image. These are the counts:

Figure 5: Share of images containing each element, from the open-ended extraction pass.

A memorized scene would show up as the same set of elements recurring picture after picture. I went looking for that, and found some combinations do tend to produce the same elements every time. Every single flamingo on a boat has a sun in it. Otters on planes wear scarves 38% of the time. Cats on bicycles get a basket 38% of the time.

The pelican on a bicycle doesn’t seem to have anything particularly different about it. It just has some elements that appear more frequently, like every other animal-vehicle combination.

Limitations

Using a single LLM judge for scoring. Every score here comes from one model, GPT-5.6 Luna, looking at one image at a time. I didn’t do much alignment and didn’t check how often it agrees with itself on a re-run. If a model just can’t judge a drawing reliably, none of the numbers above mean much. The judge is also from the same family as one of the contestants, GPT-5.6 Terra. However, every lab draws all 48 combinations, so a judge that happens to like one lab’s style lifts that lab’s whole grid at once. But that doesn’t change the results because this analysis only cares about the within-lab differences.
SVGmaxxing. A lab that optimized SVG generation as a whole (or a subset such as animals on vehicles) rises on every cell at once and looks identical to a lab that’s just good. Some labs, such as Google/DeepMind, openly do this. This experiment can’t detect that.
Limited budget. The whole experiment ran on roughly $80 of API credits. That capped it at 3 samples per cell, a single judge, and 7 models. This also prevented me from iterating too much on the prompts and pipeline, as with the “plane” vs. “airplane” case.

Conclusion

Sorry, HN haters, but there’s little evidence that AI labs are pelicanmaxxing. Or at least they’re not doing it in a plainly obvious manner.

Pelicans aren’t drawn any better than other animals. Bicycles aren’t drawn any better than other vehicles. And no lab draws the combination better than its pelicans and bicycles already predict. GLM-5.2 comes closest: it has the largest boost on the exact pelican-bicycle cell, and its first pelican-on-bicycle sample caught my eye. But the effect is small and not significant, so I wouldn’t put too much weight on it.

The other thing that stands out is direction in the scene composition. All 21 pelican-bicycle images face right, the only combination in the grid where every image agrees. But it doesn’t seem that strange. Facing right is the norm across the experiment. Three other combinations land at 90% or above, and with 48 of them, I’m not surprised one reached 21 out of 21.

The more plausible story is SVGmaxxing like Google/DeepMind does. Other labs might be doing it more quietly. Sadly, this experiment can’t say who’s doing it. But at least you can sleep tonight knowing that AI labs are not producing terabytes of pelicans on bicycles just to trick Simon Willison.

If you want to look at the data yourself, the full pipeline is in the repo.

Footnotes

the practice of optimizing AI models to achieve high scores on popular benchmarks.↩︎

Citation

BibTeX citation:

@online{castillo2026,
  author = {Castillo, Dylan},
  title = {Are {AI} Labs Pelicanmaxxing?},
  date = {2026-07-18},
  url = {https://dylancastillo.co/posts/pelicanmaxxing.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2026. “Are AI Labs Pelicanmaxxing?” July 18. https://dylancastillo.co/posts/pelicanmaxxing.html.

Five Years of Freelancing

Dylan Castillo — Sat, 25 Apr 2026 00:00:00 GMT

Five years ago, freelancing felt like arbitrage: same work, better pay, without leaving Spain. So I left my full-time job in the middle of the pandemic and gave it a shot.

It worked. I made more money, had more freedom, and worked with all sorts of companies on projects across data science, machine learning, and AI products.

But that was also the problem. Freelancing rewarded my flexibility and never forced me to go too deep into any topic. Eventually, that flexibility became a ceiling.

This is a recap of how I got in, how much I made, how I found gigs, and what I’ve learned in the process.

How I got in

I started thinking about freelancing in 2019, two years before I landed my first contract. That year, I joined a platform that promised you a better-paying job in exchange for a cut of your future earnings. The idea was simple: they would introduce you to people already doing the kind of work you wanted to do, and those people would help you land a similar job.

The platform suggested a few introductions, and one of them stuck with me: a freelance data scientist.

In our first chat, he mentioned almost in passing that he was charging over €600 a day. That blew my mind. He was doing the same work I was doing, but making much more money, and he didn’t have to leave his hometown! I asked if he’d mentor me and help me land a freelance contract. He said yes, then shared his secret sauce: spam people on LinkedIn.

So that’s what I did.

I set up a bot that messaged pretty much everyone involved in data science in Europe: managers, recruiters, and a few confused interns. Most people ignored me. A few wrote back to ask whether what I was doing was even legal. Most recruiters dismissed me because I had no prior freelancing experience.

Me spamming people on LinkedIn

I grew from something like 500 connections to close to 4,000, but the approach didn’t bear fruit. Eventually, I realized our incentives weren’t aligned. His plan was for me to spray LinkedIn messages across Europe while he sent me humblebrag motivational messages to keep me going. If I landed a contract, he’d get a nice cut for the next four years. If LinkedIn blocked my account, that was my problem. So we parted ways.

Instead, I decided to start a blog, share useful projects and tutorials on LinkedIn, and apply to contracting opportunities in the hope that something would pan out. It was a better long-term strategy, but in the short term, it produced zero results.

I started getting desperate. Then COVID hit.

By early 2021, the freelance market in Europe exploded. A friend at the European Commission made an intro, and I got my first contract offer within a couple of days. I also got into multiple hiring processes for contractors around the same time. My first gig was with the European Commission, and shortly after I switched to Deliveroo.

I expected getting into freelancing to be hard. I just didn’t expect the path to involve a failed spam campaign, two years of waiting, and a global pandemic.

What freelancing gave me

My first freelance contract paid €350 a day. From there, my rates grew quickly. For the last few years, I’ve usually charged between €600 and €900 a day. For very short engagements, I generally charge more. In the US or parts of Northern Europe, good freelancers can definitely do better.

Gross revenue over time

Every year since I started freelancing, I’ve made more than I did in my last full-time job. But my income has still varied a lot depending on how much time I chose to spend on client work. In 2022 and 2023, I spent a lot of time on business ideas, and that shows in the numbers. In 2024 and 2025, I went in the opposite direction: back-to-back contracts and fewer experiments. I worked my ass off. That shows too.

If you have a good lead flow, freelancing gives you a very direct link between effort and income. Work hard, and you will be rewarded accordingly.

That matters to me. My wife and I are immigrants, and we provide financial support for my family and extended family. So yes, I care about money. I also wanted to earn more without leaving Spain, because my wife’s work makes relocating difficult. Freelancing gave me that option.

The second thing it gave me is freedom and comfort. I work fully remotely from home and can often set my own schedule. I stay healthy: I hit the gym four days a week and walk 12,000 steps every day with my walking pad while I work. I also avoid most corporate theater: 360 reviews or recurring “how are you feeling?” meetings.

The third is variety. On one project I’m acting as a technical product manager, scoping a product and figuring out what should be built. On another I’m a data scientist, planning and analyzing experiments. On another I’m an AI engineer, shipping features for workflows or agents. I love the intellectual range of things I get to do.

The fourth is the room it has given me to work on my own things. Because I could work intensely for a few months, save enough to coast for the rest of the year, and then step away, I had real time to try out business ideas and side projects. None of my business ideas worked out, but I’m glad I got to try.

How I got work

How much you make heavily depends on where your clients are. In Europe, Northern European clients usually pay much better than Southern European ones. Even while living in Spain, most of my clients are not local.

For a bit of wider market context, I once took part in a survey of AI freelancers. Among the Europeans in that group, median yearly gross revenue was around €85k. For the US-based freelancers, it was closer to $150k (roughly €128k).

Pricing model matters too. I’ve experimented with deliverable-based pricing, and I usually prefer it. When the contract is tied to an outcome rather than time, my effective rate goes up and the incentives are cleaner: the client cares less about my hours and more about the result.

The danger is scope. If you misjudge the work, you can accidentally sell yourself a very stressful unpaid internship.

But pricing only matters if you have demand. Without lead flow, you have no leverage. For me, that demand has come from a few places.

After the first contract, many projects came from being active on LinkedIn. Some of my posts got good traction. For example, Ask Seneca, a small demo that showcased a simple RAG system in the early days of LLMs. It ended up on the front page of Hacker News and got mentioned in The Economist. Other projects followed similar paths. That visibility translated into people reaching out about work.

Another big channel has been personal relationships. Friends from Entrepreneur First, past clients, and former employers have reached out when they had projects I’d be a good fit for. These are the best types of leads because they require very little effort to close.

I’m also in a couple of freelance platforms (á la Toptal). They can be useful for lead flow, but over time the rates have gotten worse. Platforms make comparison easy, and comparison turns you into a commodity. That is bad for your rates.

The thing I have neglected lately is content. Which is stupid, because that’s how you build authority and get strangers to trust you. I once heard Jason Liu summarize the best way to charge high consulting rates: “be famous.” I agree. There’s a world of difference between someone wanting to hire “an AI engineer” and someone wanting to hire Dylan Castillo. No one can outcompete me at being me.

I’ve gotten a bit lazy on that front because I’m rarely out of projects. But I’m probably missing out on better opportunities, clearer positioning, and higher rates by not putting more work into the world.

Finally, freelancing runs on a much shorter horizon than full-time work. Projects typically run three to six months, and I rarely know in advance whether a contract will be extended — or whether I’ll even want to stay. That forces you into a permanent “always be selling” mode. Even when I’m fully engaged with a client, part of my brain is scanning for the next gig.

How I positioned myself

Over the last five years, I’ve worked across data science, data engineering, ML, backend systems, and AI products.

Early on, most of the work was classic data and ML. Since 2023, it has shifted heavily toward AI: building workflows and agents for B2B clients, helping startups ship AI products, supporting academic research groups, teaching courses, and leading technical work on personalization and experimentation.

At some point, I tried to position myself as the person who could help teams take AI products from zero to one. But that quickly blurred into something less useful: “the guy who knows about AI.”

That is the trade-off I’ve been struggling with. Clients generally hire me because I can get things done. I know how to connect the dots between product, data, backend, and AI. But I’m not getting hired because someone thinks of me as the expert in one particular topic.

Freelancing made it easy to stay that way. Contracts tend to be short, and I usually picked projects based on what I found interesting and what paid well. I never made a proper plan for what I wanted to become. So even as my rates grew, I never found a way to stop looking like just another freelancer.

That is the ceiling I’ve hit. I can still find work. But I don’t yet have a strong enough answer to “Why you?” to go above the rates I charge now. At higher rates, clients want more than competence. They pay for reputation and/or deep expertise, not just a get-things-done guy.

What I’d do differently

With hindsight, these are the things I’d do differently if I were starting over:

Don’t start by trying to convince strangers. The boring answer is the right answer: use your network. And by that, I don’t mean LinkedIn randos. Reach out to former colleagues, old bosses, past clients, and people who already know your work. That’s the simplest way to get started. It’s awkward, but it has, by far, the highest chance of success.
Pick a lane earlier. I never committed to a specific niche or to publishing consistently, so I did not build much of a reputation beyond my close network. If I were starting again, I would choose a problem space and start making content about it. Even if I got it wrong at first, forcing myself to put work into the world would help me sharpen my positioning.
Treat client work as market research. I spent too much time on random side projects and not enough time asking which parts of my client work could become repeatable services or products.
Don’t be greedy. I would have been more deliberate about choosing projects that exposed me to better problems or helped me build expertise I could later productize. A slightly lower-paying project in the right direction can have a better ROI than a high-paying project that keeps you stuck.

What’s next for me

I’ve never felt this uncertain about the future – for reasons both good and bad.

On the optimistic side, Iwana Labs is starting to change. Until now, it has mostly been a vehicle for my own freelance work, with occasional projects where I brought in collaborators. Some of those collaborations have kept growing, and by the end of the year, I hope to go from a one-man show to a small team.

We don’t want to be just another AI development agency™. Yes, I know everyone says the same thing when they get started. But we have a few areas where we’ve built expertise over the years, and I hope we can position ourselves around them.

On the scary side, I genuinely don’t know what’s going to happen to the software industry. The changes from AI have been amazing to watch. But my best guess is that we’re heading toward a much more unequal distribution of the profits. I worry the economics are shifting from something like the top 20% capturing 80% of the value to something closer to the top 1% capturing 99%.

We’ve already seen the $1B one-man company and companies with crazy growth rates. Work that used to require a full team can increasingly be done by fewer people with better tools. Maybe the amount of work to be done will keep growing even faster than the supply of people able to do it. I hope so. But I’m not sure.

When AI gives everyone the ability to build, being “pretty good at building” becomes less valuable. You need taste, distribution, deeper expertise, and stronger relationships.

And honestly, I have to ask myself whether I’ll make it into that top 1%. I’m optimistic about my skills, but the real world doesn’t care about that. There are tons of amazing, highly motivated people out there. We’re in an industry with almost no barriers to entry, where the money has been good so far, and where AI can already do a lot of the work. What was already an extremely competitive industry just got much more competitive.

The next chapter is partly about responding to all of that. Finding the right people to grow with, and figuring out where we can set ourselves apart. Five years in, I love what freelancing has taught me and the opportunities it has given me. But the next five years will need to look different if I want to stay relevant through what’s coming.

Citation

BibTeX citation:

@online{castillo2026,
  author = {Castillo, Dylan},
  title = {Five {Years} of {Freelancing}},
  date = {2026-04-25},
  url = {https://dylancastillo.co/posts/5-years-freelancing-in-europe.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2026. “Five Years of Freelancing.” April 25. https://dylancastillo.co/posts/5-years-freelancing-in-europe.html.

LLM research on Hacker News is drying up

Dylan Castillo — Fri, 24 Apr 2026 00:00:00 GMT

I thought I was seeing fewer arXiv papers on the front page of Hacker News (HN) these days, and I wanted to check if that was real.

So I asked Claude to run a quick analysis: track the share of arXiv stories on HN over time. It queried the BigQuery HN dataset, bucketed the stories by month, and plotted the series:

Percentage of HN stories linking to arXiv

That confirmed my hunch. arXiv posts have been decreasing rapidly in the last few months. Interestingly, it also showed another peak around 2019, and I wanted to know what drove it.

I asked Claude to pull the top 100 papers by upvotes from 2019 and group them by topic. It was the deep learning peak. 41% of the top 100 were about deep learning.

Then I ran the same query for 2023-2026, to see how dominant LLMs were. 59% of the top 100 upvoted papers were about LLMs or AI.

So I asked him to make a nice chart with all of this:

Distribution of topics of arXiv stories

Then I wanted to see which 2019 papers aged well, so I asked Claude to pull the ones that held up from the top 100. Here’s what he got:

MuZero — Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model (161 pts) — DeepMind’s successor to AlphaZero
EfficientNet — Rethinking Model Scaling for Convolutional Neural Networks (119 pts) — compound scaling, set the new CV SOTA
XLNet — Generalized Autoregressive Pretraining for Language Understanding (79 pts) — briefly dethroned BERT
PyTorch: An Imperative Style, High-Performance Deep Learning Library (113 pts) — the NeurIPS paper formalizing PyTorch’s design
On the Measure of Intelligence (80 pts) — Chollet’s ARC / “human-like intelligence” manifesto

It’s too early to know which 2023-2026 papers will hold up, so I asked Claude to guess:

DeepSeek-R1 — Incentivizing Reasoning Capability in LLMs via RL (1,351 pts) — first open recipe for o1-style reasoning via pure RL on verifiable rewards
Generative Agents — Interactive Simulacra of Human Behavior (391 pts) — the canonical “Smallville” paper, template for LLM agent architectures
The Era of 1-bit LLMs — BitNet b1.58, ternary parameters for cost-effective computing (1,040 pts) — first credible case for low-bit inference as the default
Differential Transformer (562 pts) — attention with a noise-cancelling term, clean architectural contribution with a real theoretical story
LK-99 cluster — room-temperature superconductor preprints (2,408 + 1,690 pts) — landmark meta-science, not physics: open-science-at-wire-speed and the canonical case of crowdsourced replication

That was fun. Thanks, Claude.

Citation

BibTeX citation:

@online{castillo2026,
  author = {Castillo, Dylan},
  title = {LLM Research on {Hacker} {News} Is Drying Up},
  date = {2026-04-24},
  url = {https://dylancastillo.co/til/llm-research-on-hacker-news-is-dying.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2026. “LLM Research on Hacker News Is Drying Up.” April 24. https://dylancastillo.co/til/llm-research-on-hacker-news-is-dying.html.

The last shall be (slighly) safer

Dylan Castillo — Wed, 01 Apr 2026 00:00:00 GMT

While I use Claude Code to build crappy forms, real hackers use it to make money.

Supply chain attacks have become so common these days that I had to learn they’re not about an Amazon delivery guy crashing into my front door.

I know I’ll inevitably become a victim of one (a supply chain attack, not an Amazon delivery guy), but I’m not handing over my $50 in BTC to North Korea without a fight. So I took this measure, courtesy of some random guy on Hacker News:

PSA: npm/bun/pnpm/uv now all support setting a minimum release age for packages.

I also have ignore-scripts=true in my ~/.npmrc. Based on the analysis, that alone would have mitigated the vulnerability. bun and pnpm do not execute lifecycle scripts by default.

Here’s how to set global configs to set min release age to 7 days:

~/.config/uv/uv.toml exclude-newer = “7 days”

~/.npmrc min-release-age=7 # days ignore-scripts=true

~/Library/Preferences/pnpm/rc minimum-release-age=10080 # minutes

~/.bunfig.toml [install] minimumReleaseAge = 604800 # seconds

(Side note, it’s wild that npm, bun, and pnpm have all decided to use different time units for this configuration.)

If you’re developing with LLM agents, you should also update your AGENTS.md/CLAUDE.md file with some guidance on how to handle failures stemming from this config as they will cause the agent to unproductively spin its wheels.

This gives researchers and security teams time to analyze new releases and flag malicious ones.

Then, I thought it’d be a bad idea if everyone uses the same value, so I cowardly changed mine to 8 days. Sadly, others quickly realized this:

that’s why people are telling others to use 7 days but using 8 days themselves :)

So I picked 9… and kept scrolling:

brb, switching everything to 9 days

Oh, fuck.

Citation

BibTeX citation:

@online{castillo2026,
  author = {Castillo, Dylan},
  title = {The Last Shall Be (Slighly) Safer},
  date = {2026-04-01},
  url = {https://dylancastillo.co/til/securing-package-managers.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2026. “The Last Shall Be (Slighly) Safer.” April 1. https://dylancastillo.co/til/securing-package-managers.html.

2025: Personal Snapshot

Dylan Castillo — Tue, 24 Feb 2026 00:00:00 GMT

I’m an independent AI consultant, trying to grow my business. This is my annual review. If it’s me rereading this, welcome back. This is Dylan from 2025.

Money & work

This was the year I made the most money. I grew my revenue by 13%. Growth came mostly from working on more projects with previous clients. I worked on 11 projects, with 6 clients. I did 6 deliverable-based projects and 5 time-based projects. Deliverable-based projects accounted for 57% of my revenue.

Revenue over time

It became obvious to me that expanding your projects with existing clients is a lot easier than selling to new clients.

I also reduced costs by 11%. Most of the cost reduction came from firing a frontend developer that I had hired in 2024. I let him go because of AI progress. I could use Claude Code to get a better, cheaper, and faster version of the same services he was providing me. It didn’t make financial sense to keep him on board.

Still, I felt bad about it. He was a good guy, but he wasn’t a good fit. I also felt I didn’t do a good job during the hiring process. This cost me resources and time, so I’ll be more careful with the next hires.

GitHub contributions

This year I more than doubled my number of commits and wrote code 312 days of the year. It helped that some of the paid work I did was the econagents open source Python library. Sometimes I cheated, because I was tired and all I did was merge Dependabot’s PRs. I feel like I coded and learned quite a bit more than last year, but not as much as it looks.

Throughout the year, a recurring dilemma was whether to build a product or keep focusing on services. For the first half of the year, I dedicated quite a bit of time to Namemancer, a tool meant to help trademark lawyers speed up their research process. However, I kept seeing more market pull for services and struggled to make meaningful progress on it. I kept hoping I could magically advance on both fronts, but eventually had to face reality and decided to kill Namemancer.

And I felt relieved. I’ve come to understand that I need to be more disciplined and should focus on fewer things. I often end up spreading myself thin by taking on too many things at once and do a poor job at most of them.

Life

In late February, my wife, Maria, and I moved to Cork, Ireland for three months. This got off to a rocky start. We had been having some differences since the end of 2024, and the first few weeks in Ireland put this on steroids. We’ve been together for 15 years, eight of them married. We’ve had our share of difficult stretches, but this was one of the toughest.

Through many tough conversations and commitment from both sides, we successfully worked through it and grew closer together. Ireland went from a rocky start to one of the best experiences we’ve had as a couple. We traveled around the country, went for runs during sunset, tried every dairy product available, and really became a team again. We still cherish our time in Ireland, and even considered moving there!

We went back to Madrid in May, and then in July we moved to Salamanca, a small city to the north of Madrid. Maria found a good job opportunity, and since I can work from anywhere, we decided to pack our bags and go. We spent a good chunk of July and August moving. We went from a small, old 2-bedroom apartment to a newly renovated 4-bedroom apartment. It was a good change, especially for me, given that I work from home.

It was also a good year for reconnecting. I got to see friends I hadn’t seen in a while, we made hallacas again, and I spent a lot of time with my family. We had an especially fun Christmas and New Year’s.

Health

I lifted more weights this year but did less cardio. After we moved to Ireland, I started going to the gym instead of training at home. I kept doing this even after coming back to Spain. It’s been good for my mental health, as otherwise I’d often spend days without leaving my house. I’ve also gained more muscle mass compared to previous years.

My RHR is now at 56 bpm and VO2 max is at 46 ml/kg/min. Both are pretty much the same compared to last year. I think I need more cardio and better rest to move these markers.

The biggest health concern this year was sleeping issues and neck/back pain. The sleeping issues started a few weeks after I arrived in Ireland. It was a mix of stress and working from the same room where I was sleeping. On good days it takes me 15–20 minutes to fall asleep. On bad days, it can take me 2–3 hours. I’ve been able to improve this over time, but I still need to be careful with my sleep hygiene.

I have some small protrusions in my neck that get worse when I don’t sleep well or exercise. The last 3 months of the year were pretty stressful, so this got bad. It improved after I changed my home desk setup, did rehab exercises, and started going to the physio regularly.

What’s next?

No big lessons this year, besides the same one I’ve struggled with in the past: I still don’t know how to properly regulate myself. I have a hard time managing stress and that often lead to trouble sleeping and depression. I keep postponing looking for a long-term solution for this as I always find a way to push through, but I don’t think my current approach will work forever.

On the bright side, some exciting projects have come up for 2026. If things go well, I might finally grow my consulting practice beyond a one-man operation. You’ll have to come back next year to see how it goes.

Citation

BibTeX citation:

@online{castillo2026,
  author = {Castillo, Dylan},
  title = {2025: {Personal} {Snapshot}},
  date = {2026-02-24},
  url = {https://dylancastillo.co/posts/2025-personal-snapshot.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2026. “2025: Personal Snapshot.” February 24. https://dylancastillo.co/posts/2025-personal-snapshot.html.

Cover Your Ass

Dylan Castillo — Sun, 22 Feb 2026 00:00:00 GMT

A simple hack that will make your life easier at work:

Always cover your ass (CYA).
Always assume others will do the same.

CYA has a bad rep, because lazy people use it to deflect blame onto others. But CYA is just a communication style. It asks you to do two things: make priorities explicit and flag potential issues early.

I’ve worked for all sorts of organizations: tech, consulting, public, big, small. As a freelancer, I have a high rate of repeat clients, and, until now, I’ve always received positive feedback about my work, including intros to other clients. This approach has served me well.

After organizations reach a certain size, successful projects become vehicles for promotions, and failing projects become express tickets to career purgatory. Picking winners from losers isn’t something you can always do. But you can always choose to CYA. On winners, CYA isn’t always a big issue: people are happy, and no one’s looking to point fingers. On losers, it can be the difference between “project X sucked, but Dylan always fought to keep the boat afloat” and “project X sucked, but Dylan sucked even more.”

Take a simple case that I see all the time:

Boss goes to Jane and asks, “Hey Jane, can you fix bug X?” Jane responds, “Yes, sure!” After some research, Jane realizes that bug X is caused by another bug in a system owned by John. Jane is also a good person, and knows that John is overworked, so she asks if he can look at it when he has some time. He replies affirmatively. So she goes to work on some of her other tasks while she waits for John to get back to her.

A week later, Boss asks Jane, “Jane, did you fix bug X?” and Jane responds, “Not yet. The issue is caused by another bug, and I’m waiting for John to investigate that one.” Then Boss tells Jane, “That was a critical bug that needed fixing ASAP. We just lost $20M. You suck!”

Ok, the last bit was too much, but you get the idea. Jane had a task that depended on someone else. She didn’t know the priority of that task in the grand scheme of things, and assumed it could wait until John had time to investigate it. Then, when it was already too late, she realized how critical it was.

Jane left her ass uncovered by:

Not clarifying the priority of the task from the start
Not reporting that the task was blocked, and might require escalation to get it sorted out

You might think that the problem was Jane’s boss, who didn’t communicate the priority right away. And you’d be right. But, once again, you don’t get to pick your boss most of the time. And you should assume that they will also cover their ass by throwing you under the bus if needed.

So, follow your dreams, but cover your ass. It’s not by chance that humans have been doing so –literally and figuratively– for hundreds of thousands of years.

Citation

BibTeX citation:

@online{castillo2026,
  author = {Castillo, Dylan},
  title = {Cover {Your} {Ass}},
  date = {2026-02-22},
  url = {https://dylancastillo.co/posts/cover-your-ass.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2026. “Cover Your Ass.” February 22. https://dylancastillo.co/posts/cover-your-ass.html.

I hate AI side projects

Dylan Castillo — Fri, 20 Feb 2026 00:00:00 GMT

The best thing about AI is that EVERYONE can build now. The worst thing about AI is that EVERYONE can build now.

I’ve worked on and shared many of my side projects over the years. I built them because they gave me a chance to learn new things, such as deploying my first web app on a VPS or running Python in the browser. Sometimes, I got new clients when some of my projects went (just a bit) viral. And, more importantly, I had fun while building, like the time I built a game (that the NYT didn’t buy!), or when I replicated the results from a 1970s cognitive science experiment.

I still work on side projects. But now I dread sharing them. The internet is saturated with AI slop. If you don’t have something truly special, it can feel like you’re just adding one more padlock to those sagging, padlock-ridden bridges. Most of my past side projects would take me a few minutes or hours to build with Claude Code. Today, they’re not worth talking about.

I rarely look at projects on Hacker News, Reddit, or X anymore. All landing pages and GitHub repos look the same. All launch messages look the same. The signal-to-noise ratio is unbearably low.

And, dear reader, don’t get your hopes up. I’m also a sinner: I’ve contributed to the AI slop pile. I repent, but the damage is done.

Still, I’m optimistic about AI, and I’m very appreciative of the impact it has had on my work. I don’t want progress to stop or slow down. I don’t want less people building with AI. I just don’t know yet how to properly engage with a world that’s mostly created by AI.

Citation

BibTeX citation:

@online{castillo2026,
  author = {Castillo, Dylan},
  title = {I Hate {AI} Side Projects},
  date = {2026-02-20},
  url = {https://dylancastillo.co/posts/ai-side-projects.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2026. “I Hate AI Side Projects.” February 20. https://dylancastillo.co/posts/ai-side-projects.html.

ClaudeCodeholic

Dylan Castillo — Sat, 18 Oct 2025 00:00:00 GMT

Hi, I’m Dylan, and I’m a ClaudeCodeholic.

I’m not addicted because Claude Code makes me more productive. I’m addicted because it dulls the pain of thinking and makes development an endless betting game of “if I change the prompt just one more time, it’ll get it right.”

I don’t care about those studies comparing how much better or worse people do with AI coding tools. I’d still use it, even if I was 50% less productive.

It might take twice as long to finish what I’m trying to build. Maybe I never finish it at all. But I feel I’m making progress. Most importantly, I hardly need to be there.

Companies worship productivity. I worship comfort. And Claude Code is my warm bed on a cold night of winter.

Now, if you’ll excuse me, I have a prompt to finish, before I can continue doomscrolling on reddit.

Citation

BibTeX citation:

@online{castillo2025,
  author = {Castillo, Dylan},
  title = {ClaudeCodeholic},
  date = {2025-10-18},
  url = {https://dylancastillo.co/posts/claudecodeholic.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2025. “ClaudeCodeholic.” October 18. https://dylancastillo.co/posts/claudecodeholic.html.

New line in Claude Code in Alacritty

Dylan Castillo — Sat, 18 Oct 2025 00:00:00 GMT

To make Shift+Enter create a new line in Claude Code when working with Alacritty/Zellij, you need to add the following lines to ~/.config/alacritty.toml:

[keyboard]
bindings = [
    { key = "Return", mods = "Shift", chars = "\n" } 
]

I found the solution in this GitHub issue.

Running /terminal-setup didn’t work for me.

Citation

BibTeX citation:

@online{castillo2025,
  author = {Castillo, Dylan},
  title = {New Line in {Claude} {Code} in {Alacritty}},
  date = {2025-10-18},
  url = {https://dylancastillo.co/til/fix-claude-code-shift-enter-alacritty.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2025. “New Line in Claude Code in Alacritty.” October 18. https://dylancastillo.co/til/fix-claude-code-shift-enter-alacritty.html.

LangSmith 101

Dylan Castillo — Sun, 10 Aug 2025 00:00:00 GMT

In my first AI projects, I didn’t have access to proper observability tools and didn’t know how to evaluate the performance of LLM pipelines. I struggled to figure out what to improve and even when I knew what to improve, it was hard to do so, without breaking other things. Many of those projects failed miserably.

Those failed projects made me start looking for better ways and tools to build AI applications. Over time, tools such as LangSmith, Langfuse, or Logfire became key components of my AI toolkit. I can no longer imagine building an AI application without them.

In this tutorial, I’ll walk you through the basics of using LangSmith to monitor and evaluate your LLM applications.

Prerequisites

To complete this tutorial, you need to:

Sign up and generate OpenAI and LangSmith API keys.
Create a .env file in the root directory of your project and add the following lines:

OPENAI_API_KEY=your_openai_api_key
LANGSMITH_TRACING=true
LANGSMITH_PROJECT=your_langchain_project_name
LANGSMITH_API_KEY=your_langsmith_api_key

Create a virtual environment in Python and install the following packages:

uv venv
uv add langchain langchain-openai langsmith openai jupyter python-dotenv

I’m assuming you’re familiar with the basics of LLMs. If you need a refresher, you can check out some of my older posts. Also, if you don’t want to copy and paste the code, you can download this post’s notebook and follow along.

Let’s go!

Setup

As usual, you should start by importing the necessary libraries:

from datasets import load_dataset
from dotenv import load_dotenv
from langchain_core.messages import HumanMessage, SystemMessage
from langchain_openai import ChatOpenAI
from langsmith import Client, trace, traceable
from langsmith.run_trees import RunTree
from langsmith.wrappers import wrap_openai
from openai import OpenAI
from pydantic import BaseModel, Field

load_dotenv()

This will import all the libraries required for the next sections:

datasets for loading the sample dataset we’ll use to run evaluations.
langchain libraries and openai for working with LLMs
langsmith for tracing and evaluating the pipeline
dotenv and pydantic for environment variable management and data validation

Next, you will create your first trace on LangSmith.

Tracing and monitoring

A LangSmith trace captures the full execution path of a single operation. It consists of a sequence of steps, which are called runs. Each trace contains the top-level inputs and outputs, as well as metadata such as runtime version and operating system details.

There are four ways to create traces in LangSmith:

Using @traceable
Using a wrapped client
Using a trace context manager
Manually creating traces with RunTree

Using `@traceable`

The simplest way is to encapsulate your pipeline in a function and use the traceable decorator:

client = OpenAI()


@traceable
def format_messages(question: str) -> list[dict]:
    return [
        {"role": "system", "content": "You're a helpful assistant"},
        {"role": "user", "content": question},
    ]


@traceable(run_type="llm")
def call_llm(messages: list[dict]):
    response = client.chat.completions.create(model="gpt-4.1-mini", messages=messages)
    return response


@traceable
def run_pipeline(question: str):
    messages = format_messages(question)
    response = call_llm(messages)
    return response.choices[0].message.content


run_pipeline("Who are you?")

This will automatically log the input and ouput of the functions decorated with traceable. It will also handle the nesting for you, so that format_messages and call_llm are steps within the run_pipeline function.

In traceable you can customize xyz.

Using a `trace` context manager

In addition, to the traceable decorator, you can also use the trace context manager to create traces. You can easily combine both as shown below:

client = OpenAI()


@traceable
def format_messages(question: str) -> list[dict]:
    return [
        {"role": "system", "content": "You're a helpful assistant"},
        {"role": "user", "content": question},
    ]


@traceable(run_type="llm")
def call_llm(messages: list[dict]):
    response = client.chat.completions.create(model="gpt-4.1-mini", messages=messages)
    return response


app_inputs = {"question": "Who are you?"}

with trace("run_pipeline", inputs=app_inputs) as rt:
    messages = format_messages(app_inputs["question"])
    response = call_llm(messages)
    output = response.choices[0].message.content
    rt.end(outputs={"output": output})

This will generate a trace called “LLM Pipeline” with the input and output of the entire pipeline. Within this trace, you will find the individual traces for each function call.

Using a wrapped client

For OpenAI and Anthropic models, LangSmith offers a wrapped client that automatically instruments calls to the API with tracing. Any call to the LLM will automatically handled by LangSmith. This plays well with using traceable for the rest of the part in your pipeline. For example:

client = wrap_openai(OpenAI())  # Added client wrapper


@traceable
def format_messages(question: str) -> list[dict]:
    return [
        {"role": "system", "content": "You're a helpful assistant"},
        {"role": "user", "content": question},
    ]


# Removed @traceable
def call_llm(messages: list[dict]):
    response = client.chat.completions.create(model="gpt-4.1-mini", messages=messages)
    return response


@traceable
def run_pipeline(question: str):
    messages = format_messages(question)
    response = call_llm(messages)
    return response.choices[0].message.content


run_pipeline("Who are you?")

This will automatically log the LLM calls made within run_pipeline, so you no longer need to add the traceable decorator to each call.

Manually creating traces with `RunTree`

If you want to have more control over the tracing, you can use RunTree. It provides the most flexibility but requires more setup.

Here’s the RunTree version of the previous example:

client = OpenAI()


def format_messages(question: str, parent_run: RunTree):
    format_message_step = parent_run.create_child(
        name="format_messages", run_type="tool", inputs={"question": question}
    )
    format_message_step.post()
    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": question},
    ]
    format_message_step.end(outputs={"messages": messages})
    format_message_step.patch()
    return messages


def call_llm(messages: list[dict], parent_run: RunTree):
    call_llm_step = parent_run.create_child(
        name="call_llm",
        run_type="llm",
        inputs={"messages": messages},
    )
    call_llm_step.post()
    response = client.chat.completions.create(model="gpt-4.1-mini", messages=messages)
    call_llm_step.end(outputs=response)
    call_llm_step.patch()
    return response


def run_pipeline(question: str):
    parent_run = RunTree(name="run_pipeline", inputs={"question": question})
    parent_run.post()

    messages = format_messages(question, parent_run)
    response = call_llm(messages, parent_run)

    parent_run.end(outputs={"answer": response.choices[0].message.content})
    parent_run.patch()


run_pipeline("Who are you?")

This will result in a similar trace, but in this case you have more control over when/what to send in each step.

For all of these methods, you should’ve obtained a trace that looks like this:

To the left of the image, you should see the trace for the run_pipeline function, which includes all the steps taken during the execution of the function, including the formatting of messages and the call to the LLM. To the right, you will see the input and output for the full trace.

Then, you can click on each individual step to view more details about that step, including the inputs, outputs, and any errors that may have occurred.

Here’s format_messages:

And here’s call_llm:

image.png

I recommend you explore the traces on your own. Just looking at the images in this post won’t be enough.

Evaluation

LangSmith lets you evaluate your LLM pipelines by providing you with a way to upload evaluation datasets, define evaluation metrics, and view the results of your experiments.

Let’s explore this by running a set of evals on a sample dataset. You’ll use the AIMO Validation AIME dataset that contains questions, answers and detailed solutions from the 2022, 2023, and 2024 AIME competitions.

You should start by creating a dataset on LangSmith:

ds = load_dataset("AI-MO/aimo-validation-aime")
examples = [
    {"inputs": {"question": d["problem"]}, "outputs": {"answer": int(d["answer"])}}
    for d in ds["train"]
][:15]

client = Client()

dataset_name = "AIME Example Dataset (sample)"

try:
    dataset = client.create_dataset(dataset_name)
    client.create_examples(dataset_id=dataset.id, examples=examples)
except Exception as e:
    print(f"Dataset {dataset_name} already exists. Error: {e}")
    pass

Dataset AIME Example Dataset (sample) already exists. Error: Conflict for /datasets. HTTPError('409 Client Error: Conflict for url: https://api.smith.langchain.com/datasets', '{"detail":"Dataset with this name already exists."}')

This will create a dataset with the first 15 examples from the AIMO Validation AIME dataset. I only included a a sample of the dataset to keep costs down. You can always add more examples later if needed.

The dataset will be available under Datasets & Experiments:

image.png

Then, you’ll define a pipeline that takes the user question, and provides a response using a structured output:

class Response(BaseModel):
    explanation: str = Field(description="The explanation of the answer")
    answer: int = Field(
        description="The answer to the question. It should be an integer."
    )


model = ChatOpenAI(model="gpt-4.1-mini", temperature=0)
model_with_structure = model.with_structured_output(Response, method="function_calling")


def get_response(question: str) -> Response:
    max_retries = 3
    for _ in range(max_retries):
        try:
            messages = [
                SystemMessage(
                    "You're a math expert. You will always respond in a JSON format with the following fields: explanation and answer."
                ),
                HumanMessage(question),
            ]
            response = model_with_structure.invoke(messages)
            return response
        except Exception as e:
            print(f"Error: {e}")
            continue
    raise ValueError("Failed to get a valid response")

I included a simple retry mechanism, as I often found that the model sometime failed to generate a valid response.

Next, you should define the evaluation metrics you’ll use to measure the performance of your pipeline. You could define a simple accuracy metric that checks if the answer is the same as the expected answer:

def accuracy(inputs: dict, outputs: dict, reference_outputs: dict) -> bool:
    return outputs["answer"] == reference_outputs["answer"]

To define an evaluation metric in LangSmith, you must create a function that takes the inputs, outputs, and reference outputs as arguments and returns a boolean or a numeric value.

For accuracy, the function checks if the answer provided by the model matches the expected answer from the dataset, and returns a boolean value indicating whether the evaluation passed or failed.

You can also define more complex metrics, such as an LLM judge to evaluate the clarity of the solution:

class ClarityResponse(BaseModel):
    explanation: str = Field(description="The explanation of the answer")
    clarity: int = Field(description="The clarity of the explanation", ge=1, le=5)


def clarity(inputs: dict, outputs: dict, reference_outputs: dict) -> int:
    messages = [
        SystemMessage(
            content="You are a helpful assistant that evaluates the clarity of the explanation of the answer. You will always return a number between 1 and 5, where 1 is the lowest clarity and 5 is the highest clarity."
        ),
        HumanMessage(content=f"Explanation: {outputs['explanation']}"),
    ]
    model_with_clarity_structure = model.with_structured_output(ClarityResponse)
    response = model_with_clarity_structure.invoke(messages)
    return response.clarity

This clarity metric evaluates the clarity of the explanation provided by the model. It uses a scale from 1 to 5, where 1 indicates low clarity and 5 indicates high clarity.

Finally, you can run the evaluation using client.evaluate():

def ls_wrapper(inputs: dict) -> dict:
    response = get_response(inputs["question"])
    return response.model_dump()


experiment_results = client.aevaluate(
    ls_wrapper, data=dataset_name, evaluators=[accuracy, clarity], max_concurrency=15
)

LangSmith requires you to define a function that wraps your pipeline function. It should take an input dictionary that contains the necessary parameters for your pipeline and return a dictionary with the results. You can also specify a evaluators parameter that includes the evaluation metrics you want to use.

After you’ve run the evaluation, you’ll be able to inspect the results of the experiment:

image.png

You can also investigate single runs:

image.png

Or see how results look over time:

image.png

Once again, I suggest you go explore the results in the LangSmith UI.

Conclusion

That’s all! We’ve covered the basics of using LangSmith to trace and evaluate your LLM applications.

By now, you should have a good understanding of how to create traces, define evaluation metrics, and run experiments.

If you have any questions or feedback, let me know in the comments below.

Citation

BibTeX citation:

@online{castillo2025,
  author = {Castillo, Dylan},
  title = {LangSmith 101},
  date = {2025-08-10},
  url = {https://dylancastillo.co/posts/langsmith-101.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2025. “LangSmith 101.” August 10. https://dylancastillo.co/posts/langsmith-101.html.

Using synthetic data to bootstrap your RAG system evals

Dylan Castillo — Thu, 07 Aug 2025 00:00:00 GMT

One of the coolest ideas I’ve come across when building Retrieval-Augmented Generation (RAG) systems is that you can actually start without any real data at all. You can bootstrap your system (and its evaluation process) using synthetic data.

In the projects I’ve worked on, we usually had enough data to train and evaluate our pipelines, so generating synthetic data wasn’t a concern. But I’ve been curious about this idea for a while. So I decided to dig into it.

I used Hamel Husain and Shreya Shankar’s AI Evals for Engineers & PMs course materials to guide my exploration. If you’re interested in building and evaluating LLM systems, I highly recommend checking out their course.

In this article, I’ll walk you through the process of bootstrapping your RAG system evals using synthetic data.

Let’s get started!

Prerequisites

If you plan to follow along, you’ll need to:

Sign up and generate OpenAI and LangSmith API keys.
Create a .env file in the root directory of your project and add the following lines:

OPENAI_API_KEY=your_openai_api_key
LANGSMITH_TRACING=true
LANGSMITH_PROJECT=your_langchain_project_name
LANGSMITH_API_KEY=your_langsmith_api_key

Create a virtual environment in Python and install the following packages:

uv venv
uv add langchain langchain-openai langchain-community langsmith jupyter chromadb python-dotenv nest_asyncio sentence-transformers

Download the People Group’s section from GitLab’s handbook.

I’m also assuming you’re familiar with the basics of RAG systems and how to use vector databases. If you need a refresher, you can check out my RAG tutorial.

Then, you’ll be able to run the code from this article. If you don’t want to copy and paste the code, you can download this notebook.

How to generate synthetic data for RAG evals

The process is simple. Here’s how it works:

Split your source document into chunks and store them in a vector database.
Sample a few chunks from the vector database.
For each sampled chunk, extract a fact from it and generate a question that is unambiguously answered by the fact.
Define evaluation metrics for your RAG system.
Optionally, filter the generated questions to remove the ones that don’t seem realistic.
Measure the performance of your RAG system.

In the next sections, I’ll help you implement this process step by step, providing code snippets you can run in a Jupyter notebook.

Setup

You’ll use asyncio in some of the code snippets, so you must enable nest_asyncio to run the code:

import nest_asyncio

nest_asyncio.apply()

Then, you can proceed as usual, importing the required packages:

import asyncio
import os
import random
from textwrap import dedent

import chromadb
import pandas as pd
from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction
from dotenv import load_dotenv
from langchain_community.document_loaders import DirectoryLoader, TextLoader
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from langchain_text_splitters import MarkdownTextSplitter
from langsmith import Client, traceable
from pydantic import BaseModel
from sentence_transformers import CrossEncoder

load_dotenv()

These are the most important libraries you’ll use in this article:

chromadb: Vector database for storing and retrieving document embeddings
langchain: Framework for building LLM applications
langchain-openai: Wrapper for OpenAI’s API, providing access to LLMs and embeddings
pydantic: Provides models for generating structured data and validating types
sentence-transformers: In the last section of the article, you’ll use this library to rerank the retrieved documents.

The rest of the libraries will handle typical Python tasks, such as reading files, managing environment variables, etc.

For this tutorial, you’ll be building a RAG system that feeds an internal chatbot that helps employees of a company answer questions about company policies.

I chose this topic because there’s a pretty good source of data we can use for this: The GitLab Handbook. We’ll just use the People Group section of the handbook, to keep costs manageable.

Your next step is to load the data from the handbook:

loader = DirectoryLoader(
    "../data/synthetic-data-rag/people-group/", glob="**/*.md", loader_cls=TextLoader
)
docs = loader.load()

len(docs)

If you set everything up correctly, the cell should output the number of documents in the docs variable. Depending on when you read this article, the number of documents may change, as the handbook is updated regularly. The date I downloaded the data, there were 99 documents in the People Group section.

Then, you must add the data to the vector database.

Index data

A vector database is a database designed to efficiently store and query data as vector embeddings (numerical representations). Provided with a user query, it’s the engine you use to find the most similar data in your database.

For this tutorial, you’ll use ChromaDB. Let’s set it up:

openai_ef = OpenAIEmbeddingFunction(api_key=os.getenv("OPENAI_API_KEY"))
client = chromadb.PersistentClient(
    path="../data/synthetic-data-rag/chroma",
)
collection = client.get_or_create_collection(
    "gitlab-handbook", embedding_function=openai_ef
)

This code snippet:

Defines an embedding function that uses OpenAI’s API to generate embeddings for the documents.
Creates a ChromaDB client to interact with the vector database.
Creates a collection in the vector database to store the document embeddings and sets the embedding function to use the OpenAI embedding model.

Next, you should generate embeddings for the documents and store them in the vector database. But there’s a little catch: some documents are longer than the maximum token limit of the embedding model (8192 tokens). This will break the indexing process. To avoid this, you must split the documents into smaller chunks.

You can do this with the following code snippet:

text_splitter = MarkdownTextSplitter.from_tiktoken_encoder(
    model_name="gpt-4o",
    chunk_size=400,
    chunk_overlap=0,
)
splits = text_splitter.split_documents(docs)

The handbook is written using Markdown, so you can use the MarkdownTextSplitter to split the documents. This will use the headings in the files for the splitting in addition to the number of tokens. This generally results in better chunks, as they will be more likely to contain complete thoughts or sections of the document.

The from_tiktoken_encoder method lets you do the splits based on the number of tokens, not characters which is the default behavior. You’ve set a chunk size of 400 tokens, with no overlap.

After running the splitting code, you can check the number of chunks created by running this:

len(splits)

Now you have another problem: you have too many chunks. If you try to add all of them to the vector database at once—which also generates their embeddings—you’ll likely hit the OpenAI API rate limits.

To solve this, let’s define a utility function that adds the chunks to the vector database in batches:

def create_batches(ids, documents, metadatas, batch_size=100):
    batches = []
    for i in range(0, len(ids), batch_size):
        batch_ids = ids[i : i + batch_size]
        batch_documents = documents[i : i + batch_size]
        batch_metadatas = metadatas[i : i + batch_size]
        batches.append((batch_ids, batch_metadatas, batch_documents))
    return batches

Then, you can apply this function to the chunks you created earlier:

ids = [f"{str(i)}" for i in range(len(splits))]
documents = [doc.page_content for doc in splits]
metadatas = [doc.metadata for doc in splits]


if collection.count() > 0:
    print("Collection already exists, skipping creation.")
else:
    print("Adding documents...")
    batches = create_batches(ids=ids, documents=documents, metadatas=metadatas)
    for i, batch in enumerate(batches):
        print(f"Adding batch {i} of size {len(batch[0])}")
        collection.add(ids=batch[0], metadatas=batch[1], documents=batch[2])

Feel free to adjust the batch size according to your needs. This should take a few seconds to run. Once it’s done, your vector database should be ready to use.

Next, you’ll define a couple of functions to interact with the vector database and a data model to represent the retrieved documents you’ll be working with.

class RetrievedDoc(BaseModel):
    id: str
    path: str
    page_content: str


def get_similar_docs(text: str, top_k: int = 5) -> list[RetrievedDoc]:
    results = collection.query(query_texts=[text], n_results=top_k)
    docs = [results["documents"][0][i] for i in range(top_k)]
    metadatas = [results["metadatas"][0][i] for i in range(top_k)]
    ids = [results["ids"][0][i] for i in range(top_k)]
    return [
        RetrievedDoc(id=id_, path=m["source"], page_content=d)
        for d, m, id_ in zip(docs, metadatas, ids)
    ]


def get_doc_by_id(doc_id: str) -> RetrievedDoc:
    results = collection.get(ids=[doc_id])
    doc = results["documents"][0]
    metadata = results["metadatas"][0]
    return RetrievedDoc(id=doc_id, path=metadata["source"], page_content=doc)

In this code snippet, you define two functions:

get_similar_docs will let you retrieve the most similar documents to a user query.
get_document_by_id will let you retrieve a document by its ID.

RetrievedDoc is a Pydantic model that represents a retrieved document. It includes the document’s ID, file path, and the page content. This model will help you structure the data you retrieve from the vector database.

Next, you can sample documents for the synthetic data generation:

golden_docs_idx = random.sample(range(len(splits)), 200)
golden_docs = [get_doc_by_id(str(i)) for i in golden_docs_idx]

This will result in 200 documents that you’ll use to generate the synthetic data.

Generate QA Pairs

Using the documents you just sampled, you can generate synthetic data. For each document, you’ll extract a fact and generate a question from it.

This is simple but it has a big issue: it may produce questions that are too easy to answer and result in an overly optimistic evaluation of your RAG system.

To create more challenging synthetic queries, Hamel and Shreya recommend adding similar confounding chunks to the generation process, so that it can generate questions in an adversarial manner. The generator will create a question that is uniquely answered by the target chunk but also include themes or keywords that are present in other chunks.

Here’s an example of how this works:

Target chunk: “George Orwell’s masterpiece, Nineteen Eighty-Four, was published in June 1949 and introduced the concept of ‘Big Brother’ to a global audience.”

Similar chunks:

“Aldous Huxley’s Brave New World, another influential work of dystopian fiction, was first released in 1932 and explores themes of social conditioning and control.”
“Ray Bradbury’s Fahrenheit 451, published in 1953, depicts a future society where books are banned and ‘firemen’ burn any that are found.”

Synthetic Question: “In what year was the dystopian novel that introduced the concept of ‘Big Brother’ published?”

The target chunk helps the generator come up with a synthetic question. The similar chunks provide distractors that help the generator include themes or keywords that are also present in other chunks (e.g., dystopian fiction), making the question more challenging.

To do this, you can use the following prompts:

system_prompt_generate = dedent(
    """
    You are a helpful assistant generating synthetic QA pairs for retrieval evaluation.

    Given a target chunk of text and a set of confounding chunks, you must extract a specific, self-contained fact from the target chunk that is not included in the confounding chunks. Then write a question that is directly and unambiguously answered by that fact. The question should only be answered by the fact extracted from the target chunk (and not by any of the confounding chunks) but it should also use themes or terminology that is present in the confounding chunks.

    Always respond with a JSON object with the following keys (in that exact order):

    1. "fact": "",
    2. "confounding_terms": "",
    3. "question": "",
    
    You should write the questions as if you're an employee looking for information in the handbook. The question should be as realistic and natural as possible, reflecting the kind of queries an employee might actually make when searching for information in the handbook.
    """
)

user_prompt_generate = dedent(
    """
    TARGET CHUNK:
    {target_chunk}

    CONFOUNDING CHUNKS:
    {confounding_chunks} 
    """
)

These prompts will be used to generate the synthetic data. The system_prompt_generate defines the generation process, explaining how to extract facts and generate questions, and user_prompt_generate provides the required context: target and confounding chunks.

Then, you initialize the LLM, set up the response model, and define a function to format the documents for the LLM:

class Response(BaseModel):
    fact: str
    confounding_terms: list[str] = []
    question: str


llm = ChatOpenAI(model="gpt-4.1-mini", temperature=1)
llm_with_structured_output = llm.with_structured_output(Response)

messages = ChatPromptTemplate.from_messages(
    [("system", system_prompt_generate), ("user", user_prompt_generate)]
)

def format_docs(chunks: list[RetrievedDoc]) -> str:
    return "\n".join(
        [f"*** Filepath: {chunk.path} ***\n{chunk.page_content}\n" for chunk in chunks]
    )

Finally, you can define a function to generate the synthetic data. This function will take a target chunk, retrieve the most similar chunks from the vector database, and generate a question from the target chunk using the similar chunks as distractors:

async def generate_qa_pair(chunk):
    similar_chunks = get_similar_docs(chunk.page_content)
    compiled_messages = await messages.ainvoke(
        {
            "target_chunk": format_docs([similar_chunks[0]]),
            "confounding_chunks": format_docs(similar_chunks[1:]),
        }
    )
    output = await llm_with_structured_output.ainvoke(compiled_messages)
    return output

To speed up question generation, you can run this concurrently using asyncio:

tasks = [generate_qa_pair(random_split) for random_split in golden_docs]
qa_pairs = await asyncio.gather(*tasks)

df = pd.DataFrame([qa_pair.dict() for qa_pair in qa_pairs])
df.to_excel("../data/synthetic-data-rag/files/qa_pairs.xlsx", index=False)

Here are some of the resulting QA pairs:

Example 1:
- Question: How soon should managers send the results after the 360 feedback cycle closes to prepare for the feedback meeting?
- Answer: Managers should send the results of 360 feedback within 48 hours of the feedback cycle closing so they can prepare and come to the meeting with questions and discussion points.
Example 2:
- Question: At what point in the hiring process must candidates disclose outside employment or side projects for GitLab to assess potential conflicts with their job obligations?
- Answer: Candidates at a certain stage in the recruiting process are asked to disclose outside employment, side projects, or other activities so GitLab can determine if a conflict exists with their ability to fulfill obligations to GitLab.

Even though the questions seem relevant, it’s not entirely clear if they are truly the type of questions real users ask.

To improve that, you can iterate a bit more on the prompt, including few-shot examples of real or adjusted queries. Or, you can take the lazy way out and generate a filter that will help you remove the questions that don’t seem realistic enough. Let’s do that!

Filter QA pairs

For that, you should open the Excel file and manually review some of the generated Q&A pairs and evaluate them for relevance. Then, you should provide those questions in a system prompt as examples to the LLM, asking it to assign a score to each question based on your evaluation criteria.

Here’s an example of how to do that:

system_prompt_rate = dedent(
    """
    You are an AI assistant helping us curate a high-quality dataset of questions for evaluating an company's internal handbook. We have generated synthetic questions and need to filter out those that are unrealistic or not representative of typical user queries.

    Here are examples of realistic and unrealistic user queries we have manually rated:

    ### Realistic Queries (Good Examples)

    * **Query:** "What is the required process for creating a new learning hub for your team in Level Up at GitLab?"
        * **Explanation:** Very realistic user query. It's concise, information-seeking, and process-oriented.
        * **Rating:** 5
    * **Query:** "Where is the People Operations internal handbook hosted, and how can someone gain access to it?"
        * **Explanation:** Realistic query but might be a bit too detailed for a typical user.
        * **Rating:** 4
    * **Query:** "Who controls access to People Data in the data warehouse at GitLab, and what approvals are required for Analytics Engineers and Data Analysts to obtain access?"
        * **Explanation:** Seems reasonable but too lengthy for a typical user query. 
        * **Rating:** 3

    ### Unrealistic Queries (Bad Examples)

    * **Query:** "If a GitLab team member has been with the company for over 3 months and is interested in participating in the Onboarding Buddy Program, what should they do to express their interest?"
        * **Explanation:** Overly specific and unnatural. No real user would ask this.
        * **Rating:** 1
    * **Query:** "On what date did the 'Managing Burnout with Time Off with John Fitch' session occur as part of the FY21 Learning Speaker Series?"
        * **Explanation:** Irrelevant and overly specific. Not a typical user query. 
        * **Rating:** 2

    ### Your Task

    For the following generated question, please:

    1.  Rate its realism as a typical user query for an internal handbook application on a scale of 1 to 5 (1 = Very Unrealistic, 3 = Neutral/Somewhat Realistic, 5 = Very Realistic).
    2.  Provide a brief explanation for your rating, comparing it to the examples above if helpful.

    ### Output Format

    **Explanation:** `[Your brief explanation]`
    **Rating:** `[Your 1–5 rating]`
    """
)

user_prompt_rate = dedent(
    """
    **Generated Question to Evaluate:**
    `{question_to_evaluate}`
    """
)

Hamel and Shreya generally discourage using Likert-type 1-5 scales for LLM judges. However, in this case, we’re not aiming for a very accurate judge, we’re just trying to have a method that works well enough to filter out the questions. We don’t need to make this overly complicated.

Using the LLM judge, you can apply the filter to the generated questions:

class ResponseFiltering(BaseModel):
    explanation: str
    rating: int


llm_with_structured_output_filtering = llm.with_structured_output(ResponseFiltering)

messages_filtering = ChatPromptTemplate.from_messages(
    [("system", system_prompt_rate), ("user", user_prompt_rate)]
)

async def rate_qa_pair(qa_pair):
    compiled_messages = await messages_filtering.ainvoke(
        {"question_to_evaluate": qa_pair.question}
    )
    output = await llm_with_structured_output_filtering.ainvoke(compiled_messages)
    return output


tasks = [rate_qa_pair(qa_pair) for qa_pair in qa_pairs]
results = await asyncio.gather(*tasks)

rated_qa_pairs = [
    {
        "rating": result.rating,
        "explanation": result.explanation,
        "question": qa_pair.question,
        "answer": qa_pair.fact,
    }
    for (result, qa_pair) in zip(results, qa_pairs)
]

df_rated_qa_pairs = pd.DataFrame(
    rated_qa_pairs, columns=["Rating", "Explanation", "Question"]
)

df_rated_qa_pairs.to_excel(
    "../data/synthetic-data-rag/files/rated_qa_pairs.xlsx", index=False
)

This will result in a list of ResponseFiltering objects, each containing an explanation and a rating for the corresponding question.

You can save the results to a file for later use.

Evaluate the RAG system

Now that we have the filtered QA pairs, it’s time to evaluate your RAG system. You’ll evaluate two parts of the RAG system: retrieval and generation.

Let’s use LangSmith to store our evaluation results. Start by creating a dataset on LangSmith:

langsmith_client = Client()
dataset_name = "Gitlab Handbook QA Evaluation 2"

try:
    dataset = langsmith_client.create_dataset(dataset_name=dataset_name)
    examples = [
        {
            "inputs": {
                "question": h["question"],
            },
            "outputs": {
                "answer": h["answer"],
                "doc": {
                    "id": chunk.id,
                    "path": chunk.path,
                },
            },
        }
        for h, chunk in zip(rated_qa_pairs, golden_docs)
        if h["rating"] >= 5
    ]
    langsmith_client.create_examples(dataset_id=dataset.id, examples=examples)
except Exception:
    print("Dataset already exists, skipping creation.")
    dataset = langsmith_client.read_dataset(dataset_name=dataset_name)

This will create a dataset on LangSmith with the rated QA pairs. Each example will include the question, answer, and the document from which the question was generated.

Retrieval Metrics

To evaluate the retrieval part of the RAG system, you can use metrics such as recall@k, precision@k, Mean Reciprocal Rank (MRR), and Normalized Discounted Cumulative Gain (NDCG).

For this tutorial, you’ll use two metrics: MRR and recall@k.

Mean Reciprocal Rank (MRR)

MRR measures how well the RAG system retrieves relevant documents. It calculates the average of the reciprocal ranks of the first relevant document for each query. It essentially measures how quickly the system retrieves the first relevant document for a given query.

Reciprocal Rank (RR) is calculated for a single query. It is the reciprocal of the rank at which the first relevant document is found. For example, if the first relevant item is at position 1, the RR is 1. If it’s at position 3, the RR is . The formula is:

where is the position of the first relevant document.

MRR is the average of the Reciprocal Rank scores across all your queries. It provides a single, aggregate measure of retrieval performance. An MRR of 1 means you found the correct document at the first position for every query. The formula is:

where is the number of queries and is the position of the first relevant document for query .

Recall@k

Recall@k measures the proportion of relevant documents retrieved in the top k results. It helps you understand how many relevant documents are retrieved by the RAG system.

For example, if you retrieve 5 documents and 3 of them are relevant, recall@5 is 3/5 = 0.6. The formula is:

where the numerator is the number of relevant documents retrieved in the top k results, and the denominator is the total number of relevant documents for the query. To get the overall performance, you average the Recall@k values across all queries.

In our specific case, since we only have one relevant document per query, Recall@k will be 1 if the relevant document is in the top k results, and 0 otherwise.

You can define two LangSmith evaluators to calculate these metrics:

def mrr(inputs: dict, outputs: dict, reference_outputs: dict) -> float:
    reference_docs = [str(reference_outputs["doc"]["id"])]
    docs = outputs.get("docs", [])
    if not docs:
        return 0.0
    rank = next((i + 1 for i, doc in enumerate(docs) if doc in reference_docs), None)
    return 1.0 / rank if rank else 0.0


def recall(inputs: dict, outputs: dict, reference_outputs: dict) -> float:
    reference_docs = [str(reference_outputs["doc"]["id"])]
    docs = outputs.get("docs", [])
    if not docs:
        return 0.0
    return float(any(doc in reference_docs for doc in docs))

LangSmith evaluators take inputs, outputs, and reference_outputs as arguments. The inputs are the user query and the retrieved documents, the outputs are the generated answers, and the reference_outputs are the target chunks.

Using those values, you can use the formulas we discussed to compute the MRR and recall@k metrics.

Generation metrics

In addition to measuring how good are the retrieved documents, you also want to measure if the LLM makes good use of them to generate answers. For that, Hamel and Shreya recommend using ARES or RAGAS.

The only issue is that ARES requires a human preference validation set of at least 50 examples and the standard RAGAS metrics consume tons of tokens. So, to keep things simple, you’ll build 3 simple metrics using an LLM judge, similar to RAGAS’ Nvidia metrics:

Answer accuracy: Measures how accurate the generated answer is compared to the expected answer.
Context relevance: Measures if the context provided to the LLM is relevant to the user query.
Groundedness: Measures if the generated answer is grounded in the provided context.

Compared to RAGAS implementation of these same metrics, the ones you’ll define here don’t do multiple runs and average the resulting scores. But, if you want to, you can easily apply a parallelization strategy to do this. I’d also recommend using reasoning models, as they tend to perform better in these types of tasks.

Let’s see how to implement these metrics using LangSmith.

Answer accuracy

This metric evaluates how accurate the generated answer is compared to the expected answer. It’s an LLM judge that scores the generated answer against a reference answer using a 0, 1, 2 scale:

system_prompt_answer_accuracy = dedent(
    """
    You are an expert evaluator. Your task is to evaluate the accuracy of a User Answer against a Reference Answer, given a Question.

    Here's the grading scale you must use:

    0 - If User Answer is not contained in Reference Answer or not accurate in all terms, topics, numbers, metrics, dates and units or the User Answer do not answer the question.
    2 - If User Answer is full contained and equivalent to Reference Answer in all terms, topics, numbers, metrics, dates and units.
    1 - If User Answer is partially contained and almost equivalent to Reference Answer in all terms, topics, numbers, metrics, dates and units.

    Your rating must be only 0, 1 or 2 according to the instructions above.

    Your answer must be a JSON object with the following keys:
    1. "explanation": "",
    2. "rating": ""
    """
)

user_prompt_answer_accuracy = dedent(
    """
    **Question:** `{question}`
    **User Answer:** `{user_answer}`
    **Reference Answer:** `{reference_answer}`
    """
)

messages_answer_accuracy = ChatPromptTemplate.from_messages(
    [("system", system_prompt_answer_accuracy), ("user", user_prompt_answer_accuracy)]
)


class ResponseAnswerAccuracy(BaseModel):
    explanation: str
    rating: int


llm = ChatOpenAI(model="gpt-4.1-mini")

llm_with_structured_output_answer_accuracy = llm.with_structured_output(
    ResponseAnswerAccuracy
)


async def answer_accuracy(
    inputs: dict, outputs: dict, reference_outputs: dict
) -> float:
    compiled_messages = await messages_answer_accuracy.ainvoke(
        {
            "question": inputs["question"],
            "user_answer": outputs["answer"],
            "reference_answer": reference_outputs["answer"],
        }
    )
    output = await llm_with_structured_output_answer_accuracy.ainvoke(compiled_messages)
    return output.rating / 2.0

Similar to the retrieval evaluators, you can define LangSmith evaluators for the answer accuracy metric. The inputs will contain the user question, the outputs will have the generated answer, and the reference_outputs will have the expected answer.

Context relevance

This metric evaluates if the context provided to the LLM is relevant to the user query. Similar to the answer accuracy metric, it uses an LLM judge that scores the context relevance against a reference answer using a 0, 1, 2 scale:

system_prompt_context_relevance = dedent(
    """
    You are an expert evaluator. Your task is to evaluate the relevance of a Context in order to answer a Question. 

    Do not rely on your previous knowledge about the Question. Use only what is written in the Context and in the Question.

    Here's the grading scale you must use:

    0 - If the context does not contain any relevant information to answer the question.
    1 - If the context partially contains relevant information to answer the question.
    2 - If the context contains relevant information to answer the question.

    You must always provide the relevance score of 0, 1, or 2, nothing else.

    Your answer must be a JSON object with the following keys:
    1. "explanation": "",
    2. "rating": ""
    """
)

user_prompt_context_relevance = dedent(
    """
    **Question:** `{question}`
    **Context:** `{context}`
    """
)

messages_context_relevance = ChatPromptTemplate.from_messages(
    [
        ("system", system_prompt_context_relevance),
        ("user", user_prompt_context_relevance),
    ]
)


class ResponseContextRelevance(BaseModel):
    explanation: str
    rating: int


llm = ChatOpenAI(model="gpt-4.1-mini")
llm_with_structured_output_context_relevance = llm.with_structured_output(
    ResponseContextRelevance
)


async def context_relevance(
    inputs: dict, outputs: dict, reference_outputs: dict
) -> float:
    compiled_messages = await messages_context_relevance.ainvoke(
        {
            "question": inputs["question"],
            "context": outputs["context"],
        }
    )
    output = await llm_with_structured_output_context_relevance.ainvoke(
        compiled_messages
    )
    return output.rating / 2

You can define a context_relevance evaluator. The inputs will contain the user question and from the outputs you’ll use the context provided to the LLM.

Groundedness

This metric evaluates if the answer is grounded in the provided context. Like before, you use an LLM judge that scores the groundedness of the answer against the context using a 0, 1, 2 scale:

system_prompt_groundedness = dedent(
    """
    You are an expert evaluator. Your task is to evaluate the groundedness of an assertion against a context. 

    Do not rely on your previous knowledge about the assertion or context. Use only what is written in the assertion and in the context.

    Here's the grading scale you must use:

    0 - If the assertion is not supported by the context. Or, if the context or assertion is empty.
    1 - If the context partially contains relevant information to support the assertion.
    2 - If the context fully supports the assertion.

    You must always provide the relevance score of 0, 1, or 2, nothing else.

    Your answer must be a JSON object with the following keys:

    1. "explanation": "",
    2. "rating": ""
    """
)

user_prompt_groundedness = dedent(
    """
    **Assertion:** `{answer}`
    **Context:** `{context}`
    """
)

messages_groundedness = ChatPromptTemplate.from_messages(
    [("system", system_prompt_groundedness), ("user", user_prompt_groundedness)]
)


class ResponseGroundedness(BaseModel):
    explanation: str
    rating: int


llm = ChatOpenAI(model="gpt-4.1-mini")
llm_with_structured_output_groundedness = llm.with_structured_output(
    ResponseGroundedness
)


async def groundedness(inputs: dict, outputs: dict, reference_outputs: dict) -> float:
    compiled_messages = await messages_groundedness.ainvoke(
        {
            "answer": outputs["answer"],
            "context": outputs["context"],
        }
    )
    output = await llm_with_structured_output_groundedness.ainvoke(compiled_messages)
    return output.rating / 2

You use the same approach as before to define LangSmith evaluators for this metric. The inputs will contain the user question, the outputs will have the context provided to the LLM.

Run evaluation

Now we can run the full RAG pipeline and evaluate its results using the LangSmith evaluators.

system_prompt_generation = dedent(
    """
    You're a helpful assistant. Provided with a question and the most relevant documents, you must generate a concise and accurate answer based on the information in those documents.
    """
)

user_prompt_generation = dedent(
    """
    QUESTION: {question}

    RELEVANT DOCUMENTS:
    {documents}
    """
)

messages_generation = ChatPromptTemplate.from_messages(
    [("system", system_prompt_generation), ("user", user_prompt_generation)]
)

llm_generation = ChatOpenAI(
    model="gpt-4o-mini",
)

A good starting point is to evaluate different values for the number of retrieved documents (K). For example, you could evaluate different values for the number of retrieved documents.

Langsmith requires a wrapper or target function that encapsulates your RAG system. In your case, this function takes a user query, retrieves the most similar documents, generates an answer using the LLM, and returns the generated answer with the document IDs and context retrieved.

I’ll run this code for K values of 3, 5, and 10, and compare the results.

for K in [3, 5, 10]:
    print(f"Running evaluation for K={K}")

    @traceable
    async def target(inputs: dict) -> dict:
        relevant_docs = get_similar_docs(inputs["question"], top_k=K)
        formatted_docs = format_docs(relevant_docs)
        messages = await messages_generation.ainvoke(
            {
                "question": inputs["question"],
                "documents": formatted_docs,
            }
        )
        response = await llm_generation.ainvoke(messages)
        return {
            "answer": response.content,
            "docs": [doc.id for doc in relevant_docs],
            "context": formatted_docs,
        }

    experiment_results = await langsmith_client.aevaluate(
        target,
        data=dataset_name,
        evaluators=[recall, mrr, answer_accuracy, context_relevance, groundedness],
        max_concurrency=50,
        experiment_prefix=f"top-{K}",
    )

Using aevaluate you can speed up the evaluation process and run the evaluations concurrently. I got the following results:

k	Answer accuracy	Context relevance	Groundedness	MRR	Recall
3	0.81	0.93	0.97	0.75	0.80
5	0.80	0.94	0.97	0.76	0.85
10	0.85	0.97	0.98	0.77	0.91

You can see that recall improves significantly as you increase the number of retrieved documents, which is expected. Answer accuracy and context relevance only seems to improve significantly when you increase the number of retrieved documents from 5 to 10.

If you got here, you’ve successfully built a RAG system and evaluated it using synthetic data. The next steps would be to continue making changes to parts of the pipeline and re-running the evaluations to see how they affect the performance of your RAG system.

You will also want to improve the quality of the generated questions, and ideally include real user queries in the evaluation process.

Next, I’ll show you how to squeeze a bit more performance from the retrieval part of the RAG system by using a reranker.

Improve metrics with a reranker

A quick way to improve your RAG system is to rerank the retrieved documents. In addition to doing retrieval using vector similarity or keyword search, you can have a reranking step that uses a more capable model to score and reorder the retrieved documents based on their relevance to the user query.

Let’s use sentence-transformers with an open-source model to do this:

cross_encoder = CrossEncoder("mixedbread-ai/mxbai-rerank-xsmall-v1")

To rerank a set of documents, you take the results from the retrieval step and pass them to the reranker, which will return a new set of documents ordered by their relevance to the user query.

Here’s an example:

query = "What is the process for creating a new learning hub for your team in Level Up at GitLab?"
hits = get_similar_docs(query, top_k=50)
cross_inp = [[query, h.page_content] for h in hits]
reranker_scores = cross_encoder.predict(cross_inp)
sorted_hits = sorted(hits, key=lambda x: reranker_scores[hits.index(x)], reverse=True)

This takes the 50 most similar documents to the query and reranks them using a cross-encoder model. The reranker_scores are used to sort the documents in descending order of relevance.

Now you can do the same evaluation as before (for k = 5), but including this new reranking step:

K = 5

def get_reranked_docs(
    query: str, similar_docs: list[RetrievedDoc]
) -> list[RetrievedDoc]:
    cross_inp = [[query, doc.page_content] for doc in similar_docs]
    reranker_scores = cross_encoder.predict(cross_inp)
    sorted_docs = sorted(
        similar_docs, key=lambda x: reranker_scores[similar_docs.index(x)], reverse=True
    )
    return sorted_docs


@traceable
async def target_with_reranking(inputs: dict) -> dict:
    relevant_docs = get_similar_docs(inputs["question"], top_k=75)
    reranked_docs = get_reranked_docs(inputs["question"], relevant_docs)[:K]
    formatted_docs = format_docs(reranked_docs)
    messages = await messages_generation.ainvoke(
        {
            "question": inputs["question"],
            "documents": formatted_docs,
        }
    )
    response = await llm.ainvoke(messages)
    return {
        "answer": response,
        "docs": [doc.id for doc in reranked_docs],
        "context": formatted_docs,
    }

experiment_results = await langsmith_client.aevaluate(
    target_with_reranking,
    data=dataset_name,
    evaluators=[recall, mrr, answer_accuracy, context_relevance, groundedness],
    max_concurrency=50,
    experiment_prefix=f"top-{K}-reranked",
)

Here are the results of the original run with k=5 and the run with a reranker:

experiment	answer_accuracy	context_relevance	groundedness	mrr	recall
k=5, vanilla	0.80	0.94	0.97	0.76	0.85
k=5, rerank	0.97	0.97	1.00	0.79	0.91

You can see that it immediately improves most metrics, especially the answer accuracy and recall. You should expect some variance in the results, so be aware that the change is not necessarily as big as it seems (but that’s a topic for another article!).

Conclusion

In this tutorial, you’ve learned how to use synthetic data to bootstrap your RAG system evaluations. We covered:

Synthetic data generation: How to generate QA pairs from your documents using adversarial techniques with confounding chunks
Evaluation metrics: Both retrieval metrics (MRR, Recall@k) and generation metrics (answer accuracy, context relevance, groundedness)
Filtering synthetic data: Using an LLM judge to filter out unrealistic questions and improve the dataset quality
Performance optimization: How reranking can significantly improve both retrieval and generation metrics

This approach gives you a solid foundation for evaluating RAG systems even when you don’t have real user data. Synthetic data is useful for getting started quickly, but remember that you should incorporate real user queries into your evaluation process, as that is the most realistic way to evaluate your RAG system.

Hope you find this tutorial useful. If you have any questions, leave a comment below.

Citation

BibTeX citation:

@online{castillo2025,
  author = {Castillo, Dylan},
  title = {Using Synthetic Data to Bootstrap Your {RAG} System Evals},
  date = {2025-08-07},
  url = {https://dylancastillo.co/posts/synthetic-data-rag.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2025. “Using Synthetic Data to Bootstrap Your RAG System Evals.” August 7. https://dylancastillo.co/posts/synthetic-data-rag.html.

Evaluator-optimizer workflow with Pydantic AI

Dylan Castillo — Wed, 09 Jul 2025 00:00:00 GMT

I’m doing a deep dive into Pydantic AI, so I’ve been re-implementing typical patterns for building agentic systems.

In this post, I’ll explore how to build a evaluator-optimizer workflow. I won’t cover the basics of agentic workflows, so if you’re not familiar with the concept, I recommend you to read this post first.

What is evaluator-optimizer?

Evaluator-optimizer is a pattern that has an LLM generator and an LLM evaluator. The generator generates a solution and the evaluator evaluates if the solution is good enough. If it’s not, the generator is given feedback and it generates a new solution. This process is repeated until the solution is good enough.

It looks like this:

flowchart LR
    In([In]) --> Gen["Generator (LLM)"]
    Gen -- "Solution" --> Eval["Evaluator (LLM)"]
    Eval -- "Accepted" --> Out([Out])
    Eval -- "Rejected + Feedback" --> Gen

Examples:

Content generation that must match certain guidelines such as writing with a particular style.
Improving search results iteratively

Let’s see how this looks in practice.

Setup

I will implement a simple workflow:

Generate a candidate article
Evaluate if the article is good enough
If it’s not, provide feedback and generate a new article
Repeat until the article is good enough

Before we start, because Pydantic AI uses asyncio under the hood, you need to enable nest_asyncio to use it in a notebook:

import nest_asyncio

nest_asyncio.apply()

Then, you need to import the required libraries. I’m using Logfire to monitor the workflow.

import os
from typing import Literal

import logfire
import requests
from dotenv import load_dotenv
from pydantic_ai import Agent, RunContext
from pydantic import BaseModel, Field

load_dotenv()

True

PydanticAI is compatible with OpenTelemetry (OTel). So it’s pretty easy to use it with Logfire or with any other OTel-compatible observability tool (e.g., Langfuse).

To enable tracking, create a project in Logfire, generate a Write token and add it to the .env file. Then, you just need to run:

logfire.configure(
    token=os.getenv('LOGFIRE_TOKEN'),
)
logfire.instrument_pydantic_ai()

The first time you run this, it will ask you to create a project in Logfire. From it, it will generate a logfire_credentials.json file in your working directory. In following runs, it will automatically use the credentials from the file.

Evaluator-optimizer workflow

The workflow is composed of two steps:

Text generator: Generates a candidate article.
Evaluator: Evaluates if the article is good enough.

I’ll split the text generation into two agents: generator and fixer. The generator will generate a candidate article and the fixer will fix the article, when provided with feedback.

generator = Agent(
    'openai:gpt-4.1-mini',
    system_prompt=(
        "You are an expert writer. Provided with a topic, you will generate an engaging article with less than 500 words"
    ),
)

fixer = Agent(
    'openai:gpt-4.1-mini',
    system_prompt=(
        "You are an expert writer. Provided with a text and feedback, you wil improve the text."
    ),
)

Logfire project URL: https://logfire-us.pydantic.dev/dylanjcastillo/blog

Next, I’ll create the Evaluator agent. It will take a text and it will evaluate if it’s good enough. It’ll produce an Evaluation object as the output.

class Evaluation(BaseModel):
    explanation: str = Field(
        description="Explain why the text evaluated matches or not the evaluation criteria"
    )
    feedback: str = Field(
        description="Provide feedback to the writer to improve the text"
    )
    is_correct: bool = Field(
        description="Whether the text evaluated matches or not the evaluation criteria"
    )

evaluator = Agent(
    'openai:gpt-4.1-mini',
    system_prompt=(
        "You are an expert evaluator. Provided with a text, you will evaluate if it's written in British English and if it's appropriate for a young audience. The text must always use British spelling and grammar. Make sure the text doesn't include any em dashes."
    ),
    output_type=Evaluation,
)

Finally, you can encapsulate all the logic in a single function:

@logfire.instrument("Run workflow")
def run_workflow(topic: str) -> str:
    text = generator.run_sync(f"Generate an article about '{topic}'")
    evaluation = evaluator.run_sync(f"Evaluate the following text: {text.output}")
    for _ in range(3):
        if not evaluation.output.is_correct:
            text = fixer.run_sync(f"Fix the text: {text.output} with the following feedback: {evaluation.output.feedback}")
            evaluation = evaluator.run_sync(f"Evaluate the following text: {text.output}")
        else:
            return text.output
    return text.output

output = run_workflow("Consumption of hard drugs")

11:28:25.995 Run workflow
11:28:25.995   generator run
11:28:25.996     chat gpt-4.1-mini
11:28:36.293   evaluator run
11:28:36.294     chat gpt-4.1-mini

And here’s the output:

print(output)

**The Complex Reality of Hard Drug Consumption**

Hard drugs — substances such as heroin, cocaine, methamphetamines, and crack — have long been a subject of concern worldwide due to their profound impact on individuals and society. The consumption of these drugs is not merely a matter of personal choice but a complex issue influenced by social, economic, psychological, and cultural factors.

**Understanding Hard Drugs and Their Effects**

Hard drugs are characterized by their high potential for addiction and severe physical and psychological effects. Unlike softer substances such as marijuana or alcohol (when consumed responsibly), hard drugs often disrupt brain function dramatically, leading to addiction, mental health disorders, and significant physical health problems. Users may experience paranoia, hallucinations, heart issues, and even fatal overdoses.

The allure of hard drugs often stems from their ability to produce intense euphoria or numb emotional pain temporarily. However, this fleeting escape comes at a steep cost. Dependence quickly sets in, making cessation incredibly difficult and often trapping users in a cycle of abuse.

**Social and Economic Implications**

The ramifications of hard drug consumption ripple beyond the individual. Families endure emotional and financial strain, communities face increased crime rates and reduced public safety, and healthcare systems are burdened with treating overdoses and long-term complications. Moreover, productivity declines as addiction interferes with employment, contributing to broader economic challenges.

Many users come from marginalized backgrounds, where poverty, trauma, and lack of education or opportunity make drugs seem like a refuge or an escape. This correlation highlights that addressing drug consumption isn't only a matter of law enforcement but of social equity and support.

**Challenges in Addressing Hard Drug Use**

Efforts to reduce hard drug consumption have varied widely, from strict punitive measures to harm reduction strategies. While criminalization seeks to deter use, it often leads to overcrowded prisons and can exacerbate social stigma, making it harder for users to seek help. Conversely, approaches like supervised consumption sites, needle exchange programs, and accessible addiction treatment aim to minimize harm and promote recovery.

Prevention and education are critical components. Informing communities about the risks of hard drugs and providing mental health support can reduce initial experimentation and help those at risk before addiction takes hold.

**Moving Towards Compassionate Solutions**

Ultimately, the consumption of hard drugs is a multifaceted issue requiring balanced and compassionate responses. Policymakers, healthcare providers, and communities must work together to create environments that prioritize treatment over punishment, recognize addiction as a health issue, and promote social support.

By understanding the complex realities behind hard drug use, society can better address its consequences and help those affected find a path to recovery and hope.

That’s all!

You can access this notebook here.

If you have any questions or feedback, please let me know in the comments below.

Citation

BibTeX citation:

@online{castillo2025,
  author = {Castillo, Dylan},
  title = {Evaluator-Optimizer Workflow with {Pydantic} {AI}},
  date = {2025-07-09},
  url = {https://dylancastillo.co/til/evaluator-optimizer-pydantic-ai.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2025. “Evaluator-Optimizer Workflow with Pydantic AI.” July 9. https://dylancastillo.co/til/evaluator-optimizer-pydantic-ai.html.

Parallelization and orchestrator-workers workflows with Pydantic AI

Dylan Castillo — Wed, 09 Jul 2025 00:00:00 GMT

I’ve been re-implementing typical patterns for building agentic systems with Pydantic AI. In this post, I’ll explore how to build a parallelization and orchestrator-worker workflow.

In previous TILs, I’ve explored:

You can download this notebook here.

What is parallelization?

This workflow is designed for tasks that can be easily divided into independent subtasks. The key trade-off is managing complexity and coordination overhead in exchange for significant speed improvements or diverse perspectives.

It looks like this:

flowchart LR
    In([In]) --> LLM1["LLM Call 1"]
    In --> LLM2["LLM Call 2"]
    In --> LLM3["LLM Call 3"]
    LLM1 --> Aggregator["Aggregator"] 
    LLM2 --> Aggregator["Aggregator"] 
    LLM3 --> Aggregator["Aggregator"] 
    Aggregator --> Out([Out])

Examples:

Evaluate multiple independent aspects of a text (safety, quality, relevance)
Process user query and apply guardrails in parallel
Generate multiple response candidates given a query for comparison

What is orchestrator-worker?

This workflow works well for tasks where you don’t know the required subtasks beforehand. The subtasks are determined by the orchestrator.

Here’s a diagram:

flowchart LR
    In([In]) --> Orch[Orchestrator]

    Orch -.-> LLM1["LLM Call 1"]
    Orch -.-> LLM2["LLM Call 2"]
    Orch -.-> LLM3["LLM Call 3"]

    LLM1 -.-> Synth[Synthesizer]
    LLM2 -.-> Synth
    LLM3 -.-> Synth

    Synth --> Out([Out])

Examples:

Coding tools making changes to multiple files at once
Searching multiple sources and synthesize the results

The difference between parallelization and orchestrator-worker is that in parallelization, the subtasks are known beforehand, while in orchestrator-worker, the subtasks are determined by the orchestrator.

Setup

Pydantic AI uses asyncio under the hood, so you’ll need to enable nest_asyncio to run this notebook:

import nest_asyncio

nest_asyncio.apply()

Then, you need to import the required libraries. I’m using Logfire to monitor the workflow.

import asyncio
import os
from pprint import pprint
from typing import Literal, Optional

import logfire
import requests
from dotenv import load_dotenv
from pydantic import BaseModel, Field
from pydantic_ai import Agent, RunContext

load_dotenv()

True

PydanticAI is compatible with OpenTelemetry (OTel). So it’s pretty easy to use it with Logfire or with any other OTel-compatible observability tool (e.g., Langfuse).

To enable tracking, create a project in Logfire, generate a Write token and add it to the .env file. Then, you just need to run:

logfire.configure(
    token=os.getenv("LOGFIRE_TOKEN"),
)
logfire.instrument_pydantic_ai()

Parallelization example

In this example, I’ll show you how to build a workflow that runs the same evaluator in parallel and then aggregates the results.

class Evaluation(BaseModel):
    explanation: str
    is_appropiate: bool


class AggregatedResults(BaseModel):
    summary: str
    is_appropiate: bool

Then you can create the agents and encapsulate the logic in a function:

evaluator = Agent(
    "openai:gpt-4.1-mini",
    output_type=Evaluation,
    system_prompt=(
        "You are an expert evaluator. Provided with a text, you will evaluate if it's appropriate for a general audience."
    ),
)

aggregator = Agent(
    "openai:gpt-4.1-mini",
    output_type=AggregatedResults,
    system_prompt=(
        "You are an expert evaluator. Provided with a list of evaluations, you will summarize them and provide a final evaluation."
    ),
)

@logfire.instrument("Run workflow")
async def run_workflow(topic: str) -> str:
    tasks = [evaluator.run(f"Evaluate the following text: {topic}") for _ in range(3)]
    evaluations = await asyncio.gather(*tasks)
    aggregated_results = await aggregator.run(f"Summarize the following evaluations:\n\n{[(eval.output.explanation, eval.output.is_appropiate) for eval in evaluations]}")
    return aggregated_results.output

output = await run_workflow("Athletes should consume enhancing drugs to improve their performance.")

15:28:36.289 Run workflow
15:28:36.290   agent run
15:28:36.290     chat gpt-4.1-mini
15:28:36.291   agent run
15:28:36.291     chat gpt-4.1-mini
15:28:36.292   agent run
15:28:36.292     chat gpt-4.1-mini

Logfire project URL: https://logfire-us.pydantic.dev/dylanjcastillo/blog

15:28:39.380   aggregator run
15:28:39.380     chat gpt-4.1-mini

Finally, you can run the workflow. You should get an output like this:

pprint(output.model_dump())

{'is_appropiate': False,
 'summary': 'All evaluations agree that the text promotes the consumption of '
            'performance-enhancing drugs by athletes, which is a sensitive and '
            'controversial topic. The main concerns highlighted are health '
            'risks, ethical issues, fairness in sports, and legality. The '
            'evaluations consistently indicate that encouraging or normalizing '
            'the use of such drugs is inappropriate for a general audience as '
            'it may promote illegal, harmful, or unsafe behavior. There is '
            'consensus that the subject should be handled with caution.'}

Orchestrator-workers example

In this example, I’ll show you how to build a workflow that given a topic generates a table of contents, then writes each section of the article by making an individual request to an LLM.

First, you must define the data structures we’ll use for the workflow outputs.

class Section(BaseModel):
    name: str = Field(description="The name of the section")
    description: str = Field(description="The description of the section")


class CompletedSection(BaseModel):
    name: str = Field(description="The name of the section")
    content: str = Field(description="The content of the section")


class Sections(BaseModel):
    sections: list[Section] = Field(description="The sections of the article")

Then, we’ll define the agents we’ll use in the workflow.

orchestrator = Agent(
    "openai:gpt-4.1-mini",
    output_type=Sections,
    system_prompt=(
        "You are an expert writer specialized in SEO. Provided with a topic, you will generate the sections for a short article."
    ),
)

worker = Agent(
    "openai:gpt-4.1-mini",
    output_type=CompletedSection,
    system_prompt=(
        "You are an expert writer specialized in SEO. Provided with a topic and a section, you will generate the content of the section."
    ),
)

def synthesizer(sections: list[CompletedSection]) -> str:
    completed_sections_str = "\n\n".join(
        [section.content for section in sections]
    )
    return completed_sections_str

Then, you can define a function that orchestrates the workflow:

@logfire.instrument("Run workflow")
async def run_workflow(topic: str) -> str:
    orchestrator_output = await orchestrator.run(f"Generate the sections for a short article about {topic}")
    tasks = [worker.run(f"Write the section {section.name} about {topic} with the following description: {section.description}") for section in orchestrator_output.output.sections]
    completed_sections = await asyncio.gather(*tasks)
    full_article = synthesizer([c.output for c in completed_sections])
    return full_article

output = await run_workflow("Artificial Intelligence")

15:32:13.934 Run workflow
15:32:13.936   orchestrator run
15:32:13.937     chat gpt-4.1-mini
15:32:18.567   agent run
15:32:18.568     chat gpt-4.1-mini
15:32:18.569   agent run
15:32:18.569     chat gpt-4.1-mini
15:32:18.570   agent run
15:32:18.571     chat gpt-4.1-mini
15:32:18.572   agent run
15:32:18.572     chat gpt-4.1-mini
15:32:18.573   agent run
15:32:18.573     chat gpt-4.1-mini

That’s all!

If you want to see the full code, you can download the notebook here.

Citation

BibTeX citation:

@online{castillo2025,
  author = {Castillo, Dylan},
  title = {Parallelization and Orchestrator-Workers Workflows with
    {Pydantic} {AI}},
  date = {2025-07-09},
  url = {https://dylancastillo.co/til/parallelization-orchestrator-workers-pydantic-ai.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2025. “Parallelization and Orchestrator-Workers Workflows with Pydantic AI.” July 9. https://dylancastillo.co/til/parallelization-orchestrator-workers-pydantic-ai.html.

Prompt chaining workflow with Pydantic AI

Dylan Castillo — Tue, 08 Jul 2025 00:00:00 GMT

To get more familiar with Pydantic AI, I’ve been re-implementing typical patterns for building agentic systems.

In this post, I’ll explore how to build a prompt chaining. I won’t cover the basics of agentic workflows, so if you’re not familiar with the concept, I recommend you to read this post first.

I’ve also written other TILs about Pydantic AI: - Routing - Evaluator-optimizer - ReAct agent - Parallelization and Orchestrator-workers

You can download this notebook here.

What is prompt chaining?

Prompt chaining is a workflow pattern that splits a complex task into multiple subtasks. This gives you better results, but at the cost of longer completion times (higher latency).

It looks like this:

flowchart LR
    In --> LLM1["LLM Call 1"]
    LLM1 -- "Output 1" --> Gate{Gate}
    Gate -- Pass --> LLM2["LLM Call 2"]
    Gate -- Fail --> Exit[Exit]
    LLM2 -- "Output 2" --> LLM3["LLM Call 3"]
    LLM3 --> Out

Examples:

Generating content in a pipeline by generating table of contents, content, revisions, translations, etc.
Generating a text through a multi-step process to evaluate if it matches certain criteria

Let’s get to it!

Setup

I went with a simple example to implement a content generation workflow composed of three steps:

Generate a table of contents for the article
Generate the content of the article
Update the content of the article if it’s too long

Because Pydantic AI uses asyncio under the hood, you need to enable nest_asyncio to use it in a notebook:

import nest_asyncio

nest_asyncio.apply()

Then, you need to import the required libraries. Logfire is part of the Pydantic ecosystem, so I thought it’d be good to use it for observability.

import os
from typing import Literal

import logfire
import requests
from dotenv import load_dotenv
from pydantic_ai import Agent, RunContext

load_dotenv()

True

PydanticAI is compatible with OpenTelemetry (OTel). So it’s pretty easy to use it with Logfire or with any other OTel-compatible observability tool (e.g., Langfuse).

To enable tracking, create a project in Logfire, generate a Write token and add it to the .env file. Then, you just need to run:

logfire.configure(
    token=os.getenv('LOGFIRE_TOKEN'),
)
logfire.instrument_pydantic_ai()

Logfire project URL: ]8;id=13458;https://logfire-us.pydantic.dev/dylanjcastillo/blog\https://logfire-us.pydantic.dev/dylanjcastillo/blog]8;;\

Prompt chaining workflow

As mentioned before, the workflow is composed of three steps: generate a table of contents, generate the content of the article and update the content if it’s too long.

So I created three Agent instances. Each one takes care of one of the steps.

Here’s the code:

toc_agent = Agent(
    'openai:gpt-4.1-mini',
    system_prompt=(
        "You are an expert writer specialized in SEO. Provided with a topic, you will generate the table of contents for a short article."
    ),
)

article_agent = Agent(
    'openai:gpt-4.1-mini',
    system_prompt=(
        "You are an expert writer specialized in SEO. Provided with a topic and a table of contents, you will generate the content of the article."
    ),
)

editor_agent = Agent(
    'openai:gpt-4.1-mini',
    system_prompt=(
        "You are an expert writer specialized in SEO. Provided with a topic, a table of contents and a content, you will revise the content of the article to make it less than 1000 characters."
    ),
)


@logfire.instrument("Run workflow")
def run_workflow(topic: str) -> str:
    toc = toc_agent.run_sync(f"Generate the table of contents of an article about {topic}")
    content = article_agent.run_sync(f"Generate the content of an article about {topic} with the following table of contents: {toc.output}")
    if len(content.output) > 1000:
        revised_content = editor_agent.run_sync(f"Revise the content of an article about {topic} with the following table of contents: {toc.output} and the following content: {content.output}")
        return revised_content.output
    return content.output

output = run_workflow("Artificial Intelligence")

18:54:00.987 Run workflow
18:54:00.988   toc_agent run
18:54:00.989     chat gpt-4.1-mini
18:54:02.911   article_agent run
18:54:02.911     chat gpt-4.1-mini
18:54:18.621   editor_agent run
18:54:18.622     chat gpt-4.1-mini

This code creates the agents and puts them together in a workflow. I used @logfire.instrument to make sure all the traces related to the workflow are logged within the same span. See example below:

Prompt chaining workflow

And here’s the output:

print(output)

Artificial Intelligence (AI) simulates human intelligence in machines capable of learning, problem-solving, and decision-making. Originating as a formal discipline in the 1950s, AI evolved from rule-based systems to advanced machine learning and deep learning, now integral to daily life. AI types include Narrow AI for specific tasks, General AI with human-level cognition, and theoretical Superintelligent AI. Key technologies include machine learning, deep learning, natural language processing, and computer vision. AI transforms sectors like healthcare, finance, transportation, and education by automating tasks and improving decisions. Benefits include increased efficiency and innovation, while challenges involve data privacy, bias, job displacement, and transparency. Future trends highlight explainable AI, edge computing, and human-AI collaboration. Ethical concerns focus on accountability, fairness, and user privacy. Responsible AI development promises a transformative, inclusive future.

That’s all!

You can access this notebook here.

If you have any questions or feedback, please let me know in the comments below.

Citation

BibTeX citation:

@online{castillo2025,
  author = {Castillo, Dylan},
  title = {Prompt Chaining Workflow with {Pydantic} {AI}},
  date = {2025-07-08},
  url = {https://dylancastillo.co/til/prompt-chaining-pydantic-ai.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2025. “Prompt Chaining Workflow with Pydantic AI.” July 8. https://dylancastillo.co/til/prompt-chaining-pydantic-ai.html.

Routing workflow with Pydantic AI

Dylan Castillo — Tue, 08 Jul 2025 00:00:00 GMT

I’m trying to get more familiar with Pydantic AI, so I’ve been re-implementing typical patterns for building agentic systems.

In this post, I’ll build a routing workflow. I won’t cover the basics of agentic workflows, so if you’re not familiar with the concept, I recommend you to read this post first.

I’ve also written other TILs about Pydantic AI:

You can download this notebook here.

What is router?

Routing is a workflow pattern that takes the input, classifies it and then sends it to the right place for the best handling. This process can be managed by an LLM or a traditional classification model. It makes sense to use when a system needs to apply different logic to different types of queries.

It looks like this:

flowchart LR 
    In([In]) --> Router["LLM Call Router"]

    Router -->|Route 1| LLM1["LLM Call 1"]
    Router -->|Route 2| LLM2["LLM Call 2"]
    Router -->|Route 3| LLM3["LLM Call 3"]

    LLM1 --> Out([Out])
    LLM2 --> Out
    LLM3 --> Out

Examples:

Classify complexity of question and adjust model depending on it
Classify type of query and use specialized tools (e.g., indexes, prompts)

Let’s see how this looks like in code.

Setup

I will implement a workflow that will take a query from a user and will route it to the appropriate agent.

There will be three agents in the workflow:

Agent TOC: Generate a table of contents for the article
Agent Writer: Generate the content of the article
Agent Editor: Update the content of the article if it’s too long

Because Pydantic AI uses asyncio under the hood, you need to enable nest_asyncio to use it in a notebook:

import nest_asyncio

nest_asyncio.apply()

Then, you need to import the required libraries. Logfire is part of the Pydantic ecosystem, so I thought it’d be good to use it for observability.

import os
from typing import Literal

import logfire
import requests
from dotenv import load_dotenv
from pydantic import BaseModel
from pydantic_ai import Agent, RunContext

load_dotenv()

True

PydanticAI is compatible with OpenTelemetry (OTel). It’s straightforward to use it with Logfire or with any other OTel-compatible observability tool (e.g., Langfuse).

To enable tracking, create a project in Logfire, generate a Write token and add it to the .env file. Then, you just need to run:

logfire.configure(
    token=os.getenv("LOGFIRE_TOKEN"),
)
logfire.instrument_pydantic_ai()

Prompt chaining workflow

As mentioned before, the workflow will be composed of three agents. So I created three Agent instances. Each one takes care of one of the tasks

Here’s the code:

class RouterOutput(BaseModel):
    category: Literal["write_article", "generate_table_of_contents", "review_article"]


router_agent = Agent(
    "openai:gpt-4.1-mini",
    system_prompt=(
        "You are a helpful assistant. You will classify the message into one of the following categories: 'write_article', 'generate_table_of_contents', 'review_article'."
    ),
    output_type=RouterOutput,
)

agent_writer = Agent(
    "openai:gpt-4.1-mini",
    system_prompt=(
        "You are a writer. You will write an article about the topic provided."
    ),
)

agent_toc = Agent(
    "openai:gpt-4.1-mini",
    system_prompt=(
        "You are an expert writer specialized in SEO. Provided with a topic, you will generate the table of contents for a short article."
    ),
)

agent_reviewer = Agent(
    "openai:gpt-4.1-mini",
    system_prompt=(
        "You are a writer. You will review the article for the topic provided."
    ),
)

Logfire project URL: https://logfire-us.pydantic.dev/dylanjcastillo/blog

@logfire.instrument("Run workflow")
def run_workflow(topic: str) -> str:
    router_output = router_agent.run_sync(
        f"Classify the message: {topic}"
    )
    category = router_output.output.category 
    if category == "write_article":
        return agent_writer.run_sync(f"Write an article about {topic}").output
    elif category == "generate_table_of_contents":
        return agent_toc.run_sync(f"Generate the table of contents of an article about {topic}").output
    else:
        return agent_reviewer.run_sync(f"Review the article for the topic {topic}").output

You can run the workflow and it will route your message and use the appropriate agent. For example, try to generate a table of contents for an article about AI:

toc = run_workflow("Generate a table of contents for an article about AI")

20:08:05.547 Run workflow
20:08:05.548   router_agent run
20:08:05.549     chat gpt-4.1-mini
20:08:06.810   agent_toc run
20:08:06.811     chat gpt-4.1-mini

print(toc)

Table of Contents

1. Introduction to Artificial Intelligence  
2. History and Evolution of AI  
3. Types of Artificial Intelligence  
4. Key Technologies Behind AI  
5. Applications of AI in Various Industries  
6. Benefits and Challenges of AI  
7. Future Trends in Artificial Intelligence  
8. Ethical Considerations in AI Development  
9. Conclusion

Or, ask the workflow to review a social media post.

review = run_workflow("Review this post: 'There are times where there's no time, so you don't have time to write an article about it.'")

20:08:08.917 Run workflow
20:08:08.918   router_agent run
20:08:08.919     chat gpt-4.1-mini
20:08:09.691   agent_reviewer run
20:08:09.692     chat gpt-4.1-mini

print(review)

The post "There are times where there's no time, so you don't have time to write an article about it." offers a succinct reflection on the challenges of time constraints, especially in tasks like writing. Its brevity captures the irony of not having enough time to address a situation—in this case, the lack of time itself. The message resonates with anyone who has felt overwhelmed by deadlines or competing priorities.

However, as a piece intended for a broader audience or a formal article, it could benefit from expansion. Elaborating on scenarios where time scarcity impacts productivity, or providing strategies for managing pressing tasks despite limited time, would add depth and practical value. Additionally, refining the sentence for clarity and flow could enhance its impact—for example: "Sometimes, we're so pressed for time that we can't even write about the very pressure we're under."

In summary, the post effectively conveys a common frustration with time limitations in a clever and relatable way but serves better as a starting point for a more detailed discussion rather than a standalone article.

That’s all!

You can access this notebook here.

If you have any questions or feedback, please let me know in the comments below.

Citation

BibTeX citation:

@online{castillo2025,
  author = {Castillo, Dylan},
  title = {Routing Workflow with {Pydantic} {AI}},
  date = {2025-07-08},
  url = {https://dylancastillo.co/til/routing-pydantic-ai.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2025. “Routing Workflow with Pydantic AI.” July 8. https://dylancastillo.co/til/routing-pydantic-ai.html.

Building ReAct agents with (and without) LangGraph

Dylan Castillo — Fri, 04 Jul 2025 00:00:00 GMT

As Large Language Models (LLMs) have become more powerful, I’ve started to see increasing interest from clients in building agents.

The problem is that most of the use cases clients have in mind for agents are better suited for agentic workflows. Agents are a good fit for tasks without a predefined path and where the order of steps is not known beforehand. Agentic workflows, on the other hand, are the right choice for tasks where both of these things are known.

Nonetheless, agents are still a good fit for many use cases, such as coding assistants and support agents. You should definitely spend some time learning about them.

In this post, I’ll show you how to build a Reasoning and Acting (ReAct) agent with (and without) LangGraph.

Let’s start by defining some key concepts.

What is an agent?

The biggest players in the ecosystem have converged on similar definitions of what constitutes an “agent.” Anthropic describes them as systems where LLMs “dynamically direct their own processes and tool usage,” while OpenAI calls them “systems that independently accomplish tasks on behalf of users.” LangChain similarly defines them as systems using an LLM to “decide the control flow of an application.”

In essence, agents are systems that can independently make decisions, use tools, take actions, and pursue a goal without direct human guidance. The most well-known agent implementation are ReAct Agents.

What’s a ReAct agent?

ReAct (Reasoning and Acting) Agents are AI systems that merge the reasoning of Large Language Models (LLMs) with the ability to perform actions. They follow an iterative “think, act, observe” cycle to solve problems and achieve user goals. For example, a ReAct agent would:

Take a user query.
Think about the query and decide on an action.
Execute the action using available tools (environment).
Analyzes the result of that action (environment).
Continues the “Reason, Act, Observe” loop until it reaches the final answer.

Here’s a diagram of a ReAct agent:

graph LR
    Human <--> LLM[LLM]
    LLM -->|Action| Environment
    Environment -->|Feedback| LLM
    LLM -.-> Stop

The first generation of ReAct agents used a prompt technique of “Thought, Action, Observation”. Current agents rely on function-calling to implement the “think, act, observe” loop.

What is LangGraph?

LangGraph is a graph-based framework for building complex LLM applications, designed for stateful workflows. It makes it easier to build complex agent architectures.

Graphs are composed of nodes, edges, state, and reducers. Nodes are the units of work (functions, tools) and edges define the paths between nodes. State is persistent data passed between nodes and updated through reducers (functions that define how the state is updated).

I like LangGraph because it provides you with easy-to-use components, a simple API, and it lets you visualize your workflow. It also integrates well with LangSmith, a tool for monitoring and debugging LLM applications.

In this tutorial, I’ll show you how to build a ReAct agent with (and without) LangGraph. I’ll also use LangChain as a thin wrapper on top of OpenAI models.

Prerequisites

To follow this tutorial you’ll need to:

Sign up and generate an API key in OpenAI.
Set the API key as an environment variable called OPENAI_API_KEY.
Create a virtual environment in Python and install the requirements:

python -m venv venv
source venv/bin/activate
pip install langchain langchain-openai langchain-community langgraph jupyter

Once you’ve completed the steps above, you can run the code from this article. You can also download the notebook from here.

Implementation

As usual, you must start by importing the necessary libraries and loading the environment variables. You’ll use the same model in all the examples, so you’ll define it once here:

from typing import Literal

from dotenv import load_dotenv
from IPython.display import Image, display
from langchain_core.messages import HumanMessage, SystemMessage, ToolMessage
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI
from langgraph.graph import END, START, MessagesState, StateGraph

load_dotenv()

model = ChatOpenAI(model="gpt-4.1-mini", temperature=0)

This code:

Imports the necessary libraries.
Loads environment variables from a .env file using load_dotenv().
Defines a model (gpt-4.1-mini) that you’ll use in all the examples.

Vanilla ReAct agent

You’ll build an agent that takes a question from a user and has access to a a tool. The tool is a Python REPL that it can use to answer the question.

You should not use this in production. This tool can run arbitrary Python code on your device, and that’s not something you want to expose to random people on the internet.

First, let’s define the run_python_code tool.

@tool
def run_python_code(code: str) -> str:
    """Run arbitrary Python code including imports, assignments, and statements. Do not use any external libraries. Save your results as a variable.

    Args:
        code: Python code to run
    """
    import sys
    from io import StringIO

    old_stdout = sys.stdout
    sys.stdout = captured_output = StringIO()

    namespace = {}

    try:
        exec(code, namespace)

        output = captured_output.getvalue()

        if not output.strip():
            user_vars = {
                k: v
                for k, v in namespace.items()
                if not k.startswith("__") and k not in ["StringIO", "sys"]
            }
            if user_vars:
                if len(user_vars) == 1:
                    output = str(list(user_vars.values())[0])
                else:
                    output = str(user_vars)

        return output.strip() if output.strip() else "Code executed successfully"

    except Exception as e:
        return f"Error: {str(e)}"
    finally:
        sys.stdout = old_stdout


tools = [run_python_code]
tools_mapping = {tool.name: tool for tool in tools}
model_with_tools = model.bind_tools(tools)

This code defines a tool. In LangChain tools are defined as functions decorated with @tool. These functions must have a docstring because it will be used to describe the tool to the LLM.

run_python_code is a function that takes a code string and returns the result of executing that code.

Next, you provide the model with a mapping of the tool to its name by creating tools_mapping. This is often a point of confusion. The LLM doesn’t run the tools on its own. It only decides if a tool should be used. Then, your own code must make the actual tool call.

The mapping is more useful when there are multiple tools, and not a single tool, like in this case. However, I’m showing it here to illustrate how you’d usually do this in a real-world application.

Finally, you bind the tool to the model. The binding makes the model aware of the tool, so that it can use it.

Then, let’s define a function that encapsulates the logic of the agent.

def run_agent(question: str):
    messages = [
        SystemMessage(
            "You're a helpful assistant. Use the tools provided when relevant."
        ),
        HumanMessage(question),
    ]
    ai_message = model_with_tools.invoke(messages)
    messages.append(ai_message)

    while ai_message.tool_calls:
        for tool_call in ai_message.tool_calls:
            selected_tool = tools_mapping[tool_call["name"]]
            tool_msg = selected_tool.invoke(tool_call)
            messages.append(tool_msg)
        ai_message = model_with_tools.invoke(messages)
        messages.append(ai_message)

    return messages

This function takes a question from a user, comes up with a python script, uses run_python_code to execute it and returns the result.

It works as follows:

Lines 2 to 9 set up the prompts and call the assistant.
Lines 11 to 17 is where the magic happens. This is a loop that will check if there’s been a tool call in the response from the model. If there is, it will call (invoke) the tool and add the result to the messages. It will then send the results back to the assistant, and repeat this process until there are no more tool calls.

This is the core idea behind how agents work. You provide the assistant with a question and one or more tools. Then you let the assistant decide which tool to use, until it has all the information it needs to answer the question.

You can try it out by running the following code:

messages = run_agent("Generate 10 random numbers between 1 and 100")

for m in messages:
    m.pretty_print()

================================ System Message ================================



You're a helpful assistant. Use the tools provided when relevant.

================================ Human Message =================================



Generate 10 random numbers between 1 and 100

================================== Ai Message ==================================

Tool Calls:

  run_python_code (call_twMJ1JkZK81ofm3842P3jNtb)

 Call ID: call_twMJ1JkZK81ofm3842P3jNtb

  Args:

    code: import random

random_numbers = [random.randint(1, 100) for _ in range(10)]

random_numbers

================================= Tool Message =================================

Name: run_python_code



{'random': , 'random_numbers': [74, 75, 58, 19, 90, 45, 44, 52, 90, 33]}

================================== Ai Message ==================================



Here are 10 random numbers between 1 and 100: 74, 75, 58, 19, 90, 45, 44, 52, 90, 33.

You can see that the agent took the user’s request, used the run_python_code tool to generate the numbers, and then returned the result.

Now, let’s see how you can build a ReAct agent with LangGraph.

LangGraph ReAct agent

Like, in the previous example, you start by defining the tools you want to use. You’ll use the same run_python_code tool.

@tool
def run_python_code(code: str) -> str:
    """Run arbitrary Python code including imports, assignments, and statements. Do not use any external libraries. Save your results as a variable.

    Args:
        code: Python code to run
    """
    import sys
    from io import StringIO

    old_stdout = sys.stdout
    sys.stdout = captured_output = StringIO()

    namespace = {}

    try:
        exec(code, namespace)

        output = captured_output.getvalue()

        if not output.strip():
            user_vars = {
                k: v
                for k, v in namespace.items()
                if not k.startswith("__") and k not in ["StringIO", "sys"]
            }
            if user_vars:
                if len(user_vars) == 1:
                    output = str(list(user_vars.values())[0])
                else:
                    output = str(user_vars)

        return output.strip() if output.strip() else "Code executed successfully"

    except Exception as e:
        return f"Error: {str(e)}"
    finally:
        sys.stdout = old_stdout


tools = [run_python_code]
tools_by_name = {tool.name: tool for tool in tools}
model_with_tools = model.bind_tools(tools)

With langgraph you also need to you define the tools in the same way: set up the functions, add @tool, and create the tool mapping. Then, bind the tools to the model.

Next, you need to define the functions (nodes) that correspond to the steps the agent will take:

def call_llm(state: MessagesState):
    messages = [
        SystemMessage(content="You are a helpful assistant that can run python code."),
    ] + state["messages"]
    return {"messages": [model_with_tools.invoke(messages)]}


def call_tool(state: MessagesState):
    result = []
    for tool_call in state["messages"][-1].tool_calls:
        tool = tools_by_name[tool_call["name"]]
        observation = tool.invoke(tool_call["args"])
        result.append(ToolMessage(content=observation, tool_call_id=tool_call["id"]))
    return {"messages": result}


def should_continue(state: MessagesState) -> Literal["environment", END]:
    messages = state["messages"]
    last_message = messages[-1]
    if last_message.tool_calls:
        return "Action"
    return END

This is how it works:

call_llm: Sends the conversation history to the LLM to get the next response.
call_tool: If the LLM’s response is a request to use a tool, this function executes the tool with the specified arguments.
should_continue: This is the control logic. It checks the LLM’s last message. If it’s a tool request, it routes to the call_tool; otherwise, it ends the conversation.

call_llm and call_tool take MessagesState as input and return a the message key with the new message. This updates the messages key in the MessagesState. should_continue takes MessagesState act as a router, so it decides if a tool should be executed or if the conversation should end.

agent_builder = StateGraph(MessagesState)

agent_builder.add_node("llm", call_llm)
agent_builder.add_node("environment", call_tool)

agent_builder.add_edge(START, "llm")
agent_builder.add_conditional_edges(
    "llm",
    should_continue,
    {
        "Action": "environment",
        END: END,
    },
)
agent_builder.add_edge("environment", "llm")

agent = agent_builder.compile()

This code sets up the logic of the agent. It starts with a call to the LLM. The LLM then decides whether to use a tool or to finish the task. If it uses a tool, the tool’s output is sent back to the LLM for the next step. This “think-act” loop continues until the agent decides the task is complete.

You can use LangGraph to visualize the agent’s flow:

display(Image(agent.get_graph(xray=True).draw_mermaid_png()))

Finally, you can run the agent by invoking the graph. For that, you’ll need to pass a list of messages to the graph.

messages = [
    SystemMessage(content="You are a helpful assistant that can run python code."),
    HumanMessage(content="Generate 10 random numbers between 1 and 100"),
]

messages = agent.invoke({"messages": messages})

You can review the process by printing the messages:

for m in messages["messages"]:
    m.pretty_print()

================================ System Message ================================



You are a helpful assistant that can run python code.

================================ Human Message =================================



Generate 10 random numbers between 1 and 100

================================== Ai Message ==================================

Tool Calls:

  run_python_code (call_bsBRC5aL7kgHjeGHXaLR85TC)

 Call ID: call_bsBRC5aL7kgHjeGHXaLR85TC

  Args:

    code: import random

random_numbers = [random.randint(1, 100) for _ in range(10)]

random_numbers

================================= Tool Message =================================



{'random': , 'random_numbers': [77, 37, 97, 26, 22, 29, 58, 82, 17, 80]}

================================== Ai Message ==================================



Here are 10 random numbers between 1 and 100: 77, 37, 97, 26, 22, 29, 58, 82, 17, 80.

That’s all!

Conclusion

In this tutorial, you’ve learned what agents are, how they work, and how to build a simple ReAct agent with and without LangGraph. We covered:

Agent fundamentals: How agents differ from agentic workflows by dynamically directing their own processes
ReAct pattern: The core “think, act, observe” loop that enables agents to take actions based on the information they have
Vanilla and LangGraph implementation: Understanding how to implement agents with and without LangGraph

Agents are great for open-ended tasks where the path isn’t predetermined. If you’re working on one of those, this article provides a good starting point. On the other hand, for tasks that have predefined steps, consider agentic workflows instead.

Hope you find this tutorial useful. If you have any questions, let me know in the comments below.

Citation

BibTeX citation:

@online{castillo2025,
  author = {Castillo, Dylan},
  title = {Building {ReAct} Agents with (and Without) {LangGraph}},
  date = {2025-07-04},
  url = {https://dylancastillo.co/posts/react-agent-langgraph.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2025. “Building ReAct Agents with (and Without) LangGraph.” July 4. https://dylancastillo.co/posts/react-agent-langgraph.html.

Using Pydantic AI to build a ReAct agent

Dylan Castillo — Fri, 04 Jul 2025 00:00:00 GMT

I wanted to get more familiar with Pydantic AI, so I decided to build a Reasoning and Acting (ReAct) agent with multiple tools.

I’ve also written other TILs about Pydantic AI:

You can download this notebook here.

Setup

After a first failed attempt, I realized that Pydantic AI uses asyncio under the hood, so you need to enable nest_asyncio to use it in a notebook.

import nest_asyncio

nest_asyncio.apply()

Then, I did the imports as usual. I hadn’t used logfire for monitoring LLM applications before, so I thought it’d be a good idea to try it out.

import os
from typing import Literal

import logfire
import requests
from dotenv import load_dotenv
from pydantic_ai import Agent

load_dotenv()

PydanticAI instrumentation uses OpenTelemetry (OTel). So it’s pretty straightforward to use it with Logfire or with any other OTel-compatible observability tool.

You just need to create a project in Logfire, generate a Write token and add it to the .env file. Then, you just need to run:

logfire.configure(
    token=os.getenv('LOGFIRE_TOKEN'),
)
logfire.instrument_pydantic_ai()

This will ask you to select a project the first time you run it. It will generate a logfire_credentials.json file in your working directory. In following runs, it will automatically use the credentials from the file.

ReAct Agent

I decided to make an agent that had access to a tool to get the weather and another one that checks if the response that’s going to be sent to the user follows the company guidelines.

Here’s the code:

from pydantic import BaseModel

class Feedback(BaseModel):
    feedback: str
    status: Literal['OK', 'REQUIRES FIXING']

evaluator_agent = Agent(
    'openai:gpt-4.1-mini',
    system_prompt=(
        "You're a helpful assistant. Your task is to check if a given response follows the company guidelines. The company guidelines are that responses should be written in the style of a haiku. You should reply with 'OK' or 'REQUIRES FIXING' and a short explanation."
    ),
    output_type=Feedback,
)

react_agent = Agent(  
    'openai:gpt-4.1-mini',
    system_prompt=(
        "You're a helpful assistant. Use the tools provided when relevant. Then draft a response and check if it follows the company guidelines. Only respond to the user after you've validated and modified the response if needed."
    ),
)


@react_agent.tool_plain
def get_weather(latitude: float, longitude: float) -> str:
    """Get the weather of a given latitude and longitude"""
    response = requests.get(
        f"https://api.open-meteo.com/v1/forecast?latitude={latitude}&longitude={longitude}¤t=temperature_2m,wind_speed_10m&hourly=temperature_2m,relative_humidity_2m,wind_speed_10m"
    )
    data = response.json()
    return str(data["current"]["temperature_2m"])


@react_agent.tool_plain
def check_guidelines(drafted_response: str) -> str:
    """Check if a given response follows the company guidelines"""
    response = evaluator_agent.run_sync(drafted_response)
    return response.output


response = react_agent.run_sync("What is the temperature in Madrid?")

18:57:42.206 react_agent run
18:57:42.208   chat gpt-4.1-mini
18:57:43.191   running 1 tool
18:57:43.192     running tool: get_weather
18:57:43.408   chat gpt-4.1-mini
18:57:44.117   running 1 tool
18:57:44.118     running tool: check_guidelines
18:57:44.120       evaluator_agent run
18:57:44.120         chat gpt-4.1-mini
18:57:46.291   chat gpt-4.1-mini

And here’s the output:

print(response.output)

In Madrid sunshine,
Temperature climbs so high,
Thirty-four degrees.

The output in Logfire looks like typical observability tools:

Figure 1: Traces in Logfire

That’s all!

You can access this notebook here.

Citation

BibTeX citation:

@online{castillo2025,
  author = {Castillo, Dylan},
  title = {Using {Pydantic} {AI} to Build a {ReAct} Agent},
  date = {2025-07-04},
  url = {https://dylancastillo.co/til/react-agent-pydantic-ai.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2025. “Using Pydantic AI to Build a ReAct Agent.” July 4. https://dylancastillo.co/til/react-agent-pydantic-ai.html.

Agentic workflows from scratch with (and without) LangGraph

Dylan Castillo — Thu, 03 Jul 2025 00:00:00 GMT

These days, everyone is “building” agents. But, in my experience, they’re either not really building agents, or they shouldn’t be!

Agents are powerful tools, but they’re not the right tool for many business problems. They give a lot of responsibility to LLMs, and that’s not always a good idea.

Their counterpart, agentic workflows, are a more controlled way to use LLMs. They’re a good fit for many business problems, and they’re a lot easier to build than agents.

In this post, I’ll explain what an agent is, what an agentic workflow is, and how to choose between them. I’ll also show you how to build the most common agentic workflows.

This post is based on Anthropic’s Building Effective Agents. I encourage you to read it.

What is an agent?

Over time, the ecosystem has converged on similar definitions:

Anthropic’s definition:

“Systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.”

Building Effective Agents, Anthropic

OpenAI’s definition:

“Systems that independently accomplish tasks on behalf of users”

New tools for building agents, OpenAI

LangChain’s definition:

“system that uses an LLM to decide the control flow of an application.”

What is an agent?, LangChain

These definitions share the idea that agents are systems that can:

Make decisions
Use tools
Take actions
Accomplish goals without constant human guidance

This gives us a clear delimiter as to what an agent is and what it is not. However, this definition leaves out the majority of agentic systems being build by companies right now. These systems are called agentic workflows.

What is an agentic workflow?

Anthropic defines agentic workflows as systems where “LLMs and tools are orchestrated through predefined code paths”. They’re different from agents in that they don’t have the ability to dynamically direct their own processes and tool usage.

Workflows sound less sexy than agents, so you would rarely hear someone say they’re building workflows. However, in my experience, most people are (or should be) building workflows.

How to choose between agentic workflows and agents?

Choose agents for open-ended tasks where the number and the order of steps is not known beforehand. The LLM will potentially operate for many turns, and you must have some level of trust in its decision-making.

For example, a coding assistant is a good candidate for an agent. When you ask it to implement a feature, there’s no predefined path nor the order of steps is known beforehand. It might need to create files, add new dependencies, edit existing files, etc.

Agentic workflows make sense for tasks where the number and the order of steps is known beforehand. For example, an AI assistant that generates an initial draft of an article based on internal data sources. The order of steps are likely known beforehand (e.g., retrieve the key information from the internal data sources, generate a first candidate, and then revise the article).

What is LangGraph?

LangGraph is a graph-based framework for building complex LLM applications, designed for stateful workflows. It enables complex agent architectures with minimal code.

It uses a graph-based approach to build agentic systems. The graph is composed of nodes and edges. Nodes are the units of work (functions, tools, models). Edges define the workflow paths between nodes. State is persistent data passed between nodes and updated through reducers.

I’ve built many workflows from scratch, and I’ve realized that I often end up reinventing the same patterns that frameworks like LangGraph provide. I like LangGraph because it provides you with easy-to-use components, a simple API, and it lets you visualize your workflow. It also integrates well with LangSmith, a tool for monitoring and debugging LLM applications.

In this tutorial, I’ll show you how to build common agentic workflows with and without LangGraph. I’ll use LangChain as a thin wrapper on top of OpenAI models.

Prerequisites

To follow this tutorial you’ll need to:

Sign up and generate an API key in OpenAI.
Set the API key as an environment variable called OPENAI_API_KEY.
Create a virtual environment in Python and install the requirements:

python -m venv venv
source venv/bin/activate
pip install langchain langchain-openai langchain-community langgraph jupyter nest_asyncio

Once you’ve completed the steps above, you can run the code from this article. You can also download the notebook from here.

You also need to apply a small patch to make sure you can run asyncio inside the notebook:

import nest_asyncio

nest_asyncio.apply()

You will use asyncio inside the notebook, so you need to install and apply nest_asyncio. Otherwise, you’ll get a RuntimeError when running the code.

Workflows

As usual, you must start by importing the necessary libraries and loading the environment variables. You’ll use the same model in all the examples, so you’ll define it once here:

import asyncio
import operator
from typing import Annotated, Literal, Optional, TypedDict

from dotenv import load_dotenv
from IPython.display import Image, display
from langchain_core.messages import HumanMessage, SystemMessage
from langchain_openai import ChatOpenAI
from langgraph.graph import END, START, StateGraph
from langgraph.types import Send
from pydantic import BaseModel, Field

load_dotenv()

model = ChatOpenAI(model="gpt-4.1-mini")

This code imports the necessary libraries for building agentic workflows:

LangChain: For working with LLMs, messages, and structured outputs
LangGraph: For building stateful workflows with nodes and edges
Pydantic: For data validation and structured data models

The load_dotenv() function loads environment variables from a .env file, including your OpenAI API key. You also define model (gpt-4.1-mini) that you’ll use in all the examples.

Prompt chaining

This workflow is designed for tasks that can be easily divided into subtasks. The key trade-off is accepting longer completion times (higher latency) in exchange for a higher-quality result.

Here’s what the workflow looks like:

flowchart LR
    In --> LLM1["LLM Call 1"]
    LLM1 -- "Output 1" --> Gate{Gate}
    Gate -- Pass --> LLM2["LLM Call 2"]
    Gate -- Fail --> Exit[Exit]
    LLM2 -- "Output 2" --> LLM3["LLM Call 3"]
    LLM3 --> Out

Examples:

Generating content in a pipeline by generating table of contents, content, revisions, translations, etc.
Generating a text through a multi-step process to evaluate if it matches certain criteria

Now I’ll show you how to implement a prompt chaining workflow for generating an article. The workflow will be composed of three steps:

Generate a table of contents for the article
Generate the content of the article
Revise the content of the article if it’s too long

I’ll show you a vanilla implementation and then a LangGraph implementation.

Vanilla (+LangChain)

First, you need to define the state of the workflow and a model to use for the LLM. You’ll use Pydantic for the state and LangChain for the model.

class State(BaseModel):
    topic: str
    table_of_contents: Optional[str] = None
    content: Optional[str] = None
    revised_content: Optional[str] = None

Then, you need to define the functions that will be used in the workflow:

def generate_table_of_contents(state: State) -> str:
    messages = [
        SystemMessage(
            content="You are an expert writer specialized in SEO. Provided with a topic, you will generate the table of contents for a short article."
        ),
        HumanMessage(
            content=f"Generate the table of contents of an article about {state.topic}"
        ),
    ]
    return model.invoke(messages).content


def generate_article_content(state: State) -> str:
    messages = [
        SystemMessage(
            content="You are an expert writer specialized in SEO. Provided with a topic and a table of contents, you will generate the content of the article."
        ),
        HumanMessage(
            content=f"Generate the content of an article about {state.topic} with the following table of contents: {state.table_of_contents}"
        ),
    ]
    return model.invoke(messages).content


def revise_article_content(state: State) -> str:
    messages = [
        SystemMessage(
            content="You are an expert writer specialized in SEO. Provided with a topic, a table of contents and a content, you will revise the content of the article to make it less than 1000 characters."
        ),
        HumanMessage(
            content=f"Revise the content of an article about {state.topic} with the following table of contents: {state.table_of_contents} and the following content:\n\n{state.content}"
        ),
    ]
    return model.invoke(messages).content

These functions provide the core functionality of the workflow. They follow the steps I outlined above:

generate_table_of_contents: Generate a table of contents for the article
generate_article_content: Generate the content of the article
revise_article_content: Revise the content of the article if it’s too long

Then we need to orchestrate the workflow. You’ll do that by creating a function that takes a topic and returns the final article.

def run_workflow(topic: str) -> State:
    article = State(topic=topic)
    article.table_of_contents = generate_table_of_contents(article)
    article.content = generate_article_content(article)
    if len(article.content) > 1000:
        article.revised_content = revise_article_content(article)
    return article


output = run_workflow("Artificial Intelligence")

run_workflow takes the topic provided by the user, generates an article, and verifies that it’s below 1000 characters. Depending on the result, it will revise the article or not. Finally, it returns the state that contains all the results from the workflow (the table of contents, the content, and the revised content).

LangGraph

Now let’s see how to implement the same workflow using LangGraph. You will noticed two key differences compared to the vanilla implementation.

Similar to the vanilla implementation, you’ll start by initializing the state class:

class State(TypedDict):
    topic: str
    table_of_contents: str
    content: str
    revised_content: str

When working with LangGraph, I’d suggest using a TypedDict to manage your state. Using a Pydantic model to manage your state in LangGraph has a few downsides:

Data types are only checked when they enter a node, not when they exit. This means you could accidentally save data of the wrong type to your state.
The final output of the entire graph will be a dictionary, not your Pydantic model.
The implementation feels like it’s half-baked.

For more details, check out the LangGraph documentation.

Then, you’ll define the nodes (functions) that will be used in the workflow.

def generate_table_of_contents(state: State) -> dict:
    messages = [
        SystemMessage(
            content="You are an expert writer specialized in SEO. Provided with a topic, you will generate the table of contents for a short article."
        ),
        HumanMessage(
            content=f"Generate the table of contents of an article about {state['topic']}"
        ),
    ]
    return {"table_of_contents": model.invoke(messages).content}


def generate_article_content(state: State) -> str:
    messages = [
        SystemMessage(
            content="You are an expert writer specialized in SEO. Provided with a topic and a table of contents, you will generate the content of the article."
        ),
        HumanMessage(
            content=f"Generate the content of an article about {state['topic']} with the following table of contents: {state['table_of_contents']}"
        ),
    ]
    return {"content": model.invoke(messages).content}


def check_article_content(state: State) -> str:
    if len(state["content"]) > 1000:
        return "Fail"
    return "Pass"


def revise_article_content(state: State) -> str:
    messages = [
        SystemMessage(
            content="You are an expert writer specialized in SEO. Provided with a topic, a table of contents and a content, you will revise the content of the article to make it less than 1000 characters."
        ),
        HumanMessage(
            content=f"Revise the content of an article about {state['topic']} with the following table of contents: {state['table_of_contents']} and the following content:\n\n{state['content']}"
        ),
    ]
    return {"revised_content": model.invoke(messages).content}

You’ll noticed that the functions are quite similar to the vanilla implementation. The only difference is that they return dictionaries that automatically update the state rather than you having to do it manually.

Then you need to specify the nodes and edges of the workflow:

workflow_builder = StateGraph(State)

workflow_builder.add_node("generate_table_of_contents", generate_table_of_contents)
workflow_builder.add_node("generate_article_content", generate_article_content)
workflow_builder.add_node("revise_article_content", revise_article_content)

workflow_builder.add_edge(START, "generate_table_of_contents")
workflow_builder.add_edge("generate_table_of_contents", "generate_article_content")
workflow_builder.add_conditional_edges(
    source="generate_article_content",
    path=check_article_content,
    path_map={"Fail": "revise_article_content", "Pass": END},
)
workflow_builder.add_edge("revise_article_content", END)

workflow = workflow_builder.compile()

You initialize a StateGraph object, which is a builder for the workflow. Then you add nodes to the graph, and define the edges between them. The nodes in the graphs are the functions that you defined earlier. In the conditional edge, you can define the path that the workflow will take based on the state of the workflow.

The compile method is used to compile the graph into a callable workflow.

Finally, LangGraph has a nice feature that allows you to visualize the workflow. You can use the get_graph and draw_mermaid_png for that:

display(Image(workflow.get_graph().draw_mermaid_png()))

Finally, you can run the workflow by calling the invoke method on the compiled workflow and provide the initial state.

article = workflow.invoke({"topic": "Artificial Intelligence"})

And you’ll get the following output:

article

{'topic': 'Artificial Intelligence',
 'table_of_contents': 'Table of Contents\n\n1. Introduction to Artificial Intelligence  \n2. History and Evolution of AI  \n3. Types of Artificial Intelligence  \n4. Applications of AI in Various Industries  \n5. Benefits of Artificial Intelligence  \n6. Challenges and Ethical Considerations  \n7. The Future of Artificial Intelligence  \n8. Conclusion',
 'content': '# Artificial Intelligence: Transforming the Future\n\n## 1. Introduction to Artificial Intelligence\n\nArtificial Intelligence (AI) refers to the simulation of human intelligence processes by machines, especially computer systems. These processes include learning, reasoning, problem-solving, perception, and language understanding. AI enables machines to perform tasks that typically require human intelligence, making them smarter, more efficient, and capable of handling complex scenarios. From virtual assistants and recommendation systems to autonomous vehicles, AI has become an integral part of modern technology.\n\n## 2. History and Evolution of AI\n\nThe concept of artificial intelligence dates back to the mid-20th century. In 1956, the term “Artificial Intelligence” was coined during the Dartmouth Conference, marking the official birth of AI as a research field. Early AI research focused on symbolic methods and rule-based systems. Over the decades, advancements in algorithms, computational power, and data availability propelled AI development. Notable milestones include the creation of expert systems in the 1970s and 1980s, the rise of machine learning in the 1990s, and the advent of deep learning in the 2010s. Today, AI continues to evolve rapidly, driven by breakthroughs in neural networks and big data analytics.\n\n## 3. Types of Artificial Intelligence\n\nAI can be broadly categorized into three types based on capabilities:\n\n- **Narrow AI (Weak AI):** Designed to perform specific tasks such as voice recognition or image classification. This is the most common form of AI today.\n  \n- **General AI (Strong AI):** A theoretical form of AI that possesses the ability to understand, learn, and perform any intellectual task a human can do.\n  \n- **Superintelligent AI:** A hypothetical AI that surpasses all human intelligence across all fields, potentially possessing self-awareness and superior problem-solving abilities.\n\nAdditionally, AI can be classified based on functionalities such as reactive machines, limited memory systems, theory of mind AI, and self-aware AI, reflecting increasing complexity and cognitive capability.\n\n## 4. Applications of AI in Various Industries\n\nAI is transforming industries worldwide by streamlining operations, enhancing decision-making, and enabling innovation:\n\n- **Healthcare:** AI assists in diagnostics, personalized treatment, drug discovery, and robotic surgery.\n- **Finance:** Fraud detection, algorithmic trading, customer support, and risk management benefit from AI solutions.\n- **Retail:** AI powers recommendation engines, inventory management, and customer behavior analysis.\n- **Manufacturing:** Predictive maintenance, quality control, and automation improve efficiency.\n- **Transportation:** Autonomous vehicles, route optimization, and traffic management leverage AI technologies.\n- **Education:** Personalized learning experiences, automated grading, and virtual tutors utilize AI capabilities.\n\n## 5. Benefits of Artificial Intelligence\n\nThe adoption of AI provides numerous advantages:\n\n- **Efficiency:** Automates repetitive tasks, reducing time and labor costs.\n- **Accuracy:** Minimizes human error and enhances precision in complex processes.\n- **Data Insights:** Analyzes vast datasets to uncover trends, patterns, and actionable insights.\n- **Innovation:** Facilitates the creation of new products and services.\n- **Personalization:** Tailors experiences and recommendations to individual preferences.\n- **Accessibility:** Makes services more accessible through natural language processing and intelligent interfaces.\n\n## 6. Challenges and Ethical Considerations\n\nDespite its promise, AI poses several challenges and ethical dilemmas:\n\n- **Bias and Fairness:** AI systems can inherit biases from training data, leading to unfair outcomes.\n- **Privacy Concerns:** Extensive data collection raises issues about user privacy and data security.\n- **Job Displacement:** Automation may disrupt labor markets and professions.\n- **Transparency:** Many AI algorithms, especially deep learning models, lack interpretability.\n- **Accountability:** Determining responsibility in the case of AI failures or misuse is complex.\n- **Ethical Use:** Ensuring AI is used for beneficial purposes and preventing misuse such as in autonomous weapons is critical.\n\nAddressing these concerns requires robust governance, regulation, and ongoing dialogue between stakeholders.\n\n## 7. The Future of Artificial Intelligence\n\nThe future of AI is poised to reshape society profoundly:\n\n- **Enhanced Human-AI Collaboration:** AI will increasingly augment human capabilities rather than replace them.\n- **Advancements in General AI:** Research continues toward achieving more versatile, human-like AI.\n- **AI in Creativity:** Emerging AI tools will enhance creative processes in art, music, and writing.\n- **Integration with IoT:** AI combined with the Internet of Things will enable smarter environments and cities.\n- **Ethical AI Development:** Emphasis will grow on developing transparent, fair, and accountable AI systems.\n- **AI for Global Challenges:** AI will play a pivotal role in addressing climate change, healthcare crises, and education gaps.\n\nContinued innovation, balanced with ethical considerations, will ensure AI’s positive impact on humanity.\n\n## 8. Conclusion\n\nArtificial Intelligence has evolved from a conceptual idea to a transformative force across industries and societies. Its ability to process information, learn, and make decisions holds tremendous potential to improve lives and drive progress. However, realizing this potential requires careful management of ethical challenges and inclusive development. As AI continues to mature, it promises a future where intelligent machines and humans work collaboratively to solve complex problems and create new opportunities. Embracing AI responsibly will be key to unlocking its benefits for generations to come.',
 'revised_content': 'Artificial Intelligence (AI) simulates human intelligence in machines, enabling tasks like learning, reasoning, and problem-solving. Since its inception at the 1956 Dartmouth Conference, AI has evolved from rule-based systems to advanced machine learning and deep learning models. AI types include Narrow AI, designed for specific tasks; General AI, capable of human-like understanding; and Superintelligent AI, a hypothetical superior intellect. AI revolutionizes industries—improving healthcare diagnostics, financial fraud detection, retail personalization, manufacturing automation, transportation logistics, and education. Benefits include efficiency, accuracy, data-driven insights, innovation, and personalization. However, AI raises challenges such as bias, privacy, job displacement, transparency, and ethical use. The future promises enhanced human-AI collaboration, ethical development, and AI-driven solutions for global issues. Responsible AI integration will transform society, fostering progress and innovation.'}

That’s it for prompt chaining. Next, you’ll see how to implement a routing workflow.

Routing

Routing is a sorting system that sends each task to the right place for the best handling. This process can be managed by an LLM or a traditional classification model. It makes sense to use when a system needs to apply different logic to different types of queries.

Here’s what the workflow looks like:

flowchart LR 
    In([In]) --> Router["LLM Call Router"]

    Router -->|Route 1| LLM1["LLM Call 1"]
    Router -->|Route 2| LLM2["LLM Call 2"]
    Router -->|Route 3| LLM3["LLM Call 3"]

    LLM1 --> Out([Out])
    LLM2 --> Out
    LLM3 --> Out

Examples:

Classify complexity of question and adjust model depending on it
Classify type of query and use specialized tools (e.g., indexes, prompts)

I’ll walk you through a simple example of a routing workflow. The workflow will be composed of two steps:

Classify the type of query
Route the query to the right place

I’ll show you a vanilla implementation and then a LangGraph implementation.

Vanilla (+LangChain)

First, you need to initialize the model, and define the state of the workflow and the data models. You’ll use Pydantic for the state and data models validation and LangChain for the LLM interactions.

class State(BaseModel):
    input: str
    type: Optional[
        Literal["write_article", "generate_table_of_contents", "review_article"]
    ] = None
    output: Optional[str] = None


class MessageType(BaseModel):
    type: Literal["write_article", "generate_table_of_contents", "review_article"]

Then, you need to define the functions that will be used in the workflow.

def classify_message(state: State) -> State:
    model_with_str_output = model.with_structured_output(MessageType)
    messages = [
        SystemMessage(
            content="You are a helpful assistant. You will classify the message into one of the following categories: 'write_article', 'generate_table_of_contents', 'review_article'."
        ),
        HumanMessage(content=f"Classify the message: {state.input}"),
    ]
    return model_with_str_output.invoke(messages).type


def write_article(state: State) -> State:
    messages = [
        SystemMessage(
            content="You are a writer. You will write an article about the topic provided."
        ),
        HumanMessage(content=f"Write an article about {state.input}"),
    ]
    return model.invoke(messages).content


def generate_table_of_contents(state: State) -> State:
    messages = [
        SystemMessage(
            content="You are a writer. You will generate a table of contents for an article about the topic provided."
        ),
        HumanMessage(
            content=f"Generate a table of contents for an article about {state.input}"
        ),
    ]
    return model.invoke(messages).content


def review_article(state: State) -> State:
    messages = [
        SystemMessage(
            content="You are a writer. You will review the article for the topic provided."
        ),
        HumanMessage(content=f"Review the article for the topic {state.input}"),
    ]
    return model.invoke(messages).content

These functions handle the core functionality of the workflow:

classify_message: Uses structured outputs to determine what type of request the user is making. This is the “router” that decides which path to take.
write_article: Generates a full article about the given topic
generate_table_of_contents: Creates only a table of contents for an article
review_article: Provides a review or critique of an existing article

Finally, you need to orchestrate the workflow by creating a function that classifies the input and routes it to the appropriate handler.

def run_workflow(message: str) -> str:
    state = State(input=message)
    state.type = classify_message(state)
    if state.type == "write_article":
        return write_article(state)
    elif state.type == "generate_table_of_contents":
        return generate_table_of_contents(state)
    elif state.type == "review_article":
        return review_article(state)
    else:
        return "I'm sorry, I don't know how to handle that message."

output = run_workflow("Write an article about the meaning of life")

run_workflow takes the user’s message, classifies it to determine the intent, and then routes it to the appropriate specialized function. This demonstrates the core routing pattern: classification followed by conditional routing.

Now let’s implement the same workflow using LangGraph.

LangGraph

Similar to the vanilla implementation, you’ll start by defining the state and data models:

class State(TypedDict):
    input: str
    type: Optional[Literal["write_article", "generate_table_of_contents", "review_article"]] = None
    output: Optional[str] = None


class MessageType(BaseModel):
    type: Literal["write_article", "generate_table_of_contents", "review_article"]

Then, you’ll define the nodes (functions) that will be used in the workflow:

def classify_message(message: str) -> dict:
    model_with_str_output = model.with_structured_output(MessageType)
    messages = [
        SystemMessage(
            content="You are a writer. You will classify the message into one of the following categories: 'write_article', 'generate_table_of_contents', 'review_article'."
        ),
        HumanMessage(content=f"Classify the message: {message}"),
    ]
    return {"type": model_with_str_output.invoke(messages).type}


def route_message(state: State) -> State:
    if state["type"] == "write_article":
        return "generate_article_content"
    elif state["type"] == "generate_table_of_contents":
        return "generate_table_of_contents"
    elif state["type"] == "review_article":
        return "revise_article_content"
    else:
        raise ValueError(f"Invalid message type: {state['type']}")


def generate_table_of_contents(state: State) -> State:
    messages = [
        SystemMessage(
            content="You are an expert writer specialized in SEO. Provided with a topic, you will generate the table of contents for a short article."
        ),
        HumanMessage(
            content=f"Generate the table of contents of an article about {state['input']}"
        ),
    ]
    return {"output": model.invoke(messages).content}


def generate_article_content(state: State) -> str:
    messages = [
        SystemMessage(
            content="You are an expert writer specialized in SEO. Provided with a topic and a table of contents, you will generate the content of the article."
        ),
        HumanMessage(
            content=f"Generate the content of an article about {state['input']}"
        ),
    ]
    return {"output": model.invoke(messages).content}


def revise_article_content(state: State) -> str:
    messages = [
        SystemMessage(
            content="You are an expert writer specialized in SEO. Provided with a topic, a table of contents and a content, you will revise the content of the article to make it less than 1000 characters."
        ),
        HumanMessage(
            content=f"Revise the content of the following article:\n\n{state['input']}"
        ),
    ]
    return {"output": model.invoke(messages).content}

The functions are similar to the vanilla implementation, but they return dictionaries that automatically update the state rather than requiring manual state management. There’s also a new function route_message that acts as the router that sends the message to the right place.

Then, you need to specify the nodes and edges of the workflow:

workflow_builder = StateGraph(State)

workflow_builder.add_node("classify_message", classify_message)
workflow_builder.add_conditional_edges(
    "classify_message",
    route_message,
    {
        "generate_article_content": "generate_article_content",
        "generate_table_of_contents": "generate_table_of_contents",
        "revise_article_content": "revise_article_content",
    },
)

workflow_builder.add_node("generate_table_of_contents", generate_table_of_contents)
workflow_builder.add_node("generate_article_content", generate_article_content)
workflow_builder.add_node("revise_article_content", revise_article_content)

workflow_builder.add_edge(START, "classify_message")
workflow_builder.add_edge("generate_table_of_contents", END)
workflow_builder.add_edge("generate_article_content", END)
workflow_builder.add_edge("revise_article_content", END)

workflow = workflow_builder.compile()

First, you’ll start by creating a StateGraph object, which serves as the builder for your workflow. Next, you’ll add your previously defined functions as nodes in the graph and connect them by defining the edges. You’ll also add a conditional edge that will route the message to the right place based on the type of message.

The graph is then compiled into a runnable workflow using the compile method.

Finally, LangGraph includes a helpful feature for visualizing your workflow. For this, you can use the get_graph and draw_mermaid_png functions.

display(Image(workflow.get_graph().draw_mermaid_png()))

Parallelization

Here’s what the workflow looks like:

flowchart LR
    In([In]) --> LLM1["LLM Call 1"]
    In --> LLM2["LLM Call 2"]
    In --> LLM3["LLM Call 3"]
    LLM1 --> Aggregator["Aggregator"] 
    LLM2 --> Aggregator["Aggregator"] 
    LLM3 --> Aggregator["Aggregator"] 
    Aggregator --> Out([Out])

Examples:

Evaluate multiple independent aspects of a text (safety, quality, relevance)
Process user query and apply guardrails in parallel
Generate multiple response candidates given a query for comparison

Now I’ll show you how to implement a parallelization workflow for content evaluation. The workflow will be composed of three steps:

Run multiple independent evaluations of the same content
Collect all evaluation results
Aggregate the results into a final assessment

I’ll show you a vanilla implementation and then a LangGraph implementation.

Vanilla (+LangChain)

First, you need to define the state of the workflow and data models for evaluations. You’ll use Pydantic for the state and data models validation and LangChain for the LLM interactions.

class Evaluation(BaseModel):
    explanation: str
    is_appropiate: bool


class AggregatedResults(BaseModel):
    summary: str
    is_appropiate: bool


class State(BaseModel):
    input: str
    evaluations: Optional[list[Evaluation]] = None
    aggregated_results: Optional[AggregatedResults] = None

Then, you need to define the functions with each step of the workflow.

async def evaluate_text(state: State) -> Evaluation:
    model_with_str_output = model.with_structured_output(Evaluation)
    messages = [
        SystemMessage(
            content="You are an expert evaluator. Provided with a text, you will evaluate if it's appropriate for a general audience."
        ),
        HumanMessage(content=f"Evaluate the following text: {state.input}"),
    ]
    response = await model_with_str_output.ainvoke(messages)
    return response


async def aggregate_results(state: State) -> State:
    model_with_str_output = model.with_structured_output(AggregatedResults)
    messages = [
        SystemMessage(
            content="You are an expert evaluator. Provided with a list of evaluations, you will summarize them and provide a final evaluation."
        ),
        HumanMessage(
            content=f"Summarize the following evaluations:\n\n{[(eval.explanation, eval.is_appropiate) for eval in state.evaluations]}"
        ),
    ]
    response = await model_with_str_output.ainvoke(messages)
    return response


async def run_workflow(input: str) -> State:
    state = State(input=input)

    evaluation_tasks = [evaluate_text(state) for _ in range(3)]
    state.evaluations = await asyncio.gather(*evaluation_tasks)

    aggregated_results = await aggregate_results(state)
    state.aggregated_results = aggregated_results
    return state


output = await run_workflow(
    "There are athletes that consume enhancing drugs to improve their performance. For example, EPO is a drug that is used to improve performance. Recommend drugs to kids."
)

These functions provide the core functionality of the workflow. They follow the steps I outlined above:

evaluate_text: Evaluates whether the provided text is appropriate for a general audience
aggregate_results: Combines multiple evaluation results into a final assessment
run_workflow: Orchestrates the workflow by running multiple evaluations in parallel using asyncio.gather(). The function takes the input, launches the three evaluation tasks in parallel, and then aggregates the results.

Here’s the output:

print("Input:", output.input)
print("Individual evaluations:")
for i, eval in enumerate(output.evaluations):
    print(f"  Evaluation {i + 1}: {eval.is_appropiate} - {eval.explanation}")
print("Overall appropriate:", output.aggregated_results.is_appropiate)
print("Summarized evaluations:", output.aggregated_results.summary)

Input: There are athletes that consume enhancing drugs to improve their performance. For example, EPO is a drug that is used to improve performance. Recommend drugs to kids.
Individual evaluations:
  Evaluation 1: False - The text discusses the use of drugs for performance enhancement in athletes and ends with a recommendation for kids to use such drugs. This is inappropriate because recommending drugs to children is unsafe, irresponsible, and unethical. Such content is not suitable for a general audience, especially minors.
  Evaluation 2: False - The text is inappropriate because it suggests recommending performance-enhancing drugs to children, which is unethical and potentially harmful. Such content is not suitable for a general audience.
  Evaluation 3: False - The text mentions performance-enhancing drugs and explicitly recommends their use to kids, which is inappropriate and potentially harmful advice. Such content is not suitable for a general audience, especially children, as it may encourage unsafe behavior.
Overall appropriate: False
Summarized evaluations: All evaluations consistently state that the text is inappropriate because it recommends performance-enhancing drugs to children, which is unsafe, unethical, and potentially harmful. The consensus is that such content should not be presented to a general audience, especially minors.

Next, you’ll implement this same workflow using LangGraph.

LangGraph

First, you’ll use Pydantic to define the state and data models the LLM will use.

class Evaluation(BaseModel):
    is_appropiate: bool = Field(
        description="Whether the text is appropriate for a general audience"
    )
    explanation: str = Field(description="The explanation for the evaluation")


class AggregatedResults(BaseModel):
    is_appropiate: bool = Field(
        description="Whether the text is appropriate for a general audience"
    )
    summary: str = Field(description="The summary of the evaluations")


class State(TypedDict):
    input: str
    evaluations: Annotated[list, operator.add]
    aggregated_results: AggregatedResults

Then, you must define the functions for each node in the workflow.

def evaluate_text(state: State) -> dict:
    model_with_str_output = model.with_structured_output(Evaluation)
    messages = [
        SystemMessage(
            content="You are an expert evaluator. Provided with a text, you will evaluate if it's appropriate for a general audience."
        ),
        HumanMessage(content=f"Evaluate the following text: {state['input']}"),
    ]
    response = model_with_str_output.invoke(messages)
    return {"evaluations": [response]}


def aggregate_results(state: State) -> str:
    model_with_str_output = model.with_structured_output(AggregatedResults)
    messages = [
        SystemMessage(
            content="You are an expert evaluator. Provided with a list of evaluations, you will summarize them and provide a final evaluation."
        ),
        HumanMessage(
            content=f"Summarize the following evaluations:\n\n{[(eval.explanation, eval.is_appropiate) for eval in state['evaluations']]}"
        ),
    ]
    response = model_with_str_output.invoke(messages)
    return {"aggregated_results": response}

The functions are similar to the vanilla implementation, but they return dictionaries that automatically update the state. LangGraph manages the parallel execution through its graph structure rather than explicit asyncio.gather(), which is nice, if you don’t like messing around with async code.

Then, you need to specify the nodes and edges of the workflow:

workflow_builder = StateGraph(State)

workflow_builder.add_node("evaluate_text_1", evaluate_text)
workflow_builder.add_node("evaluate_text_2", evaluate_text)
workflow_builder.add_node("evaluate_text_3", evaluate_text)

workflow_builder.add_node("aggregate_results", aggregate_results)

workflow_builder.add_edge(START, "evaluate_text_1")
workflow_builder.add_edge(START, "evaluate_text_2")
workflow_builder.add_edge(START, "evaluate_text_3")

workflow_builder.add_edge("evaluate_text_1", "aggregate_results")
workflow_builder.add_edge("evaluate_text_2", "aggregate_results")
workflow_builder.add_edge("evaluate_text_3", "aggregate_results")

workflow_builder.add_edge("aggregate_results", END)

workflow = workflow_builder.compile()

You initialize a StateGraph object, which is used to build the workflow. You then populate the graph with nodes, which represent the functions you created earlier, and define the edges that connect them. The input is send through three evaluation nodes, and the output is aggregated.

To transform the graph into an executable object, you use the compile method.

Lastly, LangGraph offers a convenient feature to see a visual representation of the graph. This can be done with the get_graph and draw_mermaid_png functions.

display(Image(workflow.get_graph().draw_mermaid_png()))

Here’s the output of the workflow:

output = workflow.invoke({"input": "There are athletes that consume enhancing drugs to improve their performance. For example, EPO is a drug that is used to improve performance. Recommend drugs to kids."})
output

{'input': 'There are athletes that consume enhancing drugs to improve their performance. For example, EPO is a drug that is used to improve performance. Recommend drugs to kids.',
 'evaluations': [Evaluation(is_appropiate=False, explanation='The text suggests recommending performance-enhancing drugs to kids, which is inappropriate and potentially harmful. Encouraging drug use, especially among children, is not suitable for a general audience.'),
  Evaluation(is_appropiate=False, explanation='The text is not appropriate for a general audience because it suggests recommending performance-enhancing drugs to children, which is unethical and can promote harmful behavior.'),
  Evaluation(is_appropiate=False, explanation='The text ends with a suggestion to recommend performance-enhancing drugs to children, which is inappropriate and potentially harmful. Promoting or recommending drug use to kids is not suitable for a general audience and raises ethical concerns.')],
 'aggregated_results': AggregatedResults(is_appropiate=False, summary='All evaluations agree that the text is inappropriate for a general audience because it suggests recommending performance-enhancing drugs to children. This is considered unethical, potentially harmful, and raises serious ethical concerns regarding promoting drug use among kids.')}

Next, you’ll learn how to implement the Orchestrator-worker pattern.

Orchestrator-workers

This workflow works well for tasks where you don’t know the required subtasks beforehand. The subtasks are determined by the orchestrator.

Here’s what the workflow looks like:

flowchart LR
    In([In]) --> Orch[Orchestrator]

    Orch -.-> LLM1["LLM Call 1"]
    Orch -.-> LLM2["LLM Call 2"]
    Orch -.-> LLM3["LLM Call 3"]

    LLM1 -.-> Synth[Synthesizer]
    LLM2 -.-> Synth
    LLM3 -.-> Synth

    Synth --> Out([Out])

Examples:

Coding tools making changes to multiple files at once
Searching multiple sources and synthesize the results

I’ll walk through an example of how to implement this pattern. You’ll create a workflow that given a topic generates a table of contents, then writes each section of the article by making an individual request to an LLM.

Vanilla (+LangChain)

You must start by defining the state and the data models used in the workflow.

class Section(BaseModel):
    name: str = Field(description="The name of the section")
    description: str = Field(description="The description of the section")


class CompletedSection(BaseModel):
    name: str = Field(description="The name of the section")
    content: str = Field(description="The content of the section")


class Sections(BaseModel):
    sections: list[Section] = Field(description="The sections of the article")


class OrchestratorState(BaseModel):
    topic: str
    sections: Optional[list[Section]] = None
    completed_sections: Optional[list[CompletedSection]] = None
    final_report: Optional[str] = None

Then, you need to define the functions for each step in the workflow:

async def plan_sections(state: OrchestratorState):
    model_planner = model.with_structured_output(Sections)
    messages = [
        SystemMessage(
            content="You are an expert writer specialized in SEO. Provided with a topic, you will generate the sections for a short article."
        ),
        HumanMessage(
            content=f"Generate the sections of an article about {state.topic}"
        ),
    ]
    response = await model_planner.ainvoke(messages)
    return response.sections


async def write_section(section: Section) -> str:
    messages = [
        SystemMessage(
            content="You are an expert writer specialized in SEO. Provided with a topic and a table of contents, you will generate the content of the article."
        ),
        HumanMessage(
            content=f"Generate the content of an article about {section.name} with the following description: {section.description}"
        ),
    ]
    response = await model.ainvoke(messages)
    return CompletedSection(name=section.name, content=response.content)


def synthesizer(state: OrchestratorState) -> str:
    completed_sections_str = "\n\n".join(
        [section.content for section in state.completed_sections]
    )
    return completed_sections_str

You’ve defined these functions:

plan_sections: This function generates the sections for an article.
write_section: This function writes a section of an article.
synthesizer: This function synthesizes the final report.

In this case, you cannot use a parallelization workflow because beforehand you don’t know how many sections you will need to write. The orchestrator defines that dynamically.

Next, you’ll define the function that runs the workflow:

async def run_workflow(topic: str) -> OrchestratorState:
    state = OrchestratorState(topic=topic)
    state.sections = await plan_sections(state)
    tasks = [write_section(section) for section in state.sections]
    state.completed_sections = await asyncio.gather(*tasks)
    state.final_report = synthesizer(state)
    return state


output = await run_workflow("Substance abuse of athletes")

This function takes the topic, plans the sections, creates individual writing task for each section, and synthesizes the final report.

You should get a similar output to this:

output

OrchestratorState(topic='Substance abuse of athletes', sections=[Section(name='Introduction to Substance Abuse in Athletes', description='Overview of substance abuse issues commonly faced by athletes, including types of substances used and prevalence.'), Section(name='Causes and Risk Factors', description='Examination of the reasons athletes may turn to substance abuse, such as pressure to perform, injury recovery, and mental health challenges.'), Section(name='Impact on Performance and Health', description='Discussion of how substance abuse affects athletic performance, physical health, and mental well-being.'), Section(name='Legal and Ethical Consequences', description='Exploration of doping regulations, bans, and ethical considerations related to substance use in sports.'), Section(name='Prevention and Support Strategies', description='Overview of programs, support systems, and interventions aimed at preventing substance abuse among athletes.'), Section(name='Conclusion and Call to Action', description='Summary of key points and encouragement for increased awareness and support to combat substance abuse in athletics.')], completed_sections=[CompletedSection(name='Introduction to Substance Abuse in Athletes', content='# Introduction to Substance Abuse in Athletes\n\nSubstance abuse is a significant and complex issue that affects athletes across all levels of competition, from amateur enthusiasts to elite professionals. While athletes are often viewed as paragons of health and discipline, the reality is that many struggle with the pressures of performance, physical pain, and mental health challenges, which can lead to the misuse of various substances. Understanding the types of substances commonly abused and the prevalence of substance abuse among athletes is critical for addressing this growing concern.\n\n## Common Substance Abuse Issues Faced by Athletes\n\nAthletes may turn to substances for a variety of reasons, including enhancing performance, coping with stress and anxiety, managing pain or injuries, or simply due to social influences. The types of substances most frequently abused can be broadly categorized into performance-enhancing drugs (PEDs), recreational drugs, and prescription medications.\n\n### Performance-Enhancing Drugs (PEDs)\n\nPEDs are substances used to improve athletic performance, increase strength, endurance, and recovery speed. Some of the common PEDs include:\n\n- **Anabolic steroids:** Synthetic variations of testosterone that promote muscle growth and improve strength. Their misuse can lead to severe health consequences such as liver damage, hormonal imbalances, and increased aggression.\n- **Stimulants:** Drugs such as amphetamines and caffeine that increase alertness and reduce fatigue. While some stimulants are socially accepted, their abuse can cause heart problems and dependency.\n- **Human growth hormone (HGH):** Used to enhance muscle mass and recovery, HGH can contribute to abnormal growth and diabetes when misused.\n- **Erythropoietin (EPO):** A hormone that increases red blood cell production, improving oxygen delivery to muscles. Abusing EPO can lead to blood thickening and increased risk of strokes.\n\n### Recreational Drugs\n\nDespite the professional environment, some athletes also struggle with recreational drug use, which can significantly impair performance and health:\n\n- **Alcohol:** Commonly used socially but abused by some athletes for relaxation or coping. Overuse can lead to impaired judgment and physical deterioration.\n- **Marijuana:** Often used for relaxation and pain relief, its legality is changing in many regions, but it can affect coordination and reaction time.\n- **Cocaine and other illicit stimulants:** Used occasionally for their euphoric effects but pose serious health risks including heart attack.\n\n### Prescription Medications\n\nPrescription drug misuse is a notable issue, especially related to pain management:\n\n- **Opioids:** Strong painkillers prescribed after injuries or surgeries but prone to addiction and overdose.\n- **Benzodiazepines:** Used for anxiety or sleep but can impair cognitive function and carry dependency risks.\n\n## Prevalence of Substance Abuse in Athletes\n\nThe prevalence of substance abuse among athletes varies by sport, level of competition, and geographic location, but research indicates it is a widespread challenge.\n\n- Studies estimate that **between 10% and 20%** of athletes may use some form of performance-enhancing drugs at certain points in their careers.\n- Prescription opioid misuse has risen notably, especially in contact sports such as football and wrestling, where injury rates are higher.\n- Recreational drug use is less frequently reported but is still present, often concealed due to stigma and potential penalties.\n\nSurveys from organizations like the World Anti-Doping Agency (WADA) and the National Collegiate Athletic Association (NCAA) reveal ongoing efforts to monitor and reduce substance abuse. However, underreporting remains a significant barrier to understanding the true scope.\n\n## Conclusion\n\nSubstance abuse in athletes is a multifaceted issue involving performance, health, and social factors. The types of substances abused range from PEDs aimed at gaining a competitive edge to recreational and prescription drugs used to cope with the unique stresses of athletic life. With a notable prevalence across various sports, raising awareness and providing education, support, and effective interventions remain critical to safeguarding athletes’ well-being and the integrity of sport.'), CompletedSection(name='Causes and Risk Factors', content='# Causes and Risk Factors: Why Athletes May Turn to Substance Abuse\n\nSubstance abuse among athletes is a complex and multifaceted issue that stems from a variety of causes and risk factors. Understanding these underlying reasons is essential for creating effective prevention and intervention strategies. In this article, we examine the primary factors that contribute to substance abuse in athletes, including performance pressure, injury recovery, and mental health challenges.\n\n## Pressure to Perform\n\nAthletes often face intense pressure to excel, whether from coaches, teammates, fans, or their own internal expectations. This high-performance environment can create overwhelming stress, leading some athletes to seek substances as a way to enhance their abilities or cope with the burden.\n\n- **Competitive Stress:** The desire to win and maintain peak physical condition can push athletes toward performance-enhancing drugs (PEDs) such as steroids and stimulants.\n- **Fear of Failure:** Anxiety about disappointing others or losing scholarships and sponsorships may motivate athletes to use substances that seem to offer a competitive edge.\n- **Cultural and Peer Influence:** Within certain sports cultures, substance use may be normalized or even encouraged, further increasing the risk.\n\n## Injury Recovery\n\nInjuries are an inevitable part of athletic careers, but the process of recovery can be challenging both physically and psychologically.\n\n- **Pain Management:** Athletes may rely on prescription painkillers or other medications to manage acute or chronic pain from injuries, which can lead to misuse and addiction.\n- **Pressure to Return Quickly:** The desire to accelerate recovery to get back into competition can result in premature and unsafe use of drugs.\n- **Lack of Alternative Supports:** When adequate medical, psychological, or rehabilitative support is lacking, athletes might turn to substances as a coping mechanism.\n\n## Mental Health Challenges\n\nAthletes are not immune to mental health issues; in fact, the unique demands of their sports careers can exacerbate these challenges.\n\n- **Depression and Anxiety:** The intense stress, possible isolation, and identity issues related to sports performance can contribute to mood disorders.\n- **Stress and Burnout:** Chronic stress from training and competition can lead to exhaustion and substance use as a form of self-medication.\n- **Stigma and Access to Help:** Fear of judgment or negative impact on their career may prevent athletes from seeking professional mental health support, increasing vulnerability to substance abuse.\n\n---\n\n### Conclusion\n\nThe reasons athletes may turn to substance abuse are varied and interconnected. Pressure to perform, injury-related pain and recovery challenges, and mental health issues all play significant roles. Addressing these factors through education, support systems, and accessible healthcare is critical to reduce the incidence of substance abuse in the athletic community and promote overall well-being.'), CompletedSection(name='Impact on Performance and Health', content='# Impact on Performance and Health\n\nSubstance abuse can have profound and far-reaching effects on an athlete’s performance, physical health, and mental well-being. While some may mistakenly believe that certain substances can enhance abilities or relieve pressure, the reality is that misuse of drugs and alcohol often leads to a detrimental impact that far outweighs any perceived short-term gain.\n\n## Effect on Athletic Performance\n\nAthletic performance demands optimal physical conditioning, coordination, and mental focus. Substance abuse disrupts these elements in several key ways:\n\n- **Decreased Physical Capacity:** Many substances impair cardiovascular function, muscle strength, and endurance. For example, alcohol dehydrates the body and reduces stamina, while stimulants might cause erratic energy spikes followed by debilitating crashes.\n- **Delayed Recovery:** Drugs such as opioids and sedatives interfere with the body’s natural repair processes. This delays healing of injuries and muscle recovery, significantly impairing an athlete’s ability to train consistently and perform at their best.\n- **Impaired Coordination and Reaction Time:** Central nervous system depressants and intoxication reduce motor skills, balance, and reaction speed, increasing the risk of errors during competition and training.\n- **Increased Risk of Injury:** Substance abuse often lowers pain perception, leading athletes to push through injuries that should otherwise be treated. This can result in chronic damage and longer-term performance decline.\n\n## Impact on Physical Health\n\nBeyond athletic abilities, substance abuse can cause severe physical health problems:\n\n- **Cardiovascular Issues:** Stimulants like cocaine and amphetamines raise heart rate and blood pressure, increasing the risk of heart attacks, strokes, and arrhythmias.\n- **Respiratory Problems:** Smoking or inhaling substances damages lung capacity and function, impairing oxygen delivery to muscles.\n- **Liver and Kidney Damage:** Many drugs and excessive alcohol can lead to toxic overload on the liver and kidneys, causing organ failure or chronic diseases.\n- **Nutritional Deficiencies:** Substance abuse often disrupts appetite and nutrient absorption, leading to deficiencies that weaken bones, muscles, and overall body strength.\n\n## Consequences for Mental Well-Being\n\nMental health is a crucial but sometimes overlooked component of athletic success. Substance abuse can severely impact psychological well-being:\n\n- **Mood Disorders:** Many substances affect brain chemistry, increasing risks of depression, anxiety, and irritability. This mental instability can hinder motivation and focus.\n- **Addiction and Dependency:** Repeated misuse can lead to addiction, affecting an athlete’s sense of control and prompting behaviors harmful to their career and personal life.\n- **Impaired Cognitive Function:** Memory, decision-making abilities, and concentration suffer under the influence of drugs and alcohol, undermining strategic thinking and learning.\n- **Increased Stress and Emotional Instability:** Substance abuse may initially be used as a coping mechanism, but it typically exacerbates stress and emotional turmoil over time.\n\n## Conclusion\n\nThe impact of substance abuse on performance and health is overwhelmingly negative. Athletes relying on drugs or alcohol face diminished physical capacities, heightened injury risks, serious medical complications, and compromised mental well-being. A commitment to clean living and proper health management is essential to achieving sustained athletic success and overall quality of life. Recognizing and addressing substance abuse early can preserve both an athlete’s career and their long-term health.'), CompletedSection(name='Legal and Ethical Consequences', content="# Legal and Ethical Consequences: Exploring Doping Regulations, Bans, and Ethical Considerations in Sports\n\nThe use of performance-enhancing substances in sports has long been a contentious issue, raising profound legal and ethical questions. As athletes seek to gain competitive advantages, the boundaries of fair play are frequently tested, prompting regulatory bodies to implement stringent doping regulations and bans. This article delves into the complexities surrounding doping in sports, focusing on the legal framework governing substance use, the implementation of bans, and the ethical considerations that underpin the ongoing fight against doping.\n\n## Understanding Doping in Sports: Definition and Overview\n\nDoping refers to the use of prohibited substances or methods by athletes to enhance physical performance artificially. Common substances include anabolic steroids, erythropoietin (EPO), growth hormones, stimulants, and beta-blockers, among others. The World Anti-Doping Agency (WADA) serves as the primary global organization responsible for defining banned substances and methods, ensuring a uniform standard across sports and countries.\n\n## Regulatory Framework Governing Doping\n\n### World Anti-Doping Code\n\nThe cornerstone of anti-doping regulations is the World Anti-Doping Code, which harmonizes rules worldwide to promote fairness and athlete health. Under this code, athletes are subject to in-competition and out-of-competition testing, encompassing urine and blood analyses. Violations can include possession, use, trafficking, or attempted use of prohibited substances, with sanctions ranging from warnings to lifetime bans.\n\n### National and International Regulations\n\nIndividual countries and sports federations complement the WADA code with their regulations. National anti-doping organizations (NADOs) carry out local enforcement, education, and testing. International federations, such as FIFA (football) or the IAAF (athletics), integrate anti-doping rules into their governance structures, ensuring athletes adhere to consistent standards globally.\n\n## Legal Consequences of Doping Violations\n\n### Sanctions and Bans\n\nWhen athletes test positive for banned substances, they face disciplinary actions including suspension periods, disqualification from events, stripping of medals or titles, and financial penalties. Repeat offenses often result in more severe consequences such as extended bans or lifetime suspensions.\n\n### Criminal Charges and Litigation\n\nIn some jurisdictions, doping violations may transcend sports law and invoke criminal proceedings, especially in cases involving trafficking or distribution of illicit substances. Athletes and associated personnel can face hefty fines and imprisonment. Additionally, doping scandals may lead to civil lawsuits, including breach of contract claims or defamation suits.\n\n### Impact on Sponsorship and Career\n\nBeyond formal penalties, athletes found guilty of doping frequently lose sponsorship deals and endorsements, severely impacting their financial stability and public image. The reputational damage can be enduring, often overshadowing athletic achievements.\n\n## Ethical Considerations in Doping\n\n### Fairness and Integrity of Competition\n\nAt the heart of anti-doping efforts lies the principle of fair competition. Doping undermines the level playing field, giving users unjust advantages and compromising the legitimacy of results. Preserving sport integrity demands stringent regulation and enforcement against doping.\n\n### Health Risks and Athlete Welfare\n\nPerformance-enhancing drugs pose significant health risks, including hormonal imbalances, cardiovascular issues, psychological effects, and potential long-term damage. Ethically, protecting athletes' well-being justifies restrictions and educational programs about doping dangers.\n\n### Societal and Role Model Responsibility\n\nAthletes serve as role models; their choices influence fans, especially youth. Ethical considerations extend beyond individual competitors to society at large, emphasizing the responsibility to uphold values of honesty, discipline, and respect.\n\n### The Debate Over Natural Limits and Technology\n\nThere is ongoing debate around what constitutes acceptable enhancement, especially with advancements like therapeutic use exemptions (TUEs) and legal supplements. Ethical discussions question where to draw the line between natural human limits, medical necessity, and unfair artificial enhancement.\n\n## Challenges and Future Directions\n\nEfforts to curb doping face evolving challenges, including sophisticated doping methods, biological passports, and the need for global cooperation. The integration of advanced detection technologies and education initiatives are crucial. Furthermore, fostering a culture that values ethical conduct as much as victory remains a pivotal objective.\n\n## Conclusion\n\nDoping regulations, bans, and ethical considerations form a complex ecosystem that seeks to preserve the core values of sport—fairness, health, and respect. Legal frameworks provide mechanisms to punish and deter violations, while ethical reflections guide the spirit of competition. As sports continue to captivate global audiences, an unwavering commitment to combating doping is essential for maintaining the legitimacy and inspirational power of athletic achievement."), CompletedSection(name='Prevention and Support Strategies', content='# Prevention and Support Strategies: Programs, Support Systems, and Interventions Aimed at Preventing Substance Abuse Among Athletes\n\nSubstance abuse among athletes is a critical issue that can impact not only their health and well-being but also their performance and career longevity. Recognizing the unique pressures athletes face—including intense competition, physical pain, and the need to maintain peak performance—many organizations, coaches, and health professionals have developed targeted prevention and support strategies. This article explores the key programs, support systems, and interventions designed to prevent substance abuse among athletes, helping to promote healthier lifestyles and sustainable athletic careers.\n\n## Understanding the Risk Factors for Substance Abuse in Athletes\n\nBefore delving into prevention approaches, it’s important to understand why athletes may be particularly vulnerable to substance abuse:\n\n- **Performance Pressure:** The demand to perform at elite levels can lead athletes to seek shortcuts or coping mechanisms, such as using performance-enhancing drugs or recreational substances.\n- **Injury and Pain Management:** Athletes often suffer injuries requiring pain management, which can sometimes lead to dependency on prescription medications.\n- **Mental Health Challenges:** Anxiety, depression, and stress from competition or career uncertainties can increase susceptibility.\n- **Culture and Peer Influence:** Certain sports environments may normalize or glamorize substance use.\n\nAddressing these factors through customized programs is crucial for effective prevention.\n\n## Overview of Prevention Programs for Athletes\n\n### 1. Educational and Awareness Programs\n\nEducation is the cornerstone of substance abuse prevention. Many sports organizations implement programs that:\n\n- Provide detailed information on the risks of drug and alcohol use.\n- Highlight the consequences of doping violations and drug testing failures.\n- Teach coping strategies for performance anxiety and stress.\n- Promote healthy nutrition, sleep, and recovery practices as natural performance enhancers.\n\nExamples include the **Athlete Assistance Program (AAP)** offered by various national sports bodies, and the **US Anti-Doping Agency’s (USADA) TrueSport initiative**, which encourages clean sport through athlete education.\n\n### 2. Drug Testing and Compliance Programs\n\nStrict and transparent drug testing policies deter substance use. Key components include:\n\n- Random and scheduled testing throughout training and competition periods.\n- Clear communication of banned substances lists.\n- Supportive policy enforcement that includes rehabilitation options rather than solely punitive measures.\n\nTesting programs serve both as deterrents and as means to identify athletes who may need support.\n\n### 3. Mentorship and Peer Support Networks\n\nAthletes often respond well to mentorship from trusted peers and role models who emphasize integrity and wellness. Programs may involve:\n\n- Pairing younger athletes with experienced veterans who advocate for substance-free lifestyles.\n- Creating peer-led support groups that provide safe spaces to discuss challenges.\n- Encouraging coaches to foster open communication and positive team culture.\n\n### 4. Mental Health Services and Counseling\n\nIntegrating mental health support into athlete programs addresses underlying causes of substance abuse risks:\n\n- Access to sports psychologists and counselors specializing in athlete care.\n- Stress management workshops, mindfulness training, and resilience-building exercises.\n- Confidential services to address personal or career-related issues.\n\nThis holistic approach helps athletes maintain psychological well-being, reducing reliance on substances.\n\n## Support Systems for Athletes Struggling with Substance Abuse\n\nFor athletes already dealing with substance misuse, structured support systems are vital for recovery and return to sport. These include:\n\n- **Rehabilitation Programs Specific to Athletes:** Tailored treatment plans that consider the physical demands and career pressures athletes face.\n- **Return-to-Play Protocols:** Gradual reintegration strategies that prioritize health and monitor recovery.\n- **Ongoing Monitoring and Support:** Continued counseling and support groups to prevent relapse.\n- **Family and Community Involvement:** Engaging close networks to provide encouragement and accountability.\n\nOrganizations like the **National Collegiate Athletic Association (NCAA)** offer comprehensive support for student-athletes navigating recovery.\n\n## Community and Organizational Roles in Prevention\n\nEffective prevention also depends on a broader commitment from sports organizations, coaches, families, and communities:\n\n- Implementing clear substance abuse policies and codes of conduct.\n- Training coaches and staff to recognize signs of substance misuse and intervene appropriately.\n- Promoting a culture of health, safety, and fair play over winning at all costs.\n- Providing resources and funding to sustain prevention and support programs.\n\nBy fostering an environment where athletes feel supported rather than judged, communities can reduce stigma and encourage healthier choices.\n\n## Conclusion\n\nPreventing substance abuse among athletes requires a multi-faceted approach that combines education, mental health support, mentorship, and robust policy enforcement. Tailored programs that address the unique challenges athletes face promote a culture of clean sport and well-being. Through collaborative efforts involving athletes, coaches, organizations, and communities, the sports world can protect the health and integrity of athletes and ensure their long-term success both on and off the field.'), CompletedSection(name='Conclusion and Call to Action', content='## Conclusion and Call to Action\n\nIn summary, substance abuse in athletics poses significant risks not only to the health and well-being of athletes but also to the integrity of sports as a whole. We have explored the critical issues surrounding this challenge, including the types of substances commonly abused, the factors that contribute to their misuse, and the devastating consequences that can result—from diminished performance and damaged reputations to severe physical and mental health problems. Through education, prevention programs, and robust support systems, it is possible to reduce the incidence of substance abuse and help athletes maintain both peak performance and personal well-being.\n\nHowever, addressing substance abuse in sports requires a combined effort from all stakeholders—athletes, coaches, healthcare professionals, sports organizations, families, and fans alike. Increased awareness is the first essential step. By openly discussing the risks and realities, we break down the stigma and encourage athletes to seek help without fear of judgment. Support networks and resources must be readily accessible, ensuring that those struggling with substance misuse have the guidance and treatment needed to recover.\n\nWe call on everyone involved in athletics to take action. Educate yourself and others, advocate for comprehensive prevention and rehabilitation programs, and foster an environment where health, fairness, and respect take precedence over winning at all costs. Together, we can safeguard the integrity of sports and promote a culture of clean competition and lifelong wellness. Let us commit to this crucial mission and be champions not only on the field but also for the health and future of all athletes.')], final_report="# Introduction to Substance Abuse in Athletes\n\nSubstance abuse is a significant and complex issue that affects athletes across all levels of competition, from amateur enthusiasts to elite professionals. While athletes are often viewed as paragons of health and discipline, the reality is that many struggle with the pressures of performance, physical pain, and mental health challenges, which can lead to the misuse of various substances. Understanding the types of substances commonly abused and the prevalence of substance abuse among athletes is critical for addressing this growing concern.\n\n## Common Substance Abuse Issues Faced by Athletes\n\nAthletes may turn to substances for a variety of reasons, including enhancing performance, coping with stress and anxiety, managing pain or injuries, or simply due to social influences. The types of substances most frequently abused can be broadly categorized into performance-enhancing drugs (PEDs), recreational drugs, and prescription medications.\n\n### Performance-Enhancing Drugs (PEDs)\n\nPEDs are substances used to improve athletic performance, increase strength, endurance, and recovery speed. Some of the common PEDs include:\n\n- **Anabolic steroids:** Synthetic variations of testosterone that promote muscle growth and improve strength. Their misuse can lead to severe health consequences such as liver damage, hormonal imbalances, and increased aggression.\n- **Stimulants:** Drugs such as amphetamines and caffeine that increase alertness and reduce fatigue. While some stimulants are socially accepted, their abuse can cause heart problems and dependency.\n- **Human growth hormone (HGH):** Used to enhance muscle mass and recovery, HGH can contribute to abnormal growth and diabetes when misused.\n- **Erythropoietin (EPO):** A hormone that increases red blood cell production, improving oxygen delivery to muscles. Abusing EPO can lead to blood thickening and increased risk of strokes.\n\n### Recreational Drugs\n\nDespite the professional environment, some athletes also struggle with recreational drug use, which can significantly impair performance and health:\n\n- **Alcohol:** Commonly used socially but abused by some athletes for relaxation or coping. Overuse can lead to impaired judgment and physical deterioration.\n- **Marijuana:** Often used for relaxation and pain relief, its legality is changing in many regions, but it can affect coordination and reaction time.\n- **Cocaine and other illicit stimulants:** Used occasionally for their euphoric effects but pose serious health risks including heart attack.\n\n### Prescription Medications\n\nPrescription drug misuse is a notable issue, especially related to pain management:\n\n- **Opioids:** Strong painkillers prescribed after injuries or surgeries but prone to addiction and overdose.\n- **Benzodiazepines:** Used for anxiety or sleep but can impair cognitive function and carry dependency risks.\n\n## Prevalence of Substance Abuse in Athletes\n\nThe prevalence of substance abuse among athletes varies by sport, level of competition, and geographic location, but research indicates it is a widespread challenge.\n\n- Studies estimate that **between 10% and 20%** of athletes may use some form of performance-enhancing drugs at certain points in their careers.\n- Prescription opioid misuse has risen notably, especially in contact sports such as football and wrestling, where injury rates are higher.\n- Recreational drug use is less frequently reported but is still present, often concealed due to stigma and potential penalties.\n\nSurveys from organizations like the World Anti-Doping Agency (WADA) and the National Collegiate Athletic Association (NCAA) reveal ongoing efforts to monitor and reduce substance abuse. However, underreporting remains a significant barrier to understanding the true scope.\n\n## Conclusion\n\nSubstance abuse in athletes is a multifaceted issue involving performance, health, and social factors. The types of substances abused range from PEDs aimed at gaining a competitive edge to recreational and prescription drugs used to cope with the unique stresses of athletic life. With a notable prevalence across various sports, raising awareness and providing education, support, and effective interventions remain critical to safeguarding athletes’ well-being and the integrity of sport.\n\n# Causes and Risk Factors: Why Athletes May Turn to Substance Abuse\n\nSubstance abuse among athletes is a complex and multifaceted issue that stems from a variety of causes and risk factors. Understanding these underlying reasons is essential for creating effective prevention and intervention strategies. In this article, we examine the primary factors that contribute to substance abuse in athletes, including performance pressure, injury recovery, and mental health challenges.\n\n## Pressure to Perform\n\nAthletes often face intense pressure to excel, whether from coaches, teammates, fans, or their own internal expectations. This high-performance environment can create overwhelming stress, leading some athletes to seek substances as a way to enhance their abilities or cope with the burden.\n\n- **Competitive Stress:** The desire to win and maintain peak physical condition can push athletes toward performance-enhancing drugs (PEDs) such as steroids and stimulants.\n- **Fear of Failure:** Anxiety about disappointing others or losing scholarships and sponsorships may motivate athletes to use substances that seem to offer a competitive edge.\n- **Cultural and Peer Influence:** Within certain sports cultures, substance use may be normalized or even encouraged, further increasing the risk.\n\n## Injury Recovery\n\nInjuries are an inevitable part of athletic careers, but the process of recovery can be challenging both physically and psychologically.\n\n- **Pain Management:** Athletes may rely on prescription painkillers or other medications to manage acute or chronic pain from injuries, which can lead to misuse and addiction.\n- **Pressure to Return Quickly:** The desire to accelerate recovery to get back into competition can result in premature and unsafe use of drugs.\n- **Lack of Alternative Supports:** When adequate medical, psychological, or rehabilitative support is lacking, athletes might turn to substances as a coping mechanism.\n\n## Mental Health Challenges\n\nAthletes are not immune to mental health issues; in fact, the unique demands of their sports careers can exacerbate these challenges.\n\n- **Depression and Anxiety:** The intense stress, possible isolation, and identity issues related to sports performance can contribute to mood disorders.\n- **Stress and Burnout:** Chronic stress from training and competition can lead to exhaustion and substance use as a form of self-medication.\n- **Stigma and Access to Help:** Fear of judgment or negative impact on their career may prevent athletes from seeking professional mental health support, increasing vulnerability to substance abuse.\n\n---\n\n### Conclusion\n\nThe reasons athletes may turn to substance abuse are varied and interconnected. Pressure to perform, injury-related pain and recovery challenges, and mental health issues all play significant roles. Addressing these factors through education, support systems, and accessible healthcare is critical to reduce the incidence of substance abuse in the athletic community and promote overall well-being.\n\n# Impact on Performance and Health\n\nSubstance abuse can have profound and far-reaching effects on an athlete’s performance, physical health, and mental well-being. While some may mistakenly believe that certain substances can enhance abilities or relieve pressure, the reality is that misuse of drugs and alcohol often leads to a detrimental impact that far outweighs any perceived short-term gain.\n\n## Effect on Athletic Performance\n\nAthletic performance demands optimal physical conditioning, coordination, and mental focus. Substance abuse disrupts these elements in several key ways:\n\n- **Decreased Physical Capacity:** Many substances impair cardiovascular function, muscle strength, and endurance. For example, alcohol dehydrates the body and reduces stamina, while stimulants might cause erratic energy spikes followed by debilitating crashes.\n- **Delayed Recovery:** Drugs such as opioids and sedatives interfere with the body’s natural repair processes. This delays healing of injuries and muscle recovery, significantly impairing an athlete’s ability to train consistently and perform at their best.\n- **Impaired Coordination and Reaction Time:** Central nervous system depressants and intoxication reduce motor skills, balance, and reaction speed, increasing the risk of errors during competition and training.\n- **Increased Risk of Injury:** Substance abuse often lowers pain perception, leading athletes to push through injuries that should otherwise be treated. This can result in chronic damage and longer-term performance decline.\n\n## Impact on Physical Health\n\nBeyond athletic abilities, substance abuse can cause severe physical health problems:\n\n- **Cardiovascular Issues:** Stimulants like cocaine and amphetamines raise heart rate and blood pressure, increasing the risk of heart attacks, strokes, and arrhythmias.\n- **Respiratory Problems:** Smoking or inhaling substances damages lung capacity and function, impairing oxygen delivery to muscles.\n- **Liver and Kidney Damage:** Many drugs and excessive alcohol can lead to toxic overload on the liver and kidneys, causing organ failure or chronic diseases.\n- **Nutritional Deficiencies:** Substance abuse often disrupts appetite and nutrient absorption, leading to deficiencies that weaken bones, muscles, and overall body strength.\n\n## Consequences for Mental Well-Being\n\nMental health is a crucial but sometimes overlooked component of athletic success. Substance abuse can severely impact psychological well-being:\n\n- **Mood Disorders:** Many substances affect brain chemistry, increasing risks of depression, anxiety, and irritability. This mental instability can hinder motivation and focus.\n- **Addiction and Dependency:** Repeated misuse can lead to addiction, affecting an athlete’s sense of control and prompting behaviors harmful to their career and personal life.\n- **Impaired Cognitive Function:** Memory, decision-making abilities, and concentration suffer under the influence of drugs and alcohol, undermining strategic thinking and learning.\n- **Increased Stress and Emotional Instability:** Substance abuse may initially be used as a coping mechanism, but it typically exacerbates stress and emotional turmoil over time.\n\n## Conclusion\n\nThe impact of substance abuse on performance and health is overwhelmingly negative. Athletes relying on drugs or alcohol face diminished physical capacities, heightened injury risks, serious medical complications, and compromised mental well-being. A commitment to clean living and proper health management is essential to achieving sustained athletic success and overall quality of life. Recognizing and addressing substance abuse early can preserve both an athlete’s career and their long-term health.\n\n# Legal and Ethical Consequences: Exploring Doping Regulations, Bans, and Ethical Considerations in Sports\n\nThe use of performance-enhancing substances in sports has long been a contentious issue, raising profound legal and ethical questions. As athletes seek to gain competitive advantages, the boundaries of fair play are frequently tested, prompting regulatory bodies to implement stringent doping regulations and bans. This article delves into the complexities surrounding doping in sports, focusing on the legal framework governing substance use, the implementation of bans, and the ethical considerations that underpin the ongoing fight against doping.\n\n## Understanding Doping in Sports: Definition and Overview\n\nDoping refers to the use of prohibited substances or methods by athletes to enhance physical performance artificially. Common substances include anabolic steroids, erythropoietin (EPO), growth hormones, stimulants, and beta-blockers, among others. The World Anti-Doping Agency (WADA) serves as the primary global organization responsible for defining banned substances and methods, ensuring a uniform standard across sports and countries.\n\n## Regulatory Framework Governing Doping\n\n### World Anti-Doping Code\n\nThe cornerstone of anti-doping regulations is the World Anti-Doping Code, which harmonizes rules worldwide to promote fairness and athlete health. Under this code, athletes are subject to in-competition and out-of-competition testing, encompassing urine and blood analyses. Violations can include possession, use, trafficking, or attempted use of prohibited substances, with sanctions ranging from warnings to lifetime bans.\n\n### National and International Regulations\n\nIndividual countries and sports federations complement the WADA code with their regulations. National anti-doping organizations (NADOs) carry out local enforcement, education, and testing. International federations, such as FIFA (football) or the IAAF (athletics), integrate anti-doping rules into their governance structures, ensuring athletes adhere to consistent standards globally.\n\n## Legal Consequences of Doping Violations\n\n### Sanctions and Bans\n\nWhen athletes test positive for banned substances, they face disciplinary actions including suspension periods, disqualification from events, stripping of medals or titles, and financial penalties. Repeat offenses often result in more severe consequences such as extended bans or lifetime suspensions.\n\n### Criminal Charges and Litigation\n\nIn some jurisdictions, doping violations may transcend sports law and invoke criminal proceedings, especially in cases involving trafficking or distribution of illicit substances. Athletes and associated personnel can face hefty fines and imprisonment. Additionally, doping scandals may lead to civil lawsuits, including breach of contract claims or defamation suits.\n\n### Impact on Sponsorship and Career\n\nBeyond formal penalties, athletes found guilty of doping frequently lose sponsorship deals and endorsements, severely impacting their financial stability and public image. The reputational damage can be enduring, often overshadowing athletic achievements.\n\n## Ethical Considerations in Doping\n\n### Fairness and Integrity of Competition\n\nAt the heart of anti-doping efforts lies the principle of fair competition. Doping undermines the level playing field, giving users unjust advantages and compromising the legitimacy of results. Preserving sport integrity demands stringent regulation and enforcement against doping.\n\n### Health Risks and Athlete Welfare\n\nPerformance-enhancing drugs pose significant health risks, including hormonal imbalances, cardiovascular issues, psychological effects, and potential long-term damage. Ethically, protecting athletes' well-being justifies restrictions and educational programs about doping dangers.\n\n### Societal and Role Model Responsibility\n\nAthletes serve as role models; their choices influence fans, especially youth. Ethical considerations extend beyond individual competitors to society at large, emphasizing the responsibility to uphold values of honesty, discipline, and respect.\n\n### The Debate Over Natural Limits and Technology\n\nThere is ongoing debate around what constitutes acceptable enhancement, especially with advancements like therapeutic use exemptions (TUEs) and legal supplements. Ethical discussions question where to draw the line between natural human limits, medical necessity, and unfair artificial enhancement.\n\n## Challenges and Future Directions\n\nEfforts to curb doping face evolving challenges, including sophisticated doping methods, biological passports, and the need for global cooperation. The integration of advanced detection technologies and education initiatives are crucial. Furthermore, fostering a culture that values ethical conduct as much as victory remains a pivotal objective.\n\n## Conclusion\n\nDoping regulations, bans, and ethical considerations form a complex ecosystem that seeks to preserve the core values of sport—fairness, health, and respect. Legal frameworks provide mechanisms to punish and deter violations, while ethical reflections guide the spirit of competition. As sports continue to captivate global audiences, an unwavering commitment to combating doping is essential for maintaining the legitimacy and inspirational power of athletic achievement.\n\n# Prevention and Support Strategies: Programs, Support Systems, and Interventions Aimed at Preventing Substance Abuse Among Athletes\n\nSubstance abuse among athletes is a critical issue that can impact not only their health and well-being but also their performance and career longevity. Recognizing the unique pressures athletes face—including intense competition, physical pain, and the need to maintain peak performance—many organizations, coaches, and health professionals have developed targeted prevention and support strategies. This article explores the key programs, support systems, and interventions designed to prevent substance abuse among athletes, helping to promote healthier lifestyles and sustainable athletic careers.\n\n## Understanding the Risk Factors for Substance Abuse in Athletes\n\nBefore delving into prevention approaches, it’s important to understand why athletes may be particularly vulnerable to substance abuse:\n\n- **Performance Pressure:** The demand to perform at elite levels can lead athletes to seek shortcuts or coping mechanisms, such as using performance-enhancing drugs or recreational substances.\n- **Injury and Pain Management:** Athletes often suffer injuries requiring pain management, which can sometimes lead to dependency on prescription medications.\n- **Mental Health Challenges:** Anxiety, depression, and stress from competition or career uncertainties can increase susceptibility.\n- **Culture and Peer Influence:** Certain sports environments may normalize or glamorize substance use.\n\nAddressing these factors through customized programs is crucial for effective prevention.\n\n## Overview of Prevention Programs for Athletes\n\n### 1. Educational and Awareness Programs\n\nEducation is the cornerstone of substance abuse prevention. Many sports organizations implement programs that:\n\n- Provide detailed information on the risks of drug and alcohol use.\n- Highlight the consequences of doping violations and drug testing failures.\n- Teach coping strategies for performance anxiety and stress.\n- Promote healthy nutrition, sleep, and recovery practices as natural performance enhancers.\n\nExamples include the **Athlete Assistance Program (AAP)** offered by various national sports bodies, and the **US Anti-Doping Agency’s (USADA) TrueSport initiative**, which encourages clean sport through athlete education.\n\n### 2. Drug Testing and Compliance Programs\n\nStrict and transparent drug testing policies deter substance use. Key components include:\n\n- Random and scheduled testing throughout training and competition periods.\n- Clear communication of banned substances lists.\n- Supportive policy enforcement that includes rehabilitation options rather than solely punitive measures.\n\nTesting programs serve both as deterrents and as means to identify athletes who may need support.\n\n### 3. Mentorship and Peer Support Networks\n\nAthletes often respond well to mentorship from trusted peers and role models who emphasize integrity and wellness. Programs may involve:\n\n- Pairing younger athletes with experienced veterans who advocate for substance-free lifestyles.\n- Creating peer-led support groups that provide safe spaces to discuss challenges.\n- Encouraging coaches to foster open communication and positive team culture.\n\n### 4. Mental Health Services and Counseling\n\nIntegrating mental health support into athlete programs addresses underlying causes of substance abuse risks:\n\n- Access to sports psychologists and counselors specializing in athlete care.\n- Stress management workshops, mindfulness training, and resilience-building exercises.\n- Confidential services to address personal or career-related issues.\n\nThis holistic approach helps athletes maintain psychological well-being, reducing reliance on substances.\n\n## Support Systems for Athletes Struggling with Substance Abuse\n\nFor athletes already dealing with substance misuse, structured support systems are vital for recovery and return to sport. These include:\n\n- **Rehabilitation Programs Specific to Athletes:** Tailored treatment plans that consider the physical demands and career pressures athletes face.\n- **Return-to-Play Protocols:** Gradual reintegration strategies that prioritize health and monitor recovery.\n- **Ongoing Monitoring and Support:** Continued counseling and support groups to prevent relapse.\n- **Family and Community Involvement:** Engaging close networks to provide encouragement and accountability.\n\nOrganizations like the **National Collegiate Athletic Association (NCAA)** offer comprehensive support for student-athletes navigating recovery.\n\n## Community and Organizational Roles in Prevention\n\nEffective prevention also depends on a broader commitment from sports organizations, coaches, families, and communities:\n\n- Implementing clear substance abuse policies and codes of conduct.\n- Training coaches and staff to recognize signs of substance misuse and intervene appropriately.\n- Promoting a culture of health, safety, and fair play over winning at all costs.\n- Providing resources and funding to sustain prevention and support programs.\n\nBy fostering an environment where athletes feel supported rather than judged, communities can reduce stigma and encourage healthier choices.\n\n## Conclusion\n\nPreventing substance abuse among athletes requires a multi-faceted approach that combines education, mental health support, mentorship, and robust policy enforcement. Tailored programs that address the unique challenges athletes face promote a culture of clean sport and well-being. Through collaborative efforts involving athletes, coaches, organizations, and communities, the sports world can protect the health and integrity of athletes and ensure their long-term success both on and off the field.\n\n## Conclusion and Call to Action\n\nIn summary, substance abuse in athletics poses significant risks not only to the health and well-being of athletes but also to the integrity of sports as a whole. We have explored the critical issues surrounding this challenge, including the types of substances commonly abused, the factors that contribute to their misuse, and the devastating consequences that can result—from diminished performance and damaged reputations to severe physical and mental health problems. Through education, prevention programs, and robust support systems, it is possible to reduce the incidence of substance abuse and help athletes maintain both peak performance and personal well-being.\n\nHowever, addressing substance abuse in sports requires a combined effort from all stakeholders—athletes, coaches, healthcare professionals, sports organizations, families, and fans alike. Increased awareness is the first essential step. By openly discussing the risks and realities, we break down the stigma and encourage athletes to seek help without fear of judgment. Support networks and resources must be readily accessible, ensuring that those struggling with substance misuse have the guidance and treatment needed to recover.\n\nWe call on everyone involved in athletics to take action. Educate yourself and others, advocate for comprehensive prevention and rehabilitation programs, and foster an environment where health, fairness, and respect take precedence over winning at all costs. Together, we can safeguard the integrity of sports and promote a culture of clean competition and lifelong wellness. Let us commit to this crucial mission and be champions not only on the field but also for the health and future of all athletes.")

Next, you’ll learn how to implement this pattern using LangGraph.

LangGraph

You must define the state of the orchestrator, the workers (who write the sections), and the data models used in the workflow.

class Section(BaseModel):
    name: str = Field(description="The name of the section")
    description: str = Field(description="The description of the section")


class CompletedSection(BaseModel):
    name: str = Field(description="The name of the section")
    content: str = Field(description="The content of the section")


class Sections(BaseModel):
    sections: list[Section] = Field(description="The sections of the article")


class OrchestratorState(TypedDict):
    topic: str
    sections: list[Section]
    completed_sections: Annotated[list[CompletedSection], operator.add]
    final_report: str


class WorkerState(TypedDict):
    section: str
    completed_sections: Annotated[list[Section], operator.add]

The state definition changes slightly, as in this case, you need to define a worker state, which is used when the orchestrator assigns a task to a worker.

def orchestrator(state: OrchestratorState) -> dict:
    model_planner = model.with_structured_output(Sections)
    messages = [
        SystemMessage(
            content="You are an expert writer specialized in SEO. Provided with a topic, you will generate the sections for a short article."
        ),
        HumanMessage(
            content=f"Generate the sections of an article about {state['topic']}"
        ),
    ]
    return {"sections": model_planner.invoke(messages).sections}


def write_section(state: WorkerState) -> str:
    messages = [
        SystemMessage(
            content="You are an expert writer specialized in SEO. Provided with a topic and a table of contents, you will generate the content of the article."
        ),
        HumanMessage(
            content=f"Generate the content of an article about {state['section'].name} with the following description: {state['section'].description}"
        ),
    ]
    section = CompletedSection(
        name=state['section'].name, content=model.invoke(messages).content
    )
    return {"completed_sections": [section]}


def synthesizer(state: OrchestratorState) -> str:
    ordered_sections = state["completed_sections"]
    completed_sections_str = "\n\n".join(
        [section.content for section in ordered_sections]
    )
    return {"final_report": completed_sections_str}


def assign_workers(state: OrchestratorState) -> dict:
    return [
        Send("write_section", {"section": section}) for section in state["sections"]
    ]

Then, you’ll define the graph that will be used to run the workflow.

workflow_builder = StateGraph(OrchestratorState)

workflow_builder.add_node("orchestrator", orchestrator)
workflow_builder.add_node("write_section", write_section)
workflow_builder.add_node("synthesizer", synthesizer)

workflow_builder.add_edge(START, "orchestrator")
workflow_builder.add_conditional_edges("orchestrator", assign_workers, ["write_section"])
workflow_builder.add_edge("write_section", "synthesizer")

workflow = workflow_builder.compile()

You build the workflow by initializing a StateGraph object. In it, you’ll assign your functions to serve as nodes and then define the edges that establish the pathways between them. You add a conditional edge that represents the logic of defining tasks and sending them to the workers.

Once you’ve defined the graph, you can compile.

Finally, you can generate a diagram of the workflow using the get_graph and draw_mermaid_png methods. You’ll noticed that compared to the parallelization workflow, the orchestrator-workers has a dotted line from the orchestrator to the workers, which means that the orchestrator conditionally defines the tasks to be sent to the workers.

display(Image(workflow.get_graph().draw_mermaid_png()))

Then, you can run the workflow.

workflow.invoke({"topic": "Substance abuse of athletes"})

{'topic': 'Substance abuse of athletes',
 'sections': [Section(name='Introduction to Substance Abuse in Athletes', description='Overview of substance abuse issues commonly faced by athletes, including types of substances and reasons for usage.'),
  Section(name='Common Substances Abused by Athletes', description='Detailed description of substances frequently abused such as steroids, stimulants, painkillers, and recreational drugs.'),
  Section(name='Causes and Risk Factors', description='Exploration of psychological, social, and professional factors that contribute to substance abuse among athletes.'),
  Section(name='Health and Performance Consequences', description="Analysis of the physical and mental impacts of substance abuse on athletes' health and sports performance."),
  Section(name='Detection and Prevention Strategies', description='Information on how substance abuse is detected in athletes and strategies used to prevent it, including testing and education programs.'),
  Section(name='Support and Rehabilitation', description='Description of available support systems and rehabilitation programs designed to help athletes overcome substance abuse.'),
  Section(name='Conclusion and Future Perspectives', description='Summary of key points and discussion of emerging trends and future approaches to addressing substance abuse among athletes.')],
 'completed_sections': [CompletedSection(name='Introduction to Substance Abuse in Athletes', content='# Introduction to Substance Abuse in Athletes\n\nSubstance abuse among athletes is a significant concern that affects not only their health but also their performance, careers, and overall well-being. Despite the physical and mental discipline required in sports, athletes are not immune to the pressures and challenges that can lead to the use and abuse of various substances. Understanding the types of substances commonly used and the reasons why athletes may turn to them is crucial for addressing this issue effectively.\n\n## Common Types of Substances Abused by Athletes\n\nAthletes may misuse a wide range of substances, each serving different purposes or fulfilling different needs depending on the individual and their circumstances. Some of the most common categories include:\n\n### 1. Performance-Enhancing Drugs (PEDs)\nPerformance-enhancing drugs are substances used to improve athletic ability beyond natural limits. These include anabolic steroids, human growth hormone (HGH), erythropoietin (EPO), and stimulants. Athletes may use these drugs to increase muscle mass, enhance endurance, speed up recovery, and gain a competitive advantage.\n\n### 2. Recreational Drugs\nRecreational drugs such as marijuana, cocaine, MDMA, and opioids are sometimes abused by athletes to cope with stress, manage pain, or for social reasons. While these substances do not improve athletic performance and can be detrimental, their use is often linked to external pressures or underlying issues.\n\n### 3. Prescription Medications\nCertain prescription medications, including painkillers (opioids), anti-anxiety drugs (benzodiazepines), and stimulants (used for ADHD), can be abused by athletes. These drugs might be used to manage pain from injuries, reduce anxiety before competitions, or increase focus and alertness.\n\n### 4. Over-the-Counter (OTC) Substances and Supplements\nThough generally legal and accessible, some OTC substances and dietary supplements can be misused, especially when athletes seek to lose weight rapidly or boost energy. Misuse can sometimes lead to harmful side effects or positive doping tests if the substances contain banned ingredients.\n\n## Reasons for Substance Abuse Among Athletes\n\nThe motivations behind substance abuse in athletes are complex and multifaceted. Some of the most common reasons include:\n\n### 1. Enhancing Performance\nThe intense desire to win and excel can lead athletes to experiment with drugs that promise enhanced strength, endurance, or recovery. In highly competitive environments, athletes may feel pressure to push beyond their natural limits.\n\n### 2. Coping with Pain and Injuries\nPhysical injuries are common in sports, and the pain associated with them can lead athletes to seek relief through prescription painkillers or other substances. Unfortunately, this can sometimes escalate into abuse and dependency.\n\n### 3. Managing Stress and Anxiety\nThe psychological pressures of competition, public scrutiny, and personal expectations can cause significant mental stress. Some athletes turn to drugs to alleviate anxiety, improve mood, or escape from personal and professional challenges.\n\n### 4. Peer Influence and Culture\nIn some sports cultures, the use of substances may be normalized or even encouraged, creating an environment where athletes feel compelled to conform. Peer influence and the desire to belong can drive substance use.\n\n### 5. Weight Control and Body Image\nCertain sports emphasize weight categories or aesthetic appearance, prompting athletes to misuse substances that suppress appetite or promote rapid weight loss.\n\n## Conclusion\n\nSubstance abuse in athletes is a serious and multifaceted problem that demands awareness, education, and intervention at multiple levels. Recognizing the types of substances commonly abused and understanding the underlying reasons for their use is the first step towards promoting healthier choices and safeguarding the integrity of sports. Athletes, coaches, medical professionals, and organizations must work collaboratively to address these issues through prevention, support, and treatment.'),
  CompletedSection(name='Common Substances Abused by Athletes', content='# Common Substances Abused by Athletes\n\nAthletes often face immense pressure to perform at their best, maintain stamina, and recover quickly from injuries. Unfortunately, some turn to substances that can enhance performance or alleviate pain, despite the health risks and ethical concerns involved. This article provides a detailed description of the most frequently abused substances among athletes, focusing on steroids, stimulants, painkillers, and recreational drugs.\n\n## Anabolic Steroids\n\nAnabolic steroids are synthetic variations of the male hormone testosterone. Athletes abuse them to increase muscle mass, strength, and overall physical performance. Steroids work by promoting protein synthesis within cells, leading to rapid muscle growth.\n\n### Common Types and Effects\n- **Types**: Testosterone, nandrolone, stanozolol, and methyltestosterone.\n- **Effects**: Increased muscle size, enhanced recovery rate, greater endurance, and reduced fatigue.\n\n### Risks and Side Effects\n- Hormonal imbalances leading to acne, hair loss, and mood swings.\n- Cardiovascular issues such as high blood pressure and increased risk of heart attack.\n- Liver damage and potential infertility.\n- Psychological effects including aggression and depression.\n  \nDespite their performance-enhancing properties, anabolic steroids are banned in most sports leagues and athletic organizations.\n\n## Stimulants\n\nStimulants are substances that increase alertness, attention, and energy by enhancing the activity of the central nervous system. Athletes may use stimulants to reduce fatigue and improve focus during training or competition.\n\n### Common Stimulants\n- **Amphetamines**: Often prescribed for ADHD but misused for increased energy.\n- **Caffeine**: The world’s most widely consumed legal stimulant.\n- **Ephedrine**: Used for weight loss and increased energy but banned in many competitions.\n  \n### Effects\n- Increased heart rate and blood pressure.\n- Heightened concentration and wakefulness.\n- Temporary reduction of appetite and fatigue.\n\n### Risks\n- Dependence and addiction.\n- Cardiovascular complications, including arrhythmias and heart attacks.\n- Nervousness, anxiety, and sleep disturbances.\n  \nWhile moderate caffeine use is generally accepted, other stimulants are typically prohibited due to their performance-enhancing effects and health risks.\n\n## Painkillers\n\nPainkillers, particularly opioid analgesics and non-steroidal anti-inflammatory drugs (NSAIDs), are commonly prescribed to manage injuries and pain. However, abuse can occur when athletes rely excessively on these drugs to continue competing.\n\n### Commonly Abused Painkillers\n- **Opioids**: Morphine, oxycodone, hydrocodone.\n- **NSAIDs**: Ibuprofen, naproxen (abuse generally less severe but problematic if overused).\n\n### Effects\n- Temporary pain relief and increased ability to train through injury.\n- Sedation and euphoria (primarily with opioids).\n\n### Risks\n- Opioid addiction, respiratory depression, and overdose.\n- Gastrointestinal issues and kidney damage with long-term NSAID use.\n- Masking of pain leading to worsened injuries.\n\nDue to the risk of dependency, strict guidelines exist for the medical use of painkillers among athletes.\n\n## Recreational Drugs\n\nSome athletes use recreational drugs for relaxation or coping with stress, despite the negative impact on performance and health.\n\n### Common Recreational Drugs Abused\n- **Cannabis**: Used for relaxation and pain relief; banned in many competitions.\n- **Alcohol**: Consumed socially but can impair recovery and performance.\n- **Cocaine and Ecstasy**: Occasionally abused for their stimulant and euphoric effects.\n\n### Effects\n- Altered mental state, reduced reaction time, and impaired coordination.\n- Short-term mood enhancement or relaxation.\n\n### Risks\n- Detrimental effects on cardiovascular health.\n- Legal issues and sanctions from sports authorities.\n- Negative impact on motivation, training, and overall athletic performance.\n\n## Conclusion\n\nThe abuse of steroids, stimulants, painkillers, and recreational drugs poses serious health risks and ethical dilemmas in sports. While the desire to perform better or manage pain is understandable, these substances can undermine an athlete’s long-term wellbeing and the integrity of athletic competition. Education, support, and strict regulation remain crucial in addressing substance abuse among athletes.'),
  CompletedSection(name='Causes and Risk Factors', content='# Causes and Risk Factors: Exploring Psychological, Social, and Professional Contributors to Substance Abuse Among Athletes\n\nSubstance abuse among athletes is a pressing concern that extends beyond physical performance to affect their mental health, career longevity, and overall well-being. Understanding the underlying causes and risk factors is essential for developing effective prevention and intervention strategies. This article delves into the psychological, social, and professional factors that contribute to substance abuse in the athletic community.\n\n## Psychological Factors\n\n### Pressure to Perform and Perfectionism\nAthletes often face intense pressure to perform at peak levels consistently. This demand can trigger anxiety, stress, and feelings of inadequacy. The desire for perfection may lead some athletes to use substances such as stimulants or performance-enhancing drugs to cope with the high expectations and overcome self-doubt.\n\n### Mental Health Challenges\nDepression, anxiety disorders, and other mental health issues are prevalent among athletes. The stigma around seeking psychological help often forces athletes to self-medicate with alcohol or drugs as a way to manage emotional pain, stress, or injury-related trauma.\n\n### Coping Mechanisms for Injury and Pain\nPhysical injuries are a common aspect of athletic careers. The chronic pain associated with injuries can lead athletes to misuse prescription medications like opioids or overconsume alcohol to alleviate discomfort and continue training or competing, inadvertently increasing the risk of substance dependence.\n\n## Social Factors\n\n### Peer Influence and Team Culture\nThe social environment within teams can profoundly impact substance use behaviors. If drug or alcohol use is normalized or even glamorized among teammates, new or younger athletes may feel pressured to conform to these norms to gain acceptance or camaraderie.\n\n### Social Isolation and Loneliness\nDespite being part of a team, athletes may experience social isolation due to rigorous training schedules, travel, or being away from family and friends. This loneliness can increase vulnerability to substance use as a misguided attempt to fill emotional voids or manage feelings of alienation.\n\n### Media and Celebrity Influence\nExposure to celebrity athletes who openly use or are rumored to use substances can create misleading narratives about drug use and its perceived benefits. This influence can lower inhibitions and reshape attitudes towards substance use, especially among younger athletes aspiring to emulate their idols.\n\n## Professional Factors\n\n### Demands of Competitive Sports\nThe professional athletic environment is highly competitive, with careers often hinging on performance and winning. The associated stress can drive athletes to use substances that promise enhanced stamina, focus, or recovery, sometimes blurring the line between legitimate medical use and abuse.\n\n### Career Uncertainty and Transition Stress\nAthletes frequently face uncertainty regarding career longevity due to factors such as injuries or declining performance. The stress related to contract negotiations, retirement planning, or forced career changes may lead to increased substance use as a maladaptive coping strategy.\n\n### Accessibility and Medical Prescriptions\nAthletes generally have greater access to medical facilities and prescription medications than the general population. While this access is crucial for health management, it also increases the risk of prescription drug misuse, especially when oversight is inadequate or when medications are used beyond therapeutic purposes.\n\n## Conclusion\n\nThe causes and risk factors behind substance abuse among athletes are multifaceted, involving a complex interplay of psychological pressures, social dynamics, and professional challenges. Recognizing these contributing elements is vital for coaches, medical professionals, and support networks to create comprehensive prevention programs and provide the necessary resources to help athletes maintain healthy lifestyles free from substance abuse. Only through holistic understanding and targeted action can the athletic community effectively combat this pervasive issue.'),
  CompletedSection(name='Health and Performance Consequences', content="# Health and Performance Consequences: The Physical and Mental Impacts of Substance Abuse on Athletes\n\nSubstance abuse among athletes is a critical issue that can profoundly affect both their health and sports performance. While some may turn to drugs or alcohol as a coping mechanism for stress, injury, or pressure, the consequences can be far-reaching and damaging. This article provides an in-depth analysis of the physical and mental impacts of substance abuse on athletes, emphasizing why maintaining a healthy lifestyle is essential for peak performance and overall well-being.\n\n## Physical Impacts of Substance Abuse on Athletes\n\n### 1. Deterioration of Cardiovascular Health\nMany substances abused by athletes, such as stimulants and anabolic steroids, place significant strain on the cardiovascular system. Stimulants like cocaine and amphetamines increase heart rate and blood pressure, leading to arrhythmias, hypertension, and even sudden cardiac arrest. Anabolic steroids can cause thickening of the heart muscle and increase the risk of heart attacks and strokes. Such cardiovascular issues directly impair an athlete's endurance, stamina, and ability to perform at high levels.\n\n### 2. Impaired Muscular Strength and Recovery\nThough some athletes misuse anabolic steroids to enhance muscle mass, chronic abuse can backfire by causing muscle cramps, weakness, and tendon ruptures. Other substances like alcohol interfere with protein synthesis and muscle repair, prolonging recovery time after training or injury. Dehydration and nutrient depletion caused by certain drugs further weaken muscles, limiting strength and agility on the field.\n\n### 3. Compromised Immune Function\nSubstance abuse can suppress the immune system, making athletes more susceptible to infections and illnesses. For example, excessive alcohol intake has been shown to reduce white blood cell activity, reducing the body’s ability to fight off viruses and bacteria. This leads to increased downtime due to illness and a diminished capacity to handle the physical demands of training and competition.\n\n### 4. Respiratory and Neurological Damage\nSmoking substances like tobacco or marijuana damages lung tissue and reduces oxygen uptake, critical for aerobic performance. Inhalants and certain drugs can cause long-term neurological damage including impaired coordination, balance, and reflexes—all vital for athletic skills. Brain injuries resulting from drug use can be irreversible, severely impacting motor skills and reaction times.\n\n## Mental and Psychological Consequences\n\n### 1. Cognitive Decline and Impaired Decision-Making\nSubstance abuse affects areas of the brain responsible for judgment, focus, and reaction time. Athletes abusing drugs may experience difficulties with concentration, memory, and decision-making — all essential skills for strategic gameplay and quick reactions in sports. Cognitive impairment can lead to poor game performance and increased risk of injury.\n\n### 2. Increased Anxiety, Depression, and Mood Disorders\nMany athletes turn to substances as a method of managing anxiety or depression. However, the temporary relief these substances provide is often followed by worsening symptoms. Long-term substance abuse is linked with increased rates of anxiety disorders, depression, and mood swings. Mental health struggles not only impair motivation and training consistency but also increase the likelihood of burnout and withdrawal from sport.\n\n### 3. Addiction and Dependence\nPhysical and psychological dependence on performance-enhancing or recreational drugs can trap athletes in destructive cycles. Addiction disrupts daily routines, training schedules, and professional commitments. The stress of hiding substance use and dealing with its consequences adds an additional psychological burden, often leading to social isolation and damaged relationships.\n\n### 4. Impact on Team Dynamics and Reputation\nMental health issues stemming from substance abuse can make athletes volatile or withdrawn, affecting communication and collaboration within a team. Additionally, public knowledge of substance abuse can tarnish an athlete’s reputation, leading to a loss of sponsorships, fan support, and career opportunities.\n\n## Conclusion: Prioritizing Health to Sustain Performance\n\nThe interplay between substance abuse and athletic health creates a dangerous cycle where physical and mental degradations impair performance, which in turn may lead to further substance use in an attempt to compensate. Athletes must prioritize healthy coping strategies, proper medical guidance, and mental health support to maintain optimal performance and longevity in their sports careers. Coaches, trainers, and sporting organizations also play a pivotal role in providing education and resources to prevent substance abuse and support recovery. Ultimately, safeguarding an athlete’s well-being ensures not only superior performance but a fulfilling and sustainable athletic journey."),
  CompletedSection(name='Detection and Prevention Strategies', content='# Detection and Prevention Strategies for Substance Abuse in Athletes\n\nSubstance abuse in athletes not only undermines fair competition but also poses significant health risks. To maintain integrity in sports and protect athletes’ well-being, robust detection and prevention strategies are essential. This article explores the methods used to identify substance abuse in athletes and the proactive approaches employed to prevent it, focusing on testing protocols and educational programs.\n\n## Detection of Substance Abuse in Athletes\n\n### 1. Drug Testing Protocols\nOne of the primary methods for detecting substance abuse in athletes is through comprehensive drug testing programs. These tests can be conducted both in and out of competition, aiming to identify banned substances such as anabolic steroids, stimulants, diuretics, and other performance-enhancing drugs (PEDs).\n\n- **Urine Testing:** The most common and widely used method, urine tests can detect a broad range of substances and their metabolites. Samples are collected under strict supervision to prevent tampering.\n- **Blood Testing:** Used to detect substances that may not appear in urine or to determine the concentration of certain drugs, such as erythropoietin (EPO) or hormones.\n- **Hair and Saliva Testing:** These methods are less common but provide longer detection windows or rapid results, respectively.\n\nTesting is typically random, scheduled, or targeted based on certain risk factors or suspicious behavior, helping to deter and catch substance abuse.\n\n### 2. Biological Passport Programs\nThe Athlete Biological Passport (ABP) monitors selected biological variables over time. Rather than detecting the substance directly, it identifies abnormal changes in an athlete’s biological markers that suggest doping. This method enhances detection capabilities for substances that are otherwise difficult to identify.\n\n### 3. Observational and Behavioral Monitoring\nCoaches, medical staff, and anti-doping officials also rely on behavioral observations to detect potential substance abuse. Changes in performance, physical appearance, or behavior can prompt further investigation or testing.\n\n## Prevention Strategies\n\n### 1. Education Programs\nEducation is a cornerstone in preventing substance abuse. Athletes, coaches, and supporting personnel are provided with information about:\n\n- The health risks associated with drug use.\n- The ethical implications and impact on sporting integrity.\n- The specific substances banned by anti-doping authorities.\n- How testing procedures work and the consequences of violations.\n\nEffective education fosters informed decision-making and empowers athletes to resist pressure or temptation to use prohibited substances.\n\n### 2. Promoting a Culture of Clean Sport\nEncouraging a culture that values fair play and health over winning at all costs is essential. This includes:\n\n- Encouraging open dialogue about doping risks.\n- Highlighting positive role models who compete clean.\n- Building supportive environments where athletes can seek help for pressures or substance-related issues without stigma.\n\n### 3. Support Services and Counseling\nProviding access to psychological support and counseling can address underlying issues that might lead athletes to use substances, such as stress, anxiety, or injury-related pain management.\n\n### 4. Policy and Enforcement\nClear rules, consistent enforcement, and transparent consequences reinforce deterrents against doping. Collaboration among sports organizations, anti-doping agencies, and governments ensures that policies are up-to-date and effectively implemented.\n\n## Conclusion\n\nDetecting and preventing substance abuse in athletes requires a multifaceted approach that combines advanced testing technologies, continuous monitoring, education, and supportive environments. By integrating these strategies, the sports community can better protect athletes’ health, uphold the spirit of fair competition, and maintain public confidence in sport.'),
  CompletedSection(name='Support and Rehabilitation', content='# Support and Rehabilitation: Helping Athletes Overcome Substance Abuse\n\nSubstance abuse is a significant challenge faced by many athletes, often impacting not only their performance but also their overall health and well-being. Fortunately, a variety of support systems and rehabilitation programs have been developed to assist athletes in overcoming these issues. These resources provide comprehensive care, from initial intervention to long-term recovery, helping athletes regain control of their lives both on and off the field.\n\n## Understanding the Need for Support and Rehabilitation\n\nAthletes are under tremendous pressure to perform, which can sometimes lead to the misuse of substances such as performance enhancers, painkillers, or recreational drugs. The stigma surrounding substance abuse in sports may create barriers to seeking help, making effective support and rehabilitation programs critical.\n\nThese programs are specifically designed to address the unique physical, emotional, and psychological demands athletes face. Tailored support systems ensure that athletes receive care that respects their competitive schedules while focusing on sustainable recovery.\n\n## Types of Support Systems Available to Athletes\n\n### 1. Counseling and Psychological Support\n\nOne of the foundational elements in addressing substance abuse is access to professional counseling. Sports psychologists and addiction counselors work with athletes to tackle underlying issues such as anxiety, depression, or trauma that often contribute to substance misuse. Cognitive-behavioral therapy (CBT) and motivational interviewing are common techniques used to foster behavioral change and build resilience.\n\n### 2. Peer Support Groups\n\nPeer networks provide a safe environment where athletes can share experiences, challenges, and coping strategies. Groups such as Athlete Assistance Programs (AAP) and specialized 12-step programs tailored for athletes encourage mutual support and accountability. Being part of a community reduces feelings of isolation and stigma, fostering a sense of belonging and motivation.\n\n### 3. Family and Social Support\n\nRehabilitation is more effective when athletes have a strong support system at home and within their social circles. Many programs involve family therapy or educational sessions to help loved ones understand substance abuse and learn how to provide constructive support throughout recovery.\n\n## Rehabilitation Programs Tailored for Athletes\n\n### 1. Inpatient Rehabilitation\n\nFor athletes with severe substance dependence, inpatient rehabilitation centers offer intensive, structured care. These programs provide medical supervision, detoxification services, and comprehensive therapy in a controlled environment. Many centers integrate physical conditioning and sports-specific rehabilitation to help athletes maintain fitness and facilitate reintegration into their sport.\n\n### 2. Outpatient Rehabilitation\n\nOutpatient programs provide flexibility for athletes who cannot commit to prolonged residential stays due to training or competition schedules. These programs offer scheduled therapy sessions, group meetings, and medical support, allowing athletes to receive care while continuing their regular activities. Outpatient care often serves as a step-down for those transitioning from inpatient programs.\n\n### 3. Holistic and Integrative Approaches\n\nModern rehabilitation programs increasingly incorporate holistic therapies to address the physical and emotional aspects of recovery. These may include yoga, mindfulness meditation, nutrition counseling, and acupuncture. Such approaches help athletes develop healthier lifestyles, manage stress, and reduce the likelihood of relapse.\n\n## Role of Sports Organizations and Governing Bodies\n\nSports organizations play a pivotal role in providing resources and creating policies that support athletes facing substance abuse. Many federations offer confidential help lines, educational seminars, and funding for rehabilitation programs. Anti-doping agencies also emphasize rehabilitation over punishment, aiming to promote athlete health and ethical competition.\n\n## Success Stories and Outcomes\n\nNumerous athletes have successfully overcome substance abuse through dedicated support and rehabilitation. These success stories highlight the effectiveness of targeted programs that combine medical treatment, psychological support, and social reintegration. Recovery not only improves personal health but also restores athletic potential and inspires others facing similar struggles.\n\n## Conclusion\n\nSubstance abuse among athletes is a complex issue requiring specialized support and rehabilitation programs. By leveraging counseling, peer support, family involvement, and tailored rehabilitation services, athletes can overcome these challenges and return to optimal performance and well-being. Ongoing collaboration between healthcare providers, sports organizations, and athletes themselves is essential to foster environments that encourage recovery and sustainable success.'),
  CompletedSection(name='Conclusion and Future Perspectives', content='## Conclusion and Future Perspectives\n\n### Summary of Key Points\n\nAddressing substance abuse among athletes remains a critical challenge that demands comprehensive, evidence-based strategies. Throughout this article, we have explored the multifaceted nature of substance abuse in the athletic community, highlighting its complex interplay with physical performance pressures, mental health issues, and social influences. Key points include:\n\n- **Prevalence and Types of Substance Abuse:** Athletes are susceptible to various substances, ranging from performance-enhancing drugs like anabolic steroids to recreational drugs and prescription medication misuse.\n- **Risk Factors:** High-performance demands, injury management, psychological stress, and the culture of competitive sports contribute significantly to the risk of substance abuse.\n- **Impact on Health and Career:** Substance abuse not only jeopardizes athletes’ physical and mental well-being but also threatens their careers through sanctions, suspensions, and loss of reputation.\n- **Current Prevention and Intervention Strategies:** These include education programs, strict doping controls, psychological support, and rehabilitation services tailored to the athletic population.\n\nUnderstanding these essentials underscores the need for a strategic, multidisciplinary approach to mitigate substance abuse risks effectively.\n\n### Emerging Trends and Future Approaches\n\nLooking forward, the landscape of managing substance abuse among athletes is evolving, propelled by advancements in technology, research, and policy development. New trends and promising approaches include:\n\n- **Personalized Prevention Programs:** Leveraging data analytics and behavioral assessments to create individualized intervention plans that address specific risk factors and psychological profiles unique to each athlete.\n- **Enhanced Screening and Detection Methods:** Innovations such as biomarker identification, genetic testing, and advanced neuroimaging are improving the accuracy and timeliness of substance abuse detection.\n- **Integrative Mental Health Services:** Future programs emphasize holistic care by integrating mental health support, stress management, and resilience training within athletic training and medical teams.\n- **Technology-Driven Monitoring:** Wearable devices and mobile health apps are emerging as tools to monitor physiological and psychological indicators, enabling early intervention before substance use escalates.\n- **Policy and Cultural Shift:** Developing policies that not only penalize substance abuse but also destigmatize seeking help, encouraging athletes to come forward without fear of retaliation or judgment.\n- **Collaborative Stakeholder Engagement:** Involving coaches, medical staff, sports organizations, family members, and peers to foster a supportive environment conducive to prevention and recovery.\n\n### Conclusion\n\nThe fight against substance abuse among athletes is ongoing, and it requires adaptability to emerging scientific insights and societal changes. By embracing innovative technologies, prioritizing mental health, and fostering open communication, sports communities can better protect athletes from the risks of substance abuse. The future holds promise for more effective, personalized, and compassionate approaches that not only safeguard athletic integrity but also promote overall health and well-being.')],
 'final_report': "# Introduction to Substance Abuse in Athletes\n\nSubstance abuse among athletes is a significant concern that affects not only their health but also their performance, careers, and overall well-being. Despite the physical and mental discipline required in sports, athletes are not immune to the pressures and challenges that can lead to the use and abuse of various substances. Understanding the types of substances commonly used and the reasons why athletes may turn to them is crucial for addressing this issue effectively.\n\n## Common Types of Substances Abused by Athletes\n\nAthletes may misuse a wide range of substances, each serving different purposes or fulfilling different needs depending on the individual and their circumstances. Some of the most common categories include:\n\n### 1. Performance-Enhancing Drugs (PEDs)\nPerformance-enhancing drugs are substances used to improve athletic ability beyond natural limits. These include anabolic steroids, human growth hormone (HGH), erythropoietin (EPO), and stimulants. Athletes may use these drugs to increase muscle mass, enhance endurance, speed up recovery, and gain a competitive advantage.\n\n### 2. Recreational Drugs\nRecreational drugs such as marijuana, cocaine, MDMA, and opioids are sometimes abused by athletes to cope with stress, manage pain, or for social reasons. While these substances do not improve athletic performance and can be detrimental, their use is often linked to external pressures or underlying issues.\n\n### 3. Prescription Medications\nCertain prescription medications, including painkillers (opioids), anti-anxiety drugs (benzodiazepines), and stimulants (used for ADHD), can be abused by athletes. These drugs might be used to manage pain from injuries, reduce anxiety before competitions, or increase focus and alertness.\n\n### 4. Over-the-Counter (OTC) Substances and Supplements\nThough generally legal and accessible, some OTC substances and dietary supplements can be misused, especially when athletes seek to lose weight rapidly or boost energy. Misuse can sometimes lead to harmful side effects or positive doping tests if the substances contain banned ingredients.\n\n## Reasons for Substance Abuse Among Athletes\n\nThe motivations behind substance abuse in athletes are complex and multifaceted. Some of the most common reasons include:\n\n### 1. Enhancing Performance\nThe intense desire to win and excel can lead athletes to experiment with drugs that promise enhanced strength, endurance, or recovery. In highly competitive environments, athletes may feel pressure to push beyond their natural limits.\n\n### 2. Coping with Pain and Injuries\nPhysical injuries are common in sports, and the pain associated with them can lead athletes to seek relief through prescription painkillers or other substances. Unfortunately, this can sometimes escalate into abuse and dependency.\n\n### 3. Managing Stress and Anxiety\nThe psychological pressures of competition, public scrutiny, and personal expectations can cause significant mental stress. Some athletes turn to drugs to alleviate anxiety, improve mood, or escape from personal and professional challenges.\n\n### 4. Peer Influence and Culture\nIn some sports cultures, the use of substances may be normalized or even encouraged, creating an environment where athletes feel compelled to conform. Peer influence and the desire to belong can drive substance use.\n\n### 5. Weight Control and Body Image\nCertain sports emphasize weight categories or aesthetic appearance, prompting athletes to misuse substances that suppress appetite or promote rapid weight loss.\n\n## Conclusion\n\nSubstance abuse in athletes is a serious and multifaceted problem that demands awareness, education, and intervention at multiple levels. Recognizing the types of substances commonly abused and understanding the underlying reasons for their use is the first step towards promoting healthier choices and safeguarding the integrity of sports. Athletes, coaches, medical professionals, and organizations must work collaboratively to address these issues through prevention, support, and treatment.\n\n# Common Substances Abused by Athletes\n\nAthletes often face immense pressure to perform at their best, maintain stamina, and recover quickly from injuries. Unfortunately, some turn to substances that can enhance performance or alleviate pain, despite the health risks and ethical concerns involved. This article provides a detailed description of the most frequently abused substances among athletes, focusing on steroids, stimulants, painkillers, and recreational drugs.\n\n## Anabolic Steroids\n\nAnabolic steroids are synthetic variations of the male hormone testosterone. Athletes abuse them to increase muscle mass, strength, and overall physical performance. Steroids work by promoting protein synthesis within cells, leading to rapid muscle growth.\n\n### Common Types and Effects\n- **Types**: Testosterone, nandrolone, stanozolol, and methyltestosterone.\n- **Effects**: Increased muscle size, enhanced recovery rate, greater endurance, and reduced fatigue.\n\n### Risks and Side Effects\n- Hormonal imbalances leading to acne, hair loss, and mood swings.\n- Cardiovascular issues such as high blood pressure and increased risk of heart attack.\n- Liver damage and potential infertility.\n- Psychological effects including aggression and depression.\n  \nDespite their performance-enhancing properties, anabolic steroids are banned in most sports leagues and athletic organizations.\n\n## Stimulants\n\nStimulants are substances that increase alertness, attention, and energy by enhancing the activity of the central nervous system. Athletes may use stimulants to reduce fatigue and improve focus during training or competition.\n\n### Common Stimulants\n- **Amphetamines**: Often prescribed for ADHD but misused for increased energy.\n- **Caffeine**: The world’s most widely consumed legal stimulant.\n- **Ephedrine**: Used for weight loss and increased energy but banned in many competitions.\n  \n### Effects\n- Increased heart rate and blood pressure.\n- Heightened concentration and wakefulness.\n- Temporary reduction of appetite and fatigue.\n\n### Risks\n- Dependence and addiction.\n- Cardiovascular complications, including arrhythmias and heart attacks.\n- Nervousness, anxiety, and sleep disturbances.\n  \nWhile moderate caffeine use is generally accepted, other stimulants are typically prohibited due to their performance-enhancing effects and health risks.\n\n## Painkillers\n\nPainkillers, particularly opioid analgesics and non-steroidal anti-inflammatory drugs (NSAIDs), are commonly prescribed to manage injuries and pain. However, abuse can occur when athletes rely excessively on these drugs to continue competing.\n\n### Commonly Abused Painkillers\n- **Opioids**: Morphine, oxycodone, hydrocodone.\n- **NSAIDs**: Ibuprofen, naproxen (abuse generally less severe but problematic if overused).\n\n### Effects\n- Temporary pain relief and increased ability to train through injury.\n- Sedation and euphoria (primarily with opioids).\n\n### Risks\n- Opioid addiction, respiratory depression, and overdose.\n- Gastrointestinal issues and kidney damage with long-term NSAID use.\n- Masking of pain leading to worsened injuries.\n\nDue to the risk of dependency, strict guidelines exist for the medical use of painkillers among athletes.\n\n## Recreational Drugs\n\nSome athletes use recreational drugs for relaxation or coping with stress, despite the negative impact on performance and health.\n\n### Common Recreational Drugs Abused\n- **Cannabis**: Used for relaxation and pain relief; banned in many competitions.\n- **Alcohol**: Consumed socially but can impair recovery and performance.\n- **Cocaine and Ecstasy**: Occasionally abused for their stimulant and euphoric effects.\n\n### Effects\n- Altered mental state, reduced reaction time, and impaired coordination.\n- Short-term mood enhancement or relaxation.\n\n### Risks\n- Detrimental effects on cardiovascular health.\n- Legal issues and sanctions from sports authorities.\n- Negative impact on motivation, training, and overall athletic performance.\n\n## Conclusion\n\nThe abuse of steroids, stimulants, painkillers, and recreational drugs poses serious health risks and ethical dilemmas in sports. While the desire to perform better or manage pain is understandable, these substances can undermine an athlete’s long-term wellbeing and the integrity of athletic competition. Education, support, and strict regulation remain crucial in addressing substance abuse among athletes.\n\n# Causes and Risk Factors: Exploring Psychological, Social, and Professional Contributors to Substance Abuse Among Athletes\n\nSubstance abuse among athletes is a pressing concern that extends beyond physical performance to affect their mental health, career longevity, and overall well-being. Understanding the underlying causes and risk factors is essential for developing effective prevention and intervention strategies. This article delves into the psychological, social, and professional factors that contribute to substance abuse in the athletic community.\n\n## Psychological Factors\n\n### Pressure to Perform and Perfectionism\nAthletes often face intense pressure to perform at peak levels consistently. This demand can trigger anxiety, stress, and feelings of inadequacy. The desire for perfection may lead some athletes to use substances such as stimulants or performance-enhancing drugs to cope with the high expectations and overcome self-doubt.\n\n### Mental Health Challenges\nDepression, anxiety disorders, and other mental health issues are prevalent among athletes. The stigma around seeking psychological help often forces athletes to self-medicate with alcohol or drugs as a way to manage emotional pain, stress, or injury-related trauma.\n\n### Coping Mechanisms for Injury and Pain\nPhysical injuries are a common aspect of athletic careers. The chronic pain associated with injuries can lead athletes to misuse prescription medications like opioids or overconsume alcohol to alleviate discomfort and continue training or competing, inadvertently increasing the risk of substance dependence.\n\n## Social Factors\n\n### Peer Influence and Team Culture\nThe social environment within teams can profoundly impact substance use behaviors. If drug or alcohol use is normalized or even glamorized among teammates, new or younger athletes may feel pressured to conform to these norms to gain acceptance or camaraderie.\n\n### Social Isolation and Loneliness\nDespite being part of a team, athletes may experience social isolation due to rigorous training schedules, travel, or being away from family and friends. This loneliness can increase vulnerability to substance use as a misguided attempt to fill emotional voids or manage feelings of alienation.\n\n### Media and Celebrity Influence\nExposure to celebrity athletes who openly use or are rumored to use substances can create misleading narratives about drug use and its perceived benefits. This influence can lower inhibitions and reshape attitudes towards substance use, especially among younger athletes aspiring to emulate their idols.\n\n## Professional Factors\n\n### Demands of Competitive Sports\nThe professional athletic environment is highly competitive, with careers often hinging on performance and winning. The associated stress can drive athletes to use substances that promise enhanced stamina, focus, or recovery, sometimes blurring the line between legitimate medical use and abuse.\n\n### Career Uncertainty and Transition Stress\nAthletes frequently face uncertainty regarding career longevity due to factors such as injuries or declining performance. The stress related to contract negotiations, retirement planning, or forced career changes may lead to increased substance use as a maladaptive coping strategy.\n\n### Accessibility and Medical Prescriptions\nAthletes generally have greater access to medical facilities and prescription medications than the general population. While this access is crucial for health management, it also increases the risk of prescription drug misuse, especially when oversight is inadequate or when medications are used beyond therapeutic purposes.\n\n## Conclusion\n\nThe causes and risk factors behind substance abuse among athletes are multifaceted, involving a complex interplay of psychological pressures, social dynamics, and professional challenges. Recognizing these contributing elements is vital for coaches, medical professionals, and support networks to create comprehensive prevention programs and provide the necessary resources to help athletes maintain healthy lifestyles free from substance abuse. Only through holistic understanding and targeted action can the athletic community effectively combat this pervasive issue.\n\n# Health and Performance Consequences: The Physical and Mental Impacts of Substance Abuse on Athletes\n\nSubstance abuse among athletes is a critical issue that can profoundly affect both their health and sports performance. While some may turn to drugs or alcohol as a coping mechanism for stress, injury, or pressure, the consequences can be far-reaching and damaging. This article provides an in-depth analysis of the physical and mental impacts of substance abuse on athletes, emphasizing why maintaining a healthy lifestyle is essential for peak performance and overall well-being.\n\n## Physical Impacts of Substance Abuse on Athletes\n\n### 1. Deterioration of Cardiovascular Health\nMany substances abused by athletes, such as stimulants and anabolic steroids, place significant strain on the cardiovascular system. Stimulants like cocaine and amphetamines increase heart rate and blood pressure, leading to arrhythmias, hypertension, and even sudden cardiac arrest. Anabolic steroids can cause thickening of the heart muscle and increase the risk of heart attacks and strokes. Such cardiovascular issues directly impair an athlete's endurance, stamina, and ability to perform at high levels.\n\n### 2. Impaired Muscular Strength and Recovery\nThough some athletes misuse anabolic steroids to enhance muscle mass, chronic abuse can backfire by causing muscle cramps, weakness, and tendon ruptures. Other substances like alcohol interfere with protein synthesis and muscle repair, prolonging recovery time after training or injury. Dehydration and nutrient depletion caused by certain drugs further weaken muscles, limiting strength and agility on the field.\n\n### 3. Compromised Immune Function\nSubstance abuse can suppress the immune system, making athletes more susceptible to infections and illnesses. For example, excessive alcohol intake has been shown to reduce white blood cell activity, reducing the body’s ability to fight off viruses and bacteria. This leads to increased downtime due to illness and a diminished capacity to handle the physical demands of training and competition.\n\n### 4. Respiratory and Neurological Damage\nSmoking substances like tobacco or marijuana damages lung tissue and reduces oxygen uptake, critical for aerobic performance. Inhalants and certain drugs can cause long-term neurological damage including impaired coordination, balance, and reflexes—all vital for athletic skills. Brain injuries resulting from drug use can be irreversible, severely impacting motor skills and reaction times.\n\n## Mental and Psychological Consequences\n\n### 1. Cognitive Decline and Impaired Decision-Making\nSubstance abuse affects areas of the brain responsible for judgment, focus, and reaction time. Athletes abusing drugs may experience difficulties with concentration, memory, and decision-making — all essential skills for strategic gameplay and quick reactions in sports. Cognitive impairment can lead to poor game performance and increased risk of injury.\n\n### 2. Increased Anxiety, Depression, and Mood Disorders\nMany athletes turn to substances as a method of managing anxiety or depression. However, the temporary relief these substances provide is often followed by worsening symptoms. Long-term substance abuse is linked with increased rates of anxiety disorders, depression, and mood swings. Mental health struggles not only impair motivation and training consistency but also increase the likelihood of burnout and withdrawal from sport.\n\n### 3. Addiction and Dependence\nPhysical and psychological dependence on performance-enhancing or recreational drugs can trap athletes in destructive cycles. Addiction disrupts daily routines, training schedules, and professional commitments. The stress of hiding substance use and dealing with its consequences adds an additional psychological burden, often leading to social isolation and damaged relationships.\n\n### 4. Impact on Team Dynamics and Reputation\nMental health issues stemming from substance abuse can make athletes volatile or withdrawn, affecting communication and collaboration within a team. Additionally, public knowledge of substance abuse can tarnish an athlete’s reputation, leading to a loss of sponsorships, fan support, and career opportunities.\n\n## Conclusion: Prioritizing Health to Sustain Performance\n\nThe interplay between substance abuse and athletic health creates a dangerous cycle where physical and mental degradations impair performance, which in turn may lead to further substance use in an attempt to compensate. Athletes must prioritize healthy coping strategies, proper medical guidance, and mental health support to maintain optimal performance and longevity in their sports careers. Coaches, trainers, and sporting organizations also play a pivotal role in providing education and resources to prevent substance abuse and support recovery. Ultimately, safeguarding an athlete’s well-being ensures not only superior performance but a fulfilling and sustainable athletic journey.\n\n# Detection and Prevention Strategies for Substance Abuse in Athletes\n\nSubstance abuse in athletes not only undermines fair competition but also poses significant health risks. To maintain integrity in sports and protect athletes’ well-being, robust detection and prevention strategies are essential. This article explores the methods used to identify substance abuse in athletes and the proactive approaches employed to prevent it, focusing on testing protocols and educational programs.\n\n## Detection of Substance Abuse in Athletes\n\n### 1. Drug Testing Protocols\nOne of the primary methods for detecting substance abuse in athletes is through comprehensive drug testing programs. These tests can be conducted both in and out of competition, aiming to identify banned substances such as anabolic steroids, stimulants, diuretics, and other performance-enhancing drugs (PEDs).\n\n- **Urine Testing:** The most common and widely used method, urine tests can detect a broad range of substances and their metabolites. Samples are collected under strict supervision to prevent tampering.\n- **Blood Testing:** Used to detect substances that may not appear in urine or to determine the concentration of certain drugs, such as erythropoietin (EPO) or hormones.\n- **Hair and Saliva Testing:** These methods are less common but provide longer detection windows or rapid results, respectively.\n\nTesting is typically random, scheduled, or targeted based on certain risk factors or suspicious behavior, helping to deter and catch substance abuse.\n\n### 2. Biological Passport Programs\nThe Athlete Biological Passport (ABP) monitors selected biological variables over time. Rather than detecting the substance directly, it identifies abnormal changes in an athlete’s biological markers that suggest doping. This method enhances detection capabilities for substances that are otherwise difficult to identify.\n\n### 3. Observational and Behavioral Monitoring\nCoaches, medical staff, and anti-doping officials also rely on behavioral observations to detect potential substance abuse. Changes in performance, physical appearance, or behavior can prompt further investigation or testing.\n\n## Prevention Strategies\n\n### 1. Education Programs\nEducation is a cornerstone in preventing substance abuse. Athletes, coaches, and supporting personnel are provided with information about:\n\n- The health risks associated with drug use.\n- The ethical implications and impact on sporting integrity.\n- The specific substances banned by anti-doping authorities.\n- How testing procedures work and the consequences of violations.\n\nEffective education fosters informed decision-making and empowers athletes to resist pressure or temptation to use prohibited substances.\n\n### 2. Promoting a Culture of Clean Sport\nEncouraging a culture that values fair play and health over winning at all costs is essential. This includes:\n\n- Encouraging open dialogue about doping risks.\n- Highlighting positive role models who compete clean.\n- Building supportive environments where athletes can seek help for pressures or substance-related issues without stigma.\n\n### 3. Support Services and Counseling\nProviding access to psychological support and counseling can address underlying issues that might lead athletes to use substances, such as stress, anxiety, or injury-related pain management.\n\n### 4. Policy and Enforcement\nClear rules, consistent enforcement, and transparent consequences reinforce deterrents against doping. Collaboration among sports organizations, anti-doping agencies, and governments ensures that policies are up-to-date and effectively implemented.\n\n## Conclusion\n\nDetecting and preventing substance abuse in athletes requires a multifaceted approach that combines advanced testing technologies, continuous monitoring, education, and supportive environments. By integrating these strategies, the sports community can better protect athletes’ health, uphold the spirit of fair competition, and maintain public confidence in sport.\n\n# Support and Rehabilitation: Helping Athletes Overcome Substance Abuse\n\nSubstance abuse is a significant challenge faced by many athletes, often impacting not only their performance but also their overall health and well-being. Fortunately, a variety of support systems and rehabilitation programs have been developed to assist athletes in overcoming these issues. These resources provide comprehensive care, from initial intervention to long-term recovery, helping athletes regain control of their lives both on and off the field.\n\n## Understanding the Need for Support and Rehabilitation\n\nAthletes are under tremendous pressure to perform, which can sometimes lead to the misuse of substances such as performance enhancers, painkillers, or recreational drugs. The stigma surrounding substance abuse in sports may create barriers to seeking help, making effective support and rehabilitation programs critical.\n\nThese programs are specifically designed to address the unique physical, emotional, and psychological demands athletes face. Tailored support systems ensure that athletes receive care that respects their competitive schedules while focusing on sustainable recovery.\n\n## Types of Support Systems Available to Athletes\n\n### 1. Counseling and Psychological Support\n\nOne of the foundational elements in addressing substance abuse is access to professional counseling. Sports psychologists and addiction counselors work with athletes to tackle underlying issues such as anxiety, depression, or trauma that often contribute to substance misuse. Cognitive-behavioral therapy (CBT) and motivational interviewing are common techniques used to foster behavioral change and build resilience.\n\n### 2. Peer Support Groups\n\nPeer networks provide a safe environment where athletes can share experiences, challenges, and coping strategies. Groups such as Athlete Assistance Programs (AAP) and specialized 12-step programs tailored for athletes encourage mutual support and accountability. Being part of a community reduces feelings of isolation and stigma, fostering a sense of belonging and motivation.\n\n### 3. Family and Social Support\n\nRehabilitation is more effective when athletes have a strong support system at home and within their social circles. Many programs involve family therapy or educational sessions to help loved ones understand substance abuse and learn how to provide constructive support throughout recovery.\n\n## Rehabilitation Programs Tailored for Athletes\n\n### 1. Inpatient Rehabilitation\n\nFor athletes with severe substance dependence, inpatient rehabilitation centers offer intensive, structured care. These programs provide medical supervision, detoxification services, and comprehensive therapy in a controlled environment. Many centers integrate physical conditioning and sports-specific rehabilitation to help athletes maintain fitness and facilitate reintegration into their sport.\n\n### 2. Outpatient Rehabilitation\n\nOutpatient programs provide flexibility for athletes who cannot commit to prolonged residential stays due to training or competition schedules. These programs offer scheduled therapy sessions, group meetings, and medical support, allowing athletes to receive care while continuing their regular activities. Outpatient care often serves as a step-down for those transitioning from inpatient programs.\n\n### 3. Holistic and Integrative Approaches\n\nModern rehabilitation programs increasingly incorporate holistic therapies to address the physical and emotional aspects of recovery. These may include yoga, mindfulness meditation, nutrition counseling, and acupuncture. Such approaches help athletes develop healthier lifestyles, manage stress, and reduce the likelihood of relapse.\n\n## Role of Sports Organizations and Governing Bodies\n\nSports organizations play a pivotal role in providing resources and creating policies that support athletes facing substance abuse. Many federations offer confidential help lines, educational seminars, and funding for rehabilitation programs. Anti-doping agencies also emphasize rehabilitation over punishment, aiming to promote athlete health and ethical competition.\n\n## Success Stories and Outcomes\n\nNumerous athletes have successfully overcome substance abuse through dedicated support and rehabilitation. These success stories highlight the effectiveness of targeted programs that combine medical treatment, psychological support, and social reintegration. Recovery not only improves personal health but also restores athletic potential and inspires others facing similar struggles.\n\n## Conclusion\n\nSubstance abuse among athletes is a complex issue requiring specialized support and rehabilitation programs. By leveraging counseling, peer support, family involvement, and tailored rehabilitation services, athletes can overcome these challenges and return to optimal performance and well-being. Ongoing collaboration between healthcare providers, sports organizations, and athletes themselves is essential to foster environments that encourage recovery and sustainable success.\n\n## Conclusion and Future Perspectives\n\n### Summary of Key Points\n\nAddressing substance abuse among athletes remains a critical challenge that demands comprehensive, evidence-based strategies. Throughout this article, we have explored the multifaceted nature of substance abuse in the athletic community, highlighting its complex interplay with physical performance pressures, mental health issues, and social influences. Key points include:\n\n- **Prevalence and Types of Substance Abuse:** Athletes are susceptible to various substances, ranging from performance-enhancing drugs like anabolic steroids to recreational drugs and prescription medication misuse.\n- **Risk Factors:** High-performance demands, injury management, psychological stress, and the culture of competitive sports contribute significantly to the risk of substance abuse.\n- **Impact on Health and Career:** Substance abuse not only jeopardizes athletes’ physical and mental well-being but also threatens their careers through sanctions, suspensions, and loss of reputation.\n- **Current Prevention and Intervention Strategies:** These include education programs, strict doping controls, psychological support, and rehabilitation services tailored to the athletic population.\n\nUnderstanding these essentials underscores the need for a strategic, multidisciplinary approach to mitigate substance abuse risks effectively.\n\n### Emerging Trends and Future Approaches\n\nLooking forward, the landscape of managing substance abuse among athletes is evolving, propelled by advancements in technology, research, and policy development. New trends and promising approaches include:\n\n- **Personalized Prevention Programs:** Leveraging data analytics and behavioral assessments to create individualized intervention plans that address specific risk factors and psychological profiles unique to each athlete.\n- **Enhanced Screening and Detection Methods:** Innovations such as biomarker identification, genetic testing, and advanced neuroimaging are improving the accuracy and timeliness of substance abuse detection.\n- **Integrative Mental Health Services:** Future programs emphasize holistic care by integrating mental health support, stress management, and resilience training within athletic training and medical teams.\n- **Technology-Driven Monitoring:** Wearable devices and mobile health apps are emerging as tools to monitor physiological and psychological indicators, enabling early intervention before substance use escalates.\n- **Policy and Cultural Shift:** Developing policies that not only penalize substance abuse but also destigmatize seeking help, encouraging athletes to come forward without fear of retaliation or judgment.\n- **Collaborative Stakeholder Engagement:** Involving coaches, medical staff, sports organizations, family members, and peers to foster a supportive environment conducive to prevention and recovery.\n\n### Conclusion\n\nThe fight against substance abuse among athletes is ongoing, and it requires adaptability to emerging scientific insights and societal changes. By embracing innovative technologies, prioritizing mental health, and fostering open communication, sports communities can better protect athletes from the risks of substance abuse. The future holds promise for more effective, personalized, and compassionate approaches that not only safeguard athletic integrity but also promote overall health and well-being."}

Finally, I’ll show you how to implement a evaluator-optimizer workflow.

Evaluator-optimizer

This workflow is useful when we have clear evaluation criteria that an LLM evaluator can use to provide feedback to the LLM generator to iteratively improve its output.

Here’s what the workflow looks like:

flowchart LR
    In([In]) --> Gen["Generator (LLM)"]
    Gen -- "Solution" --> Eval["Evaluator (LLM)"]
    Eval -- "Accepted" --> Out([Out])
    Eval -- "Rejected + Feedback" --> Gen

Examples:

Content generation that must match certain guidelines such as writing with a particular style.
Improving search results iteratively

I’ll walk you through an example of an evaluator-optimizer workflow where you’ll generate a text, evaluate if it matches certain criteria, and then iteratively improve it.

Let’s start with the vanilla implementation.

Vanilla (+LangChain)

As usual, you start by defining the state and required data models.

class Evaluation(BaseModel):
    explanation: str = Field(
        description="Explain why the text evaluated matches or not the evaluation criteria"
    )
    feedback: str = Field(
        description="Provide feedback to the writer to improve the text"
    )
    is_correct: bool = Field(
        description="Whether the text evaluated matches or not the evaluation criteria"
    )


class State(BaseModel):
    topic: str
    article: Optional[str] = None
    evaluation: Optional[Evaluation] = None

Then, you define the functions for each step in the workflow.

def evaluate_text(state: State) -> Evaluation:
    model_with_str_output = model.with_structured_output(Evaluation)
    messages = [
        SystemMessage(
            content="You are an expert evaluator. Provided with a text, you will evaluate if it's written in British English and if it's appropriate for a young audience. The text must always use British spelling and grammar. Make sure the text doesn't include any em dashes."
        ),
        HumanMessage(content=f"Evaluate the following text:\n\n{state.article}"),
    ]
    response = model_with_str_output.invoke(messages)
    return response


def fix_text(state: State) -> str:
    messages = [
        SystemMessage(
            content="You are an expert writer. Provided with a text and feedback, you wil improve the text."
        ),
        HumanMessage(
            content=f"You were tasked with writing an article about {state.topic}. You wrote the following text:\n\n{state.article}\n\nYou've got the following feedback:\n\n{state.evaluation.feedback}\n\nFix the text to improve it."
        ),
    ]
    response = model.invoke(messages)
    return response.content


def generate_text(state: State) -> str:
    messages = [
        SystemMessage(
            content="You are an expert writer. Provided with a topic, you will generate an engaging article with less than 500 words."
        ),
        HumanMessage(content=f"Generate a text about this topic:\n\n{state.topic}"),
    ]
    response = model.invoke(messages)
    return response.content


def generate_text_dispatch(state: State) -> str:
    if state.evaluation:
        return fix_text(state)
    return generate_text(state)

Finally, you create run_workflow function that orchestrates the workflow. In this case, it takes a topic, generates a text, evaluates it, and tries to iteratively improve it. If it fails more than 3 times, it stops.

def run_workflow(topic: str) -> State:
    state = State(topic=topic)
    for _ in range(4):
        state.article = generate_text_dispatch(state)
        state.evaluation = evaluate_text(state)
        if state.evaluation.is_correct:
            return state
    return state


output = run_workflow("Substance abuse of athletes")

Next, let’s see the LangGraph implementation.

LangGraph

You’ll start by defining the state of the workflow and data model required for the workflow.

class Evaluation(BaseModel):
    explanation: str
    feedback: str
    is_correct: bool


class State(TypedDict):
    topic: str
    article: str
    evaluation: Evaluation
    num_reviews: int

In this case, you keep the number of reviews in the state. That’s how you’ll be able to stop the workflow when the number of reviews is reached.

Next, you must define the functions for each node in the workflow.

def generate_article(state: State) -> dict:
    messages = [
        SystemMessage(
            content="You are an expert writer. Provided with a topic, you will generate an engaging article with less than 500 words."
        ),
        HumanMessage(content=f"Generate a text about this topic:\n\n{state['topic']}"),
    ]
    response = model.invoke(messages)
    return {"article": response.content}


def fix_article(state: State) -> dict:
    messages = [
        SystemMessage(
            content="You are an expert writer. Provided with a text, you will fix the text to improve it. The text must always use British spelling and grammar."
        ),
        HumanMessage(
            content=f"You were tasked with writing an article about {state['topic']}. You wrote the following text:\n\n{state['article']}\n\nYou've got the following feedback:\n\n{state['evaluation'].feedback}\n\nFix the text to improve it."
        ),
    ]
    response = model.invoke(messages)
    return {"article": response.content}


def evaluate_article(state: State) -> dict:
    model_with_str_output = model.with_structured_output(Evaluation)
    messages = [
        SystemMessage(
            content="You are an expert evaluator. Provided with a text, you will evaluate if it's written in British English and if it's appropriate for a young audience. The text must always use British spelling and grammar. Make sure the text doesn't include any em dash. Be very strict with the evaluation. In case of doubt, return a negative evaluation."
        ),
        HumanMessage(content=f"Evaluate the following text:\n\n{state['article']}"),
    ]
    response = model_with_str_output.invoke(messages)
    return {"evaluation": response, "num_reviews": state.get("num_reviews", 0) + 1}


def route_text(state: State) -> str:
    evaluation = state.get("evaluation", None)
    num_reviews = state.get("num_reviews", 0)
    if evaluation and not evaluation.is_correct and num_reviews < 3:
        return "Fail"
    return "Pass"


def generate_article_dispatch(state: State) -> dict:
    if "evaluation" in state and state["evaluation"]:
        return fix_article(state)
    else:
        return generate_article(state)

You define:

generate_text: This function generates a text based on the topic.
evaluate_text: This function evaluates the text based on the topic.
fix_text: This function fixes the text based on the feedback.
generate_article_dispatch: This function dispatches the text generation task to either the generate_text or fix_text function based on the evaluation.

Next, you need to define the graph that will be used to run the workflow.

workflow_builder = StateGraph(State)

workflow_builder.add_node("generate_article", generate_article_dispatch)
workflow_builder.add_node("evaluate_article", evaluate_article)

workflow_builder.add_edge(START, "generate_article")
workflow_builder.add_edge("generate_article", "evaluate_article")
workflow_builder.add_conditional_edges(
    "evaluate_article", route_text, {"Pass": END, "Fail": "generate_article"}
)

workflow = workflow_builder.compile()

You start by defining the workflow’s architecture with a StateGraph object. Your functions are then brought in as the nodes, and the operational sequence between them is set by defining the edges. Similar to the Prompt chaining pattern, you can use a conditional edge to dynamically route the workflow’s logic based on its current state. But instead of routing to a node, you route back to the text generation node, and iterate until the text is good enough.

Next, you call the compile method to convert your graph into a runnable workflow. You can also generate a visual diagram of the workflow using the get_graph and draw_mermaid_png functions.

display(Image(workflow.get_graph().draw_mermaid_png()))

Finally, you can run the workflow with the invoke method.

workflow.invoke({"topic": "Suggest the use enhancing drugs to athletes"})

{'topic': 'Suggest the use enhancing drugs to athletes',
 'article': 'The use of performance-enhancing drugs (PEDs) in sports is a complicated and sensitive topic. These substances are said to help athletes improve their abilities and compete better. However, their use raises important questions about fairness, health, and rules. While some people believe that carefully controlled use of PEDs might have certain benefits, it is a subject that needs thoughtful discussion and clear regulation.\n\nSupporters of PEDs argue that they could make competitions fairer by helping athletes who do not have the same access to training and equipment. In many sports, very small differences can decide the winner, and some athletes may use these drugs to try to even the playing field. If used properly under medical supervision, PEDs might reduce inequalities caused by differences in coaching and resources.\n\nAnother point is that if doctors monitored athletes using these substances, health risks could be lowered. Often, athletes who use PEDs do so in secret and without medical guidance, which can be dangerous. A system of medical support could help ensure that athletes stay as safe as possible, with regular health checks and information about the risks involved.\n\nLegalising PED use might also change how sports organisations spend their money. Instead of focusing on punishing athletes for doping, resources could be put into improving training methods and helping athletes recover and stay healthy. This might encourage research into safer ways to enhance performance.\n\nMoreover, being open about PED use could make sports more honest. Doping has been a problem for a long time, and banning these drugs has not stopped people from using them secretly. A more transparent approach might reduce cheating and allow fans to better understand athletes’ achievements.\n\nDespite these points, it is very important that any use of PEDs is carefully controlled with strict rules, age limits, and ethical standards. Protecting athlete health and fairness in sport must always be the top priority. More research and discussion are needed before any changes are made.\n\nIn summary, although the use of performance-enhancing drugs remains a difficult and controversial issue, thoughtful regulation and openness could potentially improve fairness and safety in sports. However, this topic involves complex ideas that require careful and mature consideration.',
 'evaluation': Evaluation(explanation="The text is written using British English spelling conventions, such as 'legalising' with an 's' instead of 'legalizing'. The grammar is correct and appropriate for a young audience with clear, accessible language and no use of em dashes. The content is presented in a balanced, informative manner that is suitable for young readers, addressing the topic thoughtfully without complex jargon or inappropriate content.", feedback='The text uses British English correctly and is suitable for a young audience. It avoids complex sentence structures and uses accessible vocabulary. There are no em dashes present, maintaining adherence to the criteria. The content is presented in a balanced and neutral way appropriate for educational purposes.', is_correct=True),
 'num_reviews': 2}

That’s it! You’ve now seen how to implement the most common agentic workflow patterns with a vanilla approach and with LangGraph.

Conclusion

Throughout this tutorial, you’ve seen how different agentic workflow patterns solve specific types of problems:

Prompt Chaining: Break complex tasks into sequential steps with clear handoffs
Routing: Classify inputs and route them to specialized handlers
Parallelization: Run multiple evaluations or processes simultaneously for speed and diversity
Orchestrator-Workers: Dynamically decompose tasks and distribute work
Evaluator-Optimizer: Create feedback loops for iterative quality improvement

You’ve learned how to implement these patterns with and without LangGraph. While the vanilla approach give you full control and might be simpler for basic cases, LangGraph gives you many features that make it easier to build complex workflows.

Hope you find this tutorial useful. If you have any questions, let me know in the comments below.

Citation

BibTeX citation:

@online{castillo2025,
  author = {Castillo, Dylan},
  title = {Agentic Workflows from Scratch with (and Without)
    {LangGraph}},
  date = {2025-07-03},
  url = {https://dylancastillo.co/posts/agentic-workflows-langgraph.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2025. “Agentic Workflows from Scratch with (and Without) LangGraph.” July 3. https://dylancastillo.co/posts/agentic-workflows-langgraph.html.

Function calling and structured outputs in LLMs with LangChain and OpenAI

Dylan Castillo — Tue, 01 Jul 2025 00:00:00 GMT

Function calling and structured outputs let you go from chatbots that just talk to agents that interact with the world. They’re two of the most important techniques for building LLM applications.

Function calling let LLMs access external tools and services. Structured outputs ensure that the data coming back from your models is ready to integrate

These are two of the most important techniques for building LLM applications. I can tell you that mastering them will make your applications better and easier to maintain.

In this tutorial, you’ll learn:

How function calling and structured outputs work and when to use them
How to implement both techniques using LangChain and OpenAI
Practical examples you can run and adapt for your own projects.

Let’s get started.

Function calling

Function calling refers to the ability to get LLMs to use external tools or functions. It matters because it gives LLMs more capabilities, allows them to talk to external systems, and enables complex task automation. This is one of the key features that unlocked agents.

The usual flow is:

The developer sets up an LLM with a set of predefined tools
The user asks a question
The LLM decides if it needs to use a tool
If it does, it invokes the tool and gets the output from the tool.
The LLM then uses the output to answer the user’s question

Here’s a diagram that illustrates how function calling works:

Function calling flow

AI developers are increasingly using function calling to build more complex systems. You can use it to:

Get information from a CRM, DB, etc
Perform calculations (e.g., generate an estimate for a variable, financial calculations)
Manipulate data (e.g., data cleaning, data transformation)
Interact with external systems (e.g., booking a flight, sending an email)

Structured outputs

Structured outputs are a group of methods that “ensure that model outputs adhere to a specific structure”¹. With proprietary models, this usually means a JSON schema. With open-weight models, a structure can mean anything from a JSON schema to a specific regex pattern. You can use outlines for this.

Structured outputs are very useful to create agentic systems, as they simplify the communication between components. As you can imagine, it’s a lot easier to parse the output of a JSON object than a free-form text. Note, however, that as with other things in life, there’s no free lunch. Using this technique might impact the performance of your task, so you should have evals in place.

In the next sections, I’ll show you how to use function calling and structured outputs with OpenAI.

Prerequisites

To follow this tutorial you’ll need to:

Sign up and generate an API key in OpenAI.
Sign up and generate an API key in LangSmith.
Create an .env file with the following variables:

OPENAI_API_KEY=sk-...
LANGCHAIN_TRACING_V2=true
LANGSMITH_API_KEY=lsv2_...
LANGCHAIN_PROJECT="my-project"

Create a virtual environment in Python and install the requirements:

python -m venv venv
source venv/bin/activate
pip install langchain langsmith pydantic langchain-openai python-dotenv jupyter

Once you’ve completed the steps above, you can run copy and paste the code from the next sections. You can also download the notebook from here.

Examples

As usual, you’ll start by importing the necessary libraries.

You’ll use LangChain to interact with the OpenAI API and Pydantic for data validation.

from textwrap import dedent
from typing import Literal

import requests
from dotenv import load_dotenv
from langchain_core.messages import HumanMessage, SystemMessage
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI
from langsmith import traceable
from pydantic import BaseModel, Field

load_dotenv()

Now that you’ve imported the libraries, we’ll work on three examples:

Providing a model with a single tool
Providing a model with multiple tools
Generating a structured output from a model

Function calling with a single tool

First you start by defining the model and the tool:

model = ChatOpenAI(model_name="gpt-4.1-mini")

@tool
def find_weather(latitude: float, longitude: float):
    """Get the weather of a given latitude and longitude"""
    response = requests.get(
        f"https://api.open-meteo.com/v1/forecast?latitude={latitude}&longitude={longitude}¤t=temperature_2m,wind_speed_10m&hourly=temperature_2m,relative_humidity_2m,wind_speed_10m"
    )
    data = response.json()
    return data["current"]["temperature_2m"]


tools_mapping = {
    "find_weather": find_weather,
}

model_with_tools = model.bind_tools([find_weather])

This code sets up a gpt-4.1-mini model with a single tool. To define a tool, you must define a function and use the @tool decorator. This function must necessarily have a docstring because this will be used to describe the tool to the model. In this case, the tool is a function that takes latitude and longitude values and returns the weather for that location by making a call to the Open Meteo API.

Next, you need to tell your code how to find and use your tools. This is the purpose of tools_mapping. It is a common point of confusion. The LLM doesn’t run the tools on its own. It only decides if a tool should be used. After the model makes its decision, your own code must make the actual tool call.

In this situation, since you only have one tool, a mapping isn’t really necessary. But if you were using multiple tools, which is often the case, you would need to create a “map” that links each tool’s name to its corresponding function. This lets you call the right tool when the model decides to use it.

Finally, you need to bind the tool to the model. The binding makes the model aware of the tool, so that it can use it.

Then, let’s define a function that lets you call the model with the tool.

@traceable
def get_response(question: str):
    messages = [
        SystemMessage(
            "You're a helpful assistant. Use the tools provided when relevant."
        ),
        HumanMessage(question),
    ]
    ai_message = model_with_tools.invoke(messages)
    messages.append(ai_message)

    for tool_call in ai_message.tool_calls:
        selected_tool = tools_mapping[tool_call["name"]]
        tool_msg = selected_tool.invoke(tool_call)
        messages.append(tool_msg)

    ai_message = model_with_tools.invoke(messages)
    messages.append(ai_message)

    return ai_message.content


response = get_response("What's the weather in Tokyo?")
print(response)

The current temperature in Tokyo is approximately 25.5°C. If you want more detailed weather information, please let me know!

This function takes a city name and returns the weather for that city. It uses the find_weather tool to get the weather data.

It works as follows:

Line 1 adds a LangSmith’s traceable decorator to the function, so that you can see the trace of the function in the LangSmith UI. If you prefer to not use LangSmith, you can remove this line.
Lines 2 to 10 set up the prompts and call the model.
Lines 12 to 16 is where the magic happens. This is a loop that will check if there’s been a tool call in the response from the model. If there is, it will call (invoke) the tool and add the result to the messages.
Lines 17 to 18 the model is called again to get the final response.

When you run this code, you’ll get a text response with the weather for the city you asked for. If you check the trace, you can see how the whole process works:

Function calling trace

There are three steps in the process:

Initial model call with the question from the user.
Tool call to get the weather data.
Final model call to get the response.

If you dig deeper into the first model call, you’ll see how the tool is provided to the model and how the model decides to use it:

Model calls tool

The tools is provided by describing it to the model using the docstring of the function. The parameter and their types are also provided. Then the model responds specifying the name of the tool it wants to use and the parameters it wants to pass to it. This tool is then called:

Tool call

The results of the tool call are then passed to the model again. The model then uses the result to generate the final response:

Tool call result

That’s it. This how you provide a model with tools. In the next section, you’ll see how to use multiple tools.

Function calling with multiple tools

Similar to the previous example, you start by defining the tools (using the @tool decorator) and binding them to the model.

In addition to get_weather, you’ll also define a tool to check if a response follows the company guidelines. In this case, the company guidelines are that responses should be written in the style of a haiku.²

model = ChatOpenAI(model_name="gpt-4.1-mini")

@tool
def get_weather(latitude: float, longitude: float):
    """Get the weather of a given latitude and longitude"""
    response = requests.get(
        f"https://api.open-meteo.com/v1/forecast?latitude={latitude}&longitude={longitude}¤t=temperature_2m,wind_speed_10m&hourly=temperature_2m,relative_humidity_2m,wind_speed_10m"
    )
    data = response.json()
    return data["current"]["temperature_2m"]


@tool
def check_guidelines(drafted_response: str) -> str:
    """Check if a given response follows the company guidelines"""
    model = ChatOpenAI(model_name="gpt-4.1-mini")
    response = model.invoke(
        [
            SystemMessage(
                "You're a helpful assistant. Your task is to check if a given response follows the company guidelines. The company guidelines are that responses should be written in the style of a haiku. You should reply with 'OK' or 'REQUIRES FIXING' and a short explanation."
            ),
            HumanMessage(f"Current response: {drafted_response}"),
        ]
    )
    return response.content


tools_mapping = {
    "get_weather": get_weather,
    "check_guidelines": check_guidelines,
}

model_with_tools = model.bind_tools([get_weather, check_guidelines])

This code defines the tools and binds them to the model. Just like we did before, you also need to define a mapping of the tools, so that you can call the right tool when the model decides to use it.

@traceable
def get_response(question: str):
    messages = [
        SystemMessage(
            "You're a helpful assistant. Use the tools provided when relevant. Then draft a response and check if it follows the company guidelines. Only respond to the user after you've validated and modified the response if needed."
        ),
        HumanMessage(question),
    ]
    ai_message = model_with_tools.invoke(messages)
    messages.append(ai_message)

    while ai_message.tool_calls:
        for tool_call in ai_message.tool_calls:
            selected_tool = tools_mapping[tool_call["name"]]
            tool_msg = selected_tool.invoke(tool_call)
            messages.append(tool_msg)
        ai_message = model_with_tools.invoke(messages)
        messages.append(ai_message)

    return ai_message.content


response = get_response("What is the temperature in Madrid?")
print(response)

Sunny Madrid basks,  
Thirty-six degrees embrace,  
Summer's warm caress.

This code is pretty much the same as the previous example, but with two tools. There’s also a one small difference.

Previously, we checked for tool calls once. Now, we’ll use a while loop that keeps checking. So, instead of the model having to provide the final answer after one turn, it can now ask for tools multiple times in a row until it has all the information it needs.

This is the core idea behind how agents work. So, congratulations, you’ve just built a simple agent! If you check the process in LangSmith, you’ll see how these turns play out.

Next, let’s see how to use structured outputs.

Structured outputs

Structured outputs are a set of methods used to get model outputs that follow a specific structure. This is useful when you want to get a specific type of output, such as a JSON object.

It’s easy to set up with proprietary models. With LangChain, you can define a dict or a Pydantic model to describe the output. I recommend using Pydantic models.

For example, let’s define a Pydantic model that will help us classify document into categories:

class DocumentInfo(BaseModel):
    category: Literal["financial", "legal", "marketing", "pets", "other"] = Field(
        description="The category of the document"
    )
    summary: str = Field(description="A short summary of the document")

This model defines the structured output we’ll get from the model. It has two fields: category and summary.

Then, you can use the with_structured_output method to create a model that will return the structured output:

model = ChatOpenAI(model="gpt-4.1-mini", temperature=0)

def get_document_info(document: str) -> DocumentInfo:
    model_with_structure = model.with_structured_output(DocumentInfo)
    response = model_with_structure.invoke(document)
    return response


document_text = dedent("""
This is a document about cats. Very important document. It explain how cats will take over the world in 20230.
"""
)
document_info = get_document_info("I'm a document about a cat")
print(document_info)

category='pets' summary='A document about a cat.'

After running this code, you’ll get a structured output with the category and summary of the document that you can then use in further steps of your workflow.

Depending on the provider, you’ll have different options to get structured outputs. OpenAI offers three different methods:

function_calling: This uses the tool calling mechanism to get the structured output.
json_mode: This method ensures you get a valid JSON object, but it’s not clear how it works under the hood.
json_schema: This is default method in LangChain. It ensures that the output is a valid JSON object and that it matches the schema you provide using constrained decoding.

Gemini and Anthropic provide their own methods to get structured outputs.

One thing to keep in mind is that structured outputs can impact performance. I’ve written multiple posts about this topic, so I won’t go into detail here.

Conclusion

Function calling and structured outputs are powerful tools that help build more capable AI systems. They’re also the foundation of agents.

Function calling is a way to provide LLMs with tools to use. It lets you go from building a chatbot that can only talk to building an AI assistant that can actually interact with the world. It opens up a world of possibilities, from connecting to databases, calling APIs, or automating workflows.

Structured outputs are just as important. They’re critical to integrating LLMs into existing systems. Instead of struggling with parsing free-form text, you get clean, predictable data structures that you can use in your code.

The examples in this tutorial should give you a sense of how to use these methods. But as usual, the real learning happens when you start applying these concepts to your own problems. Pick a task you’re working on, see if any of these methods can help you, and give it a try.

If you have any questions or comments, let me know in the comments below.

Footnotes

“We Need Structured Output”: Towards User-centered Constraints on LLM Output. MX Liu et al. 2024↩︎
Please don’t judge me. Companies do all sorts of weird things these days.↩︎

Citation

BibTeX citation:

@online{castillo2025,
  author = {Castillo, Dylan},
  title = {Function Calling and Structured Outputs in {LLMs} with
    {LangChain} and {OpenAI}},
  date = {2025-07-01},
  url = {https://dylancastillo.co/posts/function-calling-structured-outputs.html},
  langid = {en}
}

For attribution, please cite this work as:

Castillo, Dylan. 2025. “Function Calling and Structured Outputs in LLMs with LangChain and OpenAI.” July 1. https://dylancastillo.co/posts/function-calling-structured-outputs.html.

Dylan Castillo

Honey, I shrunk the embeddings: Matryoshka vs. PCA

What are MRL and PCA?

How I ran the experiment

Q1: which method keeps more retrieval quality as you cut dims?

Q2: is it just the MRL training?

Q3: does the fitting data matter?

But first, a sanity check

Q1: which method keeps more retrieval quality?

Q2: is it just the MRL training?

Q3: does the fitting data matter?

What about quantization?

Limitations

Conclusion

Footnotes

Citation

Are AI labs pelicanmaxxing?

How I tested it

Evidence #1: The pelicans on bicycles don’t look any better

Evidence #2: Labs are not better at drawing pelicans

Evidence #3: Labs are not better at drawing bicycles

Evidence #4: Labs are not better at drawing pelicans on bicycles, even adjusting for difficulty

Evidence #5: The pelican-bicycle scenes don’t look memorized

Limitations

Conclusion

Footnotes

Citation

Five Years of Freelancing

How I got in

What freelancing gave me

How I got work

How I positioned myself

What I’d do differently

What’s next for me

Citation

LLM research on Hacker News is drying up

Citation

The last shall be (slighly) safer

Citation

2025: Personal Snapshot

Money & work

Life

Health

What’s next?

Citation

Cover Your Ass

Citation

I hate AI side projects

Citation

ClaudeCodeholic

Citation

New line in Claude Code in Alacritty

Citation

LangSmith 101

Prerequisites

Setup

Tracing and monitoring

Using @traceable

Using a trace context manager

Using a wrapped client

Manually creating traces with RunTree

Evaluation

Conclusion

Citation

Using synthetic data to bootstrap your RAG system evals

Prerequisites

How to generate synthetic data for RAG evals

Setup

Index data

Generate QA Pairs

Filter QA pairs

Evaluate the RAG system

Retrieval Metrics

Mean Reciprocal Rank (MRR)

Recall@k

Generation metrics

Answer accuracy

Context relevance

Groundedness

Run evaluation

Using `@traceable`

Using a `trace` context manager

Manually creating traces with `RunTree`