Building the Machine: From Concepts to Agents, Tools & Workflows

Stay curious. I believe curiosity is the beginning of every meaningful breakthrough. — Marvette Johnson

I had finally gotten my head around what made an AI system agentic. An agent could perceive its environment, reason, plan, act, observe what happened, and decide what to do next.

Okay. I got it. Or at least I thought I did.

Because understanding what an agent is and understanding how to build one are two very different things.

And that's where the program was headed next.

The third course, Designing and Building Agentic Systems, was the practical implementation phase of the journey. We were moving beyond the conceptual foundations of agentic AI and into the architecture, tools, memory, reasoning mechanisms, and workflows that make an agentic system actually work.

In other words, it was time to start building the machine.

And before I could build it, I needed to understand how all of its pieces fit together.

How do you get an agent to think ahead?

We explored several ways an agent could approach a problem, from Chain of Thought and ReAct to more complex approaches such as Tree of Thoughts, Graph of Thoughts, and Reflection/Reflexion.

In other words, there wasn't just one way to get an agent to reason.

There were different ways to structure the thinking.

One approach was essentially: make a plan, then execute it.

That sounded perfectly reasonable. Until we introduced the possibility that reality might not cooperate.

What happens when the plan hits an unexpected condition?

That's where adaptive ReAct became interesting. Rather than blindly following the original plan, the agent could use what it observed to decide what to do next.

I started to understand that intelligence isn't just having a plan. It's knowing what to do when the plan doesn't work.

Okay. If an agent is going to adapt when things don't go according to plan, what does it need to remember, and what does it need to be able to do?

Give the brain a memory and a toolbox

If an agent is going to adjust its actions based on what it observes, it needs to keep track of what it knows.

And if it's going to actually do something with that knowledge, it needs the tools to do it.

Okay… but what does the brain need to remember? And what does it need in its toolbox?

We explored the difference between short-term memory, which helps maintain the current context, and long-term memory, which can persist information beyond the immediate interaction.

An agent needs to keep track of what's happening now, but some systems also need a way to retain information from previous interactions or experiences.

And then came the tools: code interpreters, web search, calendar routing, databases, even tools that can interact with a computer.

The brain wasn't just getting hands anymore. It was getting a toolbox.

Okay… but how does it know which tool to use when, what to do first, and what to do next?

From chains to graphs

But there was still a problem.

How do you control what happens next?

I was beginning to see that connecting a few tools together wasn't necessarily the hard part. The harder question was what should happen if something changes along the way.

That brought us to orchestration.

We moved from thinking about workflows as a simple sequence of steps to thinking about them as a graph, with state at the center.

Instead of forcing the system down one predetermined path, the workflow could respond to what was happening in the moment.

Suddenly, the workflow wasn't necessarily a straight line. It could loop.

And that was a pretty big shift in my mental model.

When the system has to figure things out

Now I could see how a workflow could change course.

But that raised another question: what tells the system it needs to change course in the first place?

This brought me back to something I had first encountered in the first course: Retrieval-Augmented Generation, or RAG.

I understood the basic idea: retrieve relevant information and use it to help an LLM generate a more grounded response.

But now we were looking at RAG differently.

What if the first retrieval wasn't good enough?

Instead of simply accepting whatever came back, the agent could evaluate the quality of what it retrieved.

If the information wasn't sufficient, the workflow could trigger a recovery path: refine the search, retrieve again, evaluate again, and continue until the result met the required standard of factual grounding.

That was different.

This wasn't simply a retrieval system anymore. The system could evaluate what it found, recognize when it wasn't good enough, and decide what to do next.

And suddenly, that loop we'd just been talking about made a whole lot more sense.

It's one thing to understand it. It's another to build it.

By this point in the program, I had spent weeks learning about reasoning, planning, memory, tools, state, workflows, orchestration, and ways for an agent to adapt when things didn't go according to plan. I had a much better understanding of how the pieces fit together.

I understood the pieces. I understood how they were supposed to work together.

And then came the project. I had to plan, design, and build one.

Gulp!

That was my first thought when I realized what was coming.

Let me be clear: this was not my first project. Throughout the program, we had been doing hands-on exercises, activities, and projects in each course and module. I had been practicing what I was learning all along.

But this was different. This project, our second graded project of the program, was the BIG one. This was where I had to take everything I had been learning and actually use it.

Suddenly there was a lot to prove, to myself more than anyone else.

And for the first time in a while, that familiar little voice showed up: What in the world am I doing?

Yes, I'd learned enough to know how much I didn't know, and now they wanted me to "show my work"?

Don't get me wrong, I wasn't looking for an escape hatch this time. That door was already closed and locked. And I wasn't looking to phone a friend.

I had spent weeks pulling on individual threads. Now I had to figure out how to weave them into something that actually worked.

That was scary!

I had to remind myself of something I've been trying to practice throughout this entire journey: stay curious. That includes being willing to embrace the unknown.

So I took a breath, put on my imaginary lab coat, grabbed my metaphorical jumper cables, and said to myself, "Okay, Marvette, let's build that machine. To the lab!"

Gulp!

In the lab!

As I stared at my Jupyter notebook, my mind went straight to Young Frankenstein. "Abby Normal." I thought to myself, I hope this brain I'm building doesn't turn out to be A.B. Normal. 😂

Jokes aside, that thought turned out to be strangely relevant. Building an agent isn't simply a matter of connecting a language model to a few tools and setting it loose. The choices we make about its goals, reasoning, tools, constraints, and decision-making all matter.

When things don't go according to plan

Of course, things didn't go according to plan.

For our second project, we built an Enhanced Agent, an expanded version of our financial research agent that brought together multiple tools, RAG-powered research, and LangGraph orchestration to analyze companies from both a financial-performance and innovation perspective.

One of the first challenges I encountered was getting the agent to do all the things I had asked it to do without taking forever to do them.

When I tested more complex investment-analysis questions, the Enhanced Agent could take a very long time to execute. It wasn't necessarily doing one thing slowly. It was doing a lot of things: invoking multiple tools, retrieving large amounts of information from the vector database, and processing increasingly extensive prompts.

And then there was the context.

How much information is enough? More importantly, how much is too much?

I had assumed that giving the model more context would naturally make the analysis better. After all, if the agent had more information available, wouldn't that help it make a better-informed decision?

Not necessarily.

So I started experimenting with chunk sizes, retrieval parameters, prompt construction, and agent execution settings to find a better balance.

This was where something I had been learning throughout the program started to become much more tangible: more isn't always better.

More context can provide richer information, but it also increases the amount the system has to process, and that can affect how efficiently the agent performs.

I was beginning to see that building an effective agent wasn't simply about giving it more information, more tools, or more instructions.

It was about making deliberate choices about what it needed, when it needed it, and how much was enough.

That was a very different way of thinking about AI than the one I had started the program with.

Wait… now I see it.

As I worked through those performance issues, something else started to click.

I had learned about tools. I had learned about RAG. I had learned about reasoning and planning. I had learned about state and graphs.

But this was the first time I could see them working together as a system.

The agent wasn't simply running a predetermined sequence of steps. Through LangGraph, it could orchestrate different tools within the workflow: select and run tools based on the question, gather information from multiple sources, and bring those results together into a response.

That was the moment the architecture started to make sense to me.

I had spent so much time learning what each piece did that I hadn't fully appreciated what happened when the pieces began working together.

The agent was no longer just a language model with some tools attached. It was a system.

Now I wasn't just asking, "What can this tool do?"

I was starting to ask, "What should the system do next?"

It's alive! ⚡

And then it happened.

It worked. It actually worked.

I stared at the output for a second and thought, "It's alive! ⚡"

Okay, maybe not quite alive.

But it was working.

And for someone who had started this program wondering whether she had any business opening a Jupyter notebook, that felt like a pretty big deal.

What building an agent taught me

By the time I finished the project, I realized I had learned something that wasn't really about financial analysis at all.

I had learned the difference between knowing the pieces and designing how the pieces work together.

I could give an agent access to multiple tools, but that didn't mean it would use every tool. The agent had to determine which tools were relevant based on the question and its reasoning process.

That meant the workflow itself mattered.

The architecture mattered.

And suddenly, concepts like state, orchestration, conditional logic, and graph-based workflows didn't feel like abstract technical vocabulary anymore.

They were design decisions.

I was beginning to understand that building an effective agent wasn't just about making individual components work.

It was about designing the system that connects them.

That was a different kind of thinking, and one that felt surprisingly familiar to me as an instructional designer.

I wasn't simply creating a collection of learning objects.

I was thinking about the system, the flow, the decisions, the conditions, and what happens next.

Looking back

Looking back at the project, I could already see how much further the system could go.

More financial data sources. Better retrieval. More sophisticated monitoring and tracing. Better ways to manage tool usage and performance.

What started as a course project could, with enough development, become something much more comprehensive.

But what interested me most wasn't where the project could go. It was where my own thinking had gone.

I had started this program wanting to understand what was under the hood. Now I had actually built something, and I was already wondering how to make it better.

And then another question popped into my head: "If this is what one agent can do, what happens when you have more than one?"

Apparently, I wasn't done pulling on the thread. 😂

Up next in this series: When One Agent Isn't Enough.