0:00
This video is sponsored by FreelyGive. If you want to help have help with Drupal and AI, get in contact with us or in AI in general.
0:09
So, in this video, I will showcase how you can get your Drupal CMS assistant or basically anything to search for uh images on your website. So, I'm just going to go ahead here and ask my
0:22
assistant, "Can you find images of someone with white beard?" So, not santa.jpeg and not in the old text that says Santa Claus, but it's still going to find this
0:37
santa.jpeg image then. This is available now as a recipe and it works. There's two recipes. One recipe if you can't install the vector database and another one if you can install the
0:51
vector database. So, let's go through it. So, the video will be split up into two parts. I will talk a little bit about the the recipes and what they actually install on your website. And the second
1:05
part will just show how you install it and how you can set it up with an assistant for instance because it is just in the end installing one agent that is supposed to
1:14
mainly be a sub agent for other other agents. So, there are two agent recipes. There's a image search agent for database search and there's a image search agent for
1:26
vector search and those in their turn are using another recipe. So, one for image search for DB uh and one for image search for vector and both of those in the end are then
1:41
using this the AI image classification recipe that takes care of classifying uh actual content. So, it's a three-level uh structure where the agent is the um well, it's the end goal, so to speak,
1:59
but you can, of course, if you want, you can just uh do the image classification recipe, uh and that helps with making the media search, the internal media search, a lot
2:12
better. Uh it also adds a taxonomy, so you can get categorize the actual media. On top of that, then, is two different uh um search recipes, and what those does is
2:26
that they take the classification, the extra fields that the classification adds. So, it adds a description of the image, it adds a taxonomy, and it indexes this in a search uh API index.
2:39
And the difference is then that this is using the normal database search, and this is using a vector search.
2:46
And then, in the end, we have a agent wrapped around it. So, a agent that can use a tool that searches in either the database search or searches in the vector search,
2:57
and they have a little bit different system prompts. So, mainly the difference between a normal traditional uh uh search and then vector search or semantic search
3:08
is that in traditional search, you have to find the search words, and you have to use those search words and try to find have the agent read it and kind of find a meaning in it, while semantic
3:20
search, you don't actually need to fit uh uh hit on the actual search word. You just have to hit on the meaning or something similar to what what you're looking for.
3:32
Uh so, they work a little bit different, but what is important to know is that if you're using the database search one, the only thing you need uh prepared on your website
3:45
is that you have set up a AI provider that can do chat with tools, and you have set this up as the default chat with tools model.
3:53
That is the only requirement. So, we actually have in our recipes a check. It will check this before it actually starts applying.
4:01
Uh and tell you something you know, something is wrong. You haven't added this. So, you need to add this. But, this also means that it's very flexible because you can choose your own model.
4:11
You can choose your own provider. If you want to use Ollama locally for this, no problems. If you want to use OpenAI and Anthropic, no problems as well.
4:19
If you want to you know, use more enterprise-y like AWS or Azure, that's no problem as well. You can choose.
4:28
It gets a little bit more complex with the vector search because you need a vector search engine.
4:34
And you need also a embeddings embeddings provider. And please note for instance that Anthropic, while it's a very good chat provider, it doesn't actually provide
4:46
embeddings. This means that you need another provider for this. If you're using OpenAI, it can take care of both embeddings and the chat provider.
4:55
Uh but then you also need a vector database. So, you need something like Milvus or PostgreSQL with pgvector, Pinecone. We have a lot of choices you can find there as well. So, you just
5:06
need to install one of these and it's the same thing there. We have a setting under everything around default operation types, we also have a a setting for a default vector database
5:17
and a default embeddings provider. So, you need to set those up before you can actually run this specific recipe.
5:25
So, let's get started. Okay. So, I I just installed Drupal CMS and the AI assistant, but of course you can run with core and any provider that has chat with tools,
5:39
basically. But, this makes sure that well, I have to reload here. So, if I go to Let's see here. Where are we?
5:51
AI AI infrastructure and default models for AI operations, you will be able to see then uh that this sets up a chat with tools function calling by default. So, most
6:05
most popular providers actually will set this up when you you set up when you set them up give API key and so on. But, of course, as I said, you can use whatever provider you want in this case.
6:18
You don't have to use Drupal CMS. This is just uh to give example because what I want to do is have this assistant here. So, uh no, let's show here. So, here is all
6:28
the agents I have, right? So, uh what I want to do is here run the assistant and ask it something like uh what content types >> [snorts] >> do I have?
6:44
And this will then go ahead and uh call the content type agent and give me answer. There's utility page, right? So, what we want to do now is to install the recipe.
6:57
And that is just done via Well, first you have to run the composer command to get it. And then you just run Drush recipes to run it. And that's basically it. After that,
7:10
you can start searching for images depending on what you basically want to do. So, uh Well, I will show here what's going to happen, what we're going to do while this is uh running. Oh, one thing
7:28
I can say as well because we're using Drupal CMS is 15 steps because this is using Drupal CMS media uh recipe uh on top of the three recipes that's running, which means that there's less
7:39
steps than if you do it on core. We're going to go ahead and add a media here. We're just going to upload an image.
7:46
Uh let's do so. And uh I'm going to take a picture of something I don't know. Let's just take a picture of a family here.
8:01
And I'm just going to write something else just to make sure that so you can see I it's we're not getting this from the Alt text then.
8:11
Now you're going to see it takes a little bit longer to save than normally. So what's happening is here is that it's going to be looking at the image.
8:20
It's going to describe it. It's going to categorize it. So you we get this right.
8:26
And it's going to um also index it. So we have these two new fields. They came from the recipe. They have been filled out. If you happen to have medias, you can just resave the
8:38
medias and it's going to fill out. Uh So if we go to search API, we just show that yeah, we have a new index here. It has filled out one thing.
8:51
So now we want to go ahead and try this out. We're just going to go to extend. I'm just going to install two uh two different module here.
9:02
So uh you shouldn't really be using Agents Explorer if you do something complex.
9:06
There is AI Agents Debugger contrib module that you should be using. It just gives so much more information. In this case, this is a really stupid agent. So while this is loading, if we just reload
9:18
here, you're going to see that all of a sudden image search agent exists. We can just click edit on this so you can see what the agent actually is that we have installed.
9:28
So there's just a system prompt. It looks a little bit different between the um database agent and the the vector based agent, but they have the same tools and so it has a tool basically that forces
9:43
that you use this index that we're using forces Yeah, I'm going to show how the tool works. So it forces some values when using this tool to make sure you can't
9:52
prompt injected to look into other search indexes. You will also see when you read the description. This is very much a helper agent. It's supposed to be used by
10:03
assistants or other parent agents. So that's why it has a description where it tells the parent agent, you have to tell me how to how I should query and what you want back, right?
10:14
So now when we have installed this, we can just go to AI search and discovery. There's AI API Explorer. There's a lot of different explorer for everything that your providers can do
10:26
and there's tool explorer. So these are the tools that agents were uses. MCP could be using, ECA could be using and there we have them the search index.
10:36
We just search for it. We say we want to search this index and search for family in this case.
10:42
You see there's a lot of other settings here. The important one it's so that the agent can basically, you know, look for hundred of images paginate paginate and
10:52
so on. There is check access. If you know that your media library is completely public, you can tell the agent to not check access. This makes it more performance so it doesn't have to do
11:03
access check on anything. Uh but we're just going to run this tool and you're going to see it find something, right?
11:12
Uh now uh a family is a happy moment, right? If I search for that, most likely, yeah, I don't find anything. So this is the main difference between a vector search
11:25
uh uh no yeah, vector semantic search and normal search, right? Where it's just searching for the actual words.
11:34
Um So, that's how the tool works. So, now we can actually go back here and try the agent. So, we're going to click on explore. This shows up because I installed the explorer and then we can
11:47
prompt it. Can you find images of a family? Let's try a family of three. It might suggest this image anyway and tell it it's a little bit wrong.
12:00
See here. So, it's using the tool to search for family of three and then it's uh searching for family of three, family three parents and child. Gives back a response.
12:12
Uh I searched sites but not that I should be doing the family of three. The only match returned a family portrait of five. So, it actually didn't give it back because I was very specific. Let's
12:23
do instead can you find a images of a family then? Then it will search for family and give back a response right away that it found this image.
12:34
Uh this is the data we tell it to give back by default but it can also give back other uh information.
12:43
So, what we're going to do now as the last step to get it working with a Drupal CMS agent. It's supposed to be working with the CMS agent, that's why we built it so you can it can have the
12:53
CMS agent find images for the pages you're building. Uh it's going to be a recipe that also attaches to the CMS agent in the future so you don't have to do anything.
13:03
But I'm just going to showcase in this case how you can connect it then to the Drupal CMS agent. So, if you go to automation, Drupal assistant configurations and the Drupal CMS
13:13
assistant we're just editing this and we're just saying it can use the image search agent as well.
13:20
And in the instructions we're just adding image search. If someone asks for the for finding finding images use the image search agent and return and ask it for the URL
13:47
and the edit link of media and return this. So very simple, right? So now we can actually use this for this and we can search here then. Can you find me images of families?
14:08
And hopefully now it's going to contact the image search agent. The image search agent is going to go out and search and give back a response.
14:16
That hopefully has two links then, one to the image and one to the edit page. Yeah. So it says here is the photo.
14:28
Well, we can tell it to make this maybe a little bit nicer. But it works, right? It goes to the edit page and you all of a sudden now you have something that can actually go
14:38
ahead and search all the images for you. Yeah, that's it. Try the recipe out. It's currently in dev. It's going to be in stable any day now. It's being reviewed.
14:50
Thank you.
