In this episode we will continue showcasing the AI module. We will do a kind of Hello World for Drupal AI and AI in general.
We will summarize an article, and we will make sure to do it locally on machines that many people have access to.
We will also showcase the AI API Explorer to autogenerate code for a specific call.
0:00
as always these videos are sponsored by FreelyGive FreelyGive pays my salary and I can do a lot to the AI Community for this um we have Drupal and AI experts not just me but other people so
0:13
if you need help please get in contact with us so another video about the Drupal AI module and what it can do uh and help you with um before I start going into what I want to showcase I
0:26
just wanted to say because I didn't mention it yesterday this is still very much in Alpha hopefully the actual API that you as a developer work with should not change much so the internals might
0:37
change but not that um what I'm going to Showcase today is a very uh like a hello world example of the of AI uh so what we're going to use is the Drupal a module we're going to use something
0:51
called the AI uh API explorer that comes with the AI module and we're going to use Ollama and Ollama then is a rapper service that starts a service um for llms that you can host locally
1:06
which means that you're not going to need a a third party service or something like that you're not going to need to pay for this and I'm going to run a model called Gemma that runs on
1:15
any computer that has 16 gab of ram so I will just showcase this first now I am running this on Linux so how this setup works on Windows and Mac might differ especially on Windows uh but the general
1:29
idea with a command for lamb and so on is the same so you will see I have 16 GB of RAM if I run this nvidia-smi I don't have a graphic card in this case so we're just going to start by downloading
1:44
Ollama so this is different my Goos windows it's just executables that you install for uh Linux we have to do this um and while that's downloading I'm going to talk about what we're going to
1:56
solve today so we had this article that we created an image for yesterday that's about food uh hoarding and I now I just want to create a summary around this so
2:04
I have already created uh on my article manage Fields I have created a summary field right uh here I just have some pseudo code that I've written so if on pre-save of the node if the node type is
2:19
article then the field summary should be filled with some AI summary right that's what we're trying to solve uh so to be able to do that we can start installing the modules already
2:33
what we need is the AI Core module so that is installed already we're going to need the AI API Explorer uh to basically test out the API and we're going to need the
2:45
providers well that is AMA so we install those three things so now we have Ollama here it's going to start as a service I guess so I have to stop this this is a typical uh
3:00
well it's not even typical Linux it's typical systemd that doesn't exist on all Linux systems so I'm going to show you the pure command here instead so you can do ollama serve so now I stopped the
3:13
service um but you can do ollama serve and this would start the service the problem now then is that I'm running ddev and ddev runs in Docker and Docker uh is basically running its own virtual
3:27
Network and this only listens to local host which is problematic because in this case my DD machine is not Local Host so it will not be allowed to uh connect to this so I have to do this
3:40
lost equals 0.0.0.0 meaning that it listens to the world uh and after that I have to download the model that we want and in our case we want Gemma because Gemma is
3:53
something that we can run on 16 gab of RAM and that you can usually see in the the parameter size so you see this is 2 billion parameters very very small it's not very very uh smart of course but
4:06
hopefully enough for a summary or kind of a summary so I would just write ollama pull gemma to B so it's pulling in the model this is not built into the web interface in
4:17
Drupal yet it's hopefully going to be in the future uh I already installed this so it's already there so now I have done everything I need to do on uh online because I got everything I need need so
4:30
I'm just going to Showcase something I will just turn off the actual um Wi-Fi here so if I try to reload this page doesn't work I don't have internet anymore right um so now I just need to
4:43
set this up this is configuration AI settings provider settings for every provider that you're going to use uh Port is 11434 that's the default Port you can open on any using this uh uh command I
4:57
showed you and then you have to use this if you're inside a Docker container when you want to talk to your actual physical machine so that's host.docker.internal
5:06
and this connects to your physical machine so the actual host of Docker and after we set this up there is then under AI settings AI API explorers and these are explorers where you can
5:17
try making calls to the different providers that you have set up uh doing different operation types so with Ollama we can do embeddings and chat generation that's what it supports currently
5:30
so we're going to use the chat generation Explorer choose our provider choose our model can use some settings here if you want but we keep it as it is because it's Gemma it doesn't take a
5:41
system actually it would throw a error in this case and then I can just ask here who are you I'm the user and I'm asking who are you and this if we go back here you will
5:53
see it's going to log that a call was sent here yeah and then it's answering yes has a large language machine developed by Google blah blah blah and this happens without internet now right
6:05
so I'm just running this on my machine but what's cool here now this is the call I want to make right I can just go to normalist code example and copy this and this is the code I need to
6:17
create the exact same call uh and what I can do then the or let's go through this first so uh it first sets up the config this is default values we could actually not
6:32
do that but I I keep it there then it normalizes the input so you could also have unnormalized example here where it's less code because you can actually you know just put it in directly but the
6:47
good thing with doing normalized and as I talked about in the last episode is that you can do switch over to OpenAI for instance if you want that in the future when you think that okay Gemma is
6:57
not good enough um and okay these are using full name spaces you should usually put those up into use there right I'm just going to fix that but that's how it has to be in
7:13
this example code of course uh so it's creating a normalized input in this case of course we have to do this um summarize this article in one sentence just answer with the
7:32
summary and then we're going to do uh strip tags because it might not take HTML some models doesn't like that um we're probably going to have helper functions for this later depending on
7:46
model what you can and can't have because that's different for different mod models so anyway we're creating the instance of Ollama we're setting the configuration and then we're just doing
7:56
the call to Gemma 2B in this case we don't need to to tag it if you're going to make a contributor module please tag stuff because that means that uh anyone listening to events so they want to do
8:07
something before the request is sent or after the request is done can search for a specific tag or in the logs you can filter on these tags right the tags from module that's being used uh provider and
8:22
and operation type is set automatically then we get our normalized response and the normalized response in this case then is is um is uh uh chat message so we can just
8:36
open that it should be a helper message here if you use the VAR it would autocomplete because I'm using the Drupal pig Drupal static uh object here it doesn't actually autocomplete unless
8:47
I'm telling it what it is I'm getting in this case um so uh the response is under response get message message then right so we just do this function and we're going to put this into the field summary
9:04
and that's it now we go to our content I just show you so you see there is no summary press save oh that was a little bit too fast what did I do wrong now oh maybe this module is not
9:25
installed uh yeah AI summary local so so I'm I am having this test module it it's good to install it if I'm going to run my code I didn't think of that uh so we're just
9:39
going to edit and save it and hopefully it takes yeah now it starts thinking and then hopefully Gemma is not the smartest model so sometimes it answers like sure here's a summary and
9:51
then the summary which is of course not what you want there that's why I asked it specifically just answer with the summary but it's not the smartest model and this is what you can get without
10:01
having a graphic card or a third party summary so yeah unfortunately it didn't understand that but still you see it actually created something and this is actually something that you then can
10:13
have on your servers uh that you don't need a very expensive GPU or a third party service for and it doesn't need to leave internet uh or go out on public internet
10:25
so you can keep your data safe um and that's it thank you for watching
