0:00
This video is sponsored by FreelyGive. If you need help with Drupal AI, get in contact with us. First of all, happy new year. I'm going to showcase a video now
0:10
that you might feel like, okay, this is definitely not something new. I have seen all this before. Just wait 2 minutes and I will come to the kicker with this. So, this is a restaurant
0:21
review site. It has like a review title, image, some ratings you have to fill out, categories, uh, address, and homepage. I just set this up in like half an hour to showcase
0:35
something. Uh, I'm just going to paste in something I created with ChatGPT, but it's based on what I feel about Mustafa's Gemüse Kebap. If you ever go to Berlin, don't go there. It's a tourist
0:44
trap. Uh, and you can start with seeing, for instance, there's a spelling error here. So, if we set up the CKEditor or assistant correctly, we can fix spelling
0:54
using AI. So, we're just going to fix spelling. It gives us another uh version that fixes that, right? So, we're just clicking here and we can do another thing here. We say that we want
1:08
to change the tone to be, I don't know, sarcastic. You know, you can add whatever you want here with whatever context you want. So, it becomes a little bit more sarcastic. for instance.
1:20
Next thing I want to do is upload an image. I'm going to start by taking like a image of a family which has nothing to do with restaurant reviews. We're using AI now to validate this. So this
1:32
actually says that like this image should be showcasing the restaurant or some food from the restaurant which this obviously doesn't do. So we can't save.
1:40
So we're using the AI validation module for this. So let's instead then upload an actual picture of Mustafa's um Gemüse Kebap. I didn't take this picture.
1:52
It's just for showcasing here. And then we can also generate the all text using AI. So we can use a lot of AI here. It creates a kebab food stand with two workers and so on. Uh and now I'm going
2:05
to then run the automators. So this is not something that I recommend you to do. You should not set ratings on a review site using AI. I'm just going to showcase that you can do this. So wait
2:19
time rating should be very long because there's a lot of tourists in there, right? Uh very low. I mean price rating should be very low as well. They are very expensive while the food rating is
2:30
probably okay. Uh it should also categorize it. Uh uh there is no like specific doner in this case. So I'm guessing I some of German, Middle Eastern. Yeah, we'll see what it does
2:43
actually. Uh, and it's going to actually go in search Google places for the actual address to get it correct and paste the homepage on something that's here. So, we have it here. I'm just
2:56
going to click save now. And this is going to happen automatically. Of course, you can use field widget actions module if you prefer uh actual buttons for this.
3:11
So we are finished. I will just say two caveats here before I go to the kicker that I sped it up now. uh um so it is uh sped up and I've actually tried this a couple of time because what we're coming
3:27
to here is not as uh reliable on maybe more complex thing as uh for instance OpenAI provider is but you will see it has filled out the rating it has categorized it as oh Middle Eastern
3:41
street food I think the Germans would be angry for this actually the owner is German uh and it has actually found the correct address by searching Google places. It has filled out the
3:54
uh uh homepage. So now to the kicker then this was not done using one of the 54 providers we have in Drupal now. So it's not OpenAI it's not Anthropic something like that. This has actually
4:08
not left my computer but this is still not you know what we have had before. We have had LM Studio, llama, some other things that you can host or run locally. So what actually
4:21
happened here for every provider call we did this has actually returned this into my browser. So I'm running Chrome right now.
4:33
And there is actually a way to have a provider running inside Chrome. And in my case, I'm running this on my pretty crappy laptop. It has 16 GB of RAM, but that's it. There's no GPU in it, and it
4:47
just works. So, let's go through how this works. So, the links I'm going to show you now are going to be in the show notes. Um, what you actually see here is the
5:02
documentation for what's called built-in AI. So, this is still experimental, and I'm I'm going to be uh honest here.
5:09
Everything I show you now, it's experimental. So it's experimental in terms of uh how how this works in Chrome. There's flags that you have to turn on. So it's not that it works from the beginning. Uh
5:23
uh the way you get it working in Drupal is also experimental. So um there are places where you can basically read about it but you can see here how it works. So the web
5:35
application can basically call this via JavaScript and depending on your local hardware, it can actually download a mod model that adapts. So if you have GPU, you can actually get better outputs than
5:50
what I showed. Now, uh the example I showed are I actually kind of chose those uh depending on what I knew the 16 GB RAM model would handle.
6:03
uh and I have also I will note as well I have done changes to the automators to actually run that with structured output. So structured output is something that this model uh uh
6:14
supports. This is something we have a issue for anyway in 2.0 because this is probably the way to go forward with automators. I will show put this in the show notes as well. If you have uh
6:25
actual uh um if you have actual uh opinions about that you can write there. We have a discuss issue if we should be doing that. Uh anyway this download a model that is on your browser and this only
6:41
has to happen once then not per website or something. It downloads so other websites can use it.
6:47
And what happens then is when you send something to our AI providers and you specifically ask for it to use the browser provider that is called uh I will show it here. So it's the browser
7:01
provider. Uh so if you just uh get the dev version of this install it it's available and I will showcase that in a second. uh the first time you use it, if you haven't uh uh downloaded the model,
7:18
there will be a popup uh on your Drupal account that basically tells you, hey, you need to get the model. You download the model uh and what happens then compared to any
7:32
other models. We can go through it here actually. So we can use the API explorer because in the API explorer of course um you can try different browsers. So we have a typical OpenAI uh what model are
7:47
you? And now I'm I'm not going to speed up anything so you can just see the uh comparison here.
7:55
Uh wait, we also do this so it becomes uh uh comparable answer in one sentence. So it actually doesn't create you know a lot of tokens on one place and no tokens
8:09
on the other. Then you have just another provider called the browser client. The reason we have named it browser client is because in the future hopefully all browsers will uh be able to support this
8:22
um uh or there uh might be support by other mechanisms like web assembly right but right now we only have the model Google chrome uh and we can just go ahead and ask it
8:37
so what happens now is this is sent to the back end uh then we have a place in the front end that's currently polling. But what we probably want to do instead of polling
8:48
there is that uh we actually add a Node.js sidecar that has websockets or something like that. So you don't actually have to wait for it. So now it's finished. You can see I'm gma a
9:00
large language model created by Google DeepMind. Um anyway, so it starts pulling when there is something for it to work on. It starts working on this and gives a response. Now there's one
9:12
caveat here for security reasons of course this is connected to a user. So this does this does work on a anonymous user if you start creating a session because then it can store this um this
9:27
in a session but if you have a completely anonymous user you you have some kind of CDN in the way so you can't start sessions well then this currently doesn't work because I I added this as
9:41
long as we're doing this experimental this is just a very clear security mechanism to make sure that you know you don't ask something and this runs on a not the person's computer. So it always
9:54
runs on the computer of the person logged in um or the session where it's created.
10:03
Uh and that is basically how it works. So now um oh before I show you how to set it up which is actually very simple but anyway there is also this that came from the Oasis 2025 event. Uh so there
10:19
was a hackathon there and some people realize okay this is perfect for CKEditor and to be honest this is the the I I'm not going to say the real implementation but how you really want
10:32
to do it is having JavaScript talk to JavaScript right um so what you get here is some uh uh CKEditor tools as well the difference then with the browser provider is that it
10:45
gives you access to the whole uh actual uh um ecosystem of modules in Drupal. So everything we have created in Drupal uh will be available using that. And there
11:01
are obvious uh one obvious caveat that of course if you're running 16 GB of RAM, the response won't always be great. So when I'm trying to to uh for instance have it uh you know do
11:18
the the uh do this um rewrite of tone sarcastic then sometimes it answered yes I can do that and then came the text and stuff like that. Um the other um um the other caveat is of course there's a
11:42
lot of Chrome flags you have to run Chrome and so on. Um, so yeah, there there are some things you have to think about when you're using this, but there are also clear clear advantages
11:58
because yes, you can already in Drupal in production grade run uh your own local model using LM Studio, VLM. I I don't know all of the providers we support.
12:15
The problem is that first of all you if if you are using it you have to trust that that website doesn't actually send it somewhere else right so that's one thing uh the other thing is if you're
12:28
hosting this this is not super easy to scale if you if you have to have a lot of users using it. So if 20 people asks at the same time, you know, that's going to start queuing it up. It's going to
12:41
take time. But if that's 20 different people with their own uh computers where they can run the model themselves, that's awesome. Plus, you can also see that it really stays uh locally. And
12:56
this also opens up things like you know when you trust Anthropic or Gemini or OpenAI or something like this what you can do is run a prevalidation like does this actually have some PII
13:11
and that runs on the client's computer and when that is approved then it can send off to a bigger model uh in the cloud.
13:21
Um, so anyway, uh, let me show you. I'm just going to extend. I saw I need to actually update this website, but it's just running locally. It's never being pushed. Uh, so the only thing you
13:34
actually do is, you know, go here. We haven't done a release yet, and we're going to keep this in PC for a long time.
13:43
uh just copy this and uh install it as always after you gotten it with composer and as soon as you have done that there is provider settings uh there's no API keys to set up or
13:58
anything like that but you can set up timeout where you basically say uh if it hasn't finished within this something is probably wrong uh polling interval how often should it
14:09
check for new requests Uh and you can also set allowed roots where uh it should be checking for this request because you might not actually want it to pull on every Drupal website
14:21
always, right? So if you know you're just going to use it on node editing, you might just have the polling on node editing as well. Um you can actually, this is a uh uh
14:36
way you should probably not be utilizing it, but say that you actually have a MacBook. MacBooks are pretty uh strong when it comes to uh being able to host models. What you can do is you can log
14:47
in on one computer on one route and have it pulled there and this works as a proxy for your account somewhere else because it is the uh when you're logged in is it is a shared session, right? So
15:02
you could actually have two computers in that case uh utilizing this [sighs] um and that is it because otherwise as I showed you in the AI API explorer this just becomes you know another provider
15:20
as anyone else. So when you're setting up your default settings or when you're setting up automators or or when you're setting up CKEditor, you can just choose this model and that's it. Uh so
15:36
it's pretty cool. I will also say for AI 2.0, so we're on 1.0, right? Next breaking change, we're looking into maybe how this can be part of the actual core. So you don't need to actually install
15:50
external mod uh module for it. So for the 1.x branch this will always stay experimental and I think it will take time before the browser providers also makes this into something stable.
16:03
Uh but in 2.0 we might actually seriously look into how we can uh introduce APIs for this and how it can be part of the actual AI module or 3.0.
16:16
Let's see where it lands. is interesting and I I think it is already in a place where if you know at least I mean you shouldn't use it for end users but if you have editors those editor has
16:30
MacBooks and uh you have expensive builds right now because either you're hosting Ollama or LM Studio VLM or something yourself or you might actually be using some of
16:45
the the bigger models but you don't want to pay for it or you don't want to send it. As long as you know who is using it, you could actually start using this and
16:55
play around with it. Um, and as always, if you want to co-maintain, there's always that possibility. And if you find bugs, just add them to the issue cube. Thank you
17:07
for watching.
