0:01
Hey everyone, Drupal AI 1.3.0 is finally out. So this video I will just use to showcase in the shortest form some of the features that we are including in it. I just want to say a huge thank you
0:16
to everyone that has been working on this. So this is going to showcase features, but there has been a lot of people working on bugs, tasks, documentation, and so on.
0:25
So let's go. Guardrails are security functions to make sure that the AI stays aligned and ethical. It can either stop you from sending out a request or it can stop a
0:40
response or reform a response. In this case, we have a secret product information guardrail. It keeps track of all the magical models my company has like the MDX 250. If I now attach the
0:55
guardrail that we have set up to this call and ask the AI, you will see that doesn't get sent to OpenAI because we don't want to share confidential product information.
1:10
What you're looking at here isn't Drupal, but something called Honeycomb. So that's just one of many platforms that handles our OpenTelemetry input.
1:19
And right now what we're diving into is a single AI request so you can see exactly how much data you can actually extract from one request. And this is thanks to the new integration of open
1:31
telemetry and AI observability in 1.3. And because we're following open standards, you can track autonomous agents whole journey from its first reasoning to when it's
1:42
actually giving out the output. And you can graph it in any way you want depending on which platform you're using. So it doesn't need to be Honeycomb. You can use Grafana. You can
1:53
use Datadog. There's a lot of these different services that you can either pay for or self-host.
2:04
So, you might know the chatbot. What we have added is that you can actually flip out the toolbar chatbot and flip it in now.
2:17
We have also added if you prefer that, so you can run it in full screen and have it back.
2:28
Talking about the chatbot, you can also add a custom loading message to it now. LLMs work best with markdown, not with HTML, not with plain text. And because of this, we have added a
2:48
markdown editor for when you're adding system prompts for for when you're adding context in context control It's a normal WYSIWYG element. You can copy and paste anything that you want
3:01
from for instance Word or Google Docs. If you go ahead and change something here, you can go ahead and look in a diff mode what actually got changed.
3:12
You can also see the raw markdown at any time. What we have also added is that for any usage of this, you can give it back either variables or tokens. This means when I start
3:28
writing a curly bracket, I will start seeing what variables are available and I can auto complete that.
3:35
Same, if I do a bracket, I get tokens. So, you will see it's using the token for the current user in this case.
3:46
Up until now, every module that integrated with the AI module as a third-party module and wanted to have a model selector. So, provider and model selector did it their own way. There's a
3:58
form element for this now and it takes care of the Ajax of auto updating configurations. You can have it in advanced mode or you can have it in a simple picker mode.
4:12
Talking about form elements, maybe you want to debug conversation history. So, you want to actually change what has happened. Well, we have a form element for this where you can start adding
4:22
messages. Uh you can start adding even tool results and you can drag and reorder them by just dragging around. And whenever you click submit, it will be saved as a array that you can use.
4:38
AI model supports something called structured output where you can ask it to follow a JSON schema in its output.
4:45
But writing a JSON schema by hand is not easy. So, we have added a form element for this that makes sure to actually validate what you're writing. So, I added a error here and I see it right
4:56
away. The AI settings page is greatly improved. It looks a lot nicer. You can also for many models get model information for pricing and general model information.
5:13
On top of that, it also shows additional AI capabilities. So, if there's a operation type where you don't have a provider currently, you can see which providers to download to actually use
5:23
this. We have added a unified CSS library in 1.3 to make sure that anyone that develops new features can have a consistent look.
5:38
Among other things in this is that we have added a tooltip function that you can see here.
5:48
I'm showcasing metadata here. So, for every release that we have done, we have given more and more normalized metadata for the AI core to give to its contrib modules. So, in 1.2, we added tokens as
6:01
normalized data, so you could count how much usage you have. In 1.3, we're adding rate limit thresholds. And why is this important?
6:10
So, up until now, if you hit a rate limit, you would just get the exception and you would just have to say, "Well, you can't use this anymore." If we actually know that you're close to
6:19
hitting a rate limit, we can do things like failover or wait until you're getting rate limiting thresholds back.
6:27
And this is also going to be important when we start adding asynchronous calls to know basically how many calls can we run at the same time.
6:39
If you go to the bottom of the AI settings page and click on advanced settings, you will now see a HTTP request timeout where you can set your own timeout for AI providers. If you
6:51
want it longer or shorter, you do not need to hook in and use code now. You can do it right in the view.
7:02
We have a new operation type called re-ranking. And while we don't have any implementations for it, this opens up for AI search to use it. And the idea behind it is that you have something
7:12
like a vector search or a normal search that does the initial filtering and then you run a second pass. So, in this case, we've gotten these four documents back.
7:21
We're asking this query and it's going to re-rank it in a second pass where we can say exactly how many results we will have back.
7:29
And [clears throat] this makes retrieval augmented generation, a typical talk with your document chatbot, a lot better.
7:39
We have added operation type called summary. And while this doesn't feel very special at first glance, and an LLM has been able to do this for a long time, these are specialized models that
7:50
can run on CPU. This means that you can have such a model running on your web server, or even in this case, have it running inside my browser. So, the browser provider runs models inside the
8:02
Google Chrome provider. And in this case, we have added a field with its action and automator, so you can get it with a click of a robot.
8:13
We're adding object detection into 1.3, which is a traditional machine learning technique, where you can train on your own data set a model to recognize objects and where
8:24
they are located. So, in this case, with this image, we get something like this back. You can see how reliable it is and the bounding box of XY width and height.
8:36
Now, what's cool with this is while we're just having Hugging Face provider for it now, we're trying to create a local AI provider so that you can run on your normal web server.
8:47
And we have already integration with the AI validation module, where you can run validation if a label exists or if a label doesn't exist and stop the upload based on this.
9:01
For both the image field and the media field that has images in it, we have added field widget actions. So, this means that you can use some context. In this case, we're just using the title to
9:14
generate images, and they get added as either a file entity on the image field or as a media entity on the media field.
9:28
Office hours is a nifty little field. We have added field widget action, so we can actually extract unstructured data into structured data.
9:41
Meta tags are really important for SEO. Because of this, we have created a field widget action that runs the automator to create meta tags based on prompts, and
9:50
you can do this for any field. In this case, we wanted to create clickbait and joke uh jokes for the description and the title field based on the content.
10:04
We have AI processes in place that can look at content and set a moderation state based on it. So, in this case, if it's draft, published, unpublished, or flagged, it can do a pre-screening of
10:15
the actual content here. So, I have created a button, so you can actually preview the state before you save it. In this case, it's okay, it's published.
10:23
I'm just going to go ahead here and add a swear word. Click, and all of a sudden it's flagged.
10:32
After we have created our summary, what we can do is use our new field widget actions buttons to create text-to-speech.
10:42
This can be used on both file fields and media fields. So, you see, it has gone ahead and created a media from this. If I click save, we can also preview it here.
10:51
>> Drupal is a free and open-source web content management system, CMS, written in PHP, distributed under the GNU You want to extract telephone numbers with a click of a button? We have that
11:05
for you. We also have field widget actions for short fields. In this case, we have a lot of unstructured data about weight and RAM on mobile phones.
11:17
We just click plot the data. It's going to look at this and plot the data in a chart field. So, if we save this, we're going to see an actual chart field.
11:30
We have added a field widget action for extracting addresses. In this case, I've set up a lyrics website where it's going to extract all the exact addresses from
11:39
lyrics using Google Places in the background by pressing a button. So, you can see this is Marc Cohn Walking in Memphis. He's talking about Beale Street, Union Avenue in Memphis,
11:50
and it just goes ahead and chooses the correct address for us. We have support also for the FAQ field.
12:00
In this case, it's going to look at this content and try to create a couple of FAQs for that when I click the button.
12:16
The JSON field can be quite nifty if you have a API, for instance. We have added a field widget action for it. So, with a click of a button, you can extract something unstructured into something
12:26
structured in JSON. Free audio post-production by auphonic.com.
