0:00
This video is sponsored by FreelyGive. If you need help with Drupal AI, get in contact with us. Today's video is mostly uh focused on developers and testers, but also product owners and project
0:11
managers that might be interested in one saving money during the development phase and to have a solid CI pipeline.
0:20
So, a a continuous integration testing pipeline when working with the AI module.
0:27
So, I'm going to showcase two things. uh that are important when working on the AI module today. One thing is how you can mock responses.
0:37
So you might uh you might be writing a contrib module for the Drupal AI. You're writing uh you know your perfect prompt.
0:45
You have solved the prompt. So, so whatever you're using like OpenAI or something is answering correct all the time but your code is not working which means that you're trying to develop and
0:56
you're you know sending messages to them or sending requests to them all the time which a cost money b takes time which is just frustrating. So I'm going to showcase how you can mock this up
1:07
instead very easily without having to you know learn about Guzzle mock or something like that and uh and recreating a API response for one of the providers because that's just complex.
1:21
The second part will then go into how you can use this in testing uh uh pipelines for kernel testing and functional testing. So let's get started. I have I have installed three
1:32
things here. So the AI module, the AI API explorer to have somewhere where we can send our prompts and the OpenAI module. So I have now then in this uh example solved my perfect prompt. It's
1:46
very simple. In this case, write me two sentence about Lionel Messi. We just shoot it off and you see it takes time.
1:54
Uh but we get a a response back. But then maybe our code or whatever we're working on is breaking. So we have to run it again and it takes some seconds for it to run cost a fraction of a scent
2:06
which adds up. Maybe you have to you know you're doing something very complex. It takes 100 200 tries that you send off requests. So how can we um how can we solve this? So in the Drupal
2:22
module uh sorry in the AI module there's a test section of course and we're running a lot of tests and one part of what we have there is a module called AI test
2:34
and the AI testing module uh by default it's just a parrot module so I I I will showcase that. So what it can do is that you send a request to the AI test provider that is a mockup provider. it's
2:49
just going to give you an answer back with what you sent. So to be able to test it without running uh it in automated tests. Uh you have to enable it then. But the problem is you're going
3:03
to see here it's not showing up. And the reason for this is it's actually not going to show things in the test directory by default. But you can enable this by adding this settings extension
3:14
discovery scan test true. So if you just set this in your settings.php and reload, all of a sudden it's going to be visible.
3:27
So we're going to enable this now and we're going to use uh first of all then my prompt here to answer uh using the echo AI provider and that's the test provider and it has two
3:43
models doesn't really matter. is just for showcasing how listing mod works and you see it answers super fast. So it's answering hello world input write me to sens. So it's just copying this plus the
3:55
configuration we're sending. Now when you're working with your module when you're testing you of course want a response that makes sense to you because maybe you're asking for a JSON output
4:08
maybe you're asking for giving back tools or something like that. So I'm going to show now how you can do this.
4:14
So after you have installed this module, there's going to be a provider setting for this. But this is not going to be a normal provider setting because it's actually not using a provider in the
4:24
background. It's just mocking things. So what you can do here now then is you can turn on catch results.
4:31
You can also catch processing time. So if you're working on front end issues where actual response time might matter or you're working with longunning task where the processing time might lead to
4:45
timeouts and stuff like this. You can actually replicate this by uh turning this on. I'm going to showcase this and I'm going to showcase how you can you know after you have built something can
4:55
turn it off as well. So after we have turned this off, we can just uh let's reload, take our thing here, and we're just going to call OpenAI again because what we want to
5:07
mock up now is a replica of how OpenAI would answer this question. I'm just going to click ask the AI.
5:16
We get our response back. I go back to this tab and there's going to be under content AI mock provider results. This is only available as long as you have this module uh enabled. And you see we
5:30
have a result here. So if we click on that, you can see the request here. You can see the response and you have the whole raw output. So if you need, you know, you need the system fingerprint
5:42
from OpenAI for whatever reason, it's actually there. You can see it time it took. So 2.7 seconds. We can also now when we want to replay it, change it. uh operation type and tags it store as
5:56
well. But say we want to use that now the only thing we have to do is click mock enabled save it and now it can start mocking it. Oh what we should do as well we should give it uh um
6:11
uh name as well so we know which it is. Uh but what you also should do when you're happy with it just go back turn off this uh and then it's not going to catch these results anymore. So if we go back
6:30
to the chat generation explorer now and choose Echo AI do the exact same message. So it has to be the exact same input. So it's it's finding it on that.
6:42
You're going to see it's loading exactly 2.7 seconds and it's giving back the results exactly like OpenAI did.
6:51
However, if I write uh messes here, you're just going to see that all of a sudden it's using it the normal uh uh it's using the mirroring method.
7:02
So now we have created something that works exactly like OpenAI and we can try it over and over without it costing money. can also go in and edit and say now then we we don't want any sleep time
7:13
because we're just doing backend development. We don't care about the actual time it takes. If I click now, you see it's coming directly. It's super fast. So that is how you can mock up
7:25
responses. The only important thing is, you know, make sure you turn it off after uh you have catch your response so you don't end up with a lot of unnamed responses and you don't know how to find
7:37
the one you're looking for. Um so let's uh jump to the next thing that is testing.
7:47
So when it comes to testing, one thing I have done here in my response, I added 300 milliseconds sleep time. So it's enough for the Ajax to actually truly load. It's uh not long
8:01
enough that it actually becomes a problem when you start stacking tests. Uh so what we want to do in this case is that we want to try that to see if the chat generation explorer actually works.
8:14
So you have a admin user that goes to this URL, puts this into the message, clicks ask the AI and should get back this results if it's using the Echo AI GPT tests.
8:26
Uh I have already written the test. So first of all I have a runner here that I can run and then I have the test. I'm not uh going to write it live takes too long time. I will just show what it
8:38
does. So it's using the uh function JavaScript tests. There's a base class in the Drupal AI. So if you're actually using Drupal AI for this or you're writing test for Drupal AI that's
8:50
functional JavaScript test, use this base class. I'm going to showcase one. Then we install all the modules. So AI AI test file AI API explorer and the user. If you're using the base uh class
9:03
function, add a screenshot module name is good. I'm going to showcase why also. So that is just for categorizing where where screenshots go. We set it up. The only setup we do is make sure Echo AI
9:16
GPT test is the default provider for AI. So that means if you if I were to reload this page, AI Echo AI GPT test would be there. So in this case, I'm not going to
9:27
test clicking around there. Then we create a user. The important thing is that it has the uh uh permission access AI prompt because that is the thing you need to be able to see here. Uh then you
9:40
go to Drupal uh sorry you log in that user you go to this uh um URL and then you have this that take screenshot that is part of this base class. You just have to give it a name. You don't have
9:54
to think about extensions or folders or stuff. Just give it a name. So this is message one before message.
10:01
We then go to the page uh and uh we fill in the message. No sorry we get the page so we can actually start interacting with them. We fill in the message one. So that's the name of this
10:16
one. Write me to sensor about Lionel Messi. We take the screenshot again after it's filled in just to see that this is working. It's very good for debugging, but it's also good later when you're
10:29
running this in a CI platform to see where something went wrong. We click on edit submit. So that is this button. Take another screenshot while the Ajax is running. We're waiting on
10:42
the Ajax request to finish. Take another screenshot. And then the text should contain this because that is part of Yeah, you can see here should contain this. So if we run this now
10:57
uh this is just going to fail uh because this runs so a functional javascript test sets up its own environment this means that uh what we did here before with a mock provider results oh sorry I
11:15
have it there this is not available on that website so it's not working there so how do I get that in there in a consistent manner where it always run.
11:25
Well, under the drop down here, there is export to test. This is going to now export a YAML file.
11:34
And the thing I need to do now is under tests. So, this is a uh this is a this is a directory name space you have to use. So, you have to do resources AI test request and operation type chat in
11:47
this case. And then you can just uh copy in the actual file the on messy prompt and you can see the file here. So if you want to change the wait time to 200 milliseconds you can
12:02
change it save it here and then we try to reload this and now it should be aware of this.
12:10
Uh yeah by the way I did a hardcoded thing here. That's maybe not how you should be doing it.
12:17
So uh now it ran and this uh uh screenshots by the way then if you want to see it on your local environment you can use sites default files simple test screenshots and you see there's a AI API
12:32
explorer chat messy test then you can go in and see how it looks beforehand. So, we're not using we're using the base theme here, right? That's why it looks so ugly. But you can see it has loaded
12:43
it. Second picture, it has uh filled in the message board. Third picture, it's processing the Ajax.
12:57
And last picture, it got back the result. So, we we're testing end to end. So, this actually works. I could push this. We already have test for the API explorer, but now I could push this. And
13:09
the interesting thing is then at least if you're using uh the AI module, but this is probably true. I think as long as you use the GitLab template, uh if I go to any test here
13:24
and take the PHPUnit, so that's the part that runs it. We can go to browse and under web sites. You're not going to see the messy thing. We have another thing here. Default file. So, it's the
13:39
exact same place. Simple tests, screenshots, and we take the AI API explorer chat explorer test. You have the actual images here. So, for every test that's ever run, you can see exactly what's
13:54
happened with your functional test. This is not available in kernel test of course but you can use the same methods for kernel test for uh replaying data.
14:07
Um yeah so that is it. This is how you mock and write tests using the Drupal AI module. Thank you.
