0:00
this video is sponsored by FreelyGive if you need help with AI and Drupal get in contact with us we're specializing that so in this uh segment which is also for DrupalCon Atlanta what I'm going to do is
0:15
the following I have added all the different Keynotes uh from uh DrupalCon Atlanta I've used the automator to actually fetch these so this will be part of what I'm showing and put those into a rag database so we
0:33
can search for uh semantic meaning and then what's going to happen is that you can basically create a keynote recommendation show by putting in what are the needs you want so what are you looking for in in Atlanta in this case I'm a front Ender that starting out my
0:52
bla career and this is my first Ron I'm mainly interested in how to get started with the contributions and how the experience really works and and out you get to people talking about it this is
1:03
Jane your comedic Captain At Your Service folks and who's that Stern face sitting next to me and this is Pete ready to navigate through the Drupal Keynotes for our newbie front Ender friend attending their first Drupal con
1:15
aoy Pete let's set sale with some Keynotes our listener is looking to dive into the world of Drupal contributions what's your top pick I would recommend the firsttime contribution workshop on so you get the idea right it's great
1:29
thing I show showcase that you can listen to in your car or uh whatever it is it's it's mostly to uh prove multi modality uh but there are very very good use cases of this of course where you
1:43
can create uh things for people to listen to on audio that's based on the data on your website so let's get started so you actually caught me in the middle of something I I thought I would take all the uh talks and just add them
1:59
before we started this but then I realized this is going to take so long time why not use AI for this as well so we're going to do that as well I have this keynote content type you can see it here it's just the speaker the
2:12
description date and time and a title and that's it but we want to use AI now instead to import this because it takes too long time so we're going to use ScrapingBot I've installed it I've added a key AI with AI automator and
2:25
OpenAI provider we have in this case I have set up keys so if I go to OpenAI authentication I can connect that key to it and same thing if we go to scraping B config oh what is my username there I
2:44
have to check good is this so now we have set that up so if we go back to our content type Keynotes now we should be able to add a link field so we we just put in
3:00
the link instead we we don't want to fill in all this stuff well we're going to have to fill in our title because that's a uh uh uh that is a requirement when it comes to nodes so if I did some other kind of
3:15
entity types maybe it wasn't needed but in this case it's needed so we have a keynote link and then we're going to scrape that text uh uh base HTML so basically this is just
3:33
scraping the website so we can use AI to extract whatever we're doing um so we're going to use ScrapingBot crawler we're going to use the keynote link set a little bit lower
3:47
weight here maybe 95 uh set as on a batch because the weight is deciding basically when something runs so this should run first oh we're using this article segmentation I think this will work let's see the idea is there that we can basically extract uh only the important
4:07
data so this but we don't get the menus and footage and stuff that doesn't matter for what we're doing so we can just start with this let me just see which is the last one I did the future of Drupal
4:21
CNS uh okay so this is the next one so the idea now is wait let me do do something else let's just put the at the top so so it gets faster to add it keynote keynote
4:43
link sorry that is what we want to put there and now I put something it's not a good more here uh so now it's going to go ahead and fetch that oh no HTML found they might
5:02
actually have do they have scraping protection there so let's try something else then it could be that this readability is the problem here uh wait let me see so I actually have yeah there's no error so it's no no
5:21
problem with the code from scraping but this is what I wanted let's not do read ability there then let's scrape everything and we're going to say oh have we remove the no HTML found
5:44
otherwise it doesn't for you on no it actually doesn't work that's interesting let's try simple crawler then so this is a simpler CER it only does basically uh uh things that are rendered
6:09
by the server so I wouldn't use it for too many things but it might work in this case or it might at least give us an idea why it's failing so we enable it and if we enable it it's one to one so we can actually
6:28
just go back here here oh no we have to click edit maybe the create page and we do great simple craw it same thing Chrome user agent we just want to pretend that we are an actual
6:52
browser otherwise we keep it as it is and let's try again okay this works better that's interesting so we have it here right so the question now is then if we can use
7:16
readability here maybe that works better if we get all the information we need let's try that so what we can also do otherwise is that we can choose an selector okay this is too this is not enough this is good I mean if you just want the text
7:33
right but the problem with this is that you can't really uh get the dates uh in this case and the speaker which is something that we want in this case so we're going to go ahead and change this
7:46
again and we're going to try HTML selector then um and the tag we want to get is what can it be I'm guessing there is something that covers the main wrapper so it's div class main wrapper so it's like
8:14
this let's see if that works better yeah this looks about right yep we have the date we have the okay this is good so we're just going to remove this from manage display we don't care about this here right um so and we don't care about the
8:41
link either the link could stay there but now I didn't add link to the older one so now what we want to do then is that we want to create all the different data or extracted so we use a AI
8:56
automator we use the base HTML and we say look at the following context and find the speakers for this uh following context that is a keynote information speaker for this keynote and return it as
9:26
text comma separated context and here we're going to use the raw context we actually care about the HTML because that might give context to AI to figure out where this is actually located
9:46
um we don't this is speed though we don't need to extract anything I think we can just keep it as it is we keep it higher than the other one because we need to scrape before we start trying to
9:59
extract it because otherwise we don't have anything to extract from right so we got the speaker uh then date and time so in this case we're just caring about how AI reads the date and time so it doesn't need to be Sav structure we're not going to filter
10:20
by it or anything uh because it's just going to use this basically to figure out uh the the uh let me just copy here uh figure out so when it's suggesting how you're uh going about the day that it doesn't choose choose two Keynotes that
10:42
are at the same time find the date for this and return it exactly as it is as text uh oh no not create new field and we can do away the key description so we're going to get that as
11:07
well simple here as well it's just one text look at the follow find the description and return it as is in plain text form with new lights at appropriate place I mean it doesn't need to be perfect for what we're doing to
11:33
work we just need it to be there uh I saw now there was HTML there we don't need to extract HTML because we're not creating HTML in this case it's a t [Music] long we can just keep that empty we going to save and see how it
11:57
works so we know date and time did not actually work so we're going to do two things here uh including from and two time and maybe this also try to extract HTML which doesn't make sense so let's retry this so this is a little bit of
12:25
the tri line error you have to do when you're using this tool because it should not say just 9 right now and it's 9 to 950 we want there we go so now we have uh created uh something that makes this a
12:46
lot faster because I can just do this now instead of having to copy and paste everything okay that's maybe we shouldn't have there but this sh definitely be there we're just trying it end to end
13:04
now see if this works simple C we can close so uh yeah that looks about right so I will do a cut here I will add everything now but now you understand how I can add everything in a couple of minutes
13:23
instead of you know half an hour or an hour okay so this actually went a lot faster uh so probably I could have if I did this from the beginning I could probably have done this in like 10 minutes instead of spending one hour as
13:42
I spent on it now so I I I didn't even realize myself that I can use alternators for this which I should have done since I created it so shame on me anyway now we're going to
13:55
do what we actually came here to so we have all the different Keynotes in here the only keyn note I don't have is the Dre note because well you're stupid if you miss it uh uh you know that's like
14:14
going to an old Apple event and not listening to Steve Jobs that would just be crazy so it's not in there but everything else is in there so what we're going to do now we're going to install the mil
14:29
uh uh bdb provider so this is a vector database so we can the idea behind a vector databases basically that you can search via semantic meaning meaning that someone will write I want to have a program that covers blah blah blah blah
14:48
and this can create search uh queries from this and this can search on semantic meaning and find actual actual uh Keynotes that fit whatever you're writing in free text about so we have this uh we're going to
15:05
have the AI audio field so this is um a it's a file field but with a difference that you can use a text to speech model to create uh the audio file from some text that's in
15:22
there and finally we're going to use ElevenLabs ElevenLabs is without any doubt the best text beach model on the market you could use OpenAI for this as well but I I don't want to show uh uh models uh sorry I don't want to
15:39
show uh the same thing over and over okay sorry this has actually 10. zero let me fix this be right back so uh yeah I I already created a tag for this but I hav't actually published it sorry about anyone using 11 La so it should be possible to use it in
16:04
Drupal 11 as well there's nothing in the code that stops IT from being used so we have ElevenLabs here as well so we're going to go ahead and install those things now
16:22
so oh I know something we also need and uh and that is uh search API okay this got search AP itself the mid this and audio field and ElevenLabs LS and we go ahead and install those and I am going to create keys
17:00
outside here for ElevenLabs so you don't actually see my keys I don't want to share that with the world guess I could create a new one just for this but I think you understand
17:14
how adding Keys works so you go to this URL that I shown and just add a key with your API key so so now I have a ElevenLabs API key then I can go to provider settings 11 lab
17:36
settings select the key uh I can I can just show first manually maybe how this would work well no first what I want to do is the following I want to go and well I need to flush the catch we have a pug that is in the issue CU that's
17:58
been there for a long time we have to fix so it seems that the factor DB if it exists or not is connected to the providers that in cach which means it doesn't show up unless you watch the cash we need to fix that actually
18:12
um that's a reminder I'm using DV so it's just on the local here I don't need a API key because it's local so we have uh M now what I'm going to go ahead and do now is I'm going to go to the A search there you are search API at a
18:33
server that is my Keynotes I'm going to use uh OpenAI small we don't need big in this case um our MB we're going to use GPT 4 for this and I'm going to use the default database and call this then uh
18:57
Keynotes uh I am going to use very big chunks I think I put the whole keynote into a chunk actually uh that probably makes sense because this is not going to talk about a paral uh of the actual
19:14
keynote should try out try to figure it out from the whole key so we're going to take a big big uh Chunk in this case this means that the whole thing becomes one chunk so how you do usually when you
19:29
have a chatbot for instance it's good to split it up in partial depending on what kind of content it is in partials where it's just answering a part of what the content can be about but in this case actually makes sense to do this um and
19:44
we call the index keynote so this is just normal search API stuff it's just that it's uh doing semantic search I'm going to show you exactly what it is in a second so we say we just want keynote
19:57
right uh on the server Keynotes we're going to add the fields and the important field now then is the keynote description but we also want the speaker in there the title and the uh date and
20:19
time uh but those are mostly um so-called contextual content so if this were to be split up into chunks this would basically exist on every chunk and the main content is the keynote description in this case so we save this
20:38
and then we're going to go ahead and Index this uh and while that is doing that I'm going to go ahead and extend just so we can test this Explorer module so there's a AI API Explorer where you can actually
20:52
test a lot of the different kind of calls that you can do with AI and I can I can show you first the actual uh text to speech generation just to Showcase ElevenLabs I'm just going to turn on here so it records my uh uh speaker as well so
21:11
hello my name is Marcus so we can just try it and we currently we have both OpenAI and ElevenLabs because we have both of them installed so we can try first with u Els hello my name is Marcus oh sorry that was open a we going to try with 11
21:29
LS then we take the voice going use Daniel maybe for our I I don't I don't remember which is which here hello my name is Marcus that was very British let's see if we can find something else bill you can see I have
21:47
my own voice there as well hello my name is Marcus so you can record your own voice and sound like yourself but I'm not a professional speaker as you can hear right now so it's better with get a professional speaker voice hello my name
22:00
is Marcus so that guy George maybe that's a good one so this will take some time okay we're done cool because what we can do then is we go to the Ed no Vector DB Explorer and we can try try to
22:13
do a search here like I want to find key notes about AI we run this uh and you will see that these are actually all about AI so it actually understands the meaning of what I uh
22:31
write and I don't AI of course if I had AI a normal search would find it as well uh how computers can use uh rational thinking to figure out this so this is just another way of writing AI right and this gives similar
22:52
uh results okay it gets Mind Over metrics which makes kind of sense I guess r thinking uh but otherwise it understands this is connected to AI right so now we have everything we need to be able then to create our
23:12
presentation so how are we going to go ahead and do that so we're going to create a content type that is called um keynote Rec recommendations is that how you spell it I check I'm not native English speaker so uh it needs a title
23:33
unfortunately it's a old so it needs a title so we're going to start with a plane text where you as the end user adds uh uh uh let's call it needs I can't think of a better word this is what you what you want to know about basically uh oh we should set that to
23:53
the required field I realized oh didn't oh right there's a bug here that is fixed actually in one. one so you have to check this and choose something else this is fixed and it's going to be back
24:07
fored as well by um so now what we need uh let me think here what else do we need oh before I continue with anything actually another thing that you need you're going to need to be able to do this is you're going to need a M so
24:30
since I use um you should put this in your dad file but I just want to solve it quickly here so you just do this uh and it will install ffmpeg and ffmpeg is you need to be able to do media manipulation in this case it's uh taking audio files and and moving them together
24:51
so there's automate for this um but what we want to do first here is take the I guess it's under the form text might be under the pl text let's see here search results we call this uh uh uh keynote
25:10
search results maybe there so this is going to be search results uh text search results on ah sorry it is this so we need a plain text I guess it is under uh key note search results we need to
25:33
get basically search results on what we're writing there right uh so we're going to use Vector search text you could have a step in between where the AI looks at your text and tries to figure out how to rewrite
25:46
this into something but I think it's going to be good enough just doing this so we're just going to take the needs and we're just going to search for it as it is and uh we're going to say
26:00
unlimited here but here we can say we want from the keynote index we want 10 results uh and yeah it's 10 results is how many are there maybe do 15 results uh and this is going to run
26:21
first we set a lower weight on it because of this save uh oh output field that should go into content in this case and that should be our first start uh so let's maybe try this before we go
26:53
anywhere else so I'm going to create my keynote recommendations and I will just just write I want ask of as many AI thought as possible and let's see what it finds we should probably uh uh stop this
27:18
from uh showing here but that we can do later so it find found all these right all the information that the AI needs to be able to now then create uh this is the other one I almost added here I have old tab of running here we're going to do content type you notation we're going
27:40
to do the uh podcast show now so we're going to go ahead and use the file upload and then there is a AI audio field and this we call the uh show files so this is the actually show where they're talking we
28:00
need to set unlimited because in this case I don't want just one person talking it's going to be really that they you know jump back and forth back and forth like a real uh show I'm going
28:12
to enable the automator use the AI audio field generate store we're going to use uh Advanced token in this case and what we're going to do is going to browse available
28:31
tokens we're going to use the node search results I guess we called it or what should we call keynote search results there we go uh wait let let let me before I do
28:52
that I'm just G to copy and paste here because this is kind of important for what we're doing so and then we can do context is not going to be that uh based on the search results you see
29:14
under of we just write search results key notes that happens during goou on Atlanta 2025 I want you to look at what the user wants to find as Keynotes to listen
29:46
to and suggest this in uh ler ler uh uh um Talk Show format you should create a dialogue between the podcast interviewer uh Jane Jane fun with a fun loving character who likes
30:22
joking and uh Pete Pete uh Pete M he's a serious guest expert on Drupal subject subjects and who is answering start with a with an introduction say this is stain and
31:00
and this is Pete and create 10 dialogues back and forth uh recommending shows for shows Keynotes for the user thank you to the guest uh the users are what do we call them J fun so here
31:25
is the important thing you have to connect these to actual uh voice placeholder so P my so we're going to create one uh uh voice placeholder oh uh give me a second so if it's not clear I'm actually kind of doing bug testing when I'm doing
31:48
some of this stuff uh so there was a bug I had to fix in the AI audio field there's going to be a new release before you can use this um so now you can click basically here add voice
32:02
placeholder so uh let's see here I needed the token so we had Jane Jane and Pete so we have to assign voices to these now so we say here Jane we're going to use ElevenLabs and we
32:23
use let's try Alice maybe and for p we also use ElevenLabs and we try George so put this in a batch if you're creating something that takes really long time you want to create you know an half an hour chore or something you have to run this in front otherwise it's
32:45
going to uh uh just uh run out of time so it's at 102 make sure this runs after the search results and then the last thing we're going to create is one file put so this is going to be the the MP3 so this we call the
33:07
recommendations oh I also forgot something now but I'm going to go back so we do uh this is going to be MP3 where we enable the AI automator AI audio field merge audio fields we merg
33:19
the keynote recommendations this has to be higher of course uh uh grouple con M3 call the file name you don't have to set something there but you it's good to set something
33:35
and if we go back to the keynote recommendations we forgot to actually add into the prompt uh that um C I want you to look at the user to find based on the person's
34:08
needs the person is looking for and there we're going to add also the thing we call needs I think we created a bad name for it but I couldn't think of a better name right now so we add needs uh can add some separators here as
34:31
well just so you the AI knows if it's code for instance it's better to use this format so markdown format but in the case of free text you can do something like this we say and let's see let's go we had it here right so we can
34:50
just continue here if I reload you're going to see there is a uh audio field here and there is the final35 so this is going to take some time now while we're doing that we can maybe do some stuff here well no I guess
35:07
on the uh manage displays uh we let's see yeah we need this one uh we need to create a widget here so we can actually show here this there's another bu this needs and the file maybe that's what we show in the end
35:36
uh oh let me just fill up some money in my ElevenLabs account okay so we have some money then we just have to go back I'm guessing it didn't store anything yet because it has to run to the end and now we should have some money now we have to wait again
35:57
I know it's not the most interesting things to look at these videos because they are one kind of like fixing process for me because I'm running on a clean environment always when I do this so there might be dependencies missing that
36:11
I missed and so on but um I still want to showcase how long time something like this uh takes oh field should F I have to fix that but anyway this didn't work see what failed
36:30
here this is Jane oh I think I forgot something very important so set this to limited because I was reloading when we had the bike that's my fault so we're just going to re delete this um and I
36:52
think this deleted let's see y it does oh then I need to delete the final file as well because otherwise that's basically how our automator works it showcases it runs when something is
37:05
empty if something is filled that means that either it has been filled by automation process or the editor shows to fill it so the automator should ever only run when you're actually wanting
37:17
this to run uh and now uh oh did I spell p wrong because I only see Al stuff here let's see I probably yeah so one thing is you see the whole process of doing this back and forth back and forth right we have Pete magu
37:49
this Pete Peter and I have a friend named Peter I did that so we need to basically empty everything here because right now we only got alysis uh alysis results which are probably the
38:12
less interesting because she's the interviewer right let's see if it work better I thought it was a little bit fast as well um but I I think it's interesting for you to see because uh you realize approximately how long time it takes to
38:30
create a automation workflow and what what what the problems can be while you're doing it as well um and I know the product pretty well so you should probably sprinkle on
38:42
sometime on top of that until you have used it well enough but um yeah uh so I can also mention that this audio field is something that's just generated right before dupon so that's why it still has some errors that I need
38:59
to fix but in the end it should work so let's actually listen to all of it because you have stayed with me this long so we can see the final results this is Jane and I'm super
39:11
excited to talk about all things AI at Drupal con Atlanta 2025 and this is Pete ready to dive into the details of each AI keynote to help you plan your schedule all right Pete
39:24
let's kick things off with a session that promises not to replace us with robots but make us Unstoppable what do you think about AI won't steal your job but it will make you unstoppable that's
39:36
a great one Jane it's all about leveraging AI to overcome inefficiencies and enhance website performance definitely a must attend I love it and what about the session titled AI is the
39:48
new UI that sounds like we're building the future right into Drupal CMS absolutely this session explores how AI agents are integrated into Drupal CMS making it powerful for both developers and end users it's packed with valuable insights moving on there's AI with
40:05
Drupal using llm technology sounds easy but useful right so I'm just going to stop there because I realized this is not perfect we need to do some more changes so uh what we're going to do is going into our
40:21
keynote recommendations we want that them to say the name of everyone so uh uh when pet is Peter P is answering questions uh about the keyote make sure that he mentions which day they haveen and add what
40:53
time and who the speaker speaker is and let's try another voice for Jam if you can find something there uh let's try another one as well for uh for Pete just so we just get
41:13
something else I'm going to do here keynote recommendations but this time I do experience good right no I I a front Ender that is starting out my Drupal career and this is my first Drupal con
41:42
what uh con I am mainly interested in how to get starting with contributions and how the Experience Builder works so now end to end now this is going to do the search so this is going to be three steps now uh so it's doing the
42:09
search it's creating the showc show showcast and then it's concatenating and let's see now if it they mentioned the date and the uh the speaker as well on top of
42:23
talking about it I won't speak until this finished you can forward here in the video I run out of things to say so better better that the AI do talking from now maybe let's see let's see if this is a better result now then
42:59
this is Jane your comedic Captain At Your Service folks and who's that Stern face sitting next to me and this is Pete ready to navigate through the Drupal Keynotes for our newbie front Ender friend attending their first Drupal con
43:11
Ahoy Pete let's set sale with some Keynotes our listener is looking to dive into the world of Drupal contributions what's your top pick I would recommend the firsttime contribution workshop on Tuesday March 25th it's from
43:24
09c to 0950 l led by is Radcliffe perfect start for anyone new to Drupal contributions sounds like a win now for the treasure hunt they're curious about Experience Builder any hidden gems there absolutely on Tuesday March 25th there's
43:42
Experience Builder is coming are you ready by lri from 15 out to 1550 this session will be a game changer oh exciting anything for those looking to build up their experience with some design thinking yes inde indeed shaping the future of Drupal how design thinking
44:00
how collaboration is Driving Experience Builder by Boston Jillian and Rene land is on the same day March 25th from 1610 to 17 Sue fantastic so much knowledge I'm almost dizzy from all this info any
44:15
last keynote for a beginner looking at opportunities certainly Jane they shouldn't Miss creating opportunities from internships to Drupal careers by jw. vuru and Toria happening on
44:27
Wednesday March 26th from 1580 to 1550 you've nailed it Pete our listener will have a blast at Drupal con thanks to you huge thank you for joining our talk today always a pleasure Jane happy to
44:40
help our butting front Ender in their Drupal Journey so I think I will stop this video here but as you can hear there are still iterations that needs to be done you have to tell it to say the time in a
44:54
way that humans actually says time because right now now it's probably copying the value and it's the the text to speech mod models are not smart enough to uh uh understand that they
45:07
need to say it in a human matter so actually you have to write this into the script that it says 10:00 shouldn't say 10 o no one talks like that but um but you get the general idea so this is uh yeah it's a pretty cool showcase I
45:28
would say that the real use case of this because this is mostly gimmicky now but the real use case of this is of course that it uh say something like this especially I think you people in us
45:40
which uh who are at least if you're not in the big cities you're you're very dependent on course so you're uh you you can't maybe read well on your way to work so what this could do easily is that if you have a website uh where you
45:57
want to know what people are commenting on the website daily you could get a daily podcast pleas this so basically you know people talking about what people were saying on the website and
46:08
you can sit in your car and listen on to this on your way to work if you really want to know this before you come to work so there there's a lot of actual useful use cases for this as well I
46:20
think thank you
