0:01
hey there so Marcus here H and for once I'm on camera as well and that actually has a reason because we're going to use me as context so it might be important that you at least have seen me uh the
0:14
context we're going to use though is a image of me so this image where I wear glasses think about that as well when you see the end result uh and we take this web page as well which is the page
0:25
from the AI interpolators documentation that is about the the AI interpolator uh and we're just going to take this link and we're going to take this image and out from the AI inter
0:38
player we're going to have a video of a avatar of me talking in my voice in rhyme about the history of the AI Interpolator and this is completely dynamically created so you can take any link and
0:51
take any image and get the Avatar of that image talking in any kind of ElevenLabs voice uh about anywh website so while this is pretty gimmicky to be honest the D-ID module can be used for a
1:07
lot of less gimmicky stuff say for instance that you have a workflow that where you have a user that is highend for you that has registered on your website you could then have the CEO of
1:20
your company welcoming that person in a personalized video which is pretty cool right or you could actually do instead of a chat box but if someone ask a question you could actually answer with
1:33
a video answering the question so there's a lot of possibilities there um as usual this is open source this is free so this is based on Drupal which is a Content management system this is the AI
1:47
interpolator that I built on top of this uh so I will show the workflow end to end how to set it up it requires no code it's just basically clicking around to set it up uh but before we do that you
2:02
can be a judge for yourself if you want to actually continue with this so I will show the end result first in the realm of tech where ideas fuse and sprawl the AI interpolator
2:16
arose Standing Tall from merging of text it got its Birthright name now with 50 plus rules it's no longer the same it aims to create workflows chains that bind to generate content leaving none
2:29
behind with future dreams of Agents project leads in Jira and slack fulfilling all your needs finished so before I just uh start showcasing how to set this up I just
2:43
want to remind people that haven't seen it there's a website called workflows of AI uh so it has dashes in it where you can basically look at all the workflows I have done so far uh to see what you
2:58
can do with AI later so yeah look into this web page if you don't know it about it you can of course follow me subscribe on YouTube If you rather do this but on
3:09
this web page you can filter by which mod modules it has been used and so on so uh yeah look into that workflows of AI so um what we're going to use as context to start with is this page it's
3:24
part of the uh documentation for AI Interpol so uh it's if you just Google drup interpolator documentation or something you're going to find it there's a sub page here that's talking about the
3:40
history of why this was built um on top of that then the plugins can just show you here on available plugins we're going to use the ID to generate the video we're going to use ElevenLabs
3:55
field to generate the audio uh we're going to use OpenAI to generate a new image so it's going to describe my image and then it's going to generate a new image of this uh and
4:08
we're going to also use the simple crawler to basically crawl this page then so this is Drupal um standard installation I just installed these plugins then I have set
4:26
them up so if you go to web service D config for instance you have to add your D-ID uh key same with LM lab settings and so on and if you're not from the Drupal world just Google installing
4:38
modules in Drupal this is the image by the way I'm going to use as context so that's an image of me it's going to basically describe this and generate another image of this is the
4:50
idea um so we're start by adding a content type that we call Avatar video just disable some stuff here that's typical CMS stuff uh so if you're not from the Drupal
5:07
world this is the way you can do it in the gooy but if you want to create something scaling you should use something called custom entities remove the body field we don't need it in this
5:18
case we're not going to create a web page and then we start filling in the fields first of all the fields that are required to fill in so that's going to be the link so the link where it fetches
5:29
the col text we're going to use external links only we're going to disable link text save that uh sorry we're also going to I edit it again we're going to make it required because that's one of the
5:40
fields we need to fill in create another field that is required and that's the image field uh so this is IM no Source image we call that and we save uh we make this also required we we
5:59
don't need a alt field it's not used for our web page and then we're going to start interpolating so we start with the link to HTML so that's formatted text text format long we call that
6:12
HTML so where it's going to take the link now it's going to scrape it and put it into this HTML field right so we say full HTML we allow all HTML we're going to use the rule simple crawler it's
6:25
going to be in base mode based on the link field so it takes the link fields scrapes it and puts it into this field uh we add a user agent here I think. requires a use agent uh we do
6:40
article segmentation so that basically makes sure that we're just getting the actual article and not all the links here on the right side or the header and so on and we set a very low weight I'm
6:51
going to use batch mode in this case if you're doing this for real especially with the video you should do it on the CR job because it takes time to generate videos or in a q
7:02
worker so that's our first step oh let me check something I'm going to set a really low weight if I didn't do that as well no I did 50 because this has to run first right from this now we want to
7:17
generate an audio so I'm just going to do first of all I'm going to call this the real MP3 so that's the real audio that we're going to store uh uh this is because of how the ElevenLabs Field Works
7:34
I'm going to try to create ElevenLabs field to did rule so you don't have to take this uh step in between but for now I will do this basically you have to allow MP3 on this so this is going to take the
7:48
generat dialogue uh and put this in a MP3 right uh all right so this has a bug here this is not going to happen when you do it I need to fix a B here on the did
8:01
module uh and then we're going to create the ElevenLabs field and that's going to be in charge of generating a dialogue from the body of the HTML right so this we call
8:16
dialog and since this split up in many we have to set unlimited and that's why we want to concatenate in this case right uh so we're going to use the real MP3 and we're going to con concatenate
8:29
using ffmpeg enable the AI interpolator we're going to do the dialogue Creator so that creates dialogue in ElevenLabs we're going to base it on the HTML and then I'm going to write a prompt here
8:42
with the voice and here we need to use placeholders just search Marcus so I'm going to use my voice that I've already trained the levels on with the voice Mark is your hands on create a four
8:57
sentence rhing uh monologue on the history of the the thing written in the context uh and here we use the context and context in this case is the HTML so basically we're saying with this voice
9:29
create a four sentence rhyming monologue about the history uh based on the HTML right this has to be higher of course than the HTML we say bet share as well we use GPT for uh turbo because we need
9:47
good results GPT 3.5 just doesn't do good enough results in this case especially if we want rhyming uh and then we have the audio generated so then the next step comes
10:00
that we want to do a image description right so we take plain text TT plane long and this is then going to be enable AI Interpolator we're going to base this on the image so we do string long we do
10:19
Advance mode in this case oh so I have installed the token module that's something that you need to install to get the advanced mode up and running uh because in this case we're
10:29
not basing it on some other field text field we're basing it on an actual image field right so we just write a prompt here based on the image WR um write one paragraph
10:49
of uh for C actually I'm just going to do this I'm going to set this unlimited because sometimes it might generate a really it might split it up because when I'm asking for four sentences it might
11:09
read this that it wants to create four different values right so because of that I said unlimited so we don't just get one of the pieces so write one paragraph of four
11:21
sentences uh with a descriptive text descripted text describe being the person in the image in detail mention things like color of CR what type of CRS gender um
11:58
calculate a age um yeah hair color skin type glasses or other peral no per I don't know the word other uh other accessories um yeah that's that's enough also describe the background
12:39
shortly this is for an image prompt also add short uh depth of field in the end so that's what I have on my image maybe it forgets it so I just adds it there so he um
12:59
or it remembers to generate this and context then in this case no sorry I don't need a context I'm stupid we don't have a context it's just the image and under advanced settings now we set lower
13:13
weight batch it's important that you use GPT-4 Vision preview because that that is the only thing that can read images and here you can then set a image field to use and we use the SCE
13:25
image so this this will generate this then I will generate a file upload which is a image and this will be the created image then so this is what we're going to uh create the image on based on
13:45
that prompt that we had so once again I disable alt field we don't need it it's not a web page enable AI Interpolator DALL-E we're going to generate images we base it on the image description
14:02
context um and that's it we just take the image description as it is right set this to 90 batch DALL-E 3 uh we want HD and a neutral image and this will create the image and
14:24
the last thing we need to create then is another file upload which is the video Avatar which is a file that will be of type MP4 enable AI interpolator we use the ID
14:40
avatar from audio base mode we're going to use the real MP3 as the base field so that's the audio we created and then the expression of uh the person it generates
14:52
pH is neutral uh I'm going to use image field not the image URL and we're going to use the created image in this case right and we set the highest weight we set 110 for
15:05
just to be sure if I had 100 somewhere um and now it's going to chain all these things together on the manage display I will just make sure that I just show the
15:17
video we don't care about the other thing we just care about the result right uh and this should be a video then and we save that and then we're going to create this so we create the Avatar
15:30
video I have to set a title because I'm using content or notes content types in Drupal so I will just set uh video here and then link take this link this is the correct right yeah why
15:46
was it build future and uh then I'm going to take my screenshot here that I have and that's it the rest I'm not going to fill in that's going to be filled in now so now
15:58
it's going to start changing hopefully five rules yes because that's what we added four out of these five rules will be simple um to generate the video generation takes time just so you know
16:14
that and that's why you need to put it in a Chron job unless you have like me I have on my local server that it just doesn't time out the the the PHP worker or the engine x uh server so
16:29
but that comes with complications or security problems of course if you don't know much about Drupal but you thought this was cool still um and if this was too fast then please as I said before I
16:45
have this website called workflows of AI where you can look at simpler workflows to basically get the idea of what each of these field interpretations I'm doing
16:57
does and how chaining works because I do understand that the workflow I did now was very complex uh we will see if it works so I I actually when I do these videos I
17:13
don't edit and that's why you have to sit and wait now but that's good also it gives you a realistic view of how long time stuff takes to make um but that also means that if if if
17:25
there's something going wrong I will show that because with AI it's not deterministic which means that it can generate stuff that is not actually what I thought at the first
17:38
iteration you might have to reprompt and so on so because of that actually it's yeah it is what it is now something went wrong because the last rule should not take so long
17:54
time yeah so uh there was a cut there by the way I found the problem basically the image that has been generated is so big that it did won't accept it so I have added into the did Rule now that it
18:07
will compress the image so you will not run into this problem here you can by the way see that it has generated a lot of fields uh so it's just a video Avatar that we need to do so I'm just
18:21
resaving and we will see if it can also generate the video where I'm talking in rme well the description of me is talking and rying um so I just so you know I want as I
18:40
said I want to be honest when I'm showcasing stuff because I know a lot of startups are showing things that seem completely you know it's blowing your mind what it can
18:53
do with the AI but it's cherry picking results in this case I'm trying to show you what the AI Interpolator actually does and where you can run into problems so let's look at this in the
19:05
realm of tech where ideas fuse and sprawl the AI interpolator arose Standing Tall from merging of texts it got its Birthright name now with 50 plus rules it's no longer the same it aims to
19:17
create workflows chains that bind to generate content leaving none behind with future dreams of Agents project leads in Jira and slack fulfilling all your needs
19:31
finished right so we have something um for The Voice by the way you can there is ElevenLabs voice field so you can actually you could make that into a a selection here
19:48
uh actually let's try that as well I'm just going to do this makes the video a little bit longer but it might be interesting for you uh so I'm just going to create a new field here that we is
20:01
the 11 field voices and we're going to choose voice uh and let's see here uh we just take that so this is a dynamic field that gets generated automatically right
20:20
uh I'm just going to make it required I just realize we have to do to make sure it's filled out and the cool thing we can do then is that in the dialogue we can actually do copy this
20:40
but take advanced mode so context is the HTML right so we just take it in token mode and we take here value so that is filled in but then we're going to take also the voice here which is is here
21:00
voice value so now this is going to be dynamically filled in right the store is still going to be the same so I uh will just do well there's one more thing I will do
21:16
here so I'm just going to take the voice and put it up here after Source image so and I have another image I've been playing around with here that's huge Hugh Jackman Hugh Jackman not
21:40
huge uh Jackman and we do angular history maybe history of angular uh let's see here something I mean this is really long but that could be fun to see if it can take
22:03
such a long it should be able to uh and then we have the image here of Mr Jackman so it's this image where he's sitting down and we have to Cho choose a voice here so I'm going to choose uh uh
22:20
uh Scott Parker that's the voice I'm using for my blog uh for my podcast by the way so if you haven't seen this there is podcast cast workflows of ai.com where you can go and listen to a
22:34
a completely autogenerated podcast that comes weekly about Drupal news so just go to this podcast. workflows ai.com to get that uh and I'm just going to save now and this is the newest Caster's
22:49
voice in this case then and we will see what happens so this will take some time again I will just mute your myself I'm not going to say anything interesting you can just forward if you want in the
23:03
YouTube video to see the results e e right so doesn't look that much like you get one it looks like Lou Figo almost but let's see from Angular JS in its prime a
24:42
transformation we've seen over time I'll for to angular with its might making web apps a delightful site through versions and years it grew adapting as web standards flew this journey of angular's
24:57
flight a testate to its developers Insight right so that was just another example of uh this getting generated now this was very gimmicky of course but what you can use this for
25:13
because you can see it it's still pretty fast if you're a website for instance that caters to uh high paying individuals for instance you could have a personalized video of this CEO of your
25:29
company welcoming them to the website right so after they have registered there's a video the next time they log in where the CEO says hello Marcus Johansson in my case Welcome to our
25:42
website blah blah blah it's personalized right uh and there's a lot of other cool stuff I think you can do with this um so yeah thank you for watching
