0:01
hey there Marcus here um so I've been on vacation start a new job and I've been sick so it's been a while uh but now I'm going to show you something uh completely new uh that it
0:16
has some connections to AI Interpolator but it's it's a New Concept which is this part of the Hugging Face module which makes it possible to use Drupal uh not have a GPU at all and to
0:31
be able to fine-tune something and take that fine-tune model and use it inside Drupal to generate stuff uh and this is completely inside the GUI no coding needed uh no server
0:45
setup knowledge or anything like that needed um so the reason that I'm it's in the Hugging Face module is basically that anything that it generates will be hosted at Hugging Face so if you if
1:03
you're not aware of these Concepts basically Hugging Face is uh you can think of it as GitHub for AI models so it's a repository where you can host uh your private AI models uh but it also
1:16
has a lot of public AI models it has uh open source AI models and what we're going to do is we're going to take one of these models uh in this case Stable Diffusion uh any
1:30
of the different versions which is a text to image model and teach it A New Concept so that's what you can see that I have done here so I have fin you on images of
1:42
myself I've called myself Marcus Johansson JJJ so it's it's it's its own token because there is a famous Marcus Johansson NHL player so I needed to basically have my own token not to
1:56
delute with the knowledge about him and then I can write something like that a photo on me in front of the Taj Mahal and it creates this right uh and I can use the AI Interpolator to also run this
2:09
specific model and do it inside Drupal as well uh so that's basically what we're going to uh what I'm going to Showcase today so how this works for you as a user is basically you install huging
2:25
face model uh module sorry there is a page you go to that you describe your New Concept you describe which base model you want to base it on you upload a lot of
2:36
images uh you crop these to focus on whatever it is you're trying to uh generate you click on start finetuning and you wait some time and then it's going to show up on hugging
2:50
face and then you can use AI Interpolator to generate images with this um so what's happening in the background then is this that when you click generate it's going to start a VM
3:04
in Runpod and Runpod is just a cloud provider that's specializes in AI uh so it specializes in GPU servers so servers you most likely don't have available uh and that are freaky
3:21
freakishly expensive and that is the other reason why I chose Runpod because Runpod has a credit system which means that you have to upload credits um to be able to use it
3:34
because even though I'm doing open source and open source clearly states I'm not responsible for how you use my code right I don't want to have this scen scenario that someone looks at this
3:47
video starts fine-tuning something there is some bug in there that I didn't know about uh and it just dies halfway through and they don't understand the concept of cloud at all so they don't
4:02
understand that oh there was uh the garbage handling garbage collection didn't work here so there's still a server running and these servers as I said are frequently expensive and this
4:13
person at the end of the month all of a sudden you know get a bill of $2 $3,000 I don't want that to happen so because of the credit system it means that you upload $10 for instance first
4:25
case you lose $10 if there's a bug like this um but the module also has a very clear checkbox that you have to check that you're aware that this might cause costs
4:36
and that you should also manually go in and check so uh the VMS are removed automatically otherwise you should remove them manually but anyway this creates this
4:49
pod takes this Docker image as the image um and just so you know I do everything open source there is open source uh rep for this if you want to see what the docker image is doing so it's not you
5:04
know stealing your images or something like that um or more importantly it's going to have the Hugging Face API token on there so I don't steal that of course uh so you can see that it's not doing
5:17
that does the fine tuning when it's finished with the fine tuning takes this mod mod uh model and pushes it to the Hugging Face reple so that's what it's doing um and the advantage with this of course
5:33
is that you you know you don't need to Learn Python a lot of this stuff is uh it's not very user friendly so a lot of the AI scientists which are not usually developers they are very
5:48
very very very very good mathematicians uh uses python because that's usually a language scripting language that you learn and um and because of this they might write a
6:05
script that you know generates this but they don't think about usability at all so because of this you kind of need to know python to do this otherwise there are some UI tools for it but most of it
6:17
is anyway based on python at best maybe you can get a Docker image which you can just start um so in this case you don't have to think about it then plus you don't have to have the GPU server then
6:31
uh there is one more thing before I'm going to show you how to set this up then and that there there is this affiliate link um I'm not in any way trying to get twitch on this then I wouldn't be doing
6:44
open source that would be very very stupid but when it comes to most of the AI uh modules usually I can ask for like can I get a free account here just to try out some stuff and sometimes they
6:57
give it to me so I don't lose any money in this case I lose a lot of money testing around and I want to do a lot more uh with run pods in terms of one fine-tuning so currently it's only doing
7:11
image models I want to be able to do large language models which cost a lot more money because they require a lot of more uh memory better Hardware uh and the other thing I have
7:24
done privately or in the company I used to work for is working with Runpod serverless which is a way where you can take a model host it but you only pay for when you actually use it so it it
7:37
works similar to Amazon Lambda just that it has a GPU uh so I want to build open source stuff with Drupal around this as well it would be nice to not lose too much money
7:47
on this so if you're anyway interested in using this and you don't mind then please use this uh link because from everything that you spend I will get 3% in credits so it's not money I get I can
8:03
take this out as money but at least I don't lose as much money uh when I'm doing this open source stuff um so that's it let's get started then on how this works so let's start this then uh this
8:24
is a plain Drupal standard profile admin toolbar install that's all we just need the Hugging Face module um I'm running D in this case but whatever local environment you have you
8:39
just use composer to get it right uh then the Hugging Face module itself is just API module and this comes with sub modules that can use it so there's a augmentor um plugin for it a module for
8:54
it but what we're going to do in this case we're going to use the Runpod uh fine-tuning module that is a module that's using it and the AI Interpolator Hugging Face module that's using it and
9:05
these have dependencies that doesn't get loaded automatically so I will just get the base AI Interpolator in this case and also for the uh uh fine-tuning I need image widget crop to be able to uh
9:21
specify what part of the image that I want to find Jun right then I just after getting this I click extend uh then there is Runpod fine tune I need to install an AI
9:35
interpolator Hugging Face those two modules are what I need one for to fine tune and one to generate this is going to install a lot of dependencies as you see um just
9:47
approve that uh and the next step is we have to go to web services Hugging Face config so we have to add our access token so we're able to both well save the model oh sorry I did I click new
10:02
organization settings is what you want but also to upload uh well create the finetune but also to read that fine tune when we're using the a interpolator access
10:13
token uh we create a new fine tune here uh fine tune temp I'm going to remove this after so you can't copy it and use mine uh we're going to show that copy paste it here save and we need to do the
10:29
same same them for Runpod um it's important that you have money in your Runpod otherwise this will not work so you should fill up with like $10 there that's under billing I think uh anyway
10:40
I'm going to click settings API key create one read and write same thing here it's going to be gone by the time you see this uh and this needs to be able to write of course because it needs
10:53
to start servers then it's just under configuration content authoring fine tune Dreambooth uh this is going to have a fine tune models later and Dreambooth llms and so
11:04
on as a sub uh uh directory but uh yeah if you can think of a better place to put it please let me know so because I have the token it knows that I my name space is Ivan boring Swedish comedian by
11:18
the way and then I'm just going to uh here create airpods uh so what we're going to do we're going to take a lot of images of uh this is this is getting recorded in the middle
11:31
of April so if you see this in the future this might be completely off but there's rumors of a airpod earphone from Apple that's going to have a display on the case some people have rendered
11:43
images of this and I want to take those and be able to generate my own images of the so-called future airpod then visibility status I said to private uh no I actually will do this public so
11:57
if you ever go there you can actually try this afterwards to see how how great it works table diffusion 2 we're going to base it on right now it's just 1.4 and two we're going to I'm going to add
12:08
so you can add a lot more text to image models that's based on anything that Dreambooth then can uh find you on and this includes custom models as well because if I find you in this
12:20
airpods and I realize I also want to fine tune on a a a future iPhone that might get uh generated but I want those two things in the same model so I can have the airpods with display and the
12:33
iPhone next to each other uh I will be able to F tune that concept on top of the model I already did right and here we have to write something then that connects to it so I
12:46
it's already stored here a phot of the in ear earphones so this basically adds the taxonomy of what this is right this is part of the taxonomy in your earphones and I just set that custom
12:59
token to this because airpods already exists and I could of course do airpods with display but it wouldn't know be so certain if it's the airpods with display that I'm uploading or if it's the actual
13:13
concept of airpods with display so because of this I generate a unique token I don't think this uh token or this word has ever existed inside the training data so that's why
13:27
then um just just going to go here and upload a lot of images because I'm using Drupal here it doesn't have to be PNG 512 or 7 768 or whatever is the very specific things you need if you run this
13:43
as a script you can just upload any type of image then image Styles will take care of making this so this will be the next step there is advanced settings if you're using Stable Diffusion 1.4 you
13:55
could write uh RTX 309 in this case because you don't need such an expensive model you don't need so much vram uh and you can set something else there uh if you're going to use Stable Diffusion 3
14:10
was just released if this will be able to fine tune it probably needs even more stuff then there's a lot of uh detailed fine-tuning uh parameters you can set here you have to read about this if you
14:24
if you're not happy with the result start reading about this and you might get better result basically anyway we will just leave it as this is as it is click to The Next
14:35
Step which is basically the cropping then so we will make sure we just focus on the stuff this is probably not going to be the best results because sometimes as you can see the earphones are outside
14:48
of the display case sometimes like this I have to choose if I want some text in there or not uh so if you want a perfect mod model you should should um fix the images in Photoshop first uh and in this
15:05
case also everyone is guessing different you see this is a monochrome display this is a color display that's a square this is rectangle so on uh so uh this will probably not be the best
15:18
result but I will just showcase this as a example of what you can do uh and when I am finished with those there is this check box that I need to check also on so this is from my
15:33
perspective what I talked about before that you do understand that this cost money this starts machines and if something fails during this run you have to manually go in and stop these
15:43
machines but anyway I will cck uh click Start fine tuning now this will take forever I'm going to fast forward this I'm guessing with a Stable Diffusion 2 model takes around 30 minutes it's on
15:57
step two now waiting for the Runpod machine to build and start so the funny thing is if we go to Runpod and click pods now all of a sudden you have a pod here so it's actually starting this pod
16:08
now and this is then the place if you click here and click stop and then under here there's going to be terminate pod if you ever need to do that manually right uh but anyway this will take 20 30
16:21
minutes now but for you it's going to look like it takes 10 seconds so we're finished I will put a success message here I realized I forgot that that's not very good that you don't get
16:55
any feedback that it's done but uh we're just going to verify two things so first we go to my pods reload and hopefully there's no pods left so it has done the garbage collection remove the Pod double
17:09
check this anyway and remove it if it's still there for whatever reason you see it's costed me some money here uh then we're going to go to Hugging Face go to my profile and we
17:24
have a new model called Ivan boring airpods uh and actually uh because we're using Stable Diffusion in this case we can actually use the inference API here just to try
17:37
it out so we could try it here a photo of the uh in ear ear phones uh what did we call it it's here airpods with display uh on a table we just press here this free
17:56
version might or might not be a available it needs to start machine sometimes uh to be there it worked in this case so see we have it here I'm just going to show you so we can compare
18:09
so we take Stable Diffusion to Hugging Face so we're going to compare with the exact same phrase uh on the base table diffusion too let's see if this is started
18:24
otherwise it might take some time um because then you can compare and see that it's not aware of this so you see it it kind of figures out its earphones right but it's not the the airpods that
18:38
we generated so it has generated it here I mean it's not perfect as I said before uh you need to have better pictures to create a perfect uh model you need more pictures than images than
18:51
10 as well so anyway we have now uh fine tuned the model so now come comes next step we want to start using this mod uh uh model as well right so I'm just going to
19:08
create a new content type here I'm going to call it um airpods image generation right uh and uh I'm just going to disable some stuff here and I'm not going to uh do that
19:30
much chaining if you want to know how to chain stuff uh look at my previous videos if you like AI stuff please subscribe I will release a lot of Drupal AI videos in the future hopefully as
19:43
well but um if you want to know how to chain so you can have stuff like you know asking in natural language for uh the the leaked airpods and this becomes this uh token that we have airpods with
19:58
display this is something OpenAI or some other LM can fix for you before you create the prompt but right now I'm just going to create a plain text uh prompt take a long
20:11
one uh we call this image prompt and we just save this oh we should set it required maybe so like we know we have to fill this out and then I just create that image field and that's where
20:28
we want to generate this right so we take file upload image and we call this the generated image I have it here in my history and we're going to use the AI Interpolator here so in this first case
20:46
I'm going to show you uh how to use it for free so we're going to use Hugging Face text to image right uh we use base mode and we use the image prompt and we just going to cop copy it as it is so we
21:00
just use context we're not going to add anything else uh we say 100 and we're going to save it directly and uh we're going to uh use the free API inference in this case and
21:16
then we just need to basically search for our model here so Ivan boring air uh pods oh this might have a cache let's see if we can find it at least if I save here yeah so um this might actually take
21:34
some time before it shows up and now to complete because it's cached but anyway it's there it's validating that it's there uh and then we just need to create this we need to add a title here because
21:47
of uh we're using a node node is probably not what we should uh be using here uh and then we're just going to take this and on our table do the same thing again save so if this gets timeout now
22:06
it's because you're using the free version and the free version uh you know if they have service available they will generate it and that happened in this case sometimes they
22:17
don't have servers available and you will get a timeout and then you have to redo it again so say that you actually want this but you want it stable I will just show you how this works then so
22:28
what you you can do with your private mod model now is you can take deploy and uh take a inference end point dedicated uh you have to choose a machine here and this depends on the
22:41
model so 16 GB is okay I think should be okay for Stable Diffusion inference Stable Diffusion 2 we just yeah I just this are okay I just make it public protected is okay as well because you're
22:56
using the tokens um and I'm just going to create this end point and you see it cost 50 cents per hour so make sure you turn this off because here there are no credits this
23:10
will actually just start costing money uh right away and we just have to wait here and this is now going to create that end point then and the only thing that's
23:24
different is we go back to the content types manage Fields uh uh the generated image and under the settings instead of API inference we take dedicated inference and we're just going to have
23:39
to copy this endpoint URL whenever it starts let's see this might take some time to start Maybe e and yeah we have a generated image here um doesn't do text very well as you can
24:57
see so uh that's it basically uh how you can fine-tune a model and use that module model to generate custom images using some completely new uh fine tune token and also how you can set up dedicated
25:15
machines that can do it then so once again thank you for watching if you like it subscribe um and see you in next video
