0:01
hey there Marcus here uh today's workflow is very specific for people who works for Drupal uh specifically working with the media module so my idea is that we're going to take the document type so
0:15
PDF word and so on and supplement or basically Elevate the the metadata that exists for it so usually when you install it usually you just you know you have a title uh that you usually becomes the
0:31
the file name of the file you upload or if you manually fill it out then and and that's it and then you can of course add fields on top of this add it manually takes a long time so my idea here is to
0:45
basically automate the process um so what I thought I would add is uh a longer summary the whole text so the whole text from the PDF so it become searchable right because then you can
1:02
have it in Solr you can search for whatever is inside a PDF a shorter summary because maybe you need to uh preview it like one two sentences somewhere uh added search
1:16
words uh so it's you know you can add another field in Solr and make those specific search word more important because AI OpenAI is pretty good at figuring out what something
1:30
actually uh what what a document actually is talking about so so like the context of it right uh I will also create a preview image because a PDF usually comes you know with
1:47
uh like a PDF icon and there's no preview image so I will be a little bit creative and actually create an image for it as well so we can preview it nicely as well with done actual image um
2:01
so to do this well first I can show you I will just use some thing I found online which is some iPhone 14 pro PDF document uh with sales argument um and I will be using ddev so I installed
2:21
it you can go to d.org u i mean you can use Docksal you can use Lando whatever you want there but in this case I wish show you how to do it with d we're going to use two of the
2:33
inator so we're going to use AI Interpolator ConvertAPI which is a simple module that basically makes sure that we can take a PDF and generate text from that uh there are other solutions for these
2:47
but these are not native to PHP you usually have to install some application on your servers which you might not have access to actually do if you're on manage hosting so we're going to use the
3:00
ConvertAPI and then we're going to use OpenAI to basically uh fill out all these fields then so let's uh get started on our site so I can show you here I've just
3:17
generate the site basically we're going to log in uh first thing we're going to do is do DD well I'm going to copy it here so what I need to do is basically copy this
3:32
composer Command right so I'm going to get the ConvertAPI and then I'm just going to copy the open AP OpenAI here as well so ddev and ddev is a prefix because I'm using ddev so it depends on
3:48
what uh local environment you're using basically then we go to extends we're just going to enable this then um oh I will uh I have already got an admin toolbar I
4:02
always use admin toolbar you should do as well uh so we take ConvertAPI and OpenAI here and then we're going to enable media as well because I don't think that's enabled by default yeah
4:15
media and install and this will install OpenAI Core and some some other modules that uh are dependent on them then so first thing it does is complaining I don't
4:28
have open uh AI key so I'm just going to add it here so uh let's log in here we log into API oh I was already logged in it looks like so I'm just going to get the API
4:48
Keys create one and just so you know this API key is here you can try to copy and use it I'm going to remove it after you you looked at this conversion and we just call
5:02
that copy it and we just put it in here API ke save then it's the same then for our uh web services ConvertAPI config so same thing here login I'm already logged in and then it's under
5:22
authentication API secrets we just copy this and I think this is two Val no it's just one okay so um oh we can close those now close all these windows basically so now we have
5:41
set up that so we actually have a media here now so there is a document already that has the uh uh where you can upload documents right and this takes PDF so we
5:52
can just start by trying this out we're just going to add a media here that's a document I'm going to upload this um file that we had there save it and you're going to see now you know it uh
6:07
it did upload but it's doesn't look that I mean you don't have that much information you have the information that the file name is this and you have the file so now we need to add some
6:22
fields to this so we go to Media types and document manage Fields I will start with a form the TT long that is long description and I will set that to full HTML enable AI Interpolator so that's
6:43
the thing that's going to uh take some text oh now I went one step uh I have to do the conversion from PDF to text first so I made a mistake here I'm just going to save this not enable
6:57
yet because the first thing we're going to to do is that we're going to create a plain text text plain long that is the original text there is one thing you should know PDF is
7:11
horrible as a structured format so doing the conversion from PDF to uh text can uh give you very bad uh results because it's it's not very clear how everything tracks horizontal to vertical
7:29
and so on might be that it reads this first and then this because it thinks this is a row and then it reads 100% 100% 100% right uh it depends a little b a little bit about how it was set up so
7:45
it's actually a quite horrible uh format just so you know so the results might not be super but let's see so I'm going to enable AI Interpolator and I'm going to enable uh
7:57
text from document and we're going to use the document as the uh field where we're getting the text from and we make sure that the weight is 100 I'm going to turn on batch
8:09
mode because we're going to add a lot of layers now so we don't want to have a timeout so if we were to do direct it tries to store everything in one uh server request which is not optimal so
8:22
we save this now then I can go to Long description enable AI Interpolator and use OpenAI text long and we use the original text as the base and based on the context text
8:42
create uh one paragraph no create oh sorry create 16 sentences of summary of it context and the context then is just the text that we just got from the PDF so I'm just going to write copy this
9:08
here which is the context which is then the text is going to be replaced when this is run we take inter Play We 101 because this has to happen after uh we got the text from ConvertAPI right uh
9:23
I'm going to take GPT-4 Turbo because it might have a too long context uh otherwise batch because the amount of words here might be over 16,000 which is what GPT 3.5 handles otherwise GPT 3.5
9:40
is actually quite okay at summaries uh it's very bad at other stuff but summaries is very okay yet so we're going to save this and then I'm going to create another one that is then
9:53
the short description by the way Drupal just changed how everything works here so I'm still a little bit not accustomed to it yet um short description it's better but
10:09
yeah uh full HTML enable AI interpolator OpenAI takes long we base it on the original text we could in we base it on the long description as well uh actually let's do that just to save some money
10:24
because the long description is still a lot shorter than the PDF right based on the context write a two sentence summary of uh of the context I guess that's okay I mean this is prompt engineering
10:51
you can get better results when you try try around and try to get the correct pront basically so in this case I now have to set this to 102 then because the long summary has to exist before the
11:03
short summary and in this case we can just try GPT 3.5 Turbo because it's going to fit in the context bace uh next thing we wanted to add was search word so I'm going to use plain
11:15
text in this case and I'm going to do short so I'm just going to split it up in multiple words that you can uh control basically um search words and then I say unlimited here because you you you can
11:31
add as many as you want I'm going to use OpenAI string I'm going to base this on Long description as well based on the context text suggest uh let's say 10 to 15 search words that are unique and
11:52
important when searching for this document and then we add context here so same thing here this has to be 1002 at least right because the description has to uh exist
12:16
before in this case I will use GPT 4 because it's it's going to create better words basically it's going to know how to create better words save this um the last thing we want to do now well
12:34
it's two-step thing so we want to take the plane text we can take text plane long and do uh header image no sorry header image prompt is what I want so this is going to create a prompt that
12:47
we're going to use to generate the image right we enable AI Interpolator once again OpenAI string long once again long description once again based on on the context um
13:04
gener generate a one sentence image prompt uh of an artistic kind for a uh preview image of the text be abstract and and uh have a futuristic uh futuristic style let's do
13:36
this this time context once again we fill it out with a context so the long description is basically doing all the work in this case so if if that is not uh very well
13:51
written the rest will actually kind of fail in this case and we do batch we do GPT 4 turbo where are you there save this last thing we want to do then is we want to do a image field that is there
14:08
preview image um I just enable everything here and disable uh OpenAI DALL-E we're going to use the header image promp and we just take it as it is since the prompt to generate the image and
14:28
this way have to set to 103 because this has to exist even later so this is the last thing we're going to do uh yeah I'm just this yeah okay let's do horizontal we do neutral let's do HD
14:43
standard see if we can get something nicer might take a little bit longer but that's fine uh and that's it basically we can keep everything in manage form display
14:53
in manage display I don't want the header image promt to show up uh or actually I'm not sure if anything there should drop but we can wait let's just keep it there so we can actually see the
15:05
result because how you style this as I said in the beginning this is for people that already know Drupal so you know how to style this uh show which Fields you want and so on so we're just going to
15:18
Well I will do one funny thing here I will just edit this view so we're not going to take the built-in thumbnail but the preview image and and let's do thumbnail here
15:32
link image to [Music] content and we just rearrange this put this instead of the thumbnail apply save so now there's no previe image but hopefully after we resave now it's going
15:47
to fill out all these fields so you see oh this didn't look too nice though maybe we should pull it up a little bit I wonder where the fields were even uh uh let's just pull
16:00
up everything here to the top hopefully that looks better so you can see what I'm talking about here let's see there we go so you have the document uh this is well let's just
16:21
order them in the order that it's happening because I created them in the wrong order so this is then going to uh because it recognizes it exists a file and it recognizes that original text is
16:35
empty it's going to ask ConvertAPI now hey please I need a text it's going to fill out this then the next step is that the long description is going to basically ask OpenAI hey create a a
16:47
long description from the original text short description ask open a hey create a short description from long description search words hey create search words from the long description
16:57
other image hey create create the header image prompt from the long description and then the oh it should be preview image prompt I'm not consistent here anyway preview image gets generated and
17:08
so if I click save now it's going to start and this will probably take some time now so I should probably have explained that while I was waiting for this because I have no nothing
17:20
interesting to say now so let's see um because the first step is pretty fast so converting the second is probably going to take some time because the document as you can see it is pretty
17:36
pretty large um so it will take some time so one thing that's interesting maybe if you if you're running like e-commerce is what you could do in this case is that if you
17:49
have something like this already uh for all your products but you don't have very good uh selling text you could generate a text field and basically ask it based on the manual or
18:02
something just generate uh uh selling text and uh one more example for commerci while we're waiting okay now it's finished but anyway I would just mention it in the OpenAI module there is the
18:17
FAQ field so there is FAQ for Drupal right so if you want to have a frequently Asked question about your uh each product in your Commerce you could use this method as well it takes a PDF
18:33
and just generates uh frequently asked questions based on the manual of a product right so maybe there's a troubleshooting section it can just get that then anyway you see now there's a
18:44
preview image uh it started first of all here we have the full text you can see that the structure is not super it's because it's PDF if we were to use word it would work better but I wanted to
18:58
show actually the worst case scenario so you can uh see what you can expect okay this failed spectacularly actually uh let's rerun it then I don't know what happened as I said when the
19:14
long descriptions fail everything fails so we just going to remove all these because then it runs again right so so and we save again in because then it reruns again so the the initial prompt of
19:34
creating the long description there is one thing that's very important to think about when you're prompting here and I will see if it fails again I have to reprompt
19:43
basically that the AI interpolator or the open a OpenAI version and most of the plugins are made to also be able to create multiple Fields because this is important if you do uh use the SERP
19:57
plug-in for in inance you want all the different websites that it finds you don't want just one right or if you want to create a gallery of images you don't want to create 10 fields for that you
20:08
just want to create one field with multiple files of course this is true as well when you generate text so you have to be very clear where you say how if it should be split up or or or not so if
20:21
you for instance mentions create 10 paragraphs it might try to create 10 different text areas right and this will of course not work if you only set one because it will
20:34
only save the first one okay so this still fails so let's retry this oh wait wait I have to rewrite first the prompt before we go anywhere else so I want to be honest with how I'm
20:54
using the uh the AI Interpolator this is not a sales pitch you you're you know I'm not it's it's free I'm not earning any money on this right I I I just use it myself a lot so would be happy if you also use it
21:12
so because of this all the videos I do they're not pre-planned so stuff can break and and I guess it's useful that you see that sometimes it breaks and I would show you maybe how I can fix
21:25
it so based on the Contex create 16 sent well let's write like this create a summary of uh up to 16 sentences in one paragraph so we're going to be very specific that this one
21:45
paragraph now let's see what happens just remove all these remove this and run again let's see if it works better so by the way also now you see there's one less uh
22:06
process that's running so from the beginning I think it was six and you're going to see when it's finished with the first one that it did one out of five because the conversion worked right so
22:16
that we don't have to run again I never emptied that field so but if I would want to then run it again I could empty it of course so let's see if my changes now did any
22:33
difference to the output yay okay it's still not very very good this is interesting why is the summary short I probably did some error there so it's full HTML OpenAI long
23:01
description based on the context create write a two sentence summary of the context oh you see what I did there I should copy and paste so we save this and then I'm just going to remove this
23:15
so we run that again but here you see now it actually worked uh I'm just going to Res sa this so if you do this based on my video you know what not to do at least now
23:29
um so now we have a little bit longer couple of sentences here we have two sentence version Oh this is another thing so another thing went wrong here actually uh GPT
23:42
35 so basically we're running Json in the background right uh because we need a structure format to do this and J chat GPT uh sorry GPT 3.5 is not always understanding in how to do this
23:57
I'm still sitting experimenting to try to get this perfect that's one of the reasons I haven't done this production ready yet so if this ever happens that it shows you chase like this happened
24:08
again let's try again then you can just remove it and resave it and in worst case if it happens too much you can just okay this was even worse you can just run uh GPT-4
24:22
basically and here you see by the way it's just rerunning this one okay so I'm just going to change this to uh GPT 4 because GPT 3.5 seems to have problems with this
24:37
currently so GPT 4 see if that works better so yeah as I'm said I I mean I could rerun this recording now make it perfect but I want to show you how it actually is to
24:55
use it because it's not always perfect it's always better if you have the money to G use GPT-4 but I understand that people want to save money and it's a lot cheaper to use
25:07
3.5 so I I am trying to perfect this prompt that runs in the background and if someone wants to help you know the code is open source you can just download it and check what I'm trying to
25:18
do in the background with these prompts because it's not just running the prompts you see uh that you put into the U it's running a lot of stuff around that to get it uh in a structured format
25:31
so it fits fits complex Fields basically so anyway now we have the document we can put this into the search right so if you use Elasticsearch Solr search API whatever same with these search
25:48
words uh is also stuff that's basically important for this specific uh document based on what GPT for things then of course uh we have a short description that we could use for a preview we have
26:04
a preview image here wait let me click on it full screen it so yeah maybe maybe I need to prompt that a little bit better I'm not sure if that's a preview image you want to show
26:19
um but it's just to show you the idea and here you have the header image prompt by the way then um so so yeah that's it this took a lot longer than I thought but it's as I said it's good to
26:33
that I show that sometimes stuff doesn't work first try uh first time you try it and then it's you can see what you can change because this is also one thing that's
26:46
very important to note compared to any other development I have ever done uh I worked on super big projects and so on everything else is deterministic where I can say basically
26:58
based on some project managers or product owners initial idea is this possible yes this is possible you know this is not possible yes I have to research or whatever AI in that sense or GPT large
27:13
language mod models they are not deterministic if you ask for one thing it might answer something else depending on on how you ask matters as well right so that's why prompting is really really
27:25
important and basically doing trial and error on this so yeah so I yeah I hope you find this interesting uh you know now how to basically get the media model uh
27:41
sorry the media entity to get a lot of metadata to it so thank you for watching
