--- url: 'http://workflows-of-ai.com/blog/improve-media-1' title: 'Improve Media #1' author: name: 'Marcus Johansson' url: 'http://workflows-of-ai.com/about/marcus-johansson' date: '2024-02-19T14:44:28+01:00' updated: '2026-10-10T12:14:04+02:00' type: article summary: 'Extract the text of a PDF with ConvertAPI, then let OpenAI write summaries, search words and a preview image for Drupal document media.' tags: - 'Marcus Johansson' - Workflows - 'Drupal AI' image: 'http://workflows-of-ai.com/sites/default/files/previews/5895fc9309e1f359.jpg' published: true og: site_name: 'Workflows of AI' 'image:width': '1280' 'image:height': '720' 'image:alt': 'Improve Media #1' --- Drupal AI # Improve Media #1 Fill the Document media type with full text, two summaries, search words and a generated preview image, without typing any of it. Marcus Johansson February 19, 2024 ยท 2 min read ## About this video The Media module in Drupal comes with a lot of power to keep a great media library for your website. It's also fieldable, which means it's easy to add metadata for previewing and searching the data. But filling out that data takes time, and with the AI Interpolator you can automate it. In this example we will fill out a search text field with the actual content of a PDF or Word file, to make it searchable without needing Apache Tika or something similar. It's done with the AI Interpolator ConvertAPI. Then with AI Interpolator OpenAI we: 1\. Add a long summary of the text. 2\. Add a short summary of the text. 3\. Add some context-aware search words, that you can use as a weighted field in Search API. 4\. Add a preview image of the PDF. *This video is from before the AI Interpolator module became AI Automators, part of the [AI module](https://www.drupal.org/project/ai). Module and menu names have changed since, and some steps look different today.* ## Watch the video [Watch on YouTube: Improve Media #1](https://www.youtube.com/watch?v=7wh8iFJiShQ) ### Video chapters - 0:00 The plan for document media - 2:18 Installing the ConvertAPI and OpenAI modules - 4:27 Adding the API keys - 6:22 Extracting text with ConvertAPI - 8:23 Long and short summaries - 11:10 Generating search words - 12:31 Preview image with DALL-E - 15:18 Showing the preview in the media view - 19:04 Fixing the failing prompts - 25:37 Results for search and previews ## Transcript YouTube's automatic captions, lightly corrected. Click a line to jump to that moment in the video. ### Transcript - 0:01 hey there Marcus here uh today's workflow is very specific for people who works for Drupal uh specifically working with the media module so my idea is that we're going to take the document type so - 0:15 PDF word and so on and supplement or basically Elevate the the metadata that exists for it so usually when you install it usually you just you know you have a title uh that you usually becomes the - 0:31 the file name of the file you upload or if you manually fill it out then and and that's it and then you can of course add fields on top of this add it manually takes a long time so my idea here is to - 0:45 basically automate the process um so what I thought I would add is uh a longer summary the whole text so the whole text from the PDF so it become searchable right because then you can - 1:02 have it in Solr you can search for whatever is inside a PDF a shorter summary because maybe you need to uh preview it like one two sentences somewhere uh added search - 1:16 words uh so it's you know you can add another field in Solr and make those specific search word more important because AI OpenAI is pretty good at figuring out what something - 1:30 actually uh what what a document actually is talking about so so like the context of it right uh I will also create a preview image because a PDF usually comes you know with - 1:47 uh like a PDF icon and there's no preview image so I will be a little bit creative and actually create an image for it as well so we can preview it nicely as well with done actual image um - 2:01 so to do this well first I can show you I will just use some thing I found online which is some iPhone 14 pro PDF document uh with sales argument um and I will be using ddev so I installed - 2:21 it you can go to d.org u i mean you can use Docksal you can use Lando whatever you want there but in this case I wish show you how to do it with d we're going to use two of the - 2:33 inator so we're going to use AI Interpolator ConvertAPI which is a simple module that basically makes sure that we can take a PDF and generate text from that uh there are other solutions for these - 2:47 but these are not native to PHP you usually have to install some application on your servers which you might not have access to actually do if you're on manage hosting so we're going to use the - 3:00 ConvertAPI and then we're going to use OpenAI to basically uh fill out all these fields then so let's uh get started on our site so I can show you here I've just - 3:17 generate the site basically we're going to log in uh first thing we're going to do is do DD well I'm going to copy it here so what I need to do is basically copy this - 3:32 composer Command right so I'm going to get the ConvertAPI and then I'm just going to copy the open AP OpenAI here as well so ddev and ddev is a prefix because I'm using ddev so it depends on - 3:48 what uh local environment you're using basically then we go to extends we're just going to enable this then um oh I will uh I have already got an admin toolbar I - 4:02 always use admin toolbar you should do as well uh so we take ConvertAPI and OpenAI here and then we're going to enable media as well because I don't think that's enabled by default yeah - 4:15 media and install and this will install OpenAI Core and some some other modules that uh are dependent on them then so first thing it does is complaining I don't - 4:28 have open uh AI key so I'm just going to add it here so uh let's log in here we log into API oh I was already logged in it looks like so I'm just going to get the API - 4:48 Keys create one and just so you know this API key is here you can try to copy and use it I'm going to remove it after you you looked at this conversion and we just call - 5:02 that copy it and we just put it in here API ke save then it's the same then for our uh web services ConvertAPI config so same thing here login I'm already logged in and then it's under - 5:22 authentication API secrets we just copy this and I think this is two Val no it's just one okay so um oh we can close those now close all these windows basically so now we have - 5:41 set up that so we actually have a media here now so there is a document already that has the uh uh where you can upload documents right and this takes PDF so we - 5:52 can just start by trying this out we're just going to add a media here that's a document I'm going to upload this um file that we had there save it and you're going to see now you know it uh - 6:07 it did upload but it's doesn't look that I mean you don't have that much information you have the information that the file name is this and you have the file so now we need to add some - 6:22 fields to this so we go to Media types and document manage Fields I will start with a form the TT long that is long description and I will set that to full HTML enable AI Interpolator so that's - 6:43 the thing that's going to uh take some text oh now I went one step uh I have to do the conversion from PDF to text first so I made a mistake here I'm just going to save this not enable - 6:57 yet because the first thing we're going to to do is that we're going to create a plain text text plain long that is the original text there is one thing you should know PDF is - 7:11 horrible as a structured format so doing the conversion from PDF to uh text can uh give you very bad uh results because it's it's not very clear how everything tracks horizontal to vertical - 7:29 and so on might be that it reads this first and then this because it thinks this is a row and then it reads 100% 100% 100% right uh it depends a little b a little bit about how it was set up so - 7:45 it's actually a quite horrible uh format just so you know so the results might not be super but let's see so I'm going to enable AI Interpolator and I'm going to enable uh - 7:57 text from document and we're going to use the document as the uh field where we're getting the text from and we make sure that the weight is 100 I'm going to turn on batch - 8:09 mode because we're going to add a lot of layers now so we don't want to have a timeout so if we were to do direct it tries to store everything in one uh server request which is not optimal so - 8:22 we save this now then I can go to Long description enable AI Interpolator and use OpenAI text long and we use the original text as the base and based on the context text - 8:42 create uh one paragraph no create oh sorry create 16 sentences of summary of it context and the context then is just the text that we just got from the PDF so I'm just going to write copy this - 9:08 here which is the context which is then the text is going to be replaced when this is run we take inter Play We 101 because this has to happen after uh we got the text from ConvertAPI right uh - 9:23 I'm going to take GPT-4 Turbo because it might have a too long context uh otherwise batch because the amount of words here might be over 16,000 which is what GPT 3.5 handles otherwise GPT 3.5 - 9:40 is actually quite okay at summaries uh it's very bad at other stuff but summaries is very okay yet so we're going to save this and then I'm going to create another one that is then - 9:53 the short description by the way Drupal just changed how everything works here so I'm still a little bit not accustomed to it yet um short description it's better but - 10:09 yeah uh full HTML enable AI interpolator OpenAI takes long we base it on the original text we could in we base it on the long description as well uh actually let's do that just to save some money - 10:24 because the long description is still a lot shorter than the PDF right based on the context write a two sentence summary of uh of the context I guess that's okay I mean this is prompt engineering - 10:51 you can get better results when you try try around and try to get the correct pront basically so in this case I now have to set this to 102 then because the long summary has to exist before the - 11:03 short summary and in this case we can just try GPT 3.5 Turbo because it's going to fit in the context bace uh next thing we wanted to add was search word so I'm going to use plain - 11:15 text in this case and I'm going to do short so I'm just going to split it up in multiple words that you can uh control basically um search words and then I say unlimited here because you you you can - 11:31 add as many as you want I'm going to use OpenAI string I'm going to base this on Long description as well based on the context text suggest uh let's say 10 to 15 search words that are unique and - 11:52 important when searching for this document and then we add context here so same thing here this has to be 1002 at least right because the description has to uh exist - 12:16 before in this case I will use GPT 4 because it's it's going to create better words basically it's going to know how to create better words save this um the last thing we want to do now well - 12:34 it's two-step thing so we want to take the plane text we can take text plane long and do uh header image no sorry header image prompt is what I want so this is going to create a prompt that - 12:47 we're going to use to generate the image right we enable AI Interpolator once again OpenAI string long once again long description once again based on on the context um - 13:04 gener generate a one sentence image prompt uh of an artistic kind for a uh preview image of the text be abstract and and uh have a futuristic uh futuristic style let's do - 13:36 this this time context once again we fill it out with a context so the long description is basically doing all the work in this case so if if that is not uh very well - 13:51 written the rest will actually kind of fail in this case and we do batch we do GPT 4 turbo where are you there save this last thing we want to do then is we want to do a image field that is there - 14:08 preview image um I just enable everything here and disable uh OpenAI DALL-E we're going to use the header image promp and we just take it as it is since the prompt to generate the image and - 14:28 this way have to set to 103 because this has to exist even later so this is the last thing we're going to do uh yeah I'm just this yeah okay let's do horizontal we do neutral let's do HD - 14:43 standard see if we can get something nicer might take a little bit longer but that's fine uh and that's it basically we can keep everything in manage form display - 14:53 in manage display I don't want the header image promt to show up uh or actually I'm not sure if anything there should drop but we can wait let's just keep it there so we can actually see the - 15:05 result because how you style this as I said in the beginning this is for people that already know Drupal so you know how to style this uh show which Fields you want and so on so we're just going to - 15:18 Well I will do one funny thing here I will just edit this view so we're not going to take the built-in thumbnail but the preview image and and let's do thumbnail here - 15:32 link image to \[Music\] content and we just rearrange this put this instead of the thumbnail apply save so now there's no previe image but hopefully after we resave now it's going - 15:47 to fill out all these fields so you see oh this didn't look too nice though maybe we should pull it up a little bit I wonder where the fields were even uh uh let's just pull - 16:00 up everything here to the top hopefully that looks better so you can see what I'm talking about here let's see there we go so you have the document uh this is well let's just - 16:21 order them in the order that it's happening because I created them in the wrong order so this is then going to uh because it recognizes it exists a file and it recognizes that original text is - 16:35 empty it's going to ask ConvertAPI now hey please I need a text it's going to fill out this then the next step is that the long description is going to basically ask OpenAI hey create a a - 16:47 long description from the original text short description ask open a hey create a short description from long description search words hey create search words from the long description - 16:57 other image hey create create the header image prompt from the long description and then the oh it should be preview image prompt I'm not consistent here anyway preview image gets generated and - 17:08 so if I click save now it's going to start and this will probably take some time now so I should probably have explained that while I was waiting for this because I have no nothing - 17:20 interesting to say now so let's see um because the first step is pretty fast so converting the second is probably going to take some time because the document as you can see it is pretty - 17:36 pretty large um so it will take some time so one thing that's interesting maybe if you if you're running like e-commerce is what you could do in this case is that if you - 17:49 have something like this already uh for all your products but you don't have very good uh selling text you could generate a text field and basically ask it based on the manual or - 18:02 something just generate uh uh selling text and uh one more example for commerci while we're waiting okay now it's finished but anyway I would just mention it in the OpenAI module there is the - 18:17 FAQ field so there is FAQ for Drupal right so if you want to have a frequently Asked question about your uh each product in your Commerce you could use this method as well it takes a PDF - 18:33 and just generates uh frequently asked questions based on the manual of a product right so maybe there's a troubleshooting section it can just get that then anyway you see now there's a - 18:44 preview image uh it started first of all here we have the full text you can see that the structure is not super it's because it's PDF if we were to use word it would work better but I wanted to - 18:58 show actually the worst case scenario so you can uh see what you can expect okay this failed spectacularly actually uh let's rerun it then I don't know what happened as I said when the - 19:14 long descriptions fail everything fails so we just going to remove all these because then it runs again right so so and we save again in because then it reruns again so the the initial prompt of - 19:34 creating the long description there is one thing that's very important to think about when you're prompting here and I will see if it fails again I have to reprompt - 19:43 basically that the AI interpolator or the open a OpenAI version and most of the plugins are made to also be able to create multiple Fields because this is important if you do uh use the SERP - 19:57 plug-in for in inance you want all the different websites that it finds you don't want just one right or if you want to create a gallery of images you don't want to create 10 fields for that you - 20:08 just want to create one field with multiple files of course this is true as well when you generate text so you have to be very clear where you say how if it should be split up or or or not so if - 20:21 you for instance mentions create 10 paragraphs it might try to create 10 different text areas right and this will of course not work if you only set one because it will - 20:34 only save the first one okay so this still fails so let's retry this oh wait wait I have to rewrite first the prompt before we go anywhere else so I want to be honest with how I'm - 20:54 using the uh the AI Interpolator this is not a sales pitch you you're you know I'm not it's it's free I'm not earning any money on this right I I I just use it myself a lot so would be happy if you also use it - 21:12 so because of this all the videos I do they're not pre-planned so stuff can break and and I guess it's useful that you see that sometimes it breaks and I would show you maybe how I can fix - 21:25 it so based on the Contex create 16 sent well let's write like this create a summary of uh up to 16 sentences in one paragraph so we're going to be very specific that this one - 21:45 paragraph now let's see what happens just remove all these remove this and run again let's see if it works better so by the way also now you see there's one less uh - 22:06 process that's running so from the beginning I think it was six and you're going to see when it's finished with the first one that it did one out of five because the conversion worked right so - 22:16 that we don't have to run again I never emptied that field so but if I would want to then run it again I could empty it of course so let's see if my changes now did any - 22:33 difference to the output yay okay it's still not very very good this is interesting why is the summary short I probably did some error there so it's full HTML OpenAI long - 23:01 description based on the context create write a two sentence summary of the context oh you see what I did there I should copy and paste so we save this and then I'm just going to remove this - 23:15 so we run that again but here you see now it actually worked uh I'm just going to Res sa this so if you do this based on my video you know what not to do at least now - 23:29 um so now we have a little bit longer couple of sentences here we have two sentence version Oh this is another thing so another thing went wrong here actually uh GPT - 23:42 35 so basically we're running Json in the background right uh because we need a structure format to do this and J chat GPT uh sorry GPT 3.5 is not always understanding in how to do this - 23:57 I'm still sitting experimenting to try to get this perfect that's one of the reasons I haven't done this production ready yet so if this ever happens that it shows you chase like this happened - 24:08 again let's try again then you can just remove it and resave it and in worst case if it happens too much you can just okay this was even worse you can just run uh GPT-4 - 24:22 basically and here you see by the way it's just rerunning this one okay so I'm just going to change this to uh GPT 4 because GPT 3.5 seems to have problems with this - 24:37 currently so GPT 4 see if that works better so yeah as I'm said I I mean I could rerun this recording now make it perfect but I want to show you how it actually is to - 24:55 use it because it's not always perfect it's always better if you have the money to G use GPT-4 but I understand that people want to save money and it's a lot cheaper to use - 25:07 3.5 so I I am trying to perfect this prompt that runs in the background and if someone wants to help you know the code is open source you can just download it and check what I'm trying to - 25:18 do in the background with these prompts because it's not just running the prompts you see uh that you put into the U it's running a lot of stuff around that to get it uh in a structured format - 25:31 so it fits fits complex Fields basically so anyway now we have the document we can put this into the search right so if you use Elasticsearch Solr search API whatever same with these search - 25:48 words uh is also stuff that's basically important for this specific uh document based on what GPT for things then of course uh we have a short description that we could use for a preview we have - 26:04 a preview image here wait let me click on it full screen it so yeah maybe maybe I need to prompt that a little bit better I'm not sure if that's a preview image you want to show - 26:19 um but it's just to show you the idea and here you have the header image prompt by the way then um so so yeah that's it this took a lot longer than I thought but it's as I said it's good to - 26:33 that I show that sometimes stuff doesn't work first try uh first time you try it and then it's you can see what you can change because this is also one thing that's - 26:46 very important to note compared to any other development I have ever done uh I worked on super big projects and so on everything else is deterministic where I can say basically - 26:58 based on some project managers or product owners initial idea is this possible yes this is possible you know this is not possible yes I have to research or whatever AI in that sense or GPT large - 27:13 language mod models they are not deterministic if you ask for one thing it might answer something else depending on on how you ask matters as well right so that's why prompting is really really - 27:25 important and basically doing trial and error on this so yeah so I yeah I hope you find this interesting uh you know now how to basically get the media model uh - 27:41 sorry the media entity to get a lot of metadata to it so thank you for watching Filed under [Workflows](/blog?category=6) [Drupal AI](/blog?category=2) Parts of this content were produced by AI. This site has not assessed whether a label is required for this content. How was AI used?The video and the introduction are Marcus's own, first published on workflows-of-ai.com. AI wrote the subtitle, the summary and the video chapters from the transcript, and corrected misheard names in YouTube's automatic captions. Marcus Johansson Tinkers with AI and Drupal [About Marcus](/about/marcus-johansson)[Drupal.org profile](https://www.drupal.org/u/marcus_johansson)[Drupal AI](https://www.drupal.org/project/ai) ## The post in three sentences ConvertAPI turns an uploaded PDF into plain text on the Document media type, so its content becomes searchable. OpenAI then writes a long summary, a two-sentence summary, search words and an image prompt, and DALL-E makes a preview image from it. Some prompts fail on the first try and the video shows how to fix them, including switching from GPT-3.5 to GPT-4.