Using the AI Interpolator SERP module we can search the web on some specific topic to generate links to websites of interest.
We can then use those to research topics of interest with the Scraping Bot Interpolator module and the OpenAI module.
This means that you can use this to generate small text snippets or even have publishing workflows based on research.
This video is from before the AI Interpolator module became AI Automators, part of the AI module . Module and menu names have changed since, and some steps look different today.
0:01
hey there um Marcus here so uh I'm going to talk about the workflow today that I already tried to record actually uh something went wrong with the recording so I re-record it
0:13
now um and it's uh about uh The SERP plugin so search engine results page so basically a plugin that can Google for you which makes it possible to do a lot of research uh fact checking stuff like
0:31
that the workflow I show today is very simple it's just going to uh basically analyze a question and then search for actual answer so not ask uh OpenAI the question right away
0:46
because OpenAI as everyone know can hallucinate so it needs context to be trustworthy and in this case we're going to create the uh context via search as always what you see in front of you
1:02
this is Drupal open source free uh you can download it afterwards um download AI interpolator modules that I'm going to show and set this up as easy as it looks here so I'm going to start by logging
1:19
into the system um and then create that content type that we call research as always if you're going to do this as a real product or use big data or so on you should use Uh custom entity
1:39
types but in this case I just want to show how you can graphically via or sorry via the GUI set up everything so instead of title we just call this a question that's going to be the question
1:51
we want answered basically um there comes a body field with Drupal by default in the nodes the content types so we're going to delete that that's more for CMS right uh we're going to start by adding
2:10
the first field which is going to be a text plane and this is going to be the search word and I it's just going to be one search word right so that's going to be the
2:23
search word we use in Google to find information that might answer the question or so because we don't want to fill out this uh manually because we might not know what we search for we're going to
2:37
use OpenAI for this so we're going to use the OpenAI string that takes a string and does something on that string uh and generates a string uh we use base mode and we use
2:51
the question as the base field so then we create a prompt here then based on the follow in question create a search phrase for Google for finding results that might answer
3:19
it question and then we're going to copy context because that's the context of um basically the question we're ask asking right uh we're going to run this in batch so
3:34
we uh see the graphical output take GPT-4 it's a little bit better quality save this and then comes then the SERP field and what we need as the field to fill out we need
3:50
links uh we call these search results and this can be unlimited so or or you can set a fixed number basically these are the amount of search results you want back that can be defined here
4:05
in the AI Interpolator then it's just going to be external links no link text is needed and we enable the AI interpolator here as well and the rule we want to run is SERP to
4:18
link and we say I don't want it to take too long now I'm just going to ask some simple questions I just want three search results because of course it needs to scrape every page
4:30
it need to analyze every page in OpenAI and so on so it takes time but of course if what you're researching is complex you should try to get as many search results as possible you can do search of
4:41
that so you can get 10 pages in batches then start from page uh from 11th results start from 21st results and so on you can search on use you can choose which Google to search Which
4:57
languages uh you search with Which languages you want to have in the search result and which country you're searching from um we're going to do this on the base
5:10
mode and we're going to use the search word that we generate then it's important here that we set the higher interpolator W uh than the initial query because we need the search word before
5:23
we can search right and we said patch again after we have done that we need to to get the actual content from this so we're going to use text format long and we call that website
5:37
content uh that's the content we get back from the links these also have to be well as many as the links are allow full HTML enable Ai interpolator and in this case we're
5:51
going to use the ScrapingBot crawler Rule and we're going to use this from the search results so the Link's there basically uh we can get this from USA we're going
6:02
to use chromes or use a real browser uh we use readability mode so we say we just want the content that is the actual article and we have to set this to an even higher weight then because we have
6:17
to make sure that uh the links exist before we can start scraping and then basically comes the last thing we need which is a text plan long that's going to be the
6:33
answer and this is the answer to the initial question based on the scraped content right so we're going to ask OpenAI to an answer the question based on the scraped content we're going to use
6:48
advanced mode for this uh and how we write this then based on the context answer the following question in around three paragraphs uh yeah three [Music] paragraphs
7:20
question so what you can do with uh the advanced mode with tokens then is of course that you can you can basically uh use multiple inputs which is what we need in this
7:36
case so we find this under nodes and the question was on the title that's why you don't see question here so we take node title for this and then the context which is then the
7:56
website content this was called uh and this is the prompt we're going to use uh we have to set an even higher interpolator weight we do batch and we take a bigger model in this
8:15
case because we don't know how big the context is going to be so we use a 16k um model so we make sure it fits inside the context there is one thing I should probably do here as well since we're
8:29
asking for paragraphs it might actually generate multiple answers and split up each paragraph As one so this is prompt engineering of course you could write three paragraphs in one result set or
8:42
something and it would probably work so you would have to find the best prompt possible there if you want it in just one field but I just do multiple Fields just to make sure that uh you will see
8:57
the result at least uh and that's it I'm going to do another thing I'm going to hide the website content from the manage display so the end output uh because that's going to take up so big
9:14
space that I have to scroll and you might not know what the actual answer is so then we add the research uh and I have a question here that's very interesting what is Link's
9:27
personality in the Zelda games uh and you see all the other fields here those are the fields now that's going to be outo fi basically so I click save uh this is going to take a little
9:43
bit of a time um because now basically it is generating a search word searching for this to get the links scraping those links with a real browser uh and in the end asking open a
9:59
I then to answer the question based on the content inside those links oh let's see it didn't actually answer it let's just try to resave and it might work [Music]
10:20
um so maybe I set up the order wrong let's see yeah here we go so here we have an answer then links personel in the Zelda Gam where it's depending on the specific game yeah you can see the
10:33
answer here basically if we click edit you can also then see what you saw before that there's actually uh script content and this content is then what went into the
10:48
question uh to OpenAI so to make sure basically that it has a context to answer on meaning that it can answer very well I'm not going to say truthfully because of course you should
11:02
not just trust all the links you find in Google but the probability of it answering is uh correctly is a lot higher at least uh but as I said this is a very simple workflow I showed
11:17
now um if you just flex your creativity muscles I'm sure you can think of so many more use cases for this so yeah thank you for watching
