0:01
hi guys Marcus here uh today's workflow is very uh Drupal connected uh so you have a workflow where you have jior editor senior editors and maybe uni jior editors are just allowed to add a
0:14
article but a senior editor has to check check it and like uh actually check so everything that's written inside it is true so we're going to automate this I will just show you in this case I'm
0:28
going to create two article here the first one both are named who started World War II and the article just says in this case it's just one assumption but it can check multiple assumptions of
0:39
course but in this case Genghis Khan started World War II I just save this the other one says Hitler started World War I maybe I should write a Hitler uh and we save
0:52
this uh this is doing fact checks in the background and if you take the Genghis Khan example you see oh it's not published our senior editor stopped it from being published because it's not
1:04
true and then we go to the other one and it's published because it is true so we're going to do this with a service called Tavily and I will show you now how to set this
1:17
up oh actually uh one thing I will also mention why Tavily is important because you might think why can't I just do this with uh OpenAI GPT just ask it via OpenAI GPT right so one thing that's
1:32
important with GPT or any large language model is to know that they are text generators in their core they are not chatbots they are not fact machines they want to generate
1:47
text uh and this is important to have in mind so even if you use ChatGPT and you ask it something it might be that the question it creates so the text it creates in the
2:01
end doesn't have the token space for the facts that you need so it might not know about the effects uh or in uh when it's generating things it has a certain amount of
2:17
creativity and this is something you can use temperature and some other variables to set up where it might actually be creative enough to make up a fact even if it has the facts so because of this
2:31
it uh sometimes writes an answer like it actually knows that is completely wrong and this is called hallucinations right and because of this it's important to know
2:44
that you should never use a large language models or at least the ones we have today to do this kind of shore so what you want to do instead is there's two steps of doing this so either you
2:59
take the large language model and you ask it for something and then you have some service after that actually verifies this so for instance you can this is something you can do when you're
3:10
looking for uh uh geographical addresses using uh uh the address field with the AI Interpolator this will then ask OpenAI to give me something uh give me a places that you see and then it actually
3:25
uses Google Places to verify this uh but in this case we want to do the opposite then we want to have facts ready and put them as context into the question where
3:38
we're asking uh GPT so it knows it's going to base it on the facts it's not going to base it on uh the things that it's generating via its text and this is where Tavily comes in then because
3:53
Tavily is actually searching the web for your question and giving back some context that you can use in OpenAI so that's why it's important so with that being said now starts the video on how
4:06
to set this up so before I start showing you how to set this up uh I would just remind your workflows of AI exist you can check all the old videos there if you're
4:17
interested um and you can also filter and search by them so it's a little bit better than YouTube uh but of course if you're really interested in all videos just subscribe to YouTube and you will
4:27
see them there there there is also so you know AI Interpolator is completely documented nowadays so uh it's on drupal.org you can find all the rules all the different plugins uh kind of a
4:40
base idea at least what you can do with the different rules and so on uh what I am going to use in this case and I have already installed this so I'm going to use ECA because
4:54
ECA uh I can't go through it now but I will say as much as it's the most powerful third-party module that exists by far it's it's it's it helps you uh get away from writing a lot of
5:12
code and it also creates a way where you can visualize it for your in client so if you don't know it go down the rabbit hole of looking at videos and learning this also uh their Slack channel is
5:26
extremely helpful so you can ask questions there and they are super nice there and answering question so top notch mod uh module use it for stuff like what I'm going to show now uh and so for the a AI
5:41
Interpolator I have built in a a processor plugin where you can basically say that uh you want to instead of using the normal AI Interpolator processors like queuing or
5:54
batching or direct saving API saving you can say ECA takes care of it because this will make sense in this case where we want to make sure that that we don't publish stuff that's false so that's the
6:07
Interpolator ECA that will install the ECA module so I have already done this I have also then installed one of the GUI component so if you start looking at videos you know what it is which is the
6:19
BPMN.iO in this case we're going to need OpenAI uh so AI Interpolator OpenAI to make the uh uh step where we're asking questions and also validating from these questions and we're going to
6:33
need uh Tavily then Tavily is what I'm going to introduce in this case um so if you're going to do real research you should not be using Tavily Tavily is good for being it could be one of the
6:47
steps but I would say if you're going to do real research use the Ser plugin instead so that's the thing that actually Googles for you and then use one of the scraper modules to actually
6:58
get all the content but if you're going to do simple uh facts checks or simple searches where you just want the small part from a website that answers a question so you
7:12
can think of it as the widget you might see on Google sometimes where you Google something and it actually answers before you can click a link which is a horrible
7:21
practice by the way it's useful but it takes away so much traffic from the end users anyway this does this then right where it gives you back a little bit of context and this is what we're going to
7:33
use here to to get this uh working so what I have done is I have gotten all these I have installed them then also under configuration OpenAI I have added the OpenAI key and another workflows uh
7:47
sorry web services Tavily config I have added uh the Tavily config I will by the way uh remove I already test ran this one time so I'm I'm I had a ECA model model there I just remove that so I
8:02
create that from scratch so we're going to start by creating a content type now that is uh oh one more thing Tavily gives you 1,000 credits for free so if you have OpenAI account but you're not sure
8:14
if you want to pay another service you can test Tavily at least uh we're going to call this truthful article uh the important thing is it's going to be unpublished by default so
8:26
it's just going to get published if our AI senior editor says that okay everything in this thing is true right going to create a remove the article body I want to create my own Fields here
8:40
and we're going to do uh call this article uh uh oh article body let's call it that text formatted long so this is where we put in the uh article this doesn't need any AI connected to
8:54
it then comes the AI Fields then so what we're going to start with is a plain text takes plain long uh questions to ask so this is basically the questions it's going to ask the search engine for
9:09
uh to get context to see if the article is truthful right so this has to be unlimited because it might need to ask more than one questions uh it's OpenAI string long we
9:22
base it on the article body and then we just write based on the context text generate as many questions as you see fit to test the truthfulness of uh the assumptions in
9:42
the article do not overdo it usually one question per assumption is enough uh and context and context so that context is then the the actual article body that's
10:05
going to be paste in here when we send this to OpenAI you can see that here in the play Source holders available so what's important here under advanced settings our normal workers are not
10:15
important we just set the ECA we don't care about weight this ECA will take care of We Set uh I'm going to do GPT for Turbo as the model save the settings then we're going to do the next step
10:28
which is basically checking in Tavily then so we do Tavily lookup uh which will be also unlimited let me think yes because we need one per question we enable Ai interpolator and then there is this rule
10:49
Tavily search word to string and we're going to just use the questions to ask as the context here and that's it basically uh we don't need to add anything there so we just set this
11:02
to ECA and here's important I'm going to set it to concatenate because then it's just going to create one big long context field per question instead of creating multiple per
11:13
question um there are use cases where this makes sense as well but in this case I will just do that and the last thing I will create is in this case uh you could
11:25
think that a Boolean is good here because if it's true or not but is very hard to write prompts for so there is AI Interpolator OpenAI Boolean rule but in this case I'm going to use selection
11:37
list and do list text and we're going to call this truthful so is this truthful or not uh and uh the allowed value is going to be truthful right this is human readable
11:57
and uh so not truthful and in this case we can actually limit it to one question uh limit it to one value because we can ask basically OpenAI to answer everything
12:12
at once and here's another thing now I'm just showing the simplest path to this there is a custom Fields module where you can basically have multiple uh types uh of field type
12:27
values inside one field and this would be super useful if you want to do this but you also want to answer why it reacted something for instance because that might be
12:38
interesting because you might read the article and think no everything is truthful why did you actually react this right and then it will answer this I'm not going to Showcase this there are
12:47
other videos where I work with the custom uh field module if you want to figure out how this works right um going to enable the interpolator we're going to use oh happened there uh we're going to uh
13:03
use the OpenAI list string but in this case we're going to use advanced mode oh so I have token module installed by the way here as well you need that for the advanced mode so then I'm going to write
13:14
here based on the article text would you say that all assumptions in the text is uh truthful when compared to the uh uh fact context uh under here if it is truthful
13:48
answer truthful so these are the two options we have otherwise answer not truthful right uh in theory we just need a uh truthful actually because that's the only thing that makes a change because
14:08
it is unpublished by default right but anyway uh then we do article text and I just do some uh natural uh start and end here so the AI understands that take notes take the Artic
14:29
body we take the value I think because that will have I'm not sure if it strips the HTML tags it's not so important you can have HTML Texs the OpenAI can understand that and then we do uh what
14:45
did I call it fact context fact [Music] context oh I'm going to do another thing here just for uh verbosity going to do cap capitalize those two and do it here as well so it
15:03
really does that connection fact context and the fact context is the answers we got back here now then so that's what we call what do we call that Fields question to ask it's
15:19
not Tavily lookup uh and we don't need the Deltas in this case let me see so this is marked here [Music] so uh we just need the uh whole field everything right
15:37
uh and that's it same thing here connected to ECA we want GPT-4 because it's better at understanding stuff reading stuff and then we're going to go and create our ECA mod model
15:53
then uh so we create a new model we're going to do this on [Music] um on on truthful article pre- save so and we're looking for the pre- event here of content Truth for
16:18
article we're going to have the first event which was questions to ask and under here now then there is a AI Interpolator rule so this basically has all the rules that you have set up
16:31
previously available here so we're going to use questions to ask uh we don't need to save in between there we can save at the end possibly I'm not sure if that's
16:44
super important um I think it keeps the context there so the second thing we want to do is Tavily look up [Applause] right uh and then then we are also going to add the last
17:04
one which is truthful is truthful uh let me uh check think here if we need to forward that value there no it should be a aware now that this will work so then uh the last task we need
17:30
is publish and I'm going to add a rule in between there but I'm just going to do this uh publish content item right uh but in between there now we want to have a rule that says Fe compare field
17:49
value is truthful so this last step should only happen if it's truthful uh the field name I of course forgot the name of it uh let me just open in a new tab field
18:02
truthful so if field truthful equals truthful then it should go to this step right so we save this I'm not sure if I need to save there we're going to see this now actually so I'm going just
18:16
going to add content and I'm going to start with something truthful to see if that works uh uh Hitler started World War uh I'm just oh that's the title I'm just
18:32
going to copy this to article body so this should be true right uh well had a hand in starting World War II in I I think it would understand that he was the main
18:49
reason but who knows uh we're going to save this and now this ECA rule is running then uh uh oh updating let me see just did I actually do save there where are we
19:11
here or is it that I need to save it maybe uh I didn't do save entity no let's just try to save this see if it works update existing node entity while changing the ID is not
19:45
supported let me just see if this is the problem did I do something wrong here template applied pre save yeah in this should run only on content truthful questions to ask let me see so
20:25
I set all the fields here maybe I need to actually create the error State better here questions to ask is using ECA t look up is using ECA truthful is using ECA so I just turn this off
20:59
I we try now it might be that I'm not allowed to set that publish on creation but that would be weird Okay so that could be it so we I'm just going to add this now and
21:12
see if it works on update to start with oh oh this is in the way uh publish content item so so this is I'm I'm testing this actually for the first time I was testing the other thing before but I
21:28
haven't tested this publishing uh so uh uh uh this should be compare field values field truthful it was equals truthful and we save see if it works now so here you can
21:50
see by the way we can just go through what happened so far it asked the question did Hitler's action directly contribute to the onset of World War II so I took this and as this question tely
22:01
looked up this and got a lot of context here and based on this it says yes it's true so I'm just going to try to save now to see if uh no okay so this is making it fail why is this making it
22:18
fail uh I want to publish the maybe if I set uh maybe this is not allowed on pre-save I maybe need to change the actual value let's see here let's use another thing then uh value token set
22:47
value no entity set field value uh uh yeah strip tags no no no field name status uh save entity yeah we save it f well no we don't need to do field value should
23:05
be one let's see if this works better so we try to publish this way instead uh and yeah you can maybe write in the comments why this didn't work for me if someone knows uh anyway
23:23
compare uh and compare field value was what we wanted to do check if truthful so field truthful equals truthful it should only pass then right save this and we try
23:43
again yay this worked and it published right so let's try this from uh start to finish again I'm just going to copy this delete this one content add content Truth for article res set this and this
24:01
and hopefully it should now become a published article of course by the way you should not show the TBL up look up and the questions to ask and all this stuff that's just for the editorial
24:11
process right so you need to hide that from the end user uh because what you're trying to do here wait let let me just do that so we have done that uh uh you want to do this
24:25
right manage display you still want to be able to see them in the back in the form so we're just going to do this because then this is all you see uh and then we're just going to create
24:38
something that's not true uh e edible foods uh eating uh lead paint is recommended like this is not a good statement to put in article right um of course with the by the way with a
25:03
prompt you have to also add so it can be sarcastic and figure out stuff like this because this might be just be a a a statement in the middle of a sarcastic uh uh text right so you have
25:16
to engineer your prompt but here you can see it's not actually published so in this case our senior publisher stopped it uh and what you will see here it's not truthful it gets some context why
25:32
it's actually dangerous uh uh and yeah it stops it um yeah so that is it basically so uh as you can see I I set up these workflows while I'm testing them myself
25:49
just to see how stuff works so because of this um um there was some steps that failed but that's good to see that it's not always perfect I'm not it's open source
26:02
free I'm not trying to sell you a a product here I'm just doing this for fun and yeah uh trying to Showcase how you can use this so thank you for watching
