In this video we will introduce the AI Interpolator MediaWiki and the Hugging Face module, and use them together with the AI Interpolator module.
The idea is to factually answer some question that we might have about any topic.
The MediaWiki plugin makes it possible to use a MediaWiki, like Wikipedia, as a context source to make sure that your AI does not hallucinate, or can answer questions based on context it's not trained on.
The Hugging Face plugin opens up a world of AI models for the AI Interpolator, by using the Inference API they have. This makes it possible to run fine-tuned, uncensored or private models within Drupal.
This video is from before the AI Interpolator module became AI Automators, part of the AI module . Module and menu names have changed since, and some steps look different today.
0:00
hey there Marcus here today I will uh introduce two new plugins one is very minor uh and one is a lot more important uh so if we just start with the minor I'm just going to say a couple
0:15
of words about it and that's the media interpolator uh and that's basically taking a search word or a you know a product the name or whatever and filling another large text field with data from
0:30
Wikipedia um it is MediaWiki agnostic so in theory you can probably use it on fandom.com or Wikiquote or some other uh larger website that's using MediaWiki um but for most purposes it's just
0:48
going to use Wikipedia and it's already uh configured to use Wikipedia by default um and the reason for this is basically that the AI when they are trying to find you know
1:01
the next tokens in the vector space sometimes it can go wrong because of how they have been trained and there there is it is a very hard uh issue to do something about to be honest so the
1:17
AI will hallucinate the only way you can get rid of this if you want truthful answers is basically to give it context like this is a text that has a lot of information in it can you summarize or
1:30
answer this question what whatever you want right uh so that's all about the MediaWiki the more more important one is that there's a Hugging Face module now uh
1:41
this is a generic module so you could actually use it for uh or the idea is if there's more contributors doing a lot of other stuff with Hugging Face models please get in
1:55
contact with me and we I will make you maintainers as well uh so we don't have a AI interpolator Hugging Face but there's a Hugging Face mod module with a API to Hugging Face and
2:08
on top of that this has a AI interpolator Hugging Face module then um this tutorial is pretty Advanced you have to know Hugging Face you have to have account at Hugging Face you have to
2:22
know how to take care of uh or how how you find the models you want to use and so on so in Hugging Face there's 500,000 plus models that someone has you know created from scratch or scratch or
2:40
fine-tuned and so on uh but out of those 500,000 there are around 40,000 as far as I could uh count at the current date that has uh something called the hugging
2:55
face in inference API which means that they are basically able to uh be run and there's two versions of this API so I'm just we're going to be playing with Google Gemma today uh but I will
3:10
explain how this module is completely model agnostic actually so but I will show you here on the deploy so I'm logged in that's why I see this by the way there is inference API and there's
3:21
inference end points and these are the two versions that you can use with the module currently so there's a serverless version that is actually free but this comes with a lot of caveats so you know
3:35
if you're going to prototype this feel free to use it sometimes it will just fail because it's down you will get a warning message that it's down it got a 400 400 or a 5003 or something like that
3:49
that's a response um but sometimes it works uh and uh also to be clear these free apis does not exist on every model especially uh models that requires a lot of uh vam from uh from expensive gpus so
4:09
because of that there is of course also then a paid end point where you can get a dedicated machine um I will talk more about that in later videos where for this video I'm
4:23
going to set up everything so it's completely free actually you don't need you know all the videos I do usually I just you know say yeah I just change this service and this service and this
4:32
service and that's fine when you know you have a company paying for it like in my case or I privately can pay for it but that's not always the choice uh the the there's not always an option right
4:46
so because of that it's pretty nice that Hugging Face lets you try out stuff like this I will say though unless you have a workflow on production where you can wait for the generation or where one
5:00
part of a generation change is allowed to fail for maybe for hours don't use this in production pay for it if you need it to to actually always answer um I'm also not going to go in
5:16
that much to why this this is so much more important than all the other AI models uh so sorry AI plugins but the the guest of it is basically a lot of these models are open source you can't
5:34
censor them you can get them to answer like you actually write or uh you know create images with yourself inside the image and so on because you can find you these you can also create these models
5:48
from scratch uh and then just a little bit about uh how you can find these so there is huggingface.co models and under there there are a lot of categories and
6:02
these categories basically map to different rules in the AI interpolator So today we're just going to use the text generation uh uh text generation task which is here so you can see all the
6:17
modules models that exist this is based on uh trending right now so because Gemma was just released like I'm not sure if it's Gemma or Gemma I say Gemma was just released a week ago it's super
6:33
popular now of course so we're going to use Gemma to be it and it is basically to give it instruction so it's a typical shat model um so that is the instruction oh yeah
6:45
the workflow we're going to do it's very simple but it's just to give you an idea what you can actually do with these mod uh plugins so it's that we're going to
6:55
create a content type that aners uh question or fill us out information basically for our um for our website in this case I'm going to create something called software and it's going to answer
7:12
some question regarding uh well I will use Drupal in this case of course so I'm logged in here uh in Drupal then uh I haven't done the normal introduction by the way what Drupal is
7:25
and so on this is so complex that you you have to know Drupal before you start playing around with this so I'm not going to introduce Drupal in this case uh what I will switch to here is the
7:36
code right because we actually need I just started this in DDEV then what we need is this modu uh module which uh we let's see here we're going to use this one so
7:56
D and we get the Hugging Face beta and then we're going to get the AI interor media wikip beta as well AI interor MediaWiki oops Wiki ah misspelled so and you see they get installed here
8:28
the only thing you have to think about about here minimum stability has to be de because these modules are still not production ready uh so they are added I had the AI interpolator by the way since
8:39
before just so you know but that would of course be uh installed also then so we just search for Hugging Face we're going to use AI Interpolator Hugging Face this is going to install Hugging Face core and
8:50
then the MediaWiki so we just installed this it says that it's going to install the core as well uh and then what we need to do is go to configuration web service as an
9:02
Hugging Face config and there we have to put a API token so that you can find under uh let's see here it should be under settings and then it's under authentication maybe
9:17
no access token here we go so I will just create a new token here um testing it's okay for reading and we show this I'm just going to copy this this will be removed by the way by
9:35
the time you see this so you can't steal this from me we just save this and then we have set up Hugging Face so the next uh point is that we want to create a content type that we call software then
9:50
uh we disable this we don't need to preview uh and we just add Fields now then we remove the body field we're not not going to need that but what we want to do is to have a
10:03
plain text field that's text plane long that we call a wikip Wikipedia article right and this should only be one and here we enable the AI interpolator and we choose the rule uh Wikipedia media
10:21
Wiki and we're going to base this on the title so I'm going to write Drupal in the title and this is going to go into Wikipedia search for Drupal and uh basically fill out this field
10:32
then uh we can use direct because this is fast uh we just want a extract I think should be enough so I'm just going to ask you how many users that exist right we want to strip HTML because this
10:46
was a plain text field and we're going to use it for context and it's better just not to have an HTML there uh search before fetching means that it will do a search if I do some misspelling right
10:59
drle or something like that instead of Drupal then it will kind of figure out that it's Drupal I mean uh then I we create another field that is uh just uh yeah we take a form
11:13
it takes long uh so the text generation currently because most of the models are not close to OpenAI GPT 4 does not have many different uh uh field types that it can answer to because when
11:33
we ask GPT-4 we can ask in a complex Jon way so in this case it's only available for the long text Fields so um I will write members here and then we set full HTML we enable
11:50
the AI interpolator and now we're going to use the Hugging Face text generation so this is the thing that talks to text generation ation models we're going to use space mode and we're going to use
12:02
the Wikipedia article right uh and now I just copy from I written it here before so based on the following text answer the question answer the question with factual information no description is
12:16
needed in the answer a description what the text says so that's just the thing is some of these mod models want to be very verose and in my case I just want to know how many
12:29
members actually exists and that's it right uh and then we just write [Music] um text and we're going to use context which is the Wikipedia article in this case right and then we write
12:48
question how many uh how many members does Drupal have wait let me just do a Drupal Wikipedia check on how to actually write this uh let's see here members members what did I write
13:12
how many members do ruple okay one do uh 139 and then since these are cheex generation it helps when you kind of force where the next thing should come so we just write the answer and that's
13:24
what we want to fill in right under advanced settings we have to make sure that this weight is higher than the Wikipedia fetching so the Wikipedia article exists
13:34
beforehand uh these are a lot of settings that you will have to tweak around especially with smaller models if you want any result that actually is of use for you uh especially
13:47
when it actually writes text I mean in this case I think it will be fine to keep it as this I think actually the Gemma even and the Gemma 2 B that runs on a normal you know a normal laptop
14:01
with 16 GB of RAM I think that will be able to answer this um so we just leave it like this right and we're going to use the free end point uh and I'm just going to start
14:15
writing so there is a auto complete here and I take Gemma 2 B it save this and then we're just going to add this software then and we're just going to write Drupal here
14:29
save it and hopefully you see it fills in the Wikipedia article and members 1.3 39 million members including 124,000 users actively contributing and this is taken from this context which
14:44
means now that it's asking truthfully so uh this is cool from many point of views because it's asking U answering truthfully it's using a model that's free but that's I mean mean that's the
15:00
current state of Hugging Face who knows maybe they don't keep this free uh but it's also using a model that you could download and fine-tune yourself to actually write text as you
15:12
want it to be written right uh and you could also add things that you find that they sensor that you might be interested in in writing about using AI right so yeah that's basically it that's how you
15:30
use the the Hugging Face and the AI interpolator MediaWiki the MediaWiki I probably won't or it might be incorporated in some workflows in the future but it's not more complex than
15:43
what I showed you now however the Hugging Face I will come back to uh we're going to look into fine-tuning our own models and using that and other things how you
15:53
can set up a dedicated API endpoint and so on um so yeah that's it thank you for watching
