How to run LLM modal locally with Hugging-Face 🤗

Welcome Back - In this topic i would like to talk about how to download and run any LLM modal into your local machine/environment.

We again use hugging-face here 🤗. You would need hugging-face API key first.

Run below code to download google/flan-t5-large in to your local machine, it will take a while and you will see the progress in your jupyter notebook.

from langchain.llms import HuggingFacePipeline
import torch
from transformers import pipeline,AutoTokenizer,AutoModelForCausalLM,AutoModelForSeq2SeqLM
model_id= "google/flan-t5-large"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)

pipe = pipeline("text2text-generation", model=model, tokenizer=tokenizer,max_length=128)
local_llm = HuggingFacePipeline(pipeline=pipe)

prompt = PromptTemplate(
    input_variables=["name"],
    template="Can you tell me about footballer {name}",
)
chain = LLMChain(prompt=prompt, llm=local_llm)
chain.run("messi")

Comments

[Jayanath Liyanage]

Great work 👍

[Voyagergle]

let me try