txtai has long had support for workflows. Workflows connect the input and outputs of machine learning models together to create powerful transformation and processing functions.
There has been a recent surge in interest in "model prompting", which is the process of building a natural language description of a task and passing it to a large language model (LLM). txtai has recently improved support for task templating, which builds string outputs from a set of parameters.
This article demonstrates how txtai workflows can be used to apply prompt templates and chain those tasks together.
Install dependencies
Install txtai and all dependencies.
# Install txtai
pip install txtai[api]
Prompt workflow
First, we'll look at building a workflow with a series of model prompts. This workflow creates a conditional translation using a statement and target language. Another task reads that output text and detects the language.
This workflow uses a sequences pipeline. The sequences pipeline loads a Hugging Face sequence to sequence model for inference, in this case FLAN-T5. The sequences pipeline takes a prompt as input and outputs the model inference result.
It's important to note that a pipeline is simply a callable function. It can easily be replaced with a call to an external API.
from txtai.pipeline import Sequences
from txtai.workflow import Workflow, TemplateTask
# Create sequences pipeline
sequences = Sequences("google/flan-t5-large")
# Define workflow or chaining of tasks together.
workflow = Workflow([
TemplateTask(
template="Translate '{statement}' to {language} if it's English",
action=sequences
),
TemplateTask(
template="What language is the following text? {text}",
action=sequences
)
])
inputs = [
{"statement": "Hello, how are you", "language": "French"},
{"statement": "Hallo, wie geht's dir", "language": "French"}
]
print(list(workflow(inputs)))
['French', 'German']
Let's recap what happened here. The first workflow task conditionally translates text to a language if it's English.
The first statement is Hello, how are you with a target language of French. So the statement is translated to French.
The second statement is German, so it's not converted to French.
The next step asks the model what the language is and it correctly prints French and German.
Prompt Workflow as YAML
The same workflow above can be created with YAML configuration.
sequences:
path: google/flan-t5-large
workflow:
chain:
tasks:
- task: template
template: Translate '{statement}' to {language} if it's English
action: sequences
- task: template
template: What language is the following text? {text}
action: sequences
from txtai.app import Application
app = Application("workflow.yml")
print(list(app.workflow("chain", inputs)))
['French', 'German']
As expected, the same result! This is a matter of preference on how you want to create a workflow. One advantage of YAML workflows is that an API can easily be created from the workflow file.
Prompt Workflow via an API call
Let's say you want the workflow to be available via an API call. Well good news, txtai has a built in API mechanism using FastAPI.
# Start an API service
!CONFIG=workflow.yml nohup uvicorn "txtai.api:app" &> api.log &
!sleep 60
import requests
# Run API request
requests.post("http://localhost:8000/workflow", json={"name": "chain", "elements": inputs}).json()
['French', 'German']
Just like the previous steps, except through an API call. Let's run via cURL for good measure.
curl -s -X POST "http://localhost:8000/workflow" \
-H "Content-Type: application/json" \
--data @- << EOF
{
"name": "chain",
"elements": [
{"statement": "Hello, how are you", "language": "French"},
{"statement": "Hallo, wie geht's dir", "language": "French"}
]
}
EOF
["French","German"]
One last time, the same output is shown.
If your primary development environment isn't Python, txtai does have API bindings for JavaScript, Rust, Go and Java.
More information on the API is available here.
Conversational chain
Conversational search is another big area of focus in 2023. txtchat is a framework for building conversational search applications. It relies heavily on txtai. Let's see a conversational example.
writable: false
cloud:
provider: huggingface-hub
container: neuml/txtai-intro
extractor:
path: google/flan-t5-large
output: reference
workflow:
search:
tasks:
- task: extractor
template: |
Answer the following question using only the context below. Give a detailed answer.
Say 'I don't have data on that' when the question can't be answered.
Question: {text}
Context:
action: extractor
- task: template
template: "{answer}\n\nReference: {reference}"
rules:
answer: I don't have data on that
app = Application("search.yml")
print(list(app.workflow("search", ["Tell me something about North America"])))
["Canada's last fully intact ice shelf has suddenly collapsed, forming a Manhattan-sized iceberg\n\nReference: 1"]
The first thing the code above does is run an embeddings search to build a conversational context. That context is then used to build a prompt and inference is run against the LLM.
The next task formats the outputs with a reference to the best matching record. In this case, it's only an id of 1. But this can be much more useful if the id is a URL or there is logic to format the id back to a unique reference string.
The txtchat project has much more on this, check it out!
Wrapping up
This article covered how to build prompt templates and task chains through a series of results. txtai has long had a robust and efficient workflow framework for connecting models together. This can be small and simple models and/or prompting with large models. Go ahead and give it a try!
Top comments (1)
Really cool! This sounds quite easy but really powerful at the same time. Does this also come with a small interface that can be embedded into websites for example?