Ranking search results with Eevo
Reranking search results with decision models is a powerful trick for making your search smarter. In reranking, you use an initial "dumb" search algorithm, such as text matching with BM25 or vector-based semantic search, to get initial results, and then reorder those results using a smarter system, like a decision model.
The price-performance and speed of decision models make Eevo an excellent option for flexible reranking of search results for increased relevance. Reranking can also be combined with filtering the results with a decision model.
Below is a simple example of using Eevo to rerank search results:
import os
import requests
QUERY = """how do I rotate an API key"""
RESULTS = {
"r1": "Rotating API keys: Create a new key, deploy it to your services, then revoke the old key. Requests with a revoked key are rejected.",
"r2": "API keys overview: Every request carries an API key, which belongs to your organization.",
"r3": "Pricing: Decisions are billed per request, and other requests per input token.",
}
QUESTIONS = {
"r1": {
"type": "score",
"instructions": "How well does result r1 answer the query?",
"criteria": [
"Unrelated to the query",
"Related, but does not answer it",
"Answers part of it",
"Answers all of it",
],
},
"r2": {
"type": "score",
"instructions": "How well does result r2 answer the query?",
"criteria": [
"Unrelated to the query",
"Related, but does not answer it",
"Answers part of it",
"Answers all of it",
],
},
"r3": {
"type": "score",
"instructions": "How well does result r3 answer the query?",
"criteria": [
"Unrelated to the query",
"Related, but does not answer it",
"Answers part of it",
"Answers all of it",
],
},
}
response = requests.post(
"https://eevo.io/api/v1/systemone",
headers={"Authorization": f"Bearer {os.environ['EEVO_API_KEY']}"},
json={"model": "eevo-1", "state": {"query": QUERY, "results": RESULTS}, "questions": QUESTIONS},
)
response.raise_for_status()
print(response.json()["answers"])An example response is below:
{
"r1": {
"type": "score",
"score": 2.883,
"confidence": 0.9101,
"legend": {
"0": "Unrelated to the query",
"1": "Related, but does not answer it",
"2": "Answers part of it",
"3": "Answers all of it"
},
"probabilities": {
"0": 0.0079,
"1": 0.0114,
"2": 0.0706,
"3": 0.9101
}
},
"r2": {
"type": "score",
"score": 0.7806,
"confidence": 0.7253,
"legend": {
"0": "Unrelated to the query",
"1": "Related, but does not answer it",
"2": "Answers part of it",
"3": "Answers all of it"
},
"probabilities": {
"0": 0.2498,
"1": 0.7253,
"2": 0.0195,
"3": 0.0054
}
},
"r3": {
"type": "score",
"score": 0.0435,
"confidence": 0.9658,
"legend": {
"0": "Unrelated to the query",
"1": "Related, but does not answer it",
"2": "Answers part of it",
"3": "Answers all of it"
},
"probabilities": {
"0": 0.9658,
"1": 0.0273,
"2": 0.0045,
"3": 0.0024
}
}
}As usual, the decision model provides probabilities reflecting its confidence in each answer. The score is the expected level, so sorting by it puts r1 first, r2 second and r3 last. The probabilities also allow for setting a cut-off, such as leaving out results that are unlikely to be even related to the query.