PII detection with Eevo
Avoiding mishandling of personally identifiable information (PII) is a big challenge, especially when AI agents are working flexibly within organisations. The cost/performance and speed/performance of decision models make them well suited for implementing controls that seek to prevent PII leaks, for example as part of input and output safety checks on an agent's tool use.
Exactly how you implement PII detection with a decision model depends on your context. This guide serves as an indicative how-to for a typical setup that you should adapt to work in your context.
Below is an example of using Eevo to detect personally identifiable information:
import os
import requests
TEXT = """Hi, this is Maria Lindqvist. I was charged twice for order 48213, both times on the card ending in 4417. Please send the refund confirmation to maria.lindqvist@example.com."""
QUESTIONS = {
"contains_pii": {
"type": "noul",
"instructions": "The text contains personally identifiable information.",
},
"name": {
"type": "noul",
"instructions": "The text contains a person’s full name.",
},
"contact": {
"type": "noul",
"instructions": "The text contains an email address, phone number or postal address.",
},
"payment": {
"type": "noul",
"instructions": "The text contains payment card or bank account details, even partial.",
},
"health": {
"type": "noul",
"instructions": "The text contains information about a person’s health.",
},
}
response = requests.post(
"https://eevo.io/api/v1/systemone",
headers={"Authorization": f"Bearer {os.environ['EEVO_API_KEY']}"},
json={"model": "eevo-1", "state": TEXT, "questions": QUESTIONS},
)
response.raise_for_status()
print(response.json()["answers"])In the snippet, there is a general question on whether there is PII, and specific questions targeting specific types of PII. The latter kind of question is useful if your organisation has a policy that distinguishes different types of personally identifiable information.
An example response is below:
{
"contains_pii": {
"type": "noul",
"noul": 0.9367
},
"name": {
"type": "noul",
"noul": 0.9578
},
"contact": {
"type": "noul",
"noul": 0.9704
},
"payment": {
"type": "noul",
"noul": 0.9281
},
"health": {
"type": "noul",
"noul": 0.0373
}
}As usual, the decision model provides probabilities reflecting its confidence in each answer. The probabilities allow for setting thresholds that trigger safeguards when there is a substantial risk of PII leaking.