# Process photos in insurance claims with Eevo

> Eevo docs. Every page as Markdown is listed at https://eevo.io/llms.txt; the API is described at https://eevo.io/api/openapi.json.

In addition to text inputs, Eevo can also process images. This means you can flexibly ask questions about an image, and get them answered similarly to any other questions. There is also a special output type `box`, which can be used to get the coordinates of a given object within an image.

With its training in understanding consequences, Eevo is especially powerful in tasks that require the model to have a sense of consequences. In this example, we show how to use Eevo to read a water damage photo and decide how to handle the claim.

## Example

In this example, a household claims for water damage to their bathroom ceiling, caused by a leak in the flat upstairs. They send a photo, a plumber's note that the leak has been fixed, and an estimate of €3,200 for the repair.

![A bathroom ceiling with a large brown water stain, dark mould patches and a section of ceiling fallen in.](https://eevo.io/static/docs/eevo/ceiling-wet.jpg)

*Photo: USDA, public domain.*

Below is an example snippet for asking Eevo questions about the photo. The photo is sent in the `assets` field and referenced by its ID in `state`. Eevo accepts PNG or JPEG photos between 280 and 1,568 pixels.

```python
import base64
import io
import os

import requests
from PIL import Image

API = "https://eevo.io/api/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['EEVO_API_KEY']}"}

def as_asset(path):
    """A photo as a SystemOne image asset, shrunk to fit Eevo's 1,568-pixel limit."""
    image = Image.open(path).convert("RGB")
    image.thumbnail((1568, 1568))
    buffer = io.BytesIO()
    image.save(buffer, format="JPEG", quality=90)
    return {"type": "image", "format": "jpg", "data_base64": base64.b64encode(buffer.getvalue()).decode()}

QUESTIONS = {
    "cause": {
        "type": "choice",
        "instructions": "What most likely caused the damage in the photos?",
        "criteria": {
            "plumbing_leak": "A leaking pipe, fitting or appliance",
            "roof_leak": "Rain coming in through the roof",
            "condensation": "Condensation or poor ventilation over time",
            "flood": "Water coming in from outside at ground level",
            "unclear": "The photos do not show the cause",
        },
    },
    "active": {
        "type": "noul",
        "instructions": "The damage looks active and still wet, rather than old and dried.",
    },
    "if_left": {
        "type": "score",
        "instructions": "If the ceiling is not professionally dried within 72 hours, what most likely happens?",
        "criteria": [
            "No change",
            "The stain spreads but stays cosmetic",
            "Mould grows in the ceiling",
            "Part of the ceiling falls",
        ],
    },
    "sagging": {
        "type": "noul",
        "instructions": "The ceiling sags, bulges or cracks in a way that suggests it could fall.",
    },
    "priceable": {
        "type": "noul",
        "instructions": "The photos are enough to estimate the repair without an inspection.",
    },
    "worst_area": {"type": "box", "instructions": "the most damaged area of the ceiling"},
}

response = requests.post(
    f"{API}/systemone",
    headers=HEADERS,
    json={
        "model": "eevo-1",
        "state": "Home insurance claim, water damage to a bathroom ceiling reported yesterday. Water came "
                 "through from the flat upstairs; a plumber says the leak there has been fixed. Photo: photo_1",
        "questions": QUESTIONS,
        "assets": {"photo_1": as_asset("ceiling-wet.jpg")},
    },
)
response.raise_for_status()
answers = response.json()["answers"]
```

Below is an example response:

```json
{
  "cause": { "type": "choice", "choice": "plumbing_leak", "confidence": 0.7786, "probabilities": { "plumbing_leak": 0.7786, "roof_leak": 0.0458, "condensation": 0.0858, "flood": 0.0187, "unclear": 0.0711 } },
  "active": { "type": "noul", "noul": 0.2582 },
  "if_left": { "type": "score", "score": 2.2823, "confidence": 0.6664, "legend": { "0": "No change", "1": "The stain spreads but stays cosmetic", "2": "Mould grows in the ceiling", "3": "Part of the ceiling falls" }, "probabilities": { "0": 0.0042, "1": 0.0194, "2": 0.6664, "3": 0.31 } },
  "sagging": { "type": "noul", "noul": 0.7061 },
  "priceable": { "type": "noul", "noul": 0.0951 },
  "worst_area": { "type": "box", "box": { "top": 0.1032, "bottom": 0.4135, "left": 0.1708, "right": 0.5238 }, "confidence": 0.2171 }
}
```

Here, Eevo reads the damage as a plumbing leak that has already brought part of the ceiling down: if it is left, it is likely to grow mould or fall further, and it is unlikely to be priced from the photos alone. The box marks the most damaged area, and can be drawn on the photo for the adjuster or the drying crew, as shown in [Detecting hazards from photos](https://eevo.io/api/docs/how-to/site-safety-photos.md).

The claim is covered either way, so the decision is about how to handle it. Below is an example of passing the photo answers to the [Consequence-based Decision API](https://eevo.io/api/docs/reference/api-cb-decision.md), together with what the photos can't show, like costs, timings and your own rates for mould and overpayment. The handler has three options: pay from the photos, send an adjuster, or send a drying crew first and an adjuster after.

```python
a = answers
mould_if_left = 0.40  # your rate for wet ceilings left undried; 0.05 for old, dry stains

context = f"""A home insurer's claims handler is triaging a water-damage claim. The household sent a photo of their
bathroom ceiling, damaged by water from the flat upstairs, and a plumber's note that the leak is fixed. Their estimate
for the repair is €3,200. The claim is covered and will be paid at about that amount whatever is decided; what
differs is how fast, whether someone inspects it, and whether the ceiling is dried professionally.

An image check of the photos gave: likely cause {a['cause']['choice']} ({a['cause']['confidence']:.2f}); active
and still wet {a['active']['noul']:.2f}; if not dried within 72 hours, outcome score {a['if_left']['score']:.1f}
on a 0-3 scale from no change to ceiling collapse; sagging {a['sagging']['noul']:.2f}; photos enough to price the
repair {a['priceable']['noul']:.2f}.

As things stand, without professional drying there is about a {mould_if_left:.0%} chance of mould, which adds
about €4,000 to the claim and weeks of disruption for the household. A drying crew costs €650 and cuts the mould
chance to about 5%. An adjuster costs €350 and comes within 5 working days; settling then takes about 10 days.
Paying from photos takes 2 days and carries about a 15% chance of overpaying by about €1,500."""

OPTIONS = [
    {"id": "A", "description": "Fast-track: pay the €3,200 estimate from the photos within 2 days.",
     "consequences": [
         {"stakeholders": ["household"], "valence": "benefit", "label": "Paid about 8 days sooner",
          "description": "The household gets its €3,200 about 8 days sooner (in 2 days instead of about 10) and can book its own repair right away."},
         {"stakeholders": ["insurer"], "valence": "benefit", "label": "No adjuster cost",
          "description": "The insurer saves the €350 adjuster visit."},
         {"stakeholders": ["insurer"], "valence": "harm", "label": "Overpayment risk",
          "description": "About €225 of expected overpayment (a 15% chance of overpaying by about €1,500)."},
     ]},
    {"id": "B", "description": "Send an adjuster within 5 working days, then settle on the adjuster's estimate.",
     "consequences": [
         {"stakeholders": ["household"], "valence": "harm", "label": "An adjuster visit at home",
          "description": "Someone in the household stays home for about an hour for the adjuster's visit."},
         {"stakeholders": ["insurer"], "valence": "harm", "label": "Adjuster cost",
          "description": "The insurer pays €350 for the adjuster visit."},
     ]},
    {"id": "C", "description": "Send a drying crew within 24 hours and an adjuster within 5 working days, then settle.",
     "consequences": [
         {"stakeholders": ["household"], "valence": "benefit", "label": "Mould risk mostly removed",
          "description": "About €1,400 of expected mould damage and disruption avoided for the household (the chance of mould, adding about €4,000 and weeks of disruption, falls from 40% to 5%)."},
         {"stakeholders": ["household"], "valence": "harm", "label": "Crew in the home 3 days",
          "description": "Drying fans and dehumidifiers run in the bathroom for about 3 days: noise and a room out of use, plus an hour at home for the adjuster."},
         {"stakeholders": ["insurer"], "valence": "harm", "label": "Drying and adjuster costs",
          "description": "The insurer pays €650 for the drying crew and €350 for the adjuster, €1,000 in all."},
         {"stakeholders": ["insurer"], "valence": "benefit", "label": "Mould claim mostly avoided",
          "description": "About €1,400 of expected mould claim costs avoided (the chance of a €4,000 mould claim falls from 40% to 5%)."},
     ]},
]

decision = requests.post(
    f"{API}/cb-decision",
    headers=HEADERS,
    json={
        "model": "eevo-1",
        "mode": "default",
        "context": context,
        "options": OPTIONS,
        "stakeholders": [
            {"id": "household", "description": "The policyholder household whose ceiling is damaged",
             "values": "A safe, dry home, quick fair payment, little disruption."},
            {"id": "insurer", "description": "The home insurer",
             "values": "Total claim cost, accurate settlements and customer retention."},
        ],
    },
)
decision.raise_for_status()
for result in decision.json()["results"]:
    print(result["stakeholder"]["id"], result["best"], result["probabilities"])
```

In this example, both the household and the insurer are best served by sending the drying crew first.

## Describe options against the same starting point

The household is paid under every option, so the payment itself is not a consequence of paying from the photos, but being paid 8 days sooner is. Describing each option's consequences against the same starting point matters: when a first version of this decision listed "receives €3,200" as a benefit of paying from the photos, Eevo picked that option for the household.

## Read next

- [Detecting hazards from photos with Eevo](https://eevo.io/api/docs/how-to/site-safety-photos.md): Example of detecting hazards in construction site photos.
- [SystemOne decision](https://eevo.io/api/docs/reference/api-systemone.md): POST /api/v1/systemone
- [Probability, confidence and expected value](https://eevo.io/api/docs/explanation/probability-and-ev.md): Learn what the probabilities in an answer mean.
