# Caldera Re data files

> The CSV files behind the worked examples: one year event loss table per cedent at two trial counts, with row counts, sizes, checksums and download links.

Source: https://reinsurance.dev/caldera-re/data/

Every worked example on this site runs on the files listed here: one [year event loss table (YELT)](https://reinsurance.dev/toolkit/trials/#the-yelt) per cedent of [Caldera Re](https://reinsurance.dev/caldera-re/), at two trial counts. The data is fictitious. The [Caldera Re appendix](https://reinsurance.dev/caldera-re/#the-data) describes how it was generated and what each cedent's book contains, and [Demo scale](https://reinsurance.dev/toolkit/trials/#demo-scale) states what 20 trials can and cannot resolve. The figures in the tables are read from the files when the site is built.

## The 20-trial demo tier

The worked examples and the figures quoted in the text come from these five files, unless a page names the other tier.

*[Table: Data file table. See the web page: https://reinsurance.dev/caldera-re/data/]*

## The 1,000-trial tier

The same simulation at a higher trial count, for the resolution contrast that [Demo scale](https://reinsurance.dev/toolkit/trials/#demo-scale) describes.

*[Table: Data file table. See the web page: https://reinsurance.dev/caldera-re/data/]*

:::note[Caldera Re]
Neither tier holds a US earthquake tail. SunCoast and Pacific Mutual share one catalog of 300 US earthquake events, and the 1,000-trial tier contains 299 of them. Its costliest US earthquake occurrence costs Pacific Mutual \$41.3M and SunCoast \$14.5M, against \$29.8M and \$10.4M in the 20-trial tier. [Demo scale](https://reinsurance.dev/toolkit/trials/#demo-scale) states what this means for the contrast between the tiers.
:::

## Reading the tables

$N$, the number of trials simulated, is the number every exceedance probability divides by, and the file name records it. **Trials with rows** can be smaller, because a trial without occurrences has no row. Trident Re's 20-trial file holds rows for 19 trials, and its $N$ is still 20; [Building an EP curve](https://reinsurance.dev/toolkit/distributions/#building-an-ep-curve) works through the difference. An **occurrence** is one `(trial_id, timestamp, event_id)` key, and it can span several rows, one per geography and line of business. Two cedents' files can hold rows of the same occurrence (SunCoast and Pacific Mutual share the US earthquake events), so occurrence counts do not add across files and the total row leaves them out.

## The schema

All ten files share the nine columns of [the site's YELT schema](https://reinsurance.dev/toolkit/trials/#the-yelt): the four-column core `trial_id`, `timestamp`, `event_id`, `loss`, and the enrichment `peril`, `geography_id`, `lob_id`, `cedent_id`, `tiv`. `loss` and `tiv` are in millions of US dollars and strictly positive, under the site's [sign convention](https://reinsurance.dev/toolkit/trials/#sign-convention).

## Loading a file

The runnable examples load a cedent through the site's `caldera_re` package, which records $N$ on the frame:

```python
from caldera_re.data import cedents, load_cedent

yelt = cedents.suncoast                  # suncoast_20.csv
yelt = load_cedent("suncoast", 1000)     # suncoast_1000.csv
yelt.attrs["n_trials"]                   # 1000, from the file name
```

Outside the site, plain pandas reads a downloaded file. State $N$ yourself, because the rows cannot supply it:

```python
import pandas as pd

yelt = pd.read_csv("trident_20.csv", parse_dates=["timestamp"])   # downloaded from the table above
yelt.attrs["n_trials"] = 20              # not yelt["trial_id"].nunique(), which is 19
```

## Checking a download

The files are frozen, and the worked examples load them as published. The program that generated them is maintained privately, so a reader cannot rerun the simulation, only check that a downloaded copy is the file the site uses. On the frame loaded above, pandas recomputes the counts in the tables and the mean trial loss that the [dataset summary](https://reinsurance.dev/caldera-re/#the-data) on the Caldera Re page prints:

```python
yelt["trial_id"].nunique()                                         # 19 trials with rows
len(yelt.drop_duplicates(["trial_id", "timestamp", "event_id"]))   # 47 occurrences
len(yelt)                                                          # 94 rows
yelt["loss"].sum() / yelt.attrs["n_trials"]                        # 41.45, mean trial loss ($M)
```

The SHA-256 of each file, computed from the published bytes when the site is built, checks the bytes themselves:

*[Table: Data file table. See the web page: https://reinsurance.dev/caldera-re/data/]*

`sha256sum trident_20.csv` on Linux, `shasum -a 256 trident_20.csv` on macOS and `certutil -hashfile trident_20.csv SHA256` on Windows print the digest to compare.