Estimate a default rate by maximum likelihood

A bank's credit risk team must set the one-year probability of default (PD) for its book of small-business loans. Eight years of history are on file: the loans performing at the start of each year, and how many defaulted within it. The credit committee wants one PD for the book and an interval it can set limits against.

The team also covers large corporate loans, where no loan has defaulted in six years. A PD of zero with an interval of zero width is not something the committee can use.

Your task

Each function takes loans and defaults, two lists with one count per year. Treat every loan as defaulting independently, with the same probability $p$ in every year.

1. Write log_likelihood(p, loans, defaults): the log-likelihood of $p$, leaving out the binomial coefficients, which do not depend on $p$. Raise ValueError if $p$ is outside $[0, 1]$. Return -math.inf when the data are impossible under $p$, such as a default when $p = 0$.
2. Write pd_estimate(loans, defaults): the maximum likelihood estimate $\hat{p}$.
3. Write standard_error(loans, defaults): the standard error of $\hat{p}$, from the curvature of the log-likelihood at its peak.
4. Write pd_interval(loans, defaults, level=0.95): the likelihood-ratio interval (low, high). It holds every $p$ whose log-likelihood is within half of chi2.ppf(level, 1) of the peak. With no defaults low is 0, and when every loan defaulted high is 1.

All four raise ValueError when the lists differ in length or are empty, or when a count is negative or not a whole number. They also raise it when a year has more defaults than loans, or when there are no loans at all. pd_interval raises it when level is not strictly between 0 and 1.

The data

SME_LOANS and SME_DEFAULTS hold the small-business book for the years in YEARS, 2018 to 2025: 413 defaults in 17,410 loan-years. CORP_LOANS and CORP_DEFAULTS hold six years of corporate loans: 1,902 loan-years without a default. Both books are synthetic.

Hint: Where to start

Lesson 5.9.1 writes the likelihood of 7 heads in 10 tosses as $p^7(1 - p)^3$, and finds its peak at the share of heads. A year of loans is the same experiment. Independent years multiply, so their log-likelihoods add.

Hint: The standard error

Lesson 5.9.2 turns the curvature at the peak, $J = -\ell''(\hat{p})$, into the standard error $\frac{1}{\sqrt{J}}$. For a coin that is $\sqrt{\hat{p}(1 - \hat{p})/n}$.

Hint: An interval that survives zero defaults

The Wald interval $\hat{p} \pm 1.96\,\operatorname{SE}$ breaks at the edge of the range; the likelihood-ratio interval does not (lesson 5.9.2). Find each end with brentq (lesson 1.3.2), bracketed between $\hat{p}$ and a point just inside 0 or 1.