Python for Probability, Statistics, and Machine Learning

3.4 Estimation Using Maximum Likelihood 139 with corresponding logarithm as nn J = log(L( p|x)) = log( p) xi + log(1 − p) n − xi i=1 i=1 Taking the derivative of this gives dJ = 1 n (n − n xi ) dp p p i =1 xi + −1 i =1 and solving this for p leads to 1n pˆ = xi n i =1 This is our estimator for p. Up until now, we have been using Sympy to solve for this based on the data xi but now that we have it analytically we don’t have to solve for it each time. To check if this estimator is biased, we compute its expectation: =1 n 1 n n E pˆ E(xi ) = nE(xi ) i by linearity of the expectation and where E(xi ) = p Therefore, E pˆ = p This means that the estimator is unbiased. Similarly, ⎡⎤ n2 E pˆ 2 = 1 E ⎣ n2 xi ⎦ i =1 and where E xi2 = p and by the independence assumption, E xi x j = E(xi )E(x j ) = p2