| 1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071 |
- Date: 8.june 2025
- Meeting with Zahra
- Dates:
- 30.6. - Zahra has presentation
- 5.7. - Zahra visits Iran for 2-3 weeks
- Asking about accuracy measures:
- * for fit we have goodness of fit measures:
- - log likelihood
- - AIC and BIC
- * these models used for classification: [1]
- - pred Y = [ Y : f(x|Y) > 0.5 === decision function(x) > 0
- [ !Y : otherwise
- - in logit models: [2] Speech and Language Processing, Ch 5
- decision_function(x) = b + w^T x == logit (x)
- logit == log odds
- logit(x) = 1/(1 + exp(-decision_function(x))
- - ACC == accuracy score [3]
- accuracy(y, \hat{y}) = 1/n \sum_{i=0}^{n-1} delta_{\hat{y}_i, y_i}
- - AUC == area under curve ???? [4,5]
- From [6]:
- fpr, tpr, thresholds = ROC = function(y_true, y_score)
- We are systematically doing
- pred Y[i] = [ 1 : y_score[i] >= threshold
- [ 0 : otherwise
- all i
- and comparing with
- y_true[i] all i
- for all meaningful thresholds == y_score
- Target scores, can either be probability estimates of the positive class,
- confidence values, or non-thresholded measure of decisions (as returned by
- “decision_function” on some classifiers). For decision_function scores,
- values greater than or equal to zero should indicate the positive class.
- In our case y_score = prob estimate(for Y=1) as given by the model
- Ref:
- - [1] https://realpython.com/logistic-regression-python/
- - [2] https://web.stanford.edu/~jurafsky/slp3/5.pdf
- - https://en.wikipedia.org/wiki/Logistic_regression
- - [3] https://scikit-learn.org/stable/modules/model_evaluation.html#accuracy-score
- - [4] https://scikit-learn.org/stable/modules/model_evaluation.html#roc-metrics
- - [5] https://en.wikipedia.org/wiki/Receiver_operating_characteristic
- - [6] https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_curve.html
- - https://stats.stackexchange.com/questions/354709/sklearn-metrics-accuracy-score-vs-logisticregression-score
- - https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_curve.html
- - https://scikit-learn.org/stable/modules/model_evaluation.html#roc-metrics
- Plan:
- * Make notes overleaf file (ZA)
- - notes will the basis for the paper and presentations
- * Transfer as much text and results to notes (ZA)
- * Test calc. with bayesian model (MH)
- * Chat about progress at the picnic
- Ideal:
- * Have some preliminary results for bayesian model until Friday
|
