|
@@ -0,0 +1,71 @@
|
|
|
|
|
+Date: 8.june 2025
|
|
|
|
|
+
|
|
|
|
|
+Meeting with Zahra
|
|
|
|
|
+
|
|
|
|
|
+Dates:
|
|
|
|
|
+ 30.6. - Zahra has presentation
|
|
|
|
|
+ 5.7. - Zahra visits Iran for 2-3 weeks
|
|
|
|
|
+
|
|
|
|
|
+Asking about accuracy measures:
|
|
|
|
|
+ * for fit we have goodness of fit measures:
|
|
|
|
|
+ - log likelihood
|
|
|
|
|
+ - AIC and BIC
|
|
|
|
|
+ * these models used for classification: [1]
|
|
|
|
|
+ - pred Y = [ Y : f(x|Y) > 0.5 === decision function(x) > 0
|
|
|
|
|
+ [ !Y : otherwise
|
|
|
|
|
+ - in logit models: [2] Speech and Language Processing, Ch 5
|
|
|
|
|
+ decision_function(x) = b + w^T x == logit (x)
|
|
|
|
|
+ logit == log odds
|
|
|
|
|
+ logit(x) = 1/(1 + exp(-decision_function(x))
|
|
|
|
|
+ - ACC == accuracy score [3]
|
|
|
|
|
+ accuracy(y, \hat{y}) = 1/n \sum_{i=0}^{n-1} delta_{\hat{y}_i, y_i}
|
|
|
|
|
+
|
|
|
|
|
+ - AUC == area under curve ???? [4,5]
|
|
|
|
|
+ From [6]:
|
|
|
|
|
+
|
|
|
|
|
+ fpr, tpr, thresholds = ROC = function(y_true, y_score)
|
|
|
|
|
+
|
|
|
|
|
+ We are systematically doing
|
|
|
|
|
+
|
|
|
|
|
+ pred Y[i] = [ 1 : y_score[i] >= threshold
|
|
|
|
|
+ [ 0 : otherwise
|
|
|
|
|
+ all i
|
|
|
|
|
+
|
|
|
|
|
+ and comparing with
|
|
|
|
|
+
|
|
|
|
|
+ y_true[i] all i
|
|
|
|
|
+ for all meaningful thresholds == y_score
|
|
|
|
|
+
|
|
|
|
|
+ Target scores, can either be probability estimates of the positive class,
|
|
|
|
|
+ confidence values, or non-thresholded measure of decisions (as returned by
|
|
|
|
|
+ “decision_function” on some classifiers). For decision_function scores,
|
|
|
|
|
+ values greater than or equal to zero should indicate the positive class.
|
|
|
|
|
+ In our case y_score = prob estimate(for Y=1) as given by the model
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+Ref:
|
|
|
|
|
+ - [1] https://realpython.com/logistic-regression-python/
|
|
|
|
|
+ - [2] https://web.stanford.edu/~jurafsky/slp3/5.pdf
|
|
|
|
|
+ - https://en.wikipedia.org/wiki/Logistic_regression
|
|
|
|
|
+
|
|
|
|
|
+ - [3] https://scikit-learn.org/stable/modules/model_evaluation.html#accuracy-score
|
|
|
|
|
+ - [4] https://scikit-learn.org/stable/modules/model_evaluation.html#roc-metrics
|
|
|
|
|
+ - [5] https://en.wikipedia.org/wiki/Receiver_operating_characteristic
|
|
|
|
|
+ - [6] https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_curve.html
|
|
|
|
|
+
|
|
|
|
|
+ - https://stats.stackexchange.com/questions/354709/sklearn-metrics-accuracy-score-vs-logisticregression-score
|
|
|
|
|
+ - https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_curve.html
|
|
|
|
|
+ - https://scikit-learn.org/stable/modules/model_evaluation.html#roc-metrics
|
|
|
|
|
+
|
|
|
|
|
+Plan:
|
|
|
|
|
+
|
|
|
|
|
+ * Make notes overleaf file (ZA)
|
|
|
|
|
+ - notes will the basis for the paper and presentations
|
|
|
|
|
+ * Transfer as much text and results to notes (ZA)
|
|
|
|
|
+ * Test calc. with bayesian model (MH)
|
|
|
|
|
+ * Chat about progress at the picnic
|
|
|
|
|
+
|
|
|
|
|
+Ideal:
|
|
|
|
|
+ * Have some preliminary results for bayesian model until Friday
|
|
|
|
|
+
|
|
|
|
|
+
|
