From TSK to the encyclopedia
A student reading Tan, Steinbach, Karpatne, and Kumar, Introduction to Data Mining, second edition, meets a term and does not know which entries to read. For each TSK term, the entries to read in order and the chapter of Data Mining as Observation that takes the term up.
| TSK term | TSK | Entries to read | Book chapter |
|---|---|---|---|
| attribute types, nominal, ordinal, interval, ratio | 2.1 | attribute type; standardization; discretization | 2 |
| data quality, noise and outliers | 2.2 | outlier; detector; skewness | 2 |
| missing values | 2.2 | imputation; leakage | 2 |
| aggregation and sampling | 2.3 | aggregation; sampling; Simpson's paradox | 2 |
| dimensionality reduction | 2.3, appendix B | principal component analysis; truncation; explained variance; the flip | 4 |
| feature subset selection | 2.3 | read subspace; nuisance; importance | 4 |
| discretization and binarization | 2.3 | discretization; quotient | 2 |
| variable transformation, standardization | 2.3 | standardization; whitening | 2 |
| Euclidean distance, Minkowski distance | 2.4 | Euclidean distance; metric; identity reader | 3 |
| cosine similarity | 2.4 | cosine; dot product; read direction; quotient | 3 |
| Jaccard coefficient, simple matching | 2.4 | Jaccard; quotient | 3 |
| correlation | 2.4 | correlation; Spearman correlation; Kendall correlation | 3 |
| Mahalanobis distance | 2.4 | Mahalanobis distance; whitening; covariance matrix | 10 |
| decision tree induction | 3.3 | decision tree; importance; classifier | 6 |
| model overfitting | 3.4 | capacity; harness; early stopping | 6 |
| model selection, validation set | 3.5, 3.6 | split; cross-validation; leakage | 8 |
| model evaluation, holdout, cross-validation, bootstrap | 3.6 | cross-validation; bootstrap; Nadeau and Bengio correction; seed | 8 |
| confusion matrix, accuracy, precision, recall, F-measure | 3.2, 4.11 | precision, recall; F1; balanced accuracy; threshold | 6 |
| ROC curve | 4.11 | ROC curve; AUROC; Youden index; Monotone Invariance Theorem | 5 |
| rule-based classifier | 4.2 | formula search; safe pruning | 6 |
| nearest neighbor classifier | 4.3 | nearest neighbour; neighbourhood graph; hub | 6 |
| naive Bayes classifier | 4.4 | naive Bayes; Hessian | 6 |
| logistic regression | 4.6 | logistic regression; linear classifier; decision boundary | 6 |
| artificial neural network, deep learning | 4.7, 4.8 | encoder; embedding; attention; language model | 11 |
| support vector machine, margin | 4.9 | margin; linear classifier; decision boundary | 6 |
| ensemble methods, bagging, boosting, random forest | 4.10 | ensemble; bagging; boosting; random forest | 7 |
| class imbalance | 4.11 | balanced accuracy; chance level; stratification | 6 |
| itemset, support, frequent itemset | 5.1, 5.2 | itemset; support; transaction | 5 |
| Apriori principle, candidate pruning | 5.2 | Apriori principle; anti-monotonicity; safe pruning | 5 |
| association rule, confidence | 5.3 | confidence; lift; cross-support ratio | 5 |
| FP-growth, closed and maximal itemsets | 5.4, 5.5 | itemset; anti-monotonicity | 5 |
| objective interestingness measures | 5.6 | lift; output metric; Simpson's paradox | 5 |
| k-means | 7.2 | k-means; sum of squared errors; identity reader | 9 |
| agglomerative hierarchical clustering, linkage | 7.3 | hierarchical clustering; metric | 9 |
| DBSCAN, density-based clustering | 7.4 | DBSCAN; density; degree | 9 |
| cluster validity, silhouette, SSE | 7.5 | validity index; silhouette; sum of squared errors; vacuity threshold | 9 |
| spectral clustering, graph-based clustering | 8.5 | spectral clustering; spectral embedding; Laplacian; commute time | 3 |
| curse of dimensionality | 8.1 | distance concentration; intrinsic dimension; manifold | 3 |
| statistical anomaly detection | 9.3 | outlier; Gaussian; Mahalanobis distance | 10 |
| proximity-based and density-based anomaly detection | 9.4, 9.5 | detector; density; anti-hub | 10 |
| clustering-based and reconstruction-based anomaly detection | 9.6, 9.7 | detector; reconstruction error | 10 |
| evaluation of anomaly detection | 9.10 | min-over-strata; abstention; anti-hub recall | 10 |
| hypothesis testing, p-value, null hypothesis | 10.2 | p-value; null model; control; bar | 8 |
| multiple hypothesis testing, Bonferroni, false discovery rate | 10.3 | multiple comparisons; harness | 8 |
| pitfalls, p-hacking, data dredging | 10.4 | preregistration; sealed; registered; declaration | 8 |
| confidence interval, standard error | appendix C, 10.2 | confidence interval; standard error; paired; sampling distribution; normal distribution; central limit theorem | 8 |
| eigenvalues and eigenvectors, matrix rank | appendix A | eigenvalue, eigenvector; spectrum; rank; positive semidefinite; spectral decomposition; linear independence; power iteration | 0 |
| singular value decomposition, PCA | appendix B | principal component analysis; latent semantic analysis; effective rank; singular value decomposition; singular value; low-rank approximation | 4 |
| covariance matrix, variance | appendix A, C | covariance matrix; variance; trace; anisotropic; covariance; quadratic form | 0 |
| gradient, derivative | appendix E | sensitivity; finite difference; curvature; Jacobian; gradient; partial derivative | 0 |
| TF-IDF, document vectors | 2.4, 8.1 | TF-IDF; bag of words; latent semantic analysis | 12 |
| vector quantization, compression | 7.2, 2.3 | quantization; codebook; product quantization; bit; matched bits | 11 |
| scalability, indexing, nearest-neighbour search | 4.3, 8.4 | inverted file; recall at k; shard; rerank; rank certificate | 11 |
| vectors, dot product, norms | appendix A | vector; dot product; length, norm; unit vector; orthogonal | 0 |
| matrix multiplication, inverse, determinant | appendix A | matrix; transpose; inverse; determinant | 0 |
| linear independence, span, basis, null space | appendix A | linear independence; span; basis; null space; column space | 0 |
| orthogonal projection, orthonormal basis, Gram and Schmidt | appendix A | projection; orthonormal basis; Gram and Schmidt; orthogonal matrix | 0 |
| probability, conditional probability, Bayes theorem | appendix C | probability; event; conditional probability; independence; Bayes' rule; base rate | 6 |
| random variables, expectation, variance | appendix C | random variable; expectation; variance; standard deviation; z-score | 3 |
| binomial, Poisson, normal distributions | appendix C | Bernoulli, binomial; Poisson distribution; normal distribution; uniform distribution | 3 |
| sample mean, median, sample variance, skewness | appendix C | mean, median; sample variance; skewness; percentile | 3 |
| hypothesis testing, p-values, significance | 10.1 | null hypothesis; test statistic; p-value; permutation test; type I and type II error, power | 8 |
| multiple comparisons, Bonferroni | 10.3 | multiple comparisons; family-wise error rate; Bonferroni correction | 8 |
| confusion matrix, accuracy | 3 | confusion matrix; accuracy; false positive rate; precision, recall | 6 |
| entropy, information gain | 3.3 | entropy; surprise; cross-entropy; bit | 6 |
| random sampling, seeds, simulation | 2.3 | with replacement, without replacement; pseudo-random generator; Monte Carlo; seed | 3 |