Text classification is often introduced as a supervised problem: collect labeled examples, train a model, and predict a new document's label. A BERT notebook in my learning collection explores another starting point—compare representations and assign labels through similarity.
The retained example includes BERT feature extraction and cosine-similarity classification. It uses a pretrained language model as a source of features rather than as a chatbot.
From text to a vector
Once documents have numerical representations, cosine similarity compares their directions. A labeling rule can use those comparisons to assign a category.
The representation of a label matters too. Comparing a paragraph with a short category name is different from comparing it with several examples that describe the category.
This exposes several choices that a high-level classification function can hide: which model output becomes the document vector, how longer text is handled, and how the categories are represented.
Similarity needs a test set
Avoiding a new training run does not remove the need to evaluate assignments. Two comments about the same event can express opposing positions. A model may recognize their shared subject while missing the distinction the task requires.
For a useful comparison, I would keep the labels and test documents fixed, then inspect how feature and pooling choices change the assignments. Ambiguous cases help expose what a representation captures.
Where this fits
The 2018 BERT paper is a useful historical anchor for contextual language features. The notebook shows their appeal as reusable inputs to a smaller downstream task.
It remains a retained learning example, without a verified personal accuracy result. A public companion would need clear source attribution, a permitted sample, and cleared outputs.