Glossary

Projects

ActiveTigger is organized around projects. Each project consists of a dataset, a set of schemes, and additional assets such as prediction models, features, projections or topic models. A project can be shared across users for collaborating.

The dataset is a table that contains the texts that you want to work on, and optionally some other columns that will help you in the process. It can be very large, in which case you will work on smaller subsets (see the Train, Validation and Test sets).

See Project Creation page for more details.

Schemes

Each project contains one or several schemes. In short, it gathers the data you want to analyze, the corresponding annotations, and models to assist your exploration and annotation. In practice, it is a set of labels as well as the annotations, a codebook and classifier models (Quick or BERT).

Users with access to a project can see all of its schemes.

There are different types of schemes in ActiveTigger:

• Multiclass: each text will be given a single label.
• Multilabel: each text can be given several labels simultaneously.
• Span: give a label to one or more portions of the text (spans).

What you can and cannot do with each scheme type:

	Multiclass	Multilabel	Span
Create labels and a codebook	🟢	🟢	🟢
Explore your corpus with a visualisation or topic models	🟢	🟢	🟢
Train a classifier (Quick or BERT)	🟢	🟢	🔴
Use generative models	🟢	🟢	🟢
Export annotations	🟢	🟢	🟢
Export models	🟢	🟢	🔴

Evaluating Multiclass models

ActiveTigger displays 3 scores per label as well as a score weighted across all labels. Each score is bounded between 0 and 1 (the higher the better).

Measure	Data	Interpretation
Precision	Per label	How confident you can be about the positive predictions of each label
Recall	Per label	Portion of the actual positive cases that are found by the model
F1	Per label	Overall quality of the model's predictions for each label
Macro F1	Overall	Overall quality of the model on all labels

Dataset stratification

A corpus might come from different sources, or different time periods, that you might want to treat equally in an annotation workflow. Stratifying the dataset prevents working on unbalanced datasets. This is a common challenge in machine learning: training on non-representative samples leads to models failing on minority subsets.

Stratification is performed upon project creation.

Tokens and window sizes

Large language models (including sentence embedding models) do not use plain texts to process data, but first decompose them into tokens in order to make them easier to handle. Tokens are either whole words or parts of words: very common words in a given language will typically have their own token, as well as common radicals and prefixes and suffixes. Each unique symbol has its own token, so that any given word (including typos) can be read by the model.

Each language model uses its own tokenizer (the algorithm that transforms raw text into tokens).

One important feature of each language model is that it has a maximum window size: the number of tokens it is able to process for any given text. Therefore, it is important to be aware of the language model's specific window size, as tokens beyond this limit will be ignored.¹

In order to check the tokenizers of different models, you can use sites such as LLM Calculator, LLM Token Counter, Token Calculator, etc. Most of them are made for generative LLMs, but can still give you a rough estimation of how your corpus will be treated.