Note on preparing your input dataset

The input dataset should be a CSV, Parquet, or XLSX file. The text columns must be well prepared and curated before upload as ActiveTigger does not propose any text treatment option. The only text manipulation implemented is the concatenation of text columns. Cleaning your dataset can include, but is not limited to, the following:

  • Removing semanticless characters (html tags, encoding artefacts, emojis1, ...).
  • Removing any duplicates.
  • Splitting texts at the sentence/paragraph level.
  • Ensuring all inputs are written in a single language (depending on the task).

Size of the texts

If you use models, please mind that they have a limited context window. It means that if texts exceed this window (depending on the model, from a few hundred to a few thousand words), they will be truncated.

We also encourage users to carefully set up their ID column as this will ease the process of aggregating the annotations with external metadata for downstream analysis.

Troubleshoot using Parquet files

If you upload a parquet file, you can encounter issues related to the format of your parquet file. Easy fix includes:

  • Make sure that data contained is formatted as a string.
  • Make sure to not have nested columns (columns and sub-columns).

  1. Depending on your task you may want to keep them because they carry significant information, or remove them because they will act as noise. Also, models may not have been trained on emojis, meaning that they will be replaced by [unk] tokens, adding noise to your text inputs. ↩