Generative page

The panel allows to use external generative models to annotate or extract information with prompts.

Warning

This is an experimental page, dedicated to exploring the use of external generative models. Some features can evolve rapidly.

The general idea is to run a prompt for a sample of the text and get the results. Each run is characterized by a batch id.

Each LLM model is running on external servers: it is up to the user to check its parameters and the level of security for the server involved.

Add new model

Create new model allows to create a new model for the project (shared by all users).

The main point to configure a model is to have access to an API. Usually, this involves knowing the endpoint and the credentials.

Info

Several APIs are based on the OpenAI API: OpenAI, OpenRouter. Ollama runs quantized models and can be installed locally.

Ilaas is a French federation providing models for inference.

HuggingFace is a platform for open source models that can also host LLMs.

  • API: The type of API that serves the model.
    • Endpoint: Some APIs require specifying the endpoint.
  • Model: The model to call from the API.
    • Available models are specific to each API.
  • API Credentials: Authentication token (specific to the API).
  • Name: The name of the model.

Configure and run a generation

Select a model

Define the sample of elements you want to send with the prompt

  • Elements: Number of elements to send
  • From: Dataset

Select or save a prompt

  • Saved prompts: Prompt the user saved
  • ICON: Save current prompt
  • ICON: Delete selected prompt

Enter a prompt : plain text for the prompt to send to the generative model. You can add elements from the dataset : [[TEXT]] for the text, and [[CONTEXT-VARIABLE]] for each contextual variable.

Generate Launch the generation process

Results

Once a generation process has finished, generated data is displayed in the table.

  • Download all Download the generated data.
  • Clear all Clear all generated data for the current project.

Add treatment for the generated columns allows you to select filters to clean the generated text before downloading it.