For teachers and coordinators

The questions a course asks before it starts, with the answers as the service runs today. Where a topic has its own page, the answer links to it.

Accounts and credits#

Every person has their own account. Students, teachers, tutors and coordinators sign up at https://my.ai.itsincom.org with their @itsincom.it address. The account is activated on request: the request goes to the school's direction, and once the direction says yes we approve it. From then on it works with the limits below.

Credits are given at approval, not renewed on a calendar. A teacher's account starts with 5,000 credits, every other role with 1,000; whoever approves can give more to a single request. When an account runs low it is topped up on request. We keep an eye on how the credits are used, so that they go to the school's work: a course that needs more for an exercise asks, and gets them.

Everyone sees only their own keys and usage. Each person creates their own API keys, and on each key can set a monthly spending cap, a number of requests per minute and a country (Account and usage). Usage is on the Usage page, filtered by key, and through GET /v1/usage. For a course this is the right shape: every student measures their own consumption, with their own key.

Roles. Teacher, Coordinator, Tutor and Student. Today the role sets the starting credits and what is asked at sign-up (a student names their course); the features are the same for everyone. A teacher does not see the students' keys: the usage of a whole class is seen by the administrators.

No limit per session. What bounds a request is the account's credit, the model's context and at most 32,768 tokens of output per request (Every parameter).

This documentation is the official one of the school's platform. The live pages are read from the running service, so their numbers are always current.

How the engines are started#

The models are served by vLLM, except Flint (Ollama) and gpt-oss-120b (llama.cpp). For each model:

«Up to»: the same model runs on more than one machine, and some of them hold 131,072 tokens. For prompts longer than that, use qwen3-coder-30b, qwen3-coder-next or qwen3.6-35b: every machine that serves them holds 262,144.

• Context. Apertus is limited to 16,384 tokens by the memory of the machine that serves it; for long contexts use one of the others. A longer context for Apertus will come with a larger machine.
• Tool calling. --enable-auto-tool-choice is on wherever there is a parser. See Structured output and function calling.
• Reasoning. Where a model reasons, --reasoning-parser is on: the reasoning does not end up in the answer's text, and counts as output tokens. chat_template_kwargs is refused with a 400 that says to use reasoning_effort instead, which we translate for each engine: "none" switches Qwen3.8 off (Reasoning).

What you can measure#

• Tokens of every request, in the usage field. In a stream, with "stream_options": {"include_usage": true} a last chunk before [DONE] carries them (Streaming).
• The prompt tokens taken from the cache, in usage.prompt_tokens_details.cached_tokens. Prefix caching (--enable-prefix-caching) is on on every vLLM engine: a prompt that starts like the previous one is not computed again.
• Usage added up by key, model, day or client, in JSON or CSV (Account and usage).
• Constrained decoding. response_format with a JSON schema, and vLLM's own structured_outputs (EBNF grammar, regex, choice). On the NVIDIA engines with vLLM 0.32 the grammars are compiled by xgrammar 0.2.8; Apertus by the xgrammar of vLLM 0.11. gpt-oss takes JSON schemas, not EBNF grammars; Flint takes neither (Structured output).
• seed reaches the engine. With the same seed and temperature: 0 the answer is almost always the same, but not guaranteed bit for bit: with many requests at once vLLM batches them differently, the floating-point sums change order, and now and then a token that was nearly tied goes the other way. With one request at a time the differences almost vanish. A good exercise to measure in class.
• There is a gateway in front of the engines: it checks the key, counts the tokens and picks the machine. Streams leave it unbuffered (X-Accel-Buffering: no): the tokens arrive as the model writes them. What it changes in a request, and what it never touches, is in What we touch.
• /metrics is not exposed: the machines cannot be reached from outside. The state of the service is on the live fleet page.

Twenty people at once#

• The machines. NVIDIA L40S cards and NVIDIA DGX Spark (GB10), in Italy; we do not publish the inventory. The engines that have a GPU to themselves reserve 90–96% of its memory (--gpu-memory-utilization), most of it for the cache of the keys and values.
• Requests at once. Each machine runs 4 to 32 requests at the same time (--max-num-seqs, or --parallel on llama.cpp), depending on the model. The most used models run on more than one machine, and the gateway shares the requests among them.
• Above that, a request waits. It queues on the machine and the student sees a slower answer, not an error. An error comes only when no machine of the model is available (503, to retry) or when a key goes over its requests per minute (429 with Retry-After). See Errors.
• Limits. Every key has 60 requests a minute, plus a limit per endpoint. Limits counted per IP address apply only to requests without a key, and are twenty times wider from the school's own networks (Rate limits).
• Twenty coding assistants with whole files in the context fit the code models: qwen3-coder-30b and qwen3-coder-next hold 262,144 tokens on every machine, and with the prefix cache a file already read is not computed again at every request. The guides to connect the assistants are in Integrations.
• A load test before the course is welcome: write to segreteria@itsincom.it with the day and time, and we run it together.

Availability and who to call#

• The service is on all the time, during lessons and outside them.
• It restarts by itself. The engines restart on their own. Every minute the gateway checks each machine, takes one that does not answer out of the rotation and sends its requests to the others. The alarms reach the operators automatically, at any hour: usually we know before you do.
• A change of model is written in the changelog and shown at the top of the reserved area. A model is withdrawn only with notice.
• A change of configuration, such as a longer context or a parser, needs no formal request: we agree on it, and a restart of a few minutes outside lesson hours is usually enough.
• If it does not answer during a lesson, write to segreteria@itsincom.it with the time and, if you have it, the request_id of a failed request.

What is kept of the requests#

• The text of API requests and answers is kept only for the keys where you turn it on, in the Messages column of the API keys page, for debugging your program. It is stored encrypted in the platform's database, in Italy, never in clear.
• It is deleted after 30 days, with the rest of the request log (model, tokens, time taken, outcome). The usage records (tokens, model, time, no text) stay for the accounts.
• More on where the data are and who processes them: Sovereignty, Data protection, Sub-processors.

Asking for more#

• A specific model can be asked for: Flint was added on 26 October at the school's request. Write to segreteria@itsincom.it.
• Embeddings are already there: bge-m3 on /v1/embeddings (Embeddings).