Skip to main content

Session Recording Summaries

Report an issue with this page

Teleport Identity Security allows you to generate and view session recording summaries for shell and database sessions. This allows you to see at a glance what your users do and estimate legitimacy of their actions before digging deeper and reviewing the entire recording.

note

Session Recording Summaries and AI Usage Session Recording Summaries are available to Teleport Enterprise customers with Teleport Identity Security.

Important: AI-powered summarization is disabled by default in all Teleport deployments (both self-hosted and cloud). No AI processing occurs unless you explicitly enable this feature. This ensures you have complete control over whether and how AI is used with your session recordings.

When you choose to enable summarization, the following options are available:

Integration modeSelf-hosted Teleport EnterpriseTeleport Enterprise Cloud
Using OpenAI with your own OpenAI key✅✅
Using OpenAI-compatible LLM gateways such as LiteLLM✅✅ (*)
Using Amazon Bedrock models provided by your own AWS account✅❌ (**)
Using an Amazon Bedrock model provided by Teleport❌❌ (**)

(*) In case of Teleport Cloud, the Teleport Auth Service must be able to reach the LLM gateway. It means the gateway needs to be publicly accessible and properly secured. Access to the gateway should be guarded by the equivalent of OpenAI API key (e.g. LiteLLM master key).

(**) Amazon Bedrock will be available in Teleport Enterprise Cloud later this year.

Existing customers will never have AI features automatically enabled without explicit consent. Learn more about the features included in Teleport Identity Security.

How it works​

Teleport can automatically summarize any recorded interactive SSH or Kubernetes session, as well as any recorded database session.

After each session finishes, Teleport decides if it should be summarized by matching it against a set of policies which route sessions to LLMs. Exactly which sessions are summarized and which model is used can be decided on using session, resource, and user metadata, such as session kind, session participants, resource labels, user traits, etc.

If a match is found, Teleport proceeds to summarize the session recording. It uses an inference model configuration indicated by the policy to generate a summary using an external inference provider, such as OpenAI or Amazon Bedrock. There is no way to run this process on demand.

There are three fundamental concepts to that underpin session summarization:

  • Inference providers (OpenAI, Amazon Bedrock) do the heavy lifting of summarization.
  • Inference models indicate how sessions get summarized: they are Teleport configuration resources that provide model settings.
  • Inference policies indicate what gets summarized and which inference model is used in given case.
warning

This feature is powered by AI leveraging a Large Language Model, and as such, its accuracy may vary.

Teleport doesn't currently support any spend controls; controlling the LLM token budget is your responsibility.

Prerequisites​

  • A running Teleport Enterprise (Self-Hosted) cluster v18.2.0 or later with Identity Security enabled.
  • Access to an LLM API. This can be:
    • OpenAI or OpenAI-compatible inference API, either public or self-hosted.
    • Amazon Bedrock (available in self-hosted mode only; see the availability table above).

Step 1/5: Configure access to inference configuration​

In order to manage the summary inference configuration, you need a write access to inference_model, inference_secret, and inference_policy resources, which is granted by default through the preset editor role.

Alternatively, you can create a new role to make sure you have access to all necessary resources. To create a new role:

  1. Create a summarizer-admin.yaml file with the following content:

    kind: role
metadata:
  name: summarizer-admin
spec:
  allow:
    rules:
    - resources:
      - inference_model
      - inference_secret
      - inference_policy
      verbs:
      - read
      - list
      - create
      - update
      - delete
version: v7

  2. Apply your changes:

    tctl create -f summarizer-admin.yaml
    tip

    You can also create and edit roles using the Web UI. Go to Access -> Roles and click Create New Role or pick an existing role to edit.

  3. Assign the summarizer-admin role to your Teleport user by running the appropriate commands for your authentication provider:

    1. Retrieve your local user's roles as a comma-separated list:

      ROLES=$(tsh status -f json | jq -r '.active.roles | join(",")')

    2. Edit your local user to add the new role:

      tctl users update $(tsh status -f json | jq -r '.active.username') \  --set-roles "${ROLES?},summarizer-admin"

    3. Sign out of the Teleport cluster and sign in again to assume the new role.

Step 2/5 : Configure the inference model​

In this step, you will create an inference_model resource. It tells Teleport how to summarize a session by pointing it to a large language model (LLM) API. In particular, it contains an API-specific model name. Note that the Teleport inference model name and API-specific model name are two different things. This allows, for example, using the same model from providers like OpenAI or Amazon Bedrock with different temperature settings, or routing to different API gateways by creating multiple Teleport inference models that reference the same underlying API-specific model name

To communicate with OpenAI, an inference model needs an API key. It's stored separately in an inference_secret resource and referenced by the inference_model's spec.openai.api_key_secret_ref field.

warning

To protect your API key, once written, an inference_secret value cannot be retrieved through tctl or Teleport API. It can only be updated or deleted.

  1. Create an inference-model.yaml file with the following content:

    kind: inference_model
version: v1
metadata:
  name: shell-summary-model
spec:
  openai:
    # In this scenario, where we connect to OpenAI directly, openai_model_id
    # needs to be a valid OpenAI model name.
    openai_model_id: gpt-4o
    # This value refers to the `inference_secret` resource name.
    api_key_secret_ref: openai-key
  # It's a good idea to limit the session length to prevent incurring cost for
  # sessions that were too long anyway to fit in the model's context window.
  # If unset, defaults to 200kB. See the Limitations section for more details.
  max_session_length_bytes: 190000

---

kind: inference_secret
version: v1
metadata:
  name: openai-key
spec:
  value: '<paste-your-openai-api-key-here>'

  2. Apply your changes:

    tctl create -f inference-model.yaml

Step 3/5: Set up an inference policy​

In this step, you will add an inference_policy resource that routes sessions to inference models using session kinds and optional custom filters. Only one policy is allowed to match a given session to guarantee that its recording will be summarized at most once. Policies are checked in alphabetical order of names and the first one that matches is used. Here are some example scenarios that can be expressed using inference policies:

  • Only summarize sessions to your production resources using resource labels.
  • Use different models for summarizing shell and database sessions.

See Session matching examples for details on how to configure Teleport to achieve these goals.

To create the required configuration resources:

  1. Create an inference-policy.yaml file with the following content:

    kind: inference_policy
version: v1
metadata:
  name: shell-sessions
  description: Summarize all sessions using OpenAI
spec:
  # Session kinds affected by this policy. Allowed values: "ssh", "k8s", "db".
  kinds:
  - ssh
  - k8s
  - db
  # Name of the `inference_model` resource that will be used for summarizing
  # these sessions.
  model: shell-summary-model

  2. Create the inference policy in the backend:

    tctl create -f inference-policy.yaml

For full description of the inference configuration resources, see appropriate sections of Teleport Resources Reference:

Step 4/5: Configure access to session summaries​

Every user who has access to given session recording also has access to its summary through a session resource kind. The preset access role allows users to only view their sessions; in order to allow a user to view all recordings and summaries, it's easiest to use the preset auditor role.

You can either use the built-in roles or create a new one to make sure you have access to all recordings. To create a new role:

  1. Create a summary-auditor.yaml file with the following content:

    kind: role
metadata:
  name: summary-auditor
spec:
  allow:
    rules:
    - resources:
      - session
      verbs:
      - read
      - list
version: v7

  2. Apply your changes:

    tctl create -f summary-auditor.yaml

  3. Assign the summary-auditor role to your Teleport user by running the appropriate commands for your authentication provider:

    1. Retrieve your local user's roles as a comma-separated list:

      ROLES=$(tsh status -f json | jq -r '.active.roles | join(",")')

    2. Edit your local user to add the new role:

      tctl users update $(tsh status -f json | jq -r '.active.username') \  --set-roles "${ROLES?},summary-auditor"

    3. Sign out of the Teleport cluster and sign in again to assume the new role.

Step 5/5: Conduct a session and view its summary​

That's it! We are ready to conduct a test session. Connect to any of your cluster resources using tsh ssh, kubectl exec, tsh db connect, or a web terminal. Start an interactive session and execute any command, quit the session, and go to Audit -> Session Recordings. You should see the corresponding recording tile with a summary button summary button. Clicking the summary button will show the summary.

info

It can take a while to generate a summary, and the time depends on multiple factors, including session size, model, and its temperature. Usually it should appear up to a minute after the session is uploaded to the Auth service.

Limitations​

Currently, Teleport doesn't attempt to summarize sessions that are larger than the model's context window. If after tokenization, the session turns out to be larger than that, OpenAI will simply reject the request; the input tokens used will be wasted. To prevent this, Teleport allows you to customize the maximum size of session that may be sent to the inference provider by using the spec.max_session_length_bytes field of the inference_model resource. Note that there's no way to provide an exact number of bytes for a given model here; in our experiments, we observed between 2 and 4 bytes per token, depending on the model and input.

Selecting sessions to summarize​

Indiscriminately summarizing all sessions in your cluster can increase costs, so it's recommended to be specific about which sessions will be summarized and which model will be used for inference. One of these mechanisms — selecting sessions by kind — was mentioned already in the above example. In addition to matching by kind, it's also possible to add a custom filter to each inference policy, which can be used to summarize only sessions that target production resources, are attended by certain users, etc.

Filters are expressed using the Teleport predicate language. If an inference policy contains a filter, it will match only sessions that match one of the indicated kinds and the specified filter.

Session matching examples​

A typical case would be to pick only production resources for summarizing. The following example policy only matches Kubernetes sessions in any of the production clusters, picked by matching cluster labels:

kind: inference_policy
version: v1
metadata:
  name: prod-k8s-sessions
  description: Summarize production Kubernetes sessions
spec:
  kinds:
  - k8s
  filter: 'equals(resource.metadata.labels["env"], "prod")'
  model: shell-summary-model

The following example uses two policies to route shell and database sessions to two different models. It assumes that there are two inference_model resources created: database-summary-model and shell-summary-model.

kind: inference_policy
version: v1
metadata:
  name: shell-policy
spec:
  kinds: ['ssh', 'k8s']
  model: shell-summary-model

---

kind: inference_policy
version: v1
metadata:
  name: db-policy
spec:
  kinds: ['db']
  model: db-summary-model

Predicate Language​

Inference policy filters are expressed using Teleport predicate language. In this case, the language supports the following operators:

OperatorMeaningExample
&&and (all conditions must match)contains(field1, field2) && equals(field2, "val")
||or (any one condition should match)contains(field1, field2) || contains(field1, "val2")
!not (used with functions)!equals(field1, field2)
==equal toresource.metadata.labels["env"] == "prod"
!=not equal toresource.metadata.labels["env"] != "prod"

The language also supports the following functions:

FunctionsReturn typeDescription
contains(list, item)BooleanReturns true if item is included in the list
equals(a, b)BooleanReturns true if value a is equal to b
contains_all(list, items)BooleanReturns true if all items are included in the list
contains_any(list, items)BooleanReturns true if any of items is included in the list
set(value1, value2, ...)[]stringReturns a list of strings
all_end_with(a, suffix)BooleanIf a is a list, returns true if all strings in the a end with suffix. If a is a string, returns true if a ends with suffix.
all_equal(a, b)BooleanIf a is a list, returns true if all items in a are equal to b. If a is a string, returns true if it's equal to b.
is_subset(a, item1, item2, ...)BooleanIf a is a list, returns true if list(item1, item2, ...) is a subset of a. If a is a string, returns true if the list contains a.
has_prefix(s, prefix)BooleanReturns true if string s starts with prefix

It's possible to build predicates using information about target resource, session, and user.

FieldMeaningExample
userUser that created the session.
user.metadata.nameUser's nameis_subset(user.metadata.name, "alice", "bob")
user.spec.rolesUser's rolescontains(user.spec.roles, "intern")
user.spec.traitsUser's traitscontains(user.spec.traits["teams"], "alpha")
resourceResource that this session connects to (server, Kubernetes cluster, or database)
resource.kindResource kindresource.kind == "node
resource.metadata.nameResource name. In case of SSH servers, it's the server ID.resource.name == "production-cluster"
resource.metadata.labelsResource labelsequals(resource.metadata.labels["env"], "prod")
sessionThe session that is being matched. This object supports all the fields of session.end and db.session.end events, so we only name the most useful ones here.
session.participantsUsers that participated in the session (SSH and Kubernetes sessions only)contains(session.participants, "alice")
session.cluster_nameName of the Teleport clustercluster_name == "teleport.example.com"
session.db_nameName of database (the underlying one, not the Teleport resource; database sessions only)session.db_name == "orders"
session.db_serviceDatabase service name (database sessions only)session.db_service == "prod-db-service"
session.kubernetes_usersList of Kubernetes usernames (Kubernetes sessions only)contains(session.kubernetes_users, "admin")
session.kubernetes_groupsList of Kubernetes groups (Kubernetes sessions only)contains(session.kubernetes_groups, "viewers")
session.kubernetes_node_nameName of node that runs the pod (Kubernetes sessions only)session.kubernetes_node_name == "node1"
session.kubernetes_pod_nameName of the pod (Kubernetes sessions only)session.kubernetes_pod_name == "some-pod"
session.kubernetes_pod_namespaceNamespace that contains the pod (Kubernetes sessions only)session.kubernetes_pod_namespace == "default"

Using LiteLLM and other OpenAI-compatible proxies​

Although the only officially supported inference provider is currently OpenAI, it's also possible to use any OpenAI-compatible API for this purpose. For example, you may want to route some or all sessions to another provider or even host your language models on premises. One way to do it is using LiteLLM. Here's what you need to do:

  1. Set up LiteLLM Proxy Server (LLM Gateway) on a host accessible from the Teleport Auth server.
  2. Add a model of your choice to the LiteLLM configuration. In this example, let's call it my-custom-model.
  3. Configure a Teleport inference model. Use spec.openai.base_url property to point Teleport to your LiteLLM instance and put my-custom-model in spec.openai.openai_model_id.
  4. Add an inference_secret containing your LiteLLM master key.

Here is an example configuration:

kind: inference_model
version: v1
metadata:
  name: lite-llm-summary-model
spec:
  openai:
    # In case of LiteLLM Proxy, the openai_model_id is the public model name as
    # configured in LiteLLM.
    openai_model_id: my-custom-model
    api_key_secret_ref: lite-llm-master-key
    base_url: http://llm-host:4000/

---

kind: inference_secret
version: v1
metadata:
  name: lite-llm-master-key
spec:
  # In case of LiteLLM, the API key is *not* your OpenAI key; instead, use your
  # LiteLLM master key.
  value: '<paste-your-master-key-here>'

Monitoring​

Teleport exposes a couple of useful Prometheus metrics that can be used to monitor the session summarizer. Metrics are labeled by inference model name, and in case of OpenAI errors — additionally with the OpenAI API error code. See the Teleport Metrics reference for details.

Troubleshooting​

Amazon Bedrock fails to invoke a model with on-demand throughput​

Problem: Summarization fails with "ValidationException: Invocation of model ID [model-id] with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model."

Solution: Use an Amazon Bedrock inference profile instead of a model ID in the inference_model resource.

Amazon Bedrock model identifier is invalid​

Problem: Summarization fails with "ValidationException: The provided model identifier is invalid."

Solution: One possible reason is a mismatch between the region declared in the inference_model resource and available regions for the given model or inference profile. Make sure that the region is supported.

Amazon Bedrock denies access to a model or inference policy​

Problem: Summarization fails with "AccessDeniedException: User: [user] is not authorized to perform: bedrock:InvokeModel on resource: [resource-arn] because no identity-based policy allows the bedrock:InvokeModel action"

Solution: First, make sure that the AWS IAM role used by Teleport has a policy attached that allows the InvokeModel action on the model or inference policy that you specified in Teleport configuration (inference_model resource). Then make sure that if you're using an Amazon Bedrock inference profile, the policy references not only the inference profile itself, but also all Amazon Bedrock models that this inference profile is associated with. Use the following command to fetch details of the inference profile, replacing bedrock-inference-profile-arn with the inference profile ARN:

aws bedrock get-inference-profile --inference-profile-identifier="bedrock-inference-profile-arn"