• Documentation
    • Search documentation
    karat

    +

    K

    API Reference ↗
    AI Platform (AIP)AIP LogicCompute usage

    Compute usage with AIP Logic

    AIP Logic is a Palantir tool that allows you to quickly and maintainably build LLM-driven processes while interacting with your organization's data through the Ontology and computational functionality. AIP Logic is built around the concept of "blocks" of LLM instructions that can be combined linearly to create chain-of-thought workflows that can query data, execute actions and functions, and generate net new information for your use case. AIP Logic measures usage in LLM tokens, LLM block executions, and tool calls. Blocks and tools can also run Ontology queries, actions, and functions, which use their own compute-seconds.

    If you have an enterprise contract with Palantir, contact your Palantir representative before proceeding with compute usage calculations.

    Core concepts: Resource, blocks, and tools

    An AIP Logic resource is comprised of one or more AIP Logic blocks. Running a resource will run the blocks required to achieve a desired output. Blocks can use tools such as Ontology queries, functions, and actions to produce an output.

    Measuring Foundry compute with AIP Logic

    AIP LLM tokens

    LLM tokens in AIP are measured in the manner of the underlying model (such as OpenAI ↗), and depend on the size of prompts and responses as well as on the number of prompts that are made. For more information, consult the table of usage for each model type.

    LLM block execution

    On top of LLM token usage, AIP Logic uses a fixed number of compute-seconds for each LLM block execution and each tool call. These charges are additive: an LLM block that calls a tool incurs both.

    UsageCompute-secondsWhen it applies
    LLM block execution4Each time a Use LLM block runs, with or without tool calls
    Tool call8Each time the LLM in a Use LLM block calls a tool, charged on top of the LLM block execution

    The LLM decides at runtime which tools to call and how often, so one block execution can include several tool calls, each charged separately. A Use LLM block inside a loop counts as one LLM block execution per iteration. To see how many tool calls a run makes, test the function and review the Debugger panel.

    Use the following formula to calculate these charges:

    Copied!
    1
block_and_tool_compute_seconds = (llm_block_executions * 4) + (tool_calls * 8)

    Additional Foundry compute usage

    When an AIP Logic block federates computation out to external tools (such as Ontology queries or functions), additional compute may be used during the execution of these applications.

    Managing AIP Logic usage of Foundry compute

    Some operations in AIP Logic can significantly affect compute usage. Below, we provide guidance on controlling compute usage by being careful about token usage, the total number of logic block executions, and usage of Foundry compute.

    Token usage

    • Processing large amounts of text can significantly increase overall compute usage. Be aware of the size of the input prompts that are being used with LLMs. This is especially relevant when pulling in large text blocks from the Ontology.
    • To moderate token usage, you should work to pare down the amount of text that is injected into prompts and ensure that only relevant text is included in the prompts themselves. This is especially important when working with large documents.

    Total number of logic block executions

    • Running many logic blocks can use large amounts of compute, especially if the blocks are triggered programmatically via a function (that is, not triggered by human action).
    • To moderate logic block executions, consider combining multiple blocks into single prompts (where appropriate), and only using additional blocks when necessary. Action, function, and data transformation blocks do not incur compute usage on their own.

    Foundry compute

    • Running many calls to Foundry applications (such as Ontology queries, functions, or actions) can use large amounts of compute. This may occur if a logic resource requires many retries, or calls many functions or actions on each execution.
    • To moderate compute usage from other applications, ensure that you understand the number of calls to other tools that are made in your chain of logic blocks. The potential tools that can call out to other parts of Foundry are:
      • Ontology Query
      • Action Execution
      • Function Execution
      • Data Transformation

    Example of AIP Logic compute usage

    Assume an AIP Logic resource has two Use LLM blocks. The first block does not call any tools. The second block has an action configured with the Apply actions tool, and the LLM calls it once per run. The resource runs end-to-end twice.

    Copied!
    1
2
3
4
5
6
7
8
LLM block executions per run: 2
Tool calls per run: 1
Number of runs: 2

1 run = (2 LLM block executions * 4 compute-seconds) + (1 tool call * 8 compute-seconds)
1 run = 8 + 8 = 16 compute-seconds

2 runs = 2 * 16 compute-seconds = 32 compute-seconds

    The two runs use 32 compute-seconds for LLM block executions and tool calls. This amount does not include LLM token usage or the compute used by the action itself. For action compute usage, see Ontology Query Compute.

    If the LLM called the action twice in each run instead of once, each run would use (2 * 4) + (2 * 8) = 24 compute-seconds, or 48 compute-seconds for both runs.