# transforms.api

The Transforms Python API provides classes and decorators for constructing a [`Pipeline`](../../api-reference/transforms-python-library/api-pipeline/index.md#transforms.api.Pipeline). This page is a reference index. Select any name in the following tables to view the documentation for that function or class, including its parameters and examples.

#### Minimal example

A new Python transforms repository includes a `src/myproject/datasets/examples.py` file that imports from `transforms.api`. The following transform reads one dataset and writes one dataset. It runs as-is once you replace the two dataset paths:

```python
from transforms.api import Input, Output, transform, LightweightInput, LightweightOutput


@transform.using(
    output_dataset=Output("/path/to/output/dataset"),
    input_dataset=Input("/path/to/input/dataset"),
)
def compute(input_dataset: LightweightInput, output_dataset: LightweightOutput) -> None:
    output_dataset.write_table(input_dataset.polars(lazy=True))
```

For a PySpark transform that returns a DataFrame directly, use [`transform_df`](../../api-reference/transforms-python-library/api-transform-df/index.md#transforms.api.transform_df):

```python
from transforms.api import transform_df, Input, Output


@transform_df(
    Output("/path/to/output/dataset"),
    hair_eye_color=Input("/path/to/input/dataset"),
)
def filter_hair_color(hair_eye_color):
    return hair_eye_color.filter(hair_eye_color.hair == "Brown")
```

To choose between the Polars, pandas, DuckDB, and PySpark compute engines, see [Compute engines](../../transforms-python/compute-engines/index.md).

:::callout{theme="neutral" title="New to Python transforms?"}
For an end-to-end tutorial, see [Getting started](../../transforms-python/getting-started/index.md). For a walkthrough of the generated `examples.py` file, see [Project structure](../../transforms-python/project-structure/index.md#examplespy).
:::

#### Functions

| Name                                                                                                | Description                                                                                                                                        |
|-----------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------|
| [`configure`](../../api-reference/transforms-python-library/api-configure/index.md#transforms.api.configure)(\[profile, allowed\_run\_duration, ...])    | A decorator that modifies the configuration of a Spark transform.                                                                                  |
| [`incremental`](../../api-reference/transforms-python-library/api-incremental/index.md#transforms.api.incremental)(\[require\_incremental, ...])        | A decorator to convert inputs and outputs into their [`transforms.api.incremental`](../../api-reference/transforms-python-library/api-incremental/index.md#transforms.api.incremental) counterparts. |
| [`lightweight`](../../api-reference/transforms-python-library/api-lightweight/index.md#transforms.api.lightweight)(\[\_maybe\_function, cpu\_cores, ...]) | A decorator that turns a transform into a lightweight transform. |
| [`transform`](../../api-reference/transforms-python-library/api-transform/index.md#transforms.api.transform)(\*\*kwargs)                                 | Wrap a compute function as a [`Transform`](../../api-reference/transforms-python-library/api-transform/index.md#transforms.api.Transform) object.                                                                                                |
| [`transform_df`](../../api-reference/transforms-python-library/api-transform-df/index.md#transforms.api.transform_df)(output, \*\*inputs)             | Register the wrapped compute function as a DataFrame transform.                                                                                    |
| [`transform_pandas`](../../api-reference/transforms-python-library/api-transform-pandas/index.md#transforms.api.transform_pandas)(output, \*\*inputs) | Register the wrapped compute function as a pandas transform.                                                                                       |
| [`transform_polars`](../../api-reference/transforms-python-library/api-transform-polars/index.md#transforms.api.transform_polars)(output, \*\*inputs) | Register the wrapped compute function as a Polars transform.                                                                                       |

#### Classes

| Name                                                                                                                                           | Description                                                                                                                                                       |
|------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`BooleanParam`](../../api-reference/transforms-python-library/api-booleanparam/index.md#transforms.api.BooleanParam)(default, \*\[, description])                                                | Specification for the `ParameterSpec` definition used as an input to a transform.                                                                                 |
| [`Check`](../../api-reference/transforms-python-library/api-check/index.md#transforms.api.Check)(expectation, name\[, on\_error, description])                                                     | Wraps up an expectation such that it can be registered with Data Health.                                                                                          |
| [`ComputeBackend`](../../api-reference/transforms-python-library/api-computebackend/index.md#transforms.api.ComputeBackend)(\*values)                                                            | Enum class for representing the different compute backends for use in [`configure()`](../../api-reference/transforms-python-library/api-configure/index.md#transforms.api.configure).                               |
| [`ContainerTransform`](../../api-reference/transforms-python-library/api-containertransform/index.md#transforms.api.ContainerTransform)(transform, \*\[, ...])                                    | A callable object that describes a single step of a lightweight, single-node computation.                                                                         |
| [`ContainerTransformsConfiguration`](../../api-reference/transforms-python-library/api-containertransformsconfiguration/index.md#transforms.api.ContainerTransformsConfiguration)(transform, \*) | A callable object that describes a single step of a lightweight, single-node computation.                                                                         |
| [`Dataset`](../../api-reference/transforms-python-library/api-dataset/index.md#foundry.transforms.Dataset)(alias)                                                                                | A class representing the files backing a Foundry dataset view.                                                                                                    |
| [`FileStatus`](../../api-reference/transforms-python-library/api-filestatus/index.md#transforms.api.FileStatus)(path, size, modified)                                                            | A `collections.namedtuple` capturing details about a `FoundryFS` file in Spark transforms.                                                                        |
| [`FileSystem`](../../api-reference/transforms-python-library/api-filesystem/index.md#transforms.api.FileSystem)(foundry\_fs\[, read\_only])                                                         | A filesystem object for reading and writing raw dataset files in Spark transforms.                                                                                |
| [`FloatParam`](../../api-reference/transforms-python-library/api-floatparam/index.md#transforms.api.FloatParam)(default, \*\[, description])                                                      | Specification for the `ParameterSpec` definition used as an input to a transform.                                                                                 |
| [`FoundryDataSidecarFile`](../../api-reference/transforms-python-library/api-foundrydatasidecarfile/index.md#transforms.api.FoundryDataSidecarFile)(param, path, ...)                            | A file object for reading and writing raw dataset files in lightweight, single-node transforms.                                                                   |
| [`FoundryDataSidecarFileSystem`](../../api-reference/transforms-python-library/api-foundrydatasidecarfilesystem/index.md#transforms.api.FoundryDataSidecarFileSystem)(param\[, ...])              | A file system for reading and writing raw dataset files in lightweight, single-node transforms.                                                                   |
| [`FoundryInputParam`](../../api-reference/transforms-python-library/api-foundryinputparam/index.md#transforms.api.FoundryInputParam)(aliases\[, branch, type, ...])                               | A base class for transforms input parameters.                                                                                                                     |
| [`FoundryOutputParam`](../../api-reference/transforms-python-library/api-foundryoutputparam/index.md#transforms.api.FoundryOutputParam)(aliases\[, type, ...])                                    | A base class for transforms output parameters.                                                                                                                    |
| [`IncrementalLightweightInput`](../../api-reference/transforms-python-library/api-incrementallightweightinput/index.md#transforms.api.IncrementalLightweightInput)(alias, rid\[, branch])         | The input object passed into incremental [`ContainerTransform`](../../api-reference/transforms-python-library/api-containertransform/index.md#transforms.api.ContainerTransform) objects at runtime.                |
| [`IncrementalLightweightOutput`](../../api-reference/transforms-python-library/api-incrementallightweightoutput/index.md#transforms.api.IncrementalLightweightOutput)(alias, rid\[, ...])         | The output object passed into user code at runtime for incremental [`ContainerTransform`](../../api-reference/transforms-python-library/api-containertransform/index.md#transforms.api.ContainerTransform) objects. |
| [`IncrementalTableTransformInput`](../../api-reference/transforms-python-library/api-incrementaltabletransforminput/index.md#transforms.api.IncrementalTableTransformInput)(table\_tinput, ...)   | [`TableTransformInput`](../../api-reference/transforms-python-library/api-tabletransforminput/index.md#transforms.api.TableTransformInput) with added functionality for incremental computation.                    |
| [`IncrementalTransformContext`](../../api-reference/transforms-python-library/api-incrementaltransformcontext/index.md#transforms.api.IncrementalTransformContext)(is\_incremental, ...)          | [`TransformContext`](../../api-reference/transforms-python-library/api-transformcontext/index.md#transforms.api.TransformContext) with added functionality for incremental computation.                             |
| [`IncrementalTransformInput`](../../api-reference/transforms-python-library/api-incrementaltransforminput/index.md#transforms.api.IncrementalTransformInput)(tinput\[, ...])                      | [`TransformInput`](../../api-reference/transforms-python-library/api-transforminput/index.md#transforms.api.TransformInput) with added functionality for incremental computation.                                   |
| [`IncrementalTransformOutput`](../../api-reference/transforms-python-library/api-incrementaltransformoutput/index.md#transforms.api.IncrementalTransformOutput)(toutput\[, ...])                  | [`TransformOutput`](../../api-reference/transforms-python-library/api-transformoutput/index.md#transforms.api.TransformOutput) with added functionality for incremental computation.                                |
| [`Input`](../../api-reference/transforms-python-library/api-input/index.md#transforms.api.Input)(\[alias, branch, description, ...])                                                              | Specification for a transform dataset input.                                                                                                                      |
| [`InputSet`](../../api-reference/transforms-python-library/api-inputset/index.md#transforms.api.InputSet)(\[aliases, description])                                                                | Specification for a list of transform inputs.                                                                                                                     |
| [`IntegerParam`](../../api-reference/transforms-python-library/api-integerparam/index.md#transforms.api.IntegerParam)(default, \*\[, description])                                                | Specification for a `ParameterSpec` definition used as an input to a transform.                                                                                   |
| [`LightweightContext`](../../api-reference/transforms-python-library/api-lightweightcontext/index.md#transforms.api.LightweightContext)()                                                        | A context object that can optionally be injected into the compute function of a lightweight transform.                                                            |
| [`LightweightInput`](../../api-reference/transforms-python-library/api-lightweightinput/index.md#transforms.api.LightweightInput)(alias, rid\[, branch])                                          | The input object passed into [`ContainerTransform`](../../api-reference/transforms-python-library/api-containertransform/index.md#transforms.api.ContainerTransform) objects at runtime.                            |
| [`LightweightInputParam`](../../api-reference/transforms-python-library/api-lightweightinputparam/index.md#transforms.api.LightweightInputParam)()                                               | Base type for input parameters compatible with lightweight, single node transforms.                                                                               |
| [`LightweightOutput`](../../api-reference/transforms-python-library/api-lightweightoutput/index.md#transforms.api.LightweightOutput)(alias, rid\[, branch])                                       | The output object passed to user code at runtime.                                                                                                                 |
| [`LightweightOutputParam`](../../api-reference/transforms-python-library/api-lightweightoutputparam/index.md#transforms.api.LightweightOutputParam)()                                            | Base type for output parameters compatible with lightweight, single node transforms.                                                                              |
| [`Markings`](../../api-reference/transforms-python-library/api-markings/index.md#transforms.api.Markings)(marking\_ids, on\_branches)                                                              | Specification for a marking that stops propagating from input.                                                                                                    |
| [`OrgMarkings`](../../api-reference/transforms-python-library/api-orgmarkings/index.md#transforms.api.OrgMarkings)(marking\_ids, on\_branches)                                                     | Specification for a marking that is no longer required on the output.                                                                                             |
| [`Output`](../../api-reference/transforms-python-library/api-output/index.md#transforms.api.Output)(\[alias, sever\_permissions, ...])                                                             | Specification for a transform output.                                                                                                                             |
| [`OutputSet`](../../api-reference/transforms-python-library/api-outputset/index.md#transforms.api.OutputSet)(\[aliases, sever\_permissions, ...])                                                  | Specification for a list of transform outputs.                                                                                                                    |
| [`Param`](../../api-reference/transforms-python-library/api-param/index.md#transforms.api.Param)(\[description])                                                                                  | Base class for any parameter taken by the transform compute function.                                                                                             |
| [`ParamContext`](../../api-reference/transforms-python-library/api-paramcontext/index.md#transforms.api.ParamContext)(foundry\_connector, input\_specs, ...)                                       | A context object injected in the `instance` method of a parameter.                                                                                                |
| [`ParamValueInput`](../../api-reference/transforms-python-library/api-paramvalueinput/index.md#transforms.api.ParamValueInput)(value)                                                            | A wrapper around the value of a parameter spec.                                                                                                                   |
| [`Pipeline`](../../api-reference/transforms-python-library/api-pipeline/index.md#transforms.api.Pipeline)()                                                                                      | An object for grouping a collection of [`Transform`](../../api-reference/transforms-python-library/api-transform/index.md#transforms.api.Transform) objects.                                                        |
| [`StringParam`](../../api-reference/transforms-python-library/api-stringparam/index.md#transforms.api.StringParam)(default, \*\[, description, ...])                                              | Specification for the `ParameterSpec` definition used as an input to a transform.                                                                                 |
| [`TableTransformInput`](../../api-reference/transforms-python-library/api-tabletransforminput/index.md#transforms.api.TableTransformInput)(rid, branch, table\_dfreader)                          | The input object passed into transform objects at runtime for virtual table inputs.                                                                               |
| [`Transform`](../../api-reference/transforms-python-library/api-transform/index.md#transforms.api.Transform)(compute\_func\[, inputs, outputs, ...])                                               | A callable object that describes a single step of a Spark computation.                                                                                            |
| [`TransformContext`](../../api-reference/transforms-python-library/api-transformcontext/index.md#transforms.api.TransformContext)(foundry\_connector\[, ...])                                      | A context object that can optionally be injected into the compute function of a transform.                                                                        |
| [`TransformInput`](../../api-reference/transforms-python-library/api-transforminput/index.md#transforms.api.TransformInput)(rid, branch, txrange, ...\[, ...])                                    | The input object passed into [`Transform`](../../api-reference/transforms-python-library/api-transform/index.md#transforms.api.Transform) objects at runtime.                                                       |
| [`TransformOutput`](../../api-reference/transforms-python-library/api-transformoutput/index.md#transforms.api.TransformOutput)(rid, branch, txrid, ...\[, mode])                                  | The output object passed into [`Transform`](../../api-reference/transforms-python-library/api-transform/index.md#transforms.api.Transform) objects at runtime.                                                      |

#### Exceptions

| Name                                                                                                                               | Description                                                                                                                                     |
|------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------|
| [`LightweightException`](../../api-reference/transforms-python-library/api-lightweightexception/index.md#transforms.api.LightweightException)                                        | Base exception for lightweight compatibility checks.                                                                                            |
| [`LightweightNotImplementedError`](../../api-reference/transforms-python-library/api-lightweightnotimplementederror/index.md#transforms.api.LightweightNotImplementedError)(message) | Lightweight-specific [`NotImplementedError` ↗](https://docs.python.org/3/library/exceptions.html#NotImplementedError) for unsupported features. |
| [`LightweightTypeError`](../../api-reference/transforms-python-library/api-lightweighttypeerror/index.md#transforms.api.LightweightTypeError)(message)                               | Exception for type errors in lightweight compatibility checks.                                                                                  |
| [`LightweightValueError`](../../api-reference/transforms-python-library/api-lightweightvalueerror/index.md#transforms.api.LightweightValueError)(message)                            | Exception for value errors in lightweight compatibility checks.                                                                                 |