# GitHub

<!-- BEGIN GENERATED:intro -->

The GitHub connector is a [Palantir-provided driver](../../data-integration/foundry-provided-drivers/index.md) for GitHub.

To create a new GitHub source, follow the [standard setup flow for Palantir-provided drivers](../../data-integration/foundry-provided-drivers/index.md#setup), then use the sections below for GitHub-specific configuration and networking. For the complete property reference, see the [official GitHub driver documentation ↗](https://cdn.cdata.com/help/FGK/jdbc/pg_connectionj.htm).

<!-- END GENERATED:intro -->

<!-- BEGIN GENERATED:capabilities -->

## Supported capabilities

| Capability | Status |
|---|---|
| Exploration | 🟢 Generally available |
| Batch syncs | 🟢 Generally available |
| Incremental | 🟢 Generally available |
| [OAuth 2.0 authentication](#oauth-20-authentication) | 🟢 Generally available |
| [Table exports](#table-exports) | 🟢 Generally available |

<!-- END GENERATED:capabilities -->

## Introduction

The GitHub connector is best suited for **reading** data in bulk. It surfaces the GitHub data model — repositories, issues, pull requests, commits, and related entities — as relational tables. You can browse these in the source explorer and pull them into Foundry with a [batch sync](../../data-connection/set-up-sync/index.md). This makes it a good fit for mirroring an entire repository or organization into Foundry for downstream analysis.

![The GitHub source explorer previewing tables such as Commits before adding them to a sync.](../../../assets/739caa7f9312cc9060cff57f5f1465fe7811df5dab35e7c78ab209a503641bec.webp)

To create or update records in GitHub, reuse your existing GitHub source and call the [GitHub REST API ↗](https://docs.github.com/en/rest) from an [external transform](../../data-connection/external-transforms/index.md) or [external function](../../data-connection/external-functions/index.md). No second source is required. See [Use GitHub sources in code](#use-github-sources-in-code) below.

## Authentication

The connector authenticates with GitHub over OAuth 2.0. You register an application on GitHub, then enter its **Client ID** and **Client secret** on the Foundry source (`OAuthClientId` / `OAuthClientSecret`). With `InitiateOAuth` set to `REFRESH`, the driver runs GitHub's [OAuth web application flow ↗](https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/authorizing-oauth-apps#web-application-flow) on the first sync and keeps the token refreshed.

GitHub offers two application types, both built on OAuth 2.0 and configured the same way. A **GitHub App** (recommended) adds fine-grained, per-resource permissions — for example, read-only access to a single repository's issues — and short-lived tokens. An **OAuth app** uses broad, account-level scopes, such as `repo`. The steps below use a GitHub App; an OAuth app is identical apart from the provisioning permissions workflow, where you instead select **OAuth app** from **Settings > Developer settings**. For a full comparison, see [Differences between GitHub Apps and OAuth apps ↗](https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/differences-between-github-apps-and-oauth-apps).

### Create the application

In GitHub, select **Settings > Developer settings > GitHub Apps > [New GitHub App ↗](https://github.com/settings/apps/new)** and set the **Callback URL** to the value shown on the Foundry source setup page.

![The GitHub Create GitHub App form, showing the app name, homepage URL, and callback URL fields.](../../../assets/888a2c138eb3eccb07d802bd9e681047af31cfa74470e33c425bbd5e27f4f792.webp)

Under **Permissions & events**, grant only what the sync needs (for example, read-only **Contents** and **Issues**), create the app, and install it from the **Install App** tab on the target repositories or organization. Then, on the app's **General** page, copy the **Client ID** and generate a **Client secret**.

![The GitHub App General page, showing the Client ID and Client secrets sections.](../../../assets/d6e27e37be80e3791370b25df7e3d26b215531b751c2e248db5461275ff5b017.webp)

### Add your GitHub client ID and secret to the source

Navigate to Data Connection in Foundry and enter the **Client ID** and **Client secret** into the source's `OAuthClientId` and `OAuthClientSecret` properties and keep `InitiateOAuth` set to `REFRESH`.

![The Foundry GitHub source connection settings, showing the OAuth client ID, client secret, and callback URL properties alongside the approved egress policies.](../../../assets/c4bfab9d513478a4b22fc86d3f62b4207ea547ca3f33dfb21813dca741c16bd8.webp)

<!-- BEGIN GENERATED:configuration -->

## Configuration

The properties below are mandatory or recommended.

| Property | Required? | Description | Default |
|---|---|---|---|
| [`InitiateOAuth` ↗](https://cdn.cdata.com/help/FGK/jdbc/RSBGitHub_p_InitiateOAuth.htm) | Recommended | Specifies the process for obtaining or refreshing the OAuth access token, which maintains user access while an authenticated, authorized user is working. | `REFRESH` |
| [`OAuthClientId` ↗](https://cdn.cdata.com/help/FGK/jdbc/RSBGitHub_p_OAuthClientId.htm) | Recommended | Specifies the client ID (also known as the consumer key) assigned to your custom OAuth application. This ID is required to identify the application to the OAuth authorization server during authentication. | — |
| [`OAuthClientSecret` ↗](https://cdn.cdata.com/help/FGK/jdbc/RSBGitHub_p_OAuthClientSecret.htm) | Recommended | Specifies the client secret assigned to your custom OAuth application. This confidential value is used to authenticate the application to the OAuth authorization server. | — |

<!-- END GENERATED:configuration -->

<!-- BEGIN GENERATED:networking -->

## Networking

The table below lists the domains that the source must be able to access to run.

For each domain, add a corresponding [egress policy](../../administration/configure-egress/index.md). If the source is hosted on-premises and not directly reachable from Foundry, use an [agent proxy egress policy](../../administration/configure-egress/index.md#agent-proxy-egress-policies) instead. The agent host itself must also be able to reach the listed domains. See [using an agent as a proxy](../../data-connection/agent-proxy/index.md) for details.

| Domain  | Required |
|--- |--- |
| github.com | Always |
| api.github.com | Always |

<!-- END GENERATED:networking -->

<!-- BEGIN GENERATED:oauth -->

## OAuth 2.0 authentication

This connector supports OAuth 2.0 authentication. Follow the [OAuth 2.0 guidance for Palantir-provided drivers](../../data-integration/foundry-provided-drivers/index.md#oauth-20-authentication) to configure and authorize the connection.

<!-- END GENERATED:oauth -->

<!-- BEGIN GENERATED:table-exports -->

## Table exports

This connector supports table exports. Learn how to [set up a table export](../../data-connection/export-overview/index.md#table-exports).

<!-- END GENERATED:table-exports -->

## Use GitHub sources in code

To create or update records in GitHub, or to use any part of the [GitHub REST API ↗](https://docs.github.com/en/rest) beyond bulk reads, call the API from an [external transform](../../data-connection/external-transforms/index.md) or [external function](../../data-connection/external-functions/index.md). Reuse your existing GitHub source, authenticating with a short-lived (roughly one hour time-to-live) GitHub App installation token minted in code. No second source is required.

Next, import the source into your [external transforms](../../data-connection/external-transforms/index.md#prerequisite-import-a-source-into-code) or [functions](../../functions/getting-started/index.md) repository. The source does not expose a built-in HTTPS client (`get_https_connection().get_client()`), but it inherits its `api.github.com` egress, which is covered by the [network egress policies above](#networking). You can call the GitHub REST API directly with [`requests` ↗](https://requests.readthedocs.io/) in Python or the source's `fetch` in TypeScript.

### Read GitHub issues

The examples below read GitHub issues using Python or TypeScript through a GitHub App.

:::callout{theme="warning"}
This workflow is not supported by an OAuth app, which requires a personal access token.
:::

First, navigate to your source's **Connection details** page in Data Connection and add the GitHub App's private key by selecting **Add property > New encrypted property**. Name the property `Other`, enabling it to be read back with `get_secret("Other")`.

Next, set `APP_ID` in the Python examples to the GitHub App ID or client ID. This identifier is public and does not need secret storage. The TypeScript examples instead read it from the source's `OAuthClientId` property.

:::callout{theme="warning" title="Function preview failures"}
[Live preview](../../functions/getting-started/index.md#test-in-live-preview) cannot bind the [source](#configuration) to the function, so these examples *always* error during preview. Instead, deploy the function and run it from a Workshop application, or via any serverless execution, to confirm it works.
:::

```python tab="Python pipeline"
import time

import jwt
import pandas as pd
import requests
from transforms.api import LightweightOutput, Output, transform
from transforms.external.systems import ResolvedSource, Source, external_systems

API = "https://api.github.com"
APP_ID = "<github_app_id_or_client_id>"
OWNER = "<owner>"
REPOSITORY = "<repository>"


def github_headers(github: ResolvedSource) -> dict[str, str]:
    now = int(time.time())
    app_jwt = jwt.encode(
        {"iat": now - 60, "exp": now + 540, "iss": APP_ID},
        github.get_secret("Other"),
        algorithm="RS256",
    )
    headers = {
        "Accept": "application/vnd.github+json",
        "X-GitHub-Api-Version": "2022-11-28",
        "Authorization": f"Bearer {app_jwt}",
    }
    installation_id = requests.get(
        f"{API}/repos/{OWNER}/{REPOSITORY}/installation", headers=headers
    ).json()["id"]
    token = requests.post(
        f"{API}/app/installations/{installation_id}/access_tokens",
        headers=headers,
    ).json()["token"]
    return {**headers, "Authorization": f"Bearer {token}"}


@external_systems(github=Source("<source_rid>"))
@transform.using(output=Output("<issues_output_dataset_rid>"))
def get_issues(github: ResolvedSource, output: LightweightOutput) -> None:
    response = requests.get(
        f"{API}/repos/{OWNER}/{REPOSITORY}/issues",
        headers=github_headers(github),
        params={"state": "all", "per_page": 100},
    )
    response.raise_for_status()
    output.write_table(pd.DataFrame([{
        "number": issue["number"],
        "title": issue["title"],
        "body": issue["body"],
        "state": issue["state"],
    } for issue in response.json()]))
```

```python tab="Python function"
import time
from dataclasses import dataclass

import jwt
import requests
from functions.api import function
from functions.sources import get_source

API = "https://api.github.com"
APP_ID = "<github_app_id_or_client_id>"
SOURCE_ALIAS = "<source_alias>"


@dataclass
class GitHubIssue:
    number: int
    title: str
    body: str | None
    state: str


def github_headers(github, owner: str, repository: str) -> dict[str, str]:
    now = int(time.time())
    app_jwt = jwt.encode(
        {"iat": now - 60, "exp": now + 540, "iss": APP_ID},
        github.get_secret("Other"),
        algorithm="RS256",
    )
    headers = {
        "Accept": "application/vnd.github+json",
        "X-GitHub-Api-Version": "2022-11-28",
        "Authorization": f"Bearer {app_jwt}",
    }
    installation_id = requests.get(
        f"{API}/repos/{owner}/{repository}/installation", headers=headers
    ).json()["id"]
    token = requests.post(
        f"{API}/app/installations/{installation_id}/access_tokens",
        headers=headers,
    ).json()["token"]
    return {**headers, "Authorization": f"Bearer {token}"}


@function(sources=["<source_alias>"])
def get_issues(owner: str, repository: str) -> list[GitHubIssue]:
    response = requests.get(
        f"{API}/repos/{owner}/{repository}/issues",
        headers=github_headers(get_source(SOURCE_ALIAS), owner, repository),
        params={"state": "all", "per_page": 100},
    )
    response.raise_for_status()
    return [GitHubIssue(issue["number"], issue["title"], issue["body"], issue["state"])
            for issue in response.json()]
```

```typescript tab="TypeScript v1 function"
import { ExternalSystems, Function, Integer } from "@foundry/functions-api";
import { GitHubSource } from "@foundry/external-systems/sources";
import * as jwt from "jsonwebtoken";

const API = "https://api.github.com";

export interface GitHubIssue {
    number: Integer;
    title: string;
    body?: string;
    state: string;
}

async function githubHeaders(owner: string, repository: string): Promise<Record<string, string>> {
    const now = Math.floor(Date.now() / 1000);
    const appJwt = jwt.sign(
        { iat: now - 60, exp: now + 540, iss: GitHubSource.getSecret("OAuthClientId") },
        GitHubSource.getSecret("Other"),
        { algorithm: "RS256" },
    );
    const headers = {
        Accept: "application/vnd.github+json",
        "X-GitHub-Api-Version": "2022-11-28",
        Authorization: `Bearer ${appJwt}`,
    };
    const installation = await GitHubSource.fetch(
        `${API}/repos/${owner}/${repository}/installation`, { method: "GET", headers }
    );
    const { id } = await installation.json() as { id: number };
    const tokenResponse = await GitHubSource.fetch(
        `${API}/app/installations/${id}/access_tokens`, { method: "POST", headers }
    );
    const { token } = await tokenResponse.json() as { token: string };
    return { ...headers, Authorization: `Bearer ${token}` };
}

export class GitHubFunctions {
    @ExternalSystems({ sources: [GitHubSource] })
    @Function()
    public async getIssues(owner: string, repository: string): Promise<GitHubIssue[]> {
        const response = await GitHubSource.fetch(
            `${API}/repos/${owner}/${repository}/issues?state=all&per_page=100`,
            { method: "GET", headers: await githubHeaders(owner, repository) },
        );
        if (!response.ok) throw new Error(`GitHub returned ${response.status}`);
        const issues = await response.json() as Array<{
            number: number; title: string; body: string | null; state: string;
        }>;
        return issues.map(({ number, title, body, state }) => ({
            number, title, body: body ?? undefined, state,
        }));
    }
}
```

```typescript tab="TypeScript v2 function"
import { Integer } from "@osdk/functions";
import { getFetch, getSource } from "@palantir/functions-sources";
import jwt from "jsonwebtoken";

const API = "https://api.github.com";
const SOURCE_RID = "<source_rid>";
export const config = { sources: [SOURCE_RID] };

export interface GitHubIssue {
    number: Integer;
    title: string;
    body?: string;
    state: string;
}

async function githubClient(owner: string, repository: string) {
    const source = await getSource({ rid: SOURCE_RID });
    const request = await getFetch(source);
    const now = Math.floor(Date.now() / 1000);
    const appJwt = jwt.sign(
        { iat: now - 60, exp: now + 540, iss: source.secrets.OAuthClientId! },
        source.secrets.Other!,
        { algorithm: "RS256" },
    );
    const headers = {
        Accept: "application/vnd.github+json",
        "X-GitHub-Api-Version": "2022-11-28",
        Authorization: `Bearer ${appJwt}`,
    };
    const installation = await request(
        `${API}/repos/${owner}/${repository}/installation`, { method: "GET", headers }
    );
    const { id } = await installation.json() as { id: number };
    const tokenResponse = await request(
        `${API}/app/installations/${id}/access_tokens`, { method: "POST", headers }
    );
    const { token } = await tokenResponse.json() as { token: string };
    return { request, headers: { ...headers, Authorization: `Bearer ${token}` } };
}

export default async function getIssues(owner: string, repository: string): Promise<GitHubIssue[]> {
    const { request, headers } = await githubClient(owner, repository);
    const response = await request(
        `${API}/repos/${owner}/${repository}/issues?state=all&per_page=100`,
        { method: "GET", headers },
    );
    if (!response.ok) throw new Error(`GitHub returned ${response.status}`);
    const issues = await response.json() as Array<{
        number: number; title: string; body: string | null; state: string;
    }>;
    return issues.map(({ number, title, body, state }) => ({
        number, title, body: body ?? undefined, state,
    }));
}
```

### Update GitHub issues

The examples below update the body of a GitHub issue through the same GitHub App authentication flow as in the [examples for reading GitHub issues](#read-github-issues).

Each example immediately replaces the selected issue's entire body. Supply only an issue whose body you intend to overwrite.

```python tab="Python pipeline"
import time

import jwt
import pandas as pd
import requests
from transforms.api import LightweightOutput, Output, transform
from transforms.external.systems import ResolvedSource, Source, external_systems

API = "https://api.github.com"
APP_ID = "<github_app_id_or_client_id>"
OWNER = "<owner>"
REPOSITORY = "<repository>"
ISSUE_NUMBER = "<issue_number>"
UPDATED_BODY = "<updated_issue_body>"


def github_headers(github: ResolvedSource) -> dict[str, str]:
    now = int(time.time())
    app_jwt = jwt.encode(
        {"iat": now - 60, "exp": now + 540, "iss": APP_ID},
        github.get_secret("Other"),
        algorithm="RS256",
    )
    headers = {
        "Accept": "application/vnd.github+json",
        "X-GitHub-Api-Version": "2022-11-28",
        "Authorization": f"Bearer {app_jwt}",
    }
    installation_id = requests.get(
        f"{API}/repos/{OWNER}/{REPOSITORY}/installation", headers=headers
    ).json()["id"]
    token = requests.post(
        f"{API}/app/installations/{installation_id}/access_tokens",
        headers=headers,
    ).json()["token"]
    return {**headers, "Authorization": f"Bearer {token}"}


@external_systems(github=Source("<source_rid>"))
@transform.using(output=Output("<update_output_dataset_rid>"))
def patch_issue(github: ResolvedSource, output: LightweightOutput) -> None:
    response = requests.patch(
        f"{API}/repos/{OWNER}/{REPOSITORY}/issues/{ISSUE_NUMBER}",
        headers=github_headers(github),
        json={"body": UPDATED_BODY},
    )
    response.raise_for_status()
    output.write_table(pd.DataFrame([{
        "issue_number": ISSUE_NUMBER,
        "body": UPDATED_BODY,
    }]))
```

```python tab="Python function"
import time

import jwt
import requests
from functions.api import function
from functions.sources import get_source

API = "https://api.github.com"
APP_ID = "<github_app_id_or_client_id>"
SOURCE_ALIAS = "<source_alias>"


def github_headers(github, owner: str, repository: str) -> dict[str, str]:
    now = int(time.time())
    app_jwt = jwt.encode(
        {"iat": now - 60, "exp": now + 540, "iss": APP_ID},
        github.get_secret("Other"),
        algorithm="RS256",
    )
    headers = {
        "Accept": "application/vnd.github+json",
        "X-GitHub-Api-Version": "2022-11-28",
        "Authorization": f"Bearer {app_jwt}",
    }
    installation_id = requests.get(
        f"{API}/repos/{owner}/{repository}/installation", headers=headers
    ).json()["id"]
    token = requests.post(
        f"{API}/app/installations/{installation_id}/access_tokens",
        headers=headers,
    ).json()["token"]
    return {**headers, "Authorization": f"Bearer {token}"}


@function(sources=["<source_alias>"])
def patch_issue(
    owner: str, repository: str, issue_number: int, body: str
) -> str:
    response = requests.patch(
        f"{API}/repos/{owner}/{repository}/issues/{issue_number}",
        headers=github_headers(get_source(SOURCE_ALIAS), owner, repository),
        json={"body": body},
    )
    response.raise_for_status()
    return body
```

```typescript tab="TypeScript v1 function"
import { ExternalSystems, Function, Integer } from "@foundry/functions-api";
import { GitHubSource } from "@foundry/external-systems/sources";
import * as jwt from "jsonwebtoken";

const API = "https://api.github.com";

async function githubHeaders(owner: string, repository: string): Promise<Record<string, string>> {
    const now = Math.floor(Date.now() / 1000);
    const appJwt = jwt.sign(
        { iat: now - 60, exp: now + 540, iss: GitHubSource.getSecret("OAuthClientId") },
        GitHubSource.getSecret("Other"),
        { algorithm: "RS256" },
    );
    const headers = {
        Accept: "application/vnd.github+json",
        "X-GitHub-Api-Version": "2022-11-28",
        Authorization: `Bearer ${appJwt}`,
    };
    const installation = await GitHubSource.fetch(
        `${API}/repos/${owner}/${repository}/installation`, { method: "GET", headers }
    );
    const { id } = await installation.json() as { id: number };
    const tokenResponse = await GitHubSource.fetch(
        `${API}/app/installations/${id}/access_tokens`, { method: "POST", headers }
    );
    const { token } = await tokenResponse.json() as { token: string };
    return { ...headers, Authorization: `Bearer ${token}` };
}

export class GitHubFunctions {
    @ExternalSystems({ sources: [GitHubSource] })
    @Function()
    public async patchIssue(
        owner: string, repository: string, issueNumber: Integer, body: string
    ): Promise<string> {
        const response = await GitHubSource.fetch(
            `${API}/repos/${owner}/${repository}/issues/${issueNumber}`,
            {
                method: "PATCH",
                headers: { ...await githubHeaders(owner, repository), "Content-Type": "application/json" },
                body: JSON.stringify({ body }),
            },
        );
        if (!response.ok) throw new Error(`GitHub returned ${response.status}`);
        return body;
    }
}
```

```typescript tab="TypeScript v2 function"
import { Integer } from "@osdk/functions";
import { getFetch, getSource } from "@palantir/functions-sources";
import jwt from "jsonwebtoken";

const API = "https://api.github.com";
const SOURCE_RID = "<source_rid>";
export const config = { sources: [SOURCE_RID] };

async function githubClient(owner: string, repository: string) {
    const source = await getSource({ rid: SOURCE_RID });
    const request = await getFetch(source);
    const now = Math.floor(Date.now() / 1000);
    const appJwt = jwt.sign(
        { iat: now - 60, exp: now + 540, iss: source.secrets.OAuthClientId! },
        source.secrets.Other!,
        { algorithm: "RS256" },
    );
    const headers = {
        Accept: "application/vnd.github+json",
        "X-GitHub-Api-Version": "2022-11-28",
        Authorization: `Bearer ${appJwt}`,
    };
    const installation = await request(
        `${API}/repos/${owner}/${repository}/installation`, { method: "GET", headers }
    );
    const { id } = await installation.json() as { id: number };
    const tokenResponse = await request(
        `${API}/app/installations/${id}/access_tokens`, { method: "POST", headers }
    );
    const { token } = await tokenResponse.json() as { token: string };
    return { request, headers: { ...headers, Authorization: `Bearer ${token}` } };
}

export default async function patchIssue(
    owner: string, repository: string, issueNumber: Integer, body: string
): Promise<string> {
    const { request, headers } = await githubClient(owner, repository);
    const response = await request(
        `${API}/repos/${owner}/${repository}/issues/${issueNumber}`,
        {
            method: "PATCH",
            headers: { ...headers, "Content-Type": "application/json" },
            body: JSON.stringify({ body }),
        },
    );
    if (!response.ok) throw new Error(`GitHub returned ${response.status}`);
    return body;
}
```