GitHub · Creating a GitHub App for On-Premises Deployment

info

This guide walks a GitHub admin through creating a GitHub App that the Milestone data collection agent uses to read your repository, pull request, and Copilot data. You only do this once per GitHub deployment.

At a Glance

Complete these steps in order — estimated time: ~10 minutes for the app, plus ~2 minutes per organization you install on.

1. Create a new GitHub App in your org (or Enterprise) settings.
2. Grant it the 13 read-only permissions listed in the Permissions section.
3. Disable webhooks — Milestone polls; it does not receive callbacks.
4. Generate a private key and copy the App ID.
5. Install the app on each organization you want collected, and record the installation ID.
6. Send the three values (App ID, private key, installation IDs) to Milestone via your secure channel.

1. Prerequisites

• You are an Owner of the GitHub organization (Cloud) or a Site Administrator (Enterprise Server).
• For GitHub Enterprise Server: you know your instance URL, e.g. https://github.acme.internal.
• You have a secure way to deliver the private key (.pem file) to Milestone — encrypted file share, password manager, or a Milestone-provided drop link. Do not email it.

2. Open the "New GitHub App" Form

The path depends on whether you're on GitHub Cloud or GitHub Enterprise Server.

GitHub Cloud (github.com)

1. Sign in to GitHub as an org Owner.
2. Navigate to your org page → Settings → Developer settings → GitHub Apps.
3. Click New GitHub App.

https://github.com/organizations/<YOUR-ORG>/settings/apps

GitHub Enterprise Server (self-hosted)

1. Sign in to your GHES instance as a Site Administrator (or org Owner).
2. Navigate to the URL below (replace <YOUR-GHES-HOSTNAME> and <YOUR-ORG>).
3. Click New GitHub App.

https://<YOUR-GHES-HOSTNAME>/organizations/<YOUR-ORG>/settings/apps

tip

Recommended: create the app under an organization, not a personal user account. Apps owned by a user are deleted when that user leaves.

3. Fill in the Basic Details

Fill the form as follows. Anything not mentioned can be left at its default.

Field	Value	Notes
GitHub App name	Milestone Data Collector	Must be globally unique on Cloud; locally unique on GHES.
Homepage URL	https://mstone.ai	Required by GitHub; Milestone does not use it.
Callback URL	(leave blank)	Milestone does not use the OAuth user-authorization flow.
Setup URL	(leave blank)	Not used.
Request user authorization (OAuth)	Unchecked	Milestone authenticates as the app, not as users.
Webhook — Active	Unchecked — disable entirely	Milestone is pull-based; no webhook URL is needed.
Where can this app be installed?	Only on this account	Prevents installation in unrelated organizations.

4. Permissions

Milestone needs 13 read-only permissions across two categories. Set each of the following to Read-only; leave every other permission at No access.

Repository Permissions (9)

Permission	Access	Why Milestone needs it
Actions	Read-only	Workflow run history — CI throughput and build-quality signals.
Administration	Read-only	Repo settings (default branch, visibility, archive state) — identifies trunk and skips archived repos.
Commit statuses	Read-only	CI status on commits — feeds change-failure-rate.
Contents	Read-only	Read code and git history (commits, branches, blame). Core of collection.
Deployments	Read-only	Deployment events — feeds deployment-frequency.
Discussions	Read-only	Engineering discussions — used for collaboration signals where enabled.
Issues	Read-only	Issue history — cycle-time and triage analysis.
Metadata	Read-only	Mandatory — repo names, descriptions, topics. Granted automatically by GitHub.
Pull requests	Read-only	PRs, reviews, comments. The other core dataset Milestone collects.

Organization Permissions (4)

Permission	Access	Why Milestone needs it
Administration	Read-only	Org-level config (default repo permissions, visibility).
Members	Read-only	Org members and team membership — links commits to active employees.
GitHub Copilot Business	Read-only	Copilot seat assignments. Required for Copilot adoption metrics. Skip if you don't use Copilot.
Organization Copilot Metrics	Read-only	Org-level Copilot usage metrics. Skip if you don't use Copilot.

note

Account permissions: leave all at No access. Subscribe to events: leave everything unchecked.

5. Create the App and Copy the App ID

1. Click Create GitHub App at the bottom of the form.
2. You'll land on the app's General page.
3. Copy the App ID — shown near the top as App ID: 1234567. You will send this to Milestone.

6. Generate a Private Key

Milestone authenticates by signing a JWT with the app's private key.

1. On the app's General page, scroll to Private keys.
2. Click Generate a private key.
3. Your browser will immediately download a .pem file (e.g. milestone-data-collector.2026-05-17.private-key.pem). This is the only time the key is shown — save it now.
4. Treat the .pem like a password — store it in a secret manager; never in git, never in chat.

tip

Key rotation: you can generate additional keys at any time and revoke old ones from the same page. If you rotate, send the new key to Milestone before deleting the old one to avoid downtime.

7. Install the App on Your Organization(s)

The app exists, but it has no access until you install it on an organization. Repeat for every organization whose data you want Milestone to collect:

1. From the app's General page, click Install App in the left sidebar.
2. Find your organization and click Install.
3. Choose All repositories, or Only select repositories if you want to limit scope.
4. Click Install.
5. After install, you'll be redirected to a URL containing the installation ID (the trailing number in the URL). Copy it — you'll send one installation ID per installed org to Milestone.

Example URL after install:

https://github.com/organizations/<your-org>/settings/installations/87654321
                                                                    ^^^^^^^^
                                                                 installation ID

tip

Multiple orgs? Repeat steps 1–5 for each org. Send Milestone a mapping of <org name> → <installation ID>.

8. Send Credentials to Milestone

Send the following over your agreed secure channel (encrypted share, password manager, SFTP — not email, not Slack DMs):

Item	Example	Source
App ID	1234567	App General page (§ 5)
Private key	milestone-data-collector.2026-05-17.private-key.pem	Downloaded in § 6
Installation ID(s)	acme-eng → 87654321, acme-data → 87654322	URL after install (§ 7)
API base URL (GHES only)	https://github.acme.internal/api/v3	Your GHES instance hostname

Milestone will load these into the data collection agent's secret store and validate connectivity via the agent's preflight check.

9. Verify (Optional but Recommended)

If you have the Milestone DCA already deployed, you can verify the credentials yourself before handing them off:

kubectl exec -n milestone-dca deploy/milestone-dca-data-collection-agent-coordinator -- \
  agent preflight -c /config/config.yaml

A green preflight confirms the JWT signs, the installation token mints, and the GitHub API responds. If you do not have the DCA deployed yet, Milestone will run the preflight on your behalf after loading the credentials.

---

Appendix A — Permissions Quick Reference

All values are Read-only unless noted.

Scope	Permission	Access
Repository	Actions	Read-only
Repository	Administration	Read-only
Repository	Commit statuses	Read-only
Repository	Contents	Read-only
Repository	Deployments	Read-only
Repository	Discussions	Read-only
Repository	Issues	Read-only
Repository	Metadata	Read-only (mandatory, auto-granted)
Repository	Pull requests	Read-only
Organization	Administration	Read-only
Organization	Members	Read-only
Organization	GitHub Copilot Business	Read-only (Copilot customers only)
Organization	Organization Copilot Metrics	Read-only (Copilot customers only)
Account	(all)	No access
Webhook	(all)	Disabled
Events	(all)	None subscribed

Appendix B — Updating Permissions Later

If Milestone adds a new collector that requires an additional read scope, your admin will need to:

1. Open the app's Permissions & events page.
2. Change the affected scope to Read-only.
3. Click Save changes.
4. Re-approve the installation — existing installs do not automatically pick up new scopes. Go to Installed GitHub Apps in the org settings, find the app, and click Review request.

note

Milestone will give you advance notice before any scope additions.

Appendix C — Audit and Revocation

Action	How
Audit API calls	Every API call Milestone makes is attributable to the GitHub App, visible in your org's Audit log (filter actor:<app-name>).
Revoke instantly	Org settings → GitHub Apps → Configure → Uninstall. Tokens are invalidated within minutes.
Revoke a key only	The private key can be revoked from the app's General page without uninstalling the app.