Collect GitHub repository data into your warehouse

The connector has three endpoints, each landing in its own table.

1. Repository Information has 42 fields and lands in github_repository. It covers the repository’s name, owner, description, language, license, creation, update and push dates, size, stars, watchers, forks and open issues, and flags such as private, fork, archived and template.
2. Repository Statistics has 3 fields and lands in github_repository_stats. It holds a date, the number of stars and the number of contributors.
3. Repository Contributors has 9 fields and lands in github_contributors. It lists each contributor’s login, type (user or bot) and number of contributions.

Between them the three tables show how large a repository is, how fast it is growing and who contributes. The contributor type field marks users and bots, so you can leave bots out of a headcount.

Authentication is an access token. Generate a personal access token on GitHub. You need one to read private repositories. To start:

1. Add a storage to receive the data: Google BigQuery, Snowflake, AWS Redshift, AWS Athena or Databricks.
2. Create a Connector Data Mart and choose GitHub as the platform.
3. Add your access token for private repositories, publish the Data Mart and run it for the first import.
4. Add a Connector Trigger to schedule recurring imports.

The connector documentation has the full setup guide.

Once the data is in your warehouse, it behaves like any other table you own. Analysts define the Data Marts, and business users read the results in the tools they already open.

• Chart stars and contributors over time from the Repository Statistics table.
• Rank contributors by contributions from the Repository Contributors table.
• Track forks, watchers and open issues over refreshes from the Repository Information table.
• Filter out archived repositories and forks with the archived and fork flags.
• Join repository data to your commercial data in a Data Mart, so delivery sits next to revenue.
• Keep the SQL behind each number one click away.

A Report delivers one Data Mart to one destination, and each Data Mart can feed many destinations.

• Push mode: OWOX writes the data on a manual or scheduled run. Google Sheets, email, Slack, Microsoft Teams and Google Chat work this way.
• Pull mode: the tool asks OWOX for the data when a user refreshes it. Data Studio and Microsoft Excel work this way.
• AI assistants such as Claude and ChatGPT read the Data Mart, not the raw tables, so answers use your definitions.