Collect GitHub data in AWS Athena
Load repository details, contributors and daily star counts from GitHub into your own AWS Athena database with OWOX Data Marts, and query them with SQL beside the product and revenue data you already analyse in AWS.
GitHubConnectedRepository Contr…Repository Infor…Repository Stati…
AWS AthenaLoginIdNodeDaily 09:00Stars go up after a release, but nobody can show whether sign-ups and revenue went up with them.
Paste this into your AI assistant. It loads GitHub repository details, contributors and daily star counts into AWS Athena with OWOX Data Marts and lines them up with your sign-ups and revenue tables in SQL.
The prompt
Read https://www.owox.com/ai-agent-setup.md and follow it to set up OWOX Data Marts for me, starting from /connectors/github-connector-to-aws-athena.md. The task: load GitHub repository stats into AWS Athena and line up daily stars and contributors with our sign-ups and revenue for the two weeks before and after each release date. Tag the links you give me with utm_campaign=ai-task and utm_term=data-foundation. Use what you know about my context and goals
GitHub Endpoints
3 GitHub endpoints you can import. Each lands in its own table in your data warehouse.
Why GitHub data belongs in your AWS Athena database
GitHub shows a repository's stars and contributors one page at a time. Stored in your warehouse on AWS Athena, the same numbers are rows you can filter by date and join to sign-ups and revenue in a single SQL query.
- Filter stars and contributors by date and join them to sign-ups
- Keep one row of stars and contributors per day, from the first run on
- Report engineering activity next to revenue without exporting a CSV
GitHubRepository activityYour data warehouseRepositorySign-upsRevenueJoined by idYour release dateSign-ups and revenue afterOne Athena query lines up the release date with what came next.Step 1: Connect GitHub and pick the fields
Create a Data Mart, select GitHub as the source, type the repository as owner/repo, paste an access token and tick the fields you need.
- Three endpoints to choose from: Repository Information, Repository Contributors and Repository Statistics
- Authentication is a GitHub access token
Google AdsGitHub
Facebook AdsAuthorizeConnectedEndpointRepository Inform…Repository Contri…Repository Statis…Repository Inform…full_namestargazers_countforks_countopen_issues_countlanguagepushed_atStep 2: Load GitHub into an Athena table
Pick the database in your storage; the names are pre-filled and you can change them. Run it and the first load lands in your own AWS account.
- The tables are github_repository, github_contributors and github_repository_stats, in the database you pick
- The statistics table holds one row per run date, so stars and contributors filter by date like any other table
DestinationDatabasegithubTablerepositoryRunTablegithub.repositoryStep 3: Get a readable output schema
The first run produces the output schema: every field with a plain description, readable by your quality checks as well.
- archived and fork are true/false flags, so you can filter them out
- created_at, updated_at and pushed_at are dates
github.repositoryOutput schemaFieldDescriptionfull_nameFull repository name (owner/repo)stargazers_countNumber of starsforks_countNumber of forksopen_issues_countNumber of open issueslanguagePrimary programming languagepushed_atRepository last push dateStep 4: Join GitHub tables to sign-ups and revenue in Athena SQL
Athena reads the GitHub tables like any other. Join them to your own: repository name to your release records, date to sign-ups, and describe each join once in your model.
- full_name is a plain column any Athena query can join on
- The statistics date lines up with the sign-up date, day by day
Repository Inform…full_namestargazers_countlanguageRepository Statis…datestarsStep 5: Schedule the refresh and begin the history
Set it once. A scheduled refresh adds today's row to the statistics table and updates the repository and contributor rows in your Athena database.
- Run it every day; a day without a run stays empty
- GitHub returns today's counts only, so there is nothing to backfill: your Athena history starts with the first run
Step 6: Reuse the same Athena tables everywhere
The same Athena tables serve your model, , , assistants like Claude and ChatGPT, and scheduled AI insights in Slack or email.
- All reports read one set of tables, so the numbers agree
- Ask an AI assistant how many stars the repository gained last month, from your Athena tables
GitHubData MartPublished
Google Sheets
Data Studio dashboards
Claude, ChatGPT
Slack, email
- 0.41.0Rotating a warehouse key is one change: Storages share reusable Credentials
- 0.41.0Shared SQL logic stays in one place: build a Data Mart on top of another
- 0.41.0Rotating a connector key is one change: Sources share reusable Credentials
- 0.39.0Reload a year of history in one backfill, not twelve
- 0.38.0Athena query results are cleaned up when the Output Bucket names a folder
Why OWOX for GitHub data in Athena
A script that calls the GitHub API, or a weekly export to a spreadsheet, gets you numbers. OWOX keeps them in your Athena database, next to the data you already query there.
- You own the data. The data lands in your own AWS account, in a database you control and can query from anything that reads Athena.
- History you keep. GitHub's repository page shows the current count. In Athena the statistics table adds a row for every day you run it, so the trend stays.
- Ready for a data model. full_name and date are plain columns any Athena query can join on.
- Maintained for you. No script to fix when the GitHub API changes.
You are booked. The invitation is on its way to , with the calendar invite and the meeting link.