Skip to content

fix: fetch gitlab groups concurrently and improve logs#4267

Merged
mbani01 merged 4 commits into
mainfrom
chore/gitlab_oauth_error_logs
Jun 25, 2026
Merged

fix: fetch gitlab groups concurrently and improve logs#4267
mbani01 merged 4 commits into
mainfrom
chore/gitlab_oauth_error_logs

Conversation

@mbani01

@mbani01 mbani01 commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

This pull request refactors the way GitLab group projects are fetched to improve performance and error logging. The main updates include batching API calls for better concurrency and adding more detailed error logging for GitLab integration failures.

Performance improvements for fetching GitLab group projects

  • Refactored the logic to fetch projects for each group into a new helper function fetchProjectsForGroup, enabling better code organization.
  • Updated fetchGitlabGroupProjects to fetch projects in batches of 10 groups concurrently, significantly speeding up the process when handling many groups.
  • Changed the structure so that results are collected and mapped back to the correct group in the final result.

Enhanced error logging for GitLab integration

  • Added detailed error logging in the IntegrationService to capture GitLab-specific error information (status, error message, URL, and method) when a connection fails. This will help with debugging integration issues.

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
Copilot AI review requested due to automatic review settings June 25, 2026 14:31
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to improve observability of GitLab OAuth connection failures by adding structured error logging in IntegrationService.gitlabConnect.

Changes:

  • Added a structured log.error payload in the gitlabConnect catch block, including Axios/GitLab response context (status/data/url) and basic error fields (message/name/stack).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread backend/src/services/integrationService.ts
mbani01 added 2 commits June 25, 2026 15:55
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
Copilot AI review requested due to automatic review settings June 25, 2026 15:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Comment thread backend/src/serverless/integrations/usecases/gitlab/getProjects.ts
Comment thread backend/src/services/integrationService.ts
Comment thread backend/src/serverless/integrations/usecases/gitlab/getProjects.ts
Signed-off-by: Mouad BANI <mouad-mb@outlook.com>
@mbani01 mbani01 changed the title chore: add axios logs fix: fetch gitlab groups concurrently and improve logs Jun 25, 2026
@mbani01 mbani01 self-assigned this Jun 25, 2026
@mbani01 mbani01 requested a review from joanagmaia June 25, 2026 15:23
@mbani01 mbani01 marked this pull request as ready for review June 25, 2026 15:23
Copilot AI review requested due to automatic review settings June 25, 2026 15:24
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment on lines +2777 to +2788
this.options.log.error(
{
errMessage: err?.message,
errName: err?.name,
errStack: err?.stack,
gitlabStatus: err?.response?.status,
gitlabError: err?.response?.data,
gitlabUrl: err?.config?.url,
gitlabMethod: err?.config?.method,
},
'gitlabConnect failed',
)
Comment on lines +59 to +61
const results = await Promise.all(
batch.map((group) => fetchProjectsForGroup(accessToken, group)),
)
@mbani01 mbani01 merged commit 0210c52 into main Jun 25, 2026
23 checks passed
@mbani01 mbani01 deleted the chore/gitlab_oauth_error_logs branch June 25, 2026 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants