Unfriendly Grinch

Gitlab: Rebase Without Pipeline

Posted on Nov 16, 2022 | By Elif Samedin | 5 minutes read

Problem

I’ve recently been in a quite interesting situation: developers have been complaining (not a new one, right?) about the fact that the development process was being slowed down by the frequent need to perform a rebase in a feature branch which in turn triggered a pipeline.

This usually happens when developers merge (quite frequently) smaller updates to a main branch. This is how trunk-based development looks like. Despite the fact that this is aimed to streamline the merging and integration phases, when the frequency of these updates becomes too high, we might end up in a situation when it actually causes more attrition.

Solution

GitLab 15.3 (which was released in August 2022) introduced the option to skip the pipeline when rebasing from the GitLab UI.

This seems a viable approach for the above problem. However, I found the docs not clear enough on how this button is displayed.

I started digging a bit more and understood that the Rebase without pipeline button would be enabled when:

  • Merge commit with semi-linear history or Fast-forward merge merge methods are enabled.
  • Pipelines must succeed and Skipped pipelines are considered successful are enabled.

Ticking only one of these or neither will not work. I have tried it out.

  • Fast-forward merge is not possible, but a conflict-free rebase is possible.

So far, so good. These were actually the easy bits.

What happens when there are multiple groups in GitLab, each of these having numerous (sub)projects? Updating the Merge Request method & checks is a project level setting. Considering this, there would be a few options to advance with this:

  • Supposing developers have enough privileges, they could update the settings themselves, or in case this is out of question,
  • Manually go over all the projects (oh, no!), or
  • Automate the process using the GitLab API (a sparkle of hope).

Well, as you already imagine, I did resort to some sort of automation…

Exploring the GitLab API

In order to be able to interact with the GitLab API, we firstly need to be able to authenticate. There are several methods to do so. However, I usually prefer using a Personal Access Token. Thus, let’s start by generating one such token.

Getting a Personal Access Token

Under your user profile, you should select Access Tokens. Pick a name for the new token (that makes most sense to you) and select the api scope.

GitLab Personal Access Token

Using the GitLab API

Let’s start by retrieving the existing groups:

]$ curl -s -H "PRIVATE-TOKEN: ${GITLAB_API_TOKEN}" -H "Content-Type: application/json" -X GET "https://scm.ntf.ro/api/v4/groups" | jq
[
  {
    "id": 28,
    "web_url": "https://scm.ntf.ro/groups/elf",
    "name": "ELF",
    "path": "elf",
    "description": "For Testing Purposes",
    "visibility": "private",
    "share_with_group_lock": false,
    "require_two_factor_authentication": false,
    "two_factor_grace_period": 48,
    "project_creation_level": "developer",
    "auto_devops_enabled": null,
    "subgroup_creation_level": "maintainer",
    "emails_disabled": null,
    "mentions_disabled": null,
    "lfs_enabled": true,
    "default_branch_protection": 2,
    "avatar_url": "https://scm.ntf.ro/uploads/-/system/group/avatar/28/grinch.jpeg",
    "request_access_enabled": true,
    "full_name": "ELF",
    "full_path": "elf",
    "created_at": "2022-10-28T11:26:19.796Z",
    "parent_id": null
  }
]

Each group can have none, one or more subgroups:

$ curl -s -H "PRIVATE-TOKEN: ${GITLAB_API_TOKEN}" -H "Content-Type: application/json" -X GET "https://scm.ntf.ro/api/v4/groups/28/subgroups" | jq
[
  {
    "id": 30,
    "web_url": "https://scm.ntf.ro/groups/elf/backend",
    "name": "Backend",
    "path": "backend",
    "description": "",
    "visibility": "private",
    "share_with_group_lock": false,
    "require_two_factor_authentication": false,
    "two_factor_grace_period": 48,
    "project_creation_level": "developer",
    "auto_devops_enabled": null,
    "subgroup_creation_level": "maintainer",
    "emails_disabled": null,
    "mentions_disabled": null,
    "lfs_enabled": true,
    "default_branch_protection": 2,
    "avatar_url": null,
    "request_access_enabled": true,
    "full_name": "ELF / Backend",
    "full_path": "elf/backend",
    "created_at": "2022-10-28T11:32:50.152Z",
    "parent_id": 28
  },
  {
    "id": 29,
    "web_url": "https://scm.ntf.ro/groups/elf/frontend",
    "name": "Frontend",
    "path": "frontend",
    "description": "",
    "visibility": "private",
    "share_with_group_lock": false,
    "require_two_factor_authentication": false,
    "two_factor_grace_period": 48,
    "project_creation_level": "developer",
    "auto_devops_enabled": null,
    "subgroup_creation_level": "maintainer",
    "emails_disabled": null,
    "mentions_disabled": null,
    "lfs_enabled": true,
    "default_branch_protection": 2,
    "avatar_url": null,
    "request_access_enabled": true,
    "full_name": "ELF / Frontend",
    "full_path": "elf/frontend",
    "created_at": "2022-10-28T11:30:06.340Z",
    "parent_id": 28
  }
]

In this case, the group ELF has two subgroups: Backend & Frontend, each having some projects.

$ curl -s -H "PRIVATE-TOKEN: ${GITLAB_API_TOKEN}" -H "Content-Type: application/json" -X GET "https://scm.ntf.ro/api/v4/groups/30/projects" | jq
[
  {
    "id": 24,
    "description": null,
    "name": "Project-C",
    "name_with_namespace": "ELF / Backend / Project-C",
    "path": "project-c",
    "path_with_namespace": "elf/backend/project-c",
    "created_at": "2022-10-28T12:16:00.069Z",
    "default_branch": "master",
    "tag_list": [],
    "topics": [],
    "ssh_url_to_repo": "git@scm.ntf.ro:elf/backend/project-c.git",
    "http_url_to_repo": "https://scm.ntf.ro/elf/backend/project-c.git",
    "web_url": "https://scm.ntf.ro/elf/backend/project-c",
    "readme_url": "https://scm.ntf.ro/elf/backend/project-c/-/blob/master/README.md",
    "avatar_url": null,
    "forks_count": 0,
    "star_count": 0,
    "last_activity_at": "2022-10-28T12:16:00.069Z",
    "namespace": {
      "id": 30,
      "name": "Backend",
      "path": "backend",
      "kind": "group",
      "full_path": "elf/backend",
      "parent_id": 28,
      "avatar_url": null,
      "web_url": "https://scm.ntf.ro/groups/elf/backend"
    },
    "container_registry_image_prefix": "scm.ntf.ro:5050/elf/backend/project-c",
    "_links": {
      "self": "https://scm.ntf.ro/api/v4/projects/24",
      "issues": "https://scm.ntf.ro/api/v4/projects/24/issues",
      "merge_requests": "https://scm.ntf.ro/api/v4/projects/24/merge_requests",
      "repo_branches": "https://scm.ntf.ro/api/v4/projects/24/repository/branches",
      "labels": "https://scm.ntf.ro/api/v4/projects/24/labels",
      "events": "https://scm.ntf.ro/api/v4/projects/24/events",
      "members": "https://scm.ntf.ro/api/v4/projects/24/members",
      "cluster_agents": "https://scm.ntf.ro/api/v4/projects/24/cluster_agents"
    },
    "packages_enabled": true,
    "empty_repo": false,
    "archived": false,
    "visibility": "private",
    "resolve_outdated_diff_discussions": false,
    "container_expiration_policy": {
      "cadence": "1d",
      "enabled": false,
      "keep_n": 10,
      "older_than": "90d",
      "name_regex": ".*",
      "name_regex_keep": null,
      "next_run_at": "2022-10-29T12:16:00.084Z"
    },
    "issues_enabled": true,
    "merge_requests_enabled": true,
    "wiki_enabled": true,
    "jobs_enabled": true,
    "snippets_enabled": true,
    "container_registry_enabled": true,
    "service_desk_enabled": false,
    "service_desk_address": null,
    "can_create_merge_request_in": true,
    "issues_access_level": "enabled",
    "repository_access_level": "enabled",
    "merge_requests_access_level": "enabled",
    "forking_access_level": "enabled",
    "wiki_access_level": "enabled",
    "builds_access_level": "enabled",
    "snippets_access_level": "enabled",
    "pages_access_level": "private",
    "operations_access_level": "enabled",
    "analytics_access_level": "enabled",
    "container_registry_access_level": "enabled",
    "security_and_compliance_access_level": "private",
    "emails_disabled": null,
    "shared_runners_enabled": false,
    "lfs_enabled": true,
    "creator_id": 7,
    "import_url": null,
    "import_type": null,
    "import_status": "none",
    "open_issues_count": 0,
    "ci_default_git_depth": 20,
    "ci_forward_deployment_enabled": true,
    "ci_job_token_scope_enabled": false,
    "ci_separated_caches": true,
    "ci_opt_in_jwt": false,
    "ci_allow_fork_pipelines_to_run_in_parent_project": true,
    "public_jobs": true,
    "build_timeout": 3600,
    "auto_cancel_pending_pipelines": "enabled",
    "ci_config_path": "",
    "shared_with_groups": [],
    "only_allow_merge_if_pipeline_succeeds": true,
    "allow_merge_on_skipped_pipeline": true,
    "restrict_user_defined_variables": false,
    "request_access_enabled": true,
    "only_allow_merge_if_all_discussions_are_resolved": false,
    "remove_source_branch_after_merge": true,
    "printing_merge_request_link_enabled": true,
    "merge_method": "rebase_merge",
    "squash_option": "default_off",
    "enforce_auth_checks_on_uploads": true,
    "suggestion_commit_message": null,
    "merge_commit_template": null,
    "squash_commit_template": null,
    "auto_devops_enabled": false,
    "auto_devops_deploy_strategy": "continuous",
    "autoclose_referenced_issues": true,
    "keep_latest_artifact": true,
    "runner_token_expiration_interval": null
  }
]

Similarly, the subgroup Frontend has two projects: Project-D & Project-E.

The aim is to get the IDs of all projects within the parent group including the ones which reside in any of the subgroups.

$ curl -s -H "PRIVATE-TOKEN: ${GITLAB_API_TOKEN}" -H "Content-Type: application/json" -X GET "https://scm.ntf.ro/api/v4/groups/28/projects/?include_subgroups=true" | jq .[].id
26
25
24
23
22

Having now the IDs, we are able to iterate over these and update the:

  • allow_merge_on_skipped_pipeline to true. This allows Merge Requests to be merged with skipped jobs, as these are considered successful.
  • merge_method to rebase_merge. This is equivalent to Merge commit with semi-linear history.
  • only_allow_merge_if_pipeline_succeeds to true. This enforces Merge Requests to be merged with successful jobs.
$ for PROJECT in `curl -s -H "PRIVATE-TOKEN: ${GITLAB_API_TOKEN}" -H "Content-Type: application/json" -X GET "https://scm.ntf.ro/api/v4/groups/28/projects/?include_subgroups=true" | jq .[].id`; do curl -H "PRIVATE-TOKEN: ${GITLAB_API_TOKEN}" -H "Content-Type: application/json" -X PUT --data '{"allow_merge_on_skipped_pipeline": true,"merge_method": "rebase_merge", "only_allow_merge_if_pipeline_succeeds": true}' "https://scm.ntf.ro/api/v4/projects/$PROJECT" | jq; done
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3726    0  3607  100   119  19631    647 --:--:-- --:--:-- --:--:-- 20250
{
  "id": 26,
  "description": null,
  "name": "Project-E",
  "name_with_namespace": "ELF / Frontend / Project-E",
  "path": "project-e",
  "path_with_namespace": "elf/frontend/project-e",
  "created_at": "2022-10-28T12:16:35.687Z",
  "default_branch": "master",
  "tag_list": [],
  "topics": [],
  "ssh_url_to_repo": "git@scm.ntf.ro:elf/frontend/project-e.git",
  "http_url_to_repo": "https://scm.ntf.ro/elf/frontend/project-e.git",
  "web_url": "https://scm.ntf.ro/elf/frontend/project-e",
  "readme_url": "https://scm.ntf.ro/elf/frontend/project-e/-/blob/master/README.md",
  "avatar_url": null,
  "forks_count": 0,
  "star_count": 0,
  "last_activity_at": "2022-10-28T12:16:35.687Z",
  "namespace": {
    "id": 29,
    "name": "Frontend",
    "path": "frontend",
    "kind": "group",
    "full_path": "elf/frontend",
    "parent_id": 28,
    "avatar_url": null,
    "web_url": "https://scm.ntf.ro/groups/elf/frontend"
  },
  "container_registry_image_prefix": "scm.ntf.ro:5050/elf/frontend/project-e",
  "_links": {
    "self": "https://scm.ntf.ro/api/v4/projects/26",
    "issues": "https://scm.ntf.ro/api/v4/projects/26/issues",
    "merge_requests": "https://scm.ntf.ro/api/v4/projects/26/merge_requests",
    "repo_branches": "https://scm.ntf.ro/api/v4/projects/26/repository/branches",
    "labels": "https://scm.ntf.ro/api/v4/projects/26/labels",
    "events": "https://scm.ntf.ro/api/v4/projects/26/events",
    "members": "https://scm.ntf.ro/api/v4/projects/26/members",
    "cluster_agents": "https://scm.ntf.ro/api/v4/projects/26/cluster_agents"
  },
  "packages_enabled": true,
  "empty_repo": false,
  "archived": false,
  "visibility": "private",
  "resolve_outdated_diff_discussions": false,
  "container_expiration_policy": {
    "cadence": "1d",
    "enabled": false,
    "keep_n": 10,
    "older_than": "90d",
    "name_regex": ".*",
    "name_regex_keep": null,
    "next_run_at": "2022-10-29T12:16:35.701Z"
  },
  "issues_enabled": true,
  "merge_requests_enabled": true,
  "wiki_enabled": true,
  "jobs_enabled": true,
  "snippets_enabled": true,
  "container_registry_enabled": true,
  "service_desk_enabled": false,
  "service_desk_address": null,
  "can_create_merge_request_in": true,
  "issues_access_level": "enabled",
  "repository_access_level": "enabled",
  "merge_requests_access_level": "enabled",
  "forking_access_level": "enabled",
  "wiki_access_level": "enabled",
  "builds_access_level": "enabled",
  "snippets_access_level": "enabled",
  "pages_access_level": "private",
  "operations_access_level": "enabled",
  "analytics_access_level": "enabled",
  "container_registry_access_level": "enabled",
  "security_and_compliance_access_level": "private",
  "emails_disabled": null,
  "shared_runners_enabled": false,
  "lfs_enabled": true,
  "creator_id": 7,
  "import_url": null,
  "import_type": null,
  "import_status": "none",
  "import_error": null,
  "open_issues_count": 0,
  "runners_token": "GR1348941KfJgQp7MPrVeNycLd9bs",
  "ci_default_git_depth": 20,
  "ci_forward_deployment_enabled": true,
  "ci_job_token_scope_enabled": false,
  "ci_separated_caches": true,
  "ci_opt_in_jwt": false,
  "ci_allow_fork_pipelines_to_run_in_parent_project": true,
  "public_jobs": true,
  "build_git_strategy": "fetch",
  "build_timeout": 3600,
  "auto_cancel_pending_pipelines": "enabled",
  "ci_config_path": "",
  "shared_with_groups": [],
  "only_allow_merge_if_pipeline_succeeds": true,
  "allow_merge_on_skipped_pipeline": true,
  "restrict_user_defined_variables": false,
  "request_access_enabled": true,
  "only_allow_merge_if_all_discussions_are_resolved": false,
  "remove_source_branch_after_merge": true,
  "printing_merge_request_link_enabled": true,
  "merge_method": "rebase_merge",
  "squash_option": "default_off",
  "enforce_auth_checks_on_uploads": true,
  "suggestion_commit_message": null,
  "merge_commit_template": null,
  "squash_commit_template": null,
  "auto_devops_enabled": false,
  "auto_devops_deploy_strategy": "continuous",
  "autoclose_referenced_issues": true,
  "keep_latest_artifact": true,
  "runner_token_expiration_interval": null
}
<OMITTED OUTPUT>

In case there would be multiple groups as well, the above example could be extended with a nested for (wink-wink).

Additionally, one could also skip the pipeline while rebasing with the API, or by using Git push options, or by adding [ci skip] or [skip ci] (using any capitalization) in a commit message.

Conclusion

Having the possibility to perform a rebase without triggering a pipeline contributes to a higher velocity in terms of development process and the latest versions of GitLab do offer this functionality.

I did enjoy playing around with the GitLab API in order to achieve this with the least effort, especially in the context of having a considerable amount of projects.

References