Gitlab issues

cat /var/log/gitlab/gitlab-rails/repocheck.log

error in blob 31ae455954e2ab24f917f0cad3c9dd9b0e029be4: gitmodulesMissing: unable to read .gitmodules blob

Solution:

su - git
cd /home/git-data/repositories/\@hashed\..\..\...git
git fsck
git gc
git fsck

gru 28 18:36:15 gitlab systemd[1]: user@997.service: Failed to add control inotify watch descriptor for control group /user.slice/user-997.slice/user@997.service: No space left on device
gru 28 18:36:15 gitlab systemd[1]: user@997.service: Failed to add memory inotify watch descriptor for control group /user.slice/user-997.slice/user@997.service: No space left on device

rate_limit_gitlab_shell:

Gitlab runner cannot clone repository:

Permission denied, please try again.
Permission denied, please try again.
git@gitlab.example.com: Permission denied (publickey,password).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

Tracing Upon SSH connection gitlab-shell is executed. Shell makes internal API connection to http://unix/api/v4/internal/allowed endpoint. Puma web server is responsible for handling request.

2.2. Git clone over SSH

HINT: GitLab: API is not accessible

Reason: Rate limit introduced with v14.8 with (28 Jan, 2022) Enable Gitlab Shell rate limit by default.

More info:

• Git operations using SSH
• Enable and disable GitLab features deployed behind feature flags

How to disable rate limits - how to disable rate_limit_gitlab_shell:

gitlab-rails console

and then

 
Feature.enabled?(:rate_limit_gitlab_shell)
 
Feature.disable(:rate_limit_gitlab_shell)
 
Feature.enabled?(:rate_limit_gitlab_shell)

Still problems, try to figure out:

• Gitlab-shell rate limiter: Rate limiting
• There is another rate limiter in Gitaly: Gitaly also has a rate-limiter in place, but calls are never made to Gitaly if the rate limit is exceeded in GitLab Shell (Rails).
  • It is disabled by default. Need to specify RPC endpoint and params to create Gitaly limiter: L144 of code
  • Request limiting in Gitaly
  • Limit RPC concurrency
    • When these limits are reached, users are disconnected.
  • Monitor Gitaly rate limiting
  • Monitor Gitaly concurrency limiting
  • Troubleshooting Gitaly and Gitaly Cluster
  • Gitaly reference
  • Gitaly source
• Allow specific users to bypass authenticated request rate limiting