meta data for this page
  •  

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
git:gitlab:issues [2022/12/28 21:31] – niziakgit:gitlab:issues [2024/06/10 12:34] (current) – niziak
Line 1: Line 1:
 ====== Gitlab issues ====== ====== Gitlab issues ======
  
-rate_limit_gitlab_shell:+===== disallowed submodule url ===== 
 + 
 +Repocheck failes: 
 + 
 +<file log /var/log/gitlab/gitlab-rails/repocheck.log> 
 +ERROR -- : my_group/my_project: Could not fsck repository: error in blob eb87a57adefc985514ddc8ae1fa4e6749c7d8211: gitmodulesUrl: disallowed submodule url: https://gitlab.example.com:my_group/my_project.git 
 +</file> 
 + 
 + 
 + 
 + 
 +===== One project failed its last repository check.  ===== 
 + 
 +<code bash>cat /var/log/gitlab/gitlab-rails/repocheck.log</code> 
 + 
 +''error in blob 31ae455954e2ab24f917f0cad3c9dd9b0e029be4: gitmodulesMissing: unable to read .gitmodules blob'' 
 + 
 +Solution: 
 +<code bash> 
 +su - git 
 +cd /home/git-data/repositories/\@hashed\..\..\...git 
 +git fsck 
 +git gc 
 +git fsck 
 +</code> 
  
 ===== Failed to add control inotify watch descriptor for control group ===== ===== Failed to add control inotify watch descriptor for control group =====
Line 10: Line 35:
  
 ===== Permission denied, please try again. ===== ===== Permission denied, please try again. =====
 +
 +rate_limit_gitlab_shell:
  
 Gitlab runner cannot clone repository: Gitlab runner cannot clone repository:
Line 21: Line 48:
 and the repository exists. and the repository exists.
 </code> </code>
 +
 +**Tracing**
 +Upon SSH connection ''gitlab-shell'' is executed. Shell makes internal API connection to ''http://unix/api/v4/internal/allowed'' endpoint. 
 +''Puma'' web server is responsible for handling request.
 +
 +[[https://docs.gitlab.com/ee/architecture/blueprints/pods/pods-feature-git-access.html#22-git-clone-over-ssh|2.2. Git clone over SSH]]
 +
 +**HINT:** [[https://docs.gitlab.com/ee/administration/operations/puma.html#gitlab-api-is-not-accessible|GitLab: API is not accessible]]
  
 **Reason:** **Reason:**
Line 43: Line 78:
  
 Still problems, try to figure out: Still problems, try to figure out:
-  * [[https://docs.gitlab.com/ee/administration/gitaly/troubleshooting.html#500-and-fetching-folder-content-errors-on-repository-pages|Troubleshooting Gitaly and Gitaly Cluster ]]+  * Gitlab-shell rate limiter: [[https://docs.gitlab.com/ee/development/gitlab_shell/#rate-limiting|Rate limiting]] 
 +  * There is another rate limiter in Gitaly: ''Gitaly also has a rate-limiter in place, but calls are never made to Gitaly if the rate limit is exceeded in GitLab Shell (Rails).'' 
 +    * It is disabled by default. Need to specify RPC endpoint and params to create Gitaly limiter: [[https://gitlab.com/gitlab-org/gitaly/-/blob/master/internal/middleware/limithandler/concurrency_limiter.go#L144|L144 of code]] 
 +    * [[https://gitlab.com/gitlab-org/gitaly/-/blob/master/doc/backpressure.md|Request limiting in Gitaly]] 
 +    * [[https://docs.gitlab.com/ee/administration/gitaly/configure_gitaly.html#limit-rpc-concurrency|Limit RPC concurrency]] 
 +      * ''When these limits are reached, users are disconnected.'' 
 +    * [[https://docs.gitlab.com/ee/administration/gitaly/monitoring.html#monitor-gitaly-rate-limiting|Monitor Gitaly rate limiting]] 
 +    * [[https://docs.gitlab.com/ee/administration/gitaly/monitoring.html#monitor-gitaly-concurrency-limiting|Monitor Gitaly concurrency limiting]] 
 +    * [[https://docs.gitlab.com/ee/administration/gitaly/troubleshooting.html#500-and-fetching-folder-content-errors-on-repository-pages|Troubleshooting Gitaly and Gitaly Cluster]] 
 +    * [[https://docs.gitlab.com/ee/administration/gitaly/reference.html|Gitaly reference]] 
 +    * [[https://gitlab.com/gitlab-org/gitaly/-/tree/master|Gitaly source]]
   * [[https://docs.gitlab.com/ee/user/admin_area/settings/user_and_ip_rate_limits.html#allow-specific-users-to-bypass-authenticated-request-rate-limiting|Allow specific users to bypass authenticated request rate limiting]]   * [[https://docs.gitlab.com/ee/user/admin_area/settings/user_and_ip_rate_limits.html#allow-specific-users-to-bypass-authenticated-request-rate-limiting|Allow specific users to bypass authenticated request rate limiting]]