* Backoff from Scan2 which is experimental to legacy pagination API call
This commit rewrite simplifiedGitlabEnumeration to use legacy pagination API call with keyset pagination instead of Scan2 which is currently in experimental state. Note
that this doesn't promise to fix this problem it's just a test to check. It also adds a retry logic in case any 500 error occurs. I added some logs as well to keep track
of no of projects being enumerated.
* implemented builtin retry mechanism for gitlab and proper handling of next page
* fixed basic auth
* Some enhancements
Reversed the gitlab cloud logic to add membership flag, so that we use the default false for non gitlab.com instances.
This can help if the issue really was membership flag as mentioned in some gitlab issues.
Also added simple flag in list projects to get only minimal fields in response instead of big json response for each project.
Added test case as well.
* enhance the test case
* add wrapper reporter to append project details to chunk metadata
* use cache to store project details
* revert unnecessary change
* delete from cache when done with scanning in ChunkUnit, implement PR suggestions
* query project details using repo instead of having it in source unit
* revert removal of build tag
* Incorporated PR comments
Previously, the GitLab include and ignore lists were only applied during repository enumeration, which meant that they would be ignored after enumeration completed. For large environments, post-enumeration scanning can take days, and it was awkward that the include/ignore lists could effectively not be modified during that time. This PR changes things such that repositories can be configured to be skipped even post-enumeration.
Importantly, repositories cannot be "un-ignored" post-enumeration. This is unfortunate, but this PR still represents improvement on the status quo.
* gitlab groups init
* added list group projects api
* list group projects updated
* added duplicate repo scan check
* comments addressed
* added error when repo and group id flags are provided at the same time
* added test case for gitlab group projects
* Implemented gitlab inclusion globbing.
Included test.
* implemented two new flags for gitlab scan, includeRepo and excludeRepo to support globbing.
Apply globbing filter when repos is not provided.
* implemented integration test for inclusion globbing
remove test to check errors if globs are invalid.
* made changes to support glob compile errors.
modified changes to support glob compilation errors.
* removed unused context from few functions.
Muneeb Ullah Khan
Kashif Khan
Mustansir
Cody Rose
Shahzad Haider
Abdul Basit
ahrav