python-jenkins-developers team mailing list archive

[Nos Tror <1979356@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Version used: commit/d1e469649574883bc531d35fb3aee207f45c15a5

# Origin of the issue

We upgraded our Jenkins instance from v2.277.4 to v2.332.3 and did the
name and label migration from 'master' node to 'built-in' node as
introduced by version v2.319 and stated in
https://www.jenkins.io/doc/upgrade-guide/2.319/#built-in-node-name-and-
label-migration.

At this point we were using python-jenkins v1.7.0.


# Occurance of the bug

After the name/label migration we recurringly saw HTTP requests to Jenkins API endpoints resulting in 404 responses. Below is an excerpt of the NGINX log (<> are placeholders).
```
<ipaddr> - <jenkins-user> [23/May/2022:15:36:23 +0200] "GET /queue/item/1556202/ HTTP/1.1" 404 463 "" "python-requests/2.21.0" ""
<ipaddr> - <jenkins-user> [23/May/2022:15:36:24 +0200] "GET /queue/item/1556202/ HTTP/1.1" 404 463 "" "python-requests/2.21.0" ""
<ipaddr> - <jenkins-user> [23/May/2022:15:36:24 +0200] "GET /queue/item/1556202/ HTTP/1.1" 404 463 "" "python-requests/2.21.0" ""
<ipaddr> - <jenkins-user> [23/May/2022:15:36:24 +0200] "GET /queue/item/1556202/ HTTP/1.1" 404 463 "" "python-requests/2.21.0" ""
<ipaddr> - <jenkins-user> [23/May/2022:15:36:25 +0200] "GET /queue/item/1556202/ HTTP/1.1" 404 463 "" "python-requests/2.21.0" ""
<ipaddr> - <jenkins-user> [23/May/2022:15:36:25 +0200] "GET /queue/item/1556202/ HTTP/1.1" 404 463 "" "python-requests/2.21.0" ""
```

>From a tool utilizing the python-jenkins module we saw the following output:
```
{'type': 'NotFoundException', 'message': 'Requested item could not be found', 'trace': [{'filename': 'project_tools/tools/swptools.py', 'name': 'build_jenkins', 'lineno': 123}, {'filename': 'jenkins/__init__.py', 'name': 'build_job', 'lineno': 1371}, {'filename': 'jenkins/__init__.py', 'name': 'jenkins_request', 'lineno': 592}]}
```


# Origin of the bug

I traced down in the code where this error is emitted.

python-jenkins > _init_.py
    build_job() @1370-1371
    response = self.jenkins_request(requests.Request(
         'POST', self.build_job_url(name, parameters, token)))
    jenkins_requests() @578-579
    return self._response_handler(
         self._request(req))
    jenkins_requests() @591-592
    elif e.response.status_code == 404:
         raise NotFoundException('Requested item could not be found')


The tool mentioned above starts Jenkins builds by sending HTTP POST requests towards /job/<job-name>/buildWithParameters?<parameters> (<> are placeholders).

The Jenkins API endpoint returns 201 when there is not already a build of the job in the queue and 303 when one or more builds already are.
The 303 response includes a redirect to the respective endpoint of the queue item /queue/item/<queue-item-number>/. However these are no valid Jenkins endpoints because they lack api/json or api/xml etc. in the end. Requests to /queue/item/<queue-item-number>/ endpoints return 404 whereas from /queue/item/<queue-item-number>/api/json endpoints 200 is returned. That could be on purpose so that clients can choose the response format, e.g. JSON or XML.
python-jenkins blindly follows these redirects without completing the returned API endpoints resulting in 404 responses and thus the NotFoundException above.


Example API interaction:
1. req: POST /job/pyproj/buildWithParameters?branch=main&deploy=production
   res: 201 Created
2. req: POST /job/pyproj/buildWithParameters?branch=main&deploy=production
   res: 303 See Other
        Location: /queue/item/1556202/
3. req: GET /queue/item/1556202/
   res: 404 Not Found
4. NotFoundException

The request URI in step 3 needs to be /queue/item/1556202/api/json
instead. This has been observed during tests and matches the Jenkins
documentation.


# Attempts to solve the bug

We updated python-jenkins to v1.8.0.0a0 as well as the latest commit
(d1e469649574883bc531d35fb3aee207f45c15a5). Both have the same bug.

As a quick and dirty fix we ended up passing `allow_redirects=False` to
build_job_url() completely ignoring redirects. See the attached patch
file.

** Affects: python-jenkins
     Importance: Undecided
         Status: New


** Tags: api jenkins-api redirect

** Patch added: "Quick and dirty fix"
   https://bugs.launchpad.net/bugs/1979356/+attachment/5598738/+files/0001-Do-not-allow-redirects-when-triggering-Jenkins-jobs.patch

** Summary changed:

- jenkins_request() does not properly handle redirects
+ Jenkins API redirects are not handled properly

-- 
You received this bug notification because you are a member of Python
Jenkins Developers, which is subscribed to Python Jenkins.
https://bugs.launchpad.net/bugs/1979356

Title:
  Jenkins API redirects are not handled properly

Status in Python Jenkins:
  New

Bug description:
  Version used: commit/d1e469649574883bc531d35fb3aee207f45c15a5

  # Origin of the issue

  We upgraded our Jenkins instance from v2.277.4 to v2.332.3 and did the
  name and label migration from 'master' node to 'built-in' node as
  introduced by version v2.319 and stated in
  https://www.jenkins.io/doc/upgrade-guide/2.319/#built-in-node-name-
  and-label-migration.

  At this point we were using python-jenkins v1.7.0.

  
  # Occurance of the bug

  After the name/label migration we recurringly saw HTTP requests to Jenkins API endpoints resulting in 404 responses. Below is an excerpt of the NGINX log (<> are placeholders).
  ```
  <ipaddr> - <jenkins-user> [23/May/2022:15:36:23 +0200] "GET /queue/item/1556202/ HTTP/1.1" 404 463 "" "python-requests/2.21.0" ""
  <ipaddr> - <jenkins-user> [23/May/2022:15:36:24 +0200] "GET /queue/item/1556202/ HTTP/1.1" 404 463 "" "python-requests/2.21.0" ""
  <ipaddr> - <jenkins-user> [23/May/2022:15:36:24 +0200] "GET /queue/item/1556202/ HTTP/1.1" 404 463 "" "python-requests/2.21.0" ""
  <ipaddr> - <jenkins-user> [23/May/2022:15:36:24 +0200] "GET /queue/item/1556202/ HTTP/1.1" 404 463 "" "python-requests/2.21.0" ""
  <ipaddr> - <jenkins-user> [23/May/2022:15:36:25 +0200] "GET /queue/item/1556202/ HTTP/1.1" 404 463 "" "python-requests/2.21.0" ""
  <ipaddr> - <jenkins-user> [23/May/2022:15:36:25 +0200] "GET /queue/item/1556202/ HTTP/1.1" 404 463 "" "python-requests/2.21.0" ""
  ```

  From a tool utilizing the python-jenkins module we saw the following output:
  ```
  {'type': 'NotFoundException', 'message': 'Requested item could not be found', 'trace': [{'filename': 'project_tools/tools/swptools.py', 'name': 'build_jenkins', 'lineno': 123}, {'filename': 'jenkins/__init__.py', 'name': 'build_job', 'lineno': 1371}, {'filename': 'jenkins/__init__.py', 'name': 'jenkins_request', 'lineno': 592}]}
  ```

  
  # Origin of the bug

  I traced down in the code where this error is emitted.

  python-jenkins > _init_.py
      build_job() @1370-1371
      response = self.jenkins_request(requests.Request(
           'POST', self.build_job_url(name, parameters, token)))
      jenkins_requests() @578-579
      return self._response_handler(
           self._request(req))
      jenkins_requests() @591-592
      elif e.response.status_code == 404:
           raise NotFoundException('Requested item could not be found')

  
  The tool mentioned above starts Jenkins builds by sending HTTP POST requests towards /job/<job-name>/buildWithParameters?<parameters> (<> are placeholders).

  The Jenkins API endpoint returns 201 when there is not already a build of the job in the queue and 303 when one or more builds already are.
  The 303 response includes a redirect to the respective endpoint of the queue item /queue/item/<queue-item-number>/. However these are no valid Jenkins endpoints because they lack api/json or api/xml etc. in the end. Requests to /queue/item/<queue-item-number>/ endpoints return 404 whereas from /queue/item/<queue-item-number>/api/json endpoints 200 is returned. That could be on purpose so that clients can choose the response format, e.g. JSON or XML.
  python-jenkins blindly follows these redirects without completing the returned API endpoints resulting in 404 responses and thus the NotFoundException above.

  
  Example API interaction:
  1. req: POST /job/pyproj/buildWithParameters?branch=main&deploy=production
     res: 201 Created
  2. req: POST /job/pyproj/buildWithParameters?branch=main&deploy=production
     res: 303 See Other
          Location: /queue/item/1556202/
  3. req: GET /queue/item/1556202/
     res: 404 Not Found
  4. NotFoundException

  The request URI in step 3 needs to be /queue/item/1556202/api/json
  instead. This has been observed during tests and matches the Jenkins
  documentation.

  
  # Attempts to solve the bug

  We updated python-jenkins to v1.8.0.0a0 as well as the latest commit
  (d1e469649574883bc531d35fb3aee207f45c15a5). Both have the same bug.

  As a quick and dirty fix we ended up passing `allow_redirects=False`
  to build_job_url() completely ignoring redirects. See the attached
  patch file.

To manage notifications about this bug go to:
https://bugs.launchpad.net/python-jenkins/+bug/1979356/+subscriptions