GithubHelp home page GithubHelp logo

Comments (9)

dosubot avatar dosubot commented on July 17, 2024 2

The Jira plugin is likely failing with a 502 error on the "collectIssues" subtask due to an issue with the API request to Jira. A 502 error typically indicates a bad gateway, meaning the server received an invalid response from an inbound server.

To resolve this issue, you can try the following steps:

  1. Check the API Client Configuration: Ensure that the ApiClient is correctly configured and that the Jira server is reachable.

  2. Verify the JQL Query: The JQL query used in the request might be causing issues. Ensure that the JQL query is correctly formed and does not result in a large dataset that could overwhelm the server.

  3. Reduce Concurrency: The current concurrency level is set to 10. Reducing the concurrency might help if the server is being overwhelmed by too many simultaneous requests.

  4. Check the Response Parser: Ensure that the response parser is correctly handling the responses from Jira.

Here is the relevant code snippet for setting up the API collector and making the request:

err = apiCollector.InitCollector(api.ApiCollectorArgs{
    ApiClient: data.ApiClient,
    PageSize:  data.Options.PageSize,
    UrlTemplate: "agile/1.0/board/{{ .Params.BoardId }}/issue",
    Query: func(reqData *api.RequestData) (url.Values, errors.Error) {
        query := url.Values{}
        query.Set("jql", jql)
        query.Set("startAt", fmt.Sprintf("%v", reqData.Pager.Skip))
        query.Set("maxResults", fmt.Sprintf("%v", reqData.Pager.Size))
        query.Set("expand", "changelog")
        return query, nil
    },
    GetTotalPages: GetTotalPagesFromResponse,
    Concurrency:   10,
    ResponseParser: func(res *http.Response) ([]json.RawMessage, errors.Error) {
        var data struct {
            Issues []json.RawMessage `json:"issues"`
        }
        blob, err := io.ReadAll(res.Body)
        if err != nil {
            return nil, errors.Convert(err)
        }
        err = json.Unmarshal(blob, &data)
        if err != nil {
            return nil, errors.Convert(err)
        }
        return data.Issues, nil
    },
})
if err != nil {
    return err
}
return apiCollector.Execute()

Consider reducing the Concurrency value and verifying the jql query to ensure it is not causing issues. If the problem persists, further investigation into the network and server configurations might be necessary [1].

Details

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

from incubator-devlake.

klesh avatar klesh commented on July 17, 2024

Hi, seems like your JIRA server is down or broken somehow. I think you need to fix your JIRA server and make sure the API can be requested successfully before Apache DevLake can collect data from it.

from incubator-devlake.

ajayesf avatar ajayesf commented on July 17, 2024

@klesh The server is working fine. We have other boards also hosted in the same server and I am able to pull data from rest of the Board. I am only facing issue for a specific board. Evertime it is failing on the same sub task.

from incubator-devlake.

ajayesf avatar ajayesf commented on July 17, 2024

Hi,

Could anyone help us on this?

from incubator-devlake.

ajayesf avatar ajayesf commented on July 17, 2024

We have also verified the Permssion of the Jira connection with an Admin account. The issue is still persist.
@klesh

from incubator-devlake.

klesh avatar klesh commented on July 17, 2024

OK, normally 502 means Server Internal Error.
So, did you try requesting the JIRA API with curl or postman to make sure the JIRA API is up and running and working properly?

from incubator-devlake.

ajayesf avatar ajayesf commented on July 17, 2024

image
APIs are working fine. Please find the attached
@klesh

from incubator-devlake.

ajayesf avatar ajayesf commented on July 17, 2024

To be specific, The task is faling at " Http DoAsync error calling [method:GET path:agile/1.0/board/151/issue query:map[expand:[changelog] jql:[updated >= '2024/05/01 00:00' ORDER BY created ASC] maxResults:[100] startAt:[0]]]." . Follows to this we are getting the 502 issue.

All tasks before "collectissue" subtask is passing.

time="2024-05-30 14:12:43" level=info msg=" [pipeline service] [pipeline #47] [task #919] start executing task: 919"
time="2024-05-30 14:12:43" level=info msg=" [pipeline service] [pipeline #47] [task #919] start plugin"
time="2024-05-30 14:12:43" level=info msg=" [pipeline service] [pipeline #47] [task #919] [api async client] creating scheduler for api "https://jira.somatus.com/rest/\", number of workers: 13, 10000 reqs / 1h0m0s (interval: 360ms)"
time="2024-05-30 14:12:43" level=info msg=" [pipeline service] [pipeline #47] [task #919] total step: 32"
time="2024-05-30 14:12:43" level=info msg=" [pipeline service] [pipeline #47] [task #919] executing subtask collectBoardFilterBegin"
time="2024-05-30 14:12:43" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectBoardFilterBegin] collect board in collectBoardFilterBegin: 151"
time="2024-05-30 14:12:43" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectBoardFilterBegin] collect board filter:14167"
time="2024-05-30 14:12:43" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectBoardFilterBegin] collect board filter jql:project = RQC ORDER BY Rank ASC"
time="2024-05-30 14:12:43" level=info msg=" [pipeline service] [pipeline #47] [task #919] finished step: 1 / 32"
time="2024-05-30 14:12:43" level=info msg=" [pipeline service] [pipeline #47] [task #919] executing subtask collectStatus"
time="2024-05-30 14:12:43" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectStatus] start api collection"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectStatus] finished records: 1"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectStatus] end api collection without error"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] finished step: 2 / 32"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] executing subtask extractStatus"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [extractStatus] extract Status, connection_id=2, board_id=151"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [extractStatus] get data from _raw_jira_api_status where params={"ConnectionId":2,"BoardId":151} and got 42"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [extractStatus] finished records: 1"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] finished step: 3 / 32"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] executing subtask collectProjects"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectProjects] collect projects"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectProjects] start api collection"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectProjects] finished records: 1"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectProjects] end api collection without error"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] finished step: 4 / 32"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] executing subtask extractProjects"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [extractProjects] get data from _raw_jira_api_projects where params={"ConnectionId":2,"BoardId":151} and got 13"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [extractProjects] finished records: 1"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] finished step: 5 / 32"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] executing subtask collectIssueTypes"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectIssueTypes] collect issue_types"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectIssueTypes] start api collection"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectIssueTypes] finished records: 1"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectIssueTypes] end api collection without error"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] finished step: 6 / 32"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] executing subtask extractIssueType"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [extractIssueType] extract IssueType, connection_id=2, board_id=151"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [extractIssueType] get data from _raw_jira_api_issue_types where params={"ConnectionId":2,"BoardId":151} and got 6"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [extractIssueType] finished records: 1"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] finished step: 7 / 32"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] executing subtask collectIssues"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectIssues] got user's timezone: Asia/Kolkata"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline #47] [task #919] [collectIssues] start api collection"

@klesh

from incubator-devlake.

d4x1 avatar d4x1 commented on July 17, 2024

@ajayesf From the logs:

time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline https://github.com/apache/incubator-devlake/pull/47] [task https://github.com/apache/incubator-devlake/issues/919] finished step: 7 / 32"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline https://github.com/apache/incubator-devlake/pull/47] [task https://github.com/apache/incubator-devlake/issues/919] executing subtask collectIssues"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline https://github.com/apache/incubator-devlake/pull/47] [task https://github.com/apache/incubator-devlake/issues/919] [collectIssues] got user's timezone: Asia/Kolkata"
time="2024-05-30 14:12:44" level=info msg=" [pipeline service] [pipeline https://github.com/apache/incubator-devlake/pull/47] [task https://github.com/apache/incubator-devlake/issues/919] [collectIssues] start api collection"

It seems collectIssues starts successfully.

And this #7540 (comment) just show orgnization api is working, you should get the api that collectIssues are using, and try to request it with curl or postman.

from incubator-devlake.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.