GithubHelp home page GithubHelp logo

konveyor / move2kube-api Goto Github PK

View Code? Open in Web Editor NEW
6.0 13.0 18.0 1.11 MB

HTTP REST API for move2kube

Home Page: https://move2kube.konveyor.io/

License: Apache License 2.0

Dockerfile 0.57% Makefile 2.56% Go 95.89% Shell 0.36% JavaScript 0.19% HTML 0.44%
kubernetes modernization replatform move2kube hacktoberfest

move2kube-api's People

Contributors

akash-nayak avatar ashokponkumar avatar burnerlee avatar eloycoto avatar gabriel-farache avatar harikrishnanbalagopal avatar isabelladev avatar kmehant avatar rgolangh avatar rmarting avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

move2kube-api's Issues

Move2Kube-UI doesn't support directly zipped source files

Describe the bug
Move2Kube is not able to handle the cases where the source files (ex: main.py and requirements.txt) are zipped directly and uploaded for translation. Currently, it works if the source files are within a folder and the zipped folder is uploaded to the UI.

To Reproduce
Steps to reproduce the behavior:

  1. Create a New Application and upload python-zipped-source-files.zip or eStore.zip file to the UI.

Generated plan

kind: Plan
metadata:
  name: testbug
spec:
  inputs:
    rootDir: assets/src
    services: {}
  outputs:
    kubernetes:
      targetCluster:
        type: Kubernetes
  1. Click on Translate and then Click Next.

Screenshot 2021-06-17 at 4 19 58 PM

API logs-

INFO[0041] Artifact Name:testbug_1623926695                
INFO[0041] About to start translation of application testbug 
INFO[0041] Debug level: false                           
INFO[0041] Waiting for QA engine to start for app testbug  
INFO[0041] Starting Translate for testbug                  
INFO[0041] App name: tetsbug; Artifact name:testbug_1623926695; level=info msg="Detected a plan file at path /workspace/testbug/artifacts/testbug_1623926695/m2k.plan. Will translate using this plan." 
INFO[0041] App name: testbug; Artifact name:testbug_1623926695; level=fatal msg="Failed to find any services. Aborting." 
ERRO[0041] Failed to create the output zip file at path testbug/artifacts/testbug_1623926695/testbug.zip . Error: "walking testbug/artifacts/testbug_1623926695/testbug: testbug/artifacts/testbug_1623926695/testbug: stat: stat testbug/artifacts/testbug_1623926695/testbug: no such file or directory"

Clean up logs

  1. For each function check if the current logs are correct.
  2. Check if any error is being ignored it is logged if necessary.
  3. Add debug logs where it might help in functions when starting any action/function.
  4. Print informative logs whenever a new request comes and success/error messages.

bug: referencing workspace inputs from a project doesn't work anymore. Start planning fails with a 500 error.

Describe the bug

Bug added in commit

commit 4f31fe54f45b0eba426c58eb6ead7b09c2adbd72 (HEAD)
Author: Mehant Kammakomati <[email protected]>
Date:   Tue Sep 27 02:28:40 2022 +0530

    feat: api support for invokedByDefault transformers (#122)
    
    Signed-off-by: Mehant Kammakomati <[email protected]>
$ docker run --rm -it -p 8084:8080 quay.io/konveyor/move2kube-ui:v0.3.5-rc.0
....
INFO[0029] POST /api/v1/workspaces/13f0ee7e-2d6e-4039-a0a8-7dea4f1dd37f/projects/a47847a0-3320-45cb-bb87-83cfc4c68914/plan  request-id=60cfcc33-64a4-4546-8322-0071b09a01ff
ERRO[0029] failed to start plan generation. Error: "failed to create the sources directory at the path  . Error: \"mkdir : no such file or directory\""  request-id=60cfcc33-64a4-4546-8322-0071b09a01ff

To Reproduce
Steps to reproduce the behavior:

  1. Run docker run --rm -it -p 8084:8080 quay.io/konveyor/move2kube-ui:v0.3.5-rc.0
  2. Create a workspace.
  3. Create a workspace input (example: language-platforms.zip)
  4. Create a project.
  5. Create a project input referencing the workspace input.
  6. Click Start Planning button

Expected behavior

In v0.3.4 and before the planning succeeds.

Screenshots

In v0.3.5-rc.0 and above the planning fails with a 500 error

ERRO[0029] failed to start plan generation. Error: "failed to create the sources directory at the path  . Error: \"mkdir : no such file or directory\""  request-id=60cfcc33-64a4-4546-8322-0071b09a01ff

Desktop (please complete the following information):

  • OS: MacOS
  • Browser: Chrome
  • Version: v0.3.5-rc.0

Additional context

Caused by removing important checks in this PR #122
Bug is not present in the previous commit c75ce35bf619f22f9cc0c6c7c61d45ed5b349db7

Specifically this change https://github.com/konveyor/move2kube-api/pull/122/files#diff-6525fa84f37be87e15da54e5e900efbbea70e2f9e75d6b21429f9bc11842ed8fL1263-R1241

Use regex substring to get portion of string

move2kube-api/main.go

Lines 185 to 192 in 3d77a86

re := regexp.MustCompile(`artifacttype:\\s.*`)
artifactTypematch := re.FindString(plan)
artifactType := "helm"
if strings.HasSuffix(artifactTypematch, "knative") {
artifactType = "knative"
} else if strings.HasSuffix(artifactTypematch, "k8") {
artifactType = "k8"
}

Change above code to not look for specific strings. Rather just do a substring on the regex extracted string or create a bracketed regex, and identify the match and use that as artifactname

No feedback when error occur during planning execution

Describe the bug
When the plan command fails, there is no status change nor feedback given to the user; when getting the plan via the API, we still get 204 response code instead of an error

To Reproduce
Steps to reproduce the behavior:

  1. Create a workspace and a project
  2. Start a plan via the API with invalid input with remote source, ie: wrong url so the plan will fail
    Command executed should be similar to plan --name a --plan-progress-port 32881 --log-file m2kcli.log --source git+ssh://bitbucket.com/gfarache31/m2k-test@master --disable-local-execution
  3. Check the logs to see the error, something similar to:
time="2024-03-12T09:47:17Z" level=info msg="plan cmdArgs: [plan --name a --plan-progress-port 32881 --log-file m2kcli.log --source git+ssh://bitbucket.com/gfarache31/m2k-test@master --disable-local-execution]"
time="2024-03-12T09:47:17Z" level=info msg="Project: 4a3d44ae-ba55-49b9-ae67-9b0b6d66640e; level=info msg=\"Cloning the repository using git into '/tmp/move2kube3232420242/m2ksources' . This might take some time.\""
time="2024-03-12T09:47:18Z" level=info msg="Project: 4a3d44ae-ba55-49b9-ae67-9b0b6d66640e; level=fatal msg=\"failed to create the plan. Error: \\\"failed to clone the repo 'git+ssh://bitbucket.com/gfarache31/m2k-test@master'. Error: failed to clone using vcs url 'git+ssh://bitbucket.com/gfarache31/m2k-test@master' and clone options {CommitDepth:1 Overwrite:true MaxSize:-1 CloneDestinationPath:/tmp/move2kube3232420242/m2ksources}. Error: failed to perform clone operation using git. Error: repository already exists\\\"\""
time="2024-03-12T09:47:18Z" level=error msg="failed to update the database after planning finished. Error: \"failed to copy the plan file from /tmp/plan-4a3d44ae-ba55-49b9-ae67-9b0b6d66640e-4058952779/m2k.plan to /move2kube-api/data/projects/4a3d44ae-ba55-49b9-ae67-9b0b6d66640e/inputs/expanded/m2k.plan after planning finished. Error: \\\"failed to open the source file at path \\\\\\\"/tmp/plan-4a3d44ae-ba55-49b9-ae67-9b0b6d66640e-4058952779/m2k.plan\\\\\\\" Error: \\\\\\\"open /tmp/plan-4a3d44ae-ba55-49b9-ae67-9b0b6d66640e-4058952779/m2k.plan: no such file or directory\\\\\\\"\\\"\""
  1. Send a GET request to the plan and see you have a 204 response code instead of a 5XX one as the planning has failed

Expected behavior
I expect that plan status to change to failed or similar and to be able to get the error information via the API (maybe not the full reason but at least that it failed)

Screenshots
N/A

Desktop (please complete the following information):

  • OS: [e.g. iOS] Fedora
  • Browser [e.g. chrome, safari] chrome
  • Version [e.g. 22] v0.3.11

Additional context
Related to konveyor/move2kube-ui#169

bug: planning and transformation fails when the zip file has no top level folder

Bug

Overview

When the sources folder uploaded to the UI is a zip/archive with no top level folder, the planning fails.
This happens because we put a .m2kignore file in the folder, so Move2Kube doesn't find any services.

Steps to reproduce

  1. Start the UI.
  2. Create a workspace and project.
  3. Upload the golang-no-parent-folder.zip file as a source input.
    golang-no-parent-folder.zip
  4. Start planning, no services are detected.
  5. Start transformation, fails and prints error logs in the terminal because it can't find the output folder.

Actual behaviour

INFO[20468] Project: 27ea485f-bd8d-4543-a9e8-2ea7529507a2; level=info msg="Planning finished on the base directory: '/tmp/plan-27ea485f-bd8d-4543-a9e8-2ea7529507a2-3686120573/sources'" 
INFO[20468] Project: 27ea485f-bd8d-4543-a9e8-2ea7529507a2; level=info msg="Planning started on its sub directories" 
INFO[20468] Project: 27ea485f-bd8d-4543-a9e8-2ea7529507a2; level=info msg="Planning finished on its sub directories" 
INFO[20468] Project: 27ea485f-bd8d-4543-a9e8-2ea7529507a2; level=info msg="[Directory Walk] Identified 0 named services and 0 to-be-named services" 
INFO[20468] Project: 27ea485f-bd8d-4543-a9e8-2ea7529507a2; level=info msg="[Named Services] Identified 0 named services" 
INFO[20468] Project: 27ea485f-bd8d-4543-a9e8-2ea7529507a2; level=info msg="Planning done. Number of services identified: 0" 
INFO[20468] Project: 27ea485f-bd8d-4543-a9e8-2ea7529507a2; level=info msg="Plan can be found at [/tmp/plan-27ea485f-bd8d-4543-a9e8-2ea7529507a2-3686120573/m2k.plan]." 
INFO[20468] Project: 27ea485f-bd8d-4543-a9e8-2ea7529507a2; level=warning msg="Did not detect any services in the directory /tmp/plan-27ea485f-bd8d-4543-a9e8-2ea7529507a2-3686120573/sources . Also we didn't find any default transformers to run."

After this if you start transformation, Move2Kube aborts because the plan has no services inside it.
So the output folder isn't created.

https://gist.github.com/eloycoto/ad287ef001f3477d418571ecbc90d896

INFO[0030] Project: dfc29a1a-b991-44a4-b9c7-7c74d2fea661; Output:2691dc52-2c41-4635-8548-86f78570abcc; level=info msg="Detected a plan file at path /move2kube-api/data/projects/dfc29a1a-b991-44a4-b9c7-7c74d2fea661/outputs/2691dc52-2c41-4635-8548-86f78570abcc/m2k.plan. Will transform using this plan."
INFO[0030] Project: dfc29a1a-b991-44a4-b9c7-7c74d2fea661; Output:2691dc52-2c41-4635-8548-86f78570abcc; level=warning msg="Using the detected plan with specified source. If you did not want to use the plan file at /move2kube-api/data/projects/dfc29a1a-b991-44a4-b9c7-7c74d2fea661/outputs/2691dc52-2c41-4635-8548-86f78570abcc/m2k.plan, delete it and rerun the command."
INFO[0030] Project: dfc29a1a-b991-44a4-b9c7-7c74d2fea661; Output:2691dc52-2c41-4635-8548-86f78570abcc; level=fatal msg="Failed to find any services or default transformers. Aborting."
ERRO[0030] failed to copy over the m2kconfig.yaml and m2kqacache.yaml. Error: "failed to write the plan file to the path /move2kube-api/data/projects/dfc29a1a-b991-44a4-b9c7-7c74d2fea661/outputs/2691dc52-2c41-4635-8548-86f78570abcc/output/m2k.plan . Error: \"open /move2kube-api/data/projects/dfc29a1a-b991-44a4-b9c7-7c74d2fea661/outputs/2691dc52-2c41-4635-8548-86f78570abcc/output/m2k.plan: no such file or directory\""

Expected behaviour

The planning should succeed and detect a Golang service.
The transformation should succeed and produce the Dockerfile and K8s YAMLs.

Version

$ move2kube-api version -l
version: v0.1.0+unreleased
gitCommit: a88d5e4b373c7ee7f610b968faa3163938430466
gitTreeState: clean
goVersion: go1.19.2
platform: darwin/amd64

Document the entire API in openapi.json

โš ๏ธ Don't forget to read our contribution guidelines before you start working on an issue https://github.com/konveyor/move2kube-api/blob/main/contributing.md

Is your feature request related to a problem? Please describe.
Currently only part of the API has been documented: https://github.com/konveyor/move2kube-api/blob/main/assets/openapi.json
Document the entire API in openapi.json

Describe the solution you'd like

  1. Go to https://editor.swagger.io/
  2. Click File > Import URL > Paste https://raw.githubusercontent.com/konveyor/move2kube-api/main/assets/openapi.json
  3. Edit the YAML and add all the endpoints https://github.com/konveyor/move2kube-api/blob/main/internal/move2kubeapi/move2kubeapi.go#L80-L114
  4. After you are done, click File > Convert and save as JSON
  5. Make a PR with the updated openapi.json

Describe alternatives you've considered
Alternatively you can add go-swagger https://github.com/go-swagger/go-swagger annotations to the Golang code and generate the openapi.json from the annotations. This approach might be harder than the steps given above.

Make cbuild is failing with Go 1.18

Describe the bug
make cbuild fails to build the move2kube-api image as it requires Go 1.19

> [builder 12/12] RUN make build:                                                   
0.124 go mod download                                                                
0.163 make get_swagger                                                               
0.166 make[1]: Entering directory '/go/src/move2kube-api'
0.190 curl -Lo swagger-ui.tgz https://github.com/swagger-api/swagger-ui/archive/refs/tags/v3.52.3.tar.gz \
0.190     && tar -xzf swagger-ui.tgz \
0.190     && mv swagger-ui-3.52.3/dist assets/swagger \
0.190     && cp assets/openapi.json assets/swagger/openapi.json \
0.190     && cp assets/index.html assets/swagger/index.html \
0.190     && rm swagger-ui.tgz \
0.190     && rm -rf swagger-ui-3.52.3
0.195   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
0.195                                  Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 6030k    0 6030k    0     0  1777k      0 --:--:--  0:00:03 --:--:-- 3291k
3.793 make[1]: Leaving directory '/go/src/move2kube-api'
3.795 go build -ldflags '-w -s -X github.com/konveyor/move2kube-api/cmd/version.version=latest -X github.com/konveyor/move2kube-api/cmd/version.buildmetadata=unreleased -X github.com/konveyor/move2kube-api/cmd/version.gitCommit=594b44e064af54e284b15c2156d7e3adfdcca8b9 -X github.com/konveyor/move2kube-api/cmd/version.gitTreeState=dirty -extldflags "-static"' -o /go/src/move2kube-api/bin/move2kube-api ./cmd/main
5.988 # github.com/konveyor/move2kube-api/internal/move2kubeapi/handlers
5.988 internal/move2kubeapi/handlers/inputs.go:57:19: undefined: http.MaxBytesError
5.988 note: module requires Go 1.19
6.002 make: *** [Makefile:87: /go/src/move2kube-api/bin/move2kube-api] Error 2
------
Dockerfile:48
--------------------
  46 |     ARG VERSION=latest
  47 |     COPY . .
  48 | >>> RUN make build
  49 |     
  50 |     # Run image
--------------------
ERROR: failed to solve: process "/bin/sh -c make build" did not complete successfully: exit code: 2
make: *** [cbuild] Error 1

To Reproduce
Steps to reproduce the behavior:

  1. Run make cbuild

Expected behavior
It should successfully build the move2kube-api image.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.