GithubHelp home page GithubHelp logo

Tar --output-dir error about relx HOT 7 CLOSED

MarkoMin avatar MarkoMin commented on July 21, 2024
Tar --output-dir error

from relx.

Comments (7)

ferd avatar ferd commented on July 21, 2024 1

The paths searched by the install_upgrade_escript are mostly aligned with the requirements set by release_handler:unpack_release(Name), which are mostly an OTP standard outside of our control. It expects to find files in a given location to unpack and run everything fine.

Are you running a build in prod directly with the rebar3 tree structure in it? While we could just dump that tarball at _build/<profile>/rel/<release_name>/releases/ and let it do its thing, there is actually no intended relationship between the rebar3 _build directory structure and layout and what you would use in production.

The intended use is that you pick whatever tarball rebar3 builds and use that as a build artifact entirely distinct from what rebar3 outputs in _build for your deployments. None of _build should be kept otherwise. So we just output the tarball to a place that is unlikely to clash with the _build/<profile>/rel/<release_name> artifact in subsequent operations (such that generating the tarball many times, or doing relups multiple times should not contain a previous tarball by accident) and then it's up to you to put this wherever you want in production.

from relx.

MarkoMin avatar MarkoMin commented on July 21, 2024

Thanks for the explanation, but I just want to point out that even when I specify the --output-dir the tarball doesn't end up in specified directory but somewhere else.

I understand the thing about the defaults, but shouldn't user have the flexibility to put the tarball wherever he wants?

from relx.

MarkoMin avatar MarkoMin commented on July 21, 2024

I hit this problem once again - but now I your answer better, ignore my previous answer.

Everything you said is true for production and I got no problems there - but I encountered this problem when I want to start my releases while developing - not installed anywhere, just in _build/.... E.g. I run _build/default/rel/my_rel/bin/my_rel console an that succeeds, but when I run _build/default/rel/my_rel/bin/my_rel upgrade 0.1.1 it says my_rel-0.1.1.tar.gz is missing even though tarball is in _build/default/rel/my_rel.

Dunno how often do people start releases via rebar3 _build folder, but I find it convenient from time to time.

I misunderstood --output-dir before and I agree that it works properly. But I still think that it would be nice if tarball could go into releases or relases/<vsn> and that would enable upgrade/downgrade/install/unpack commands to be correctly performed in development.

P.S. sorry for close-reopen, hit my touchpad...

from relx.

ferd avatar ferd commented on July 21, 2024

Hm. I don't remember if this is a path issue, or an Erlang issue. There's an old bug in Erlang/OTP where if you started your release without a tarball first, it never really picks up on releases fine or has issues with paths in some places: erlang/otp#4041

Otherwise yeah I'd imagine just putting the tarball in the right directory would work to later call a relup. I've never really spent the time of doing it right in the _build directory though.

from relx.

MarkoMin avatar MarkoMin commented on July 21, 2024

Hm. I don't remember if this is a path issue, or an Erlang issue.

I would say that this is indeed path issue because it is resolved when I manually copy tarballs into releases or releases/<rel_vsn> directory.

I've never really spent the time of doing it right in the _build directory though.

Yeah, I find it particularly useful when experimenting with releases/upgrades. It gives you a sandbox right in your _build dir. It just saves your time - you don't need to install it somewhere if you want to try something. I usually do the following:

git checkout <prev_vsn_tag>
rebar3 tar
git checkout <HEAD>
rebar3 tar

and then you have your last 2 release versions installed in your _build and you can try to upgrade them.

2 solutions to this (if you agree that this should be taken care of) are:

  1. Add current directory in search path for install, upgrade, downgrade and unpack commands in start script
  • should cause minimal problems in backward compatibility, but this change is affecting part of the production (start script))
  1. Put tarballs in /releases or releases/<vsn> directory
  • could maybe cause backward compatibility issues (only in development!), but is not affecting start script

I'm for opt 2.

from relx.

ferd avatar ferd commented on July 21, 2024

A third option is to just use a hook to make it work:

{post_hooks, [
  {tar, "mv $REBAR_BUILD_DIR/rel/YOUR_RELNAME/*.tar.gz \"$REBAR_BUILD_DIR/rel/YOUR_RELNAME/releases/\""}
]}.

which will just take the artifact and move it over.

I'm not sure opt 2 is a good idea (possibly breaking backwards compat with a bunch of CI scripts for people uploading artifacts is not a great plan). I think it would be safe tarball-wise (the systools docs make it clear we need to pass any extra file we want packaged).

Opt 1 is possibly safer, but the current directory wouldn't necessarily be a good plan unless it's appended at the end of all search paths to avoid suddenly favoring files outside of where they'd be expected, but as you said it does impact prod-level scripts.

from relx.

MarkoMin avatar MarkoMin commented on July 21, 2024

A third option is to just use a hook to make it work:

Yup, that works! I tried something like that before, but I wasn't satisfied because I didn't know about $REBAR_BUILD_DIR variable... Now I see that there is a lot of vars exported for hooks, which I believe is worth to be in the hooks documentation.

I'll close this issue because it's now solvable via hooks, and I'll try to enhance rebar3 hooks docs :D

Many thanks for the help!

from relx.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.