Comments (35)
Makes sense to me.
from rome.
In the past it seems that the decision has always been to closest match the style of Carthage. In this case, to match documented behavior, it seems like it should be comma-separated, e.g. iOS,watchOS
.
From carthage help build
:
[--platform (platform)]
the platforms to build for (one of 'all', 'macOS', 'iOS', 'watchOS', 'tvOS', or comma-separated values of the formers except for 'all')
from rome.
@erichoracek you can try this out without breaking compatibility with previous versions.
The bucket/local cache structure was:
<git-repo-name>/<framework-name-and-version>.[framework|dSYM]
it is now:
<git-repo-name>/<platform>/<framework-name-and-version>.[framework|dSYM]
This means that versions of rome >= 0.9.0.18 won't find frameworks uploaded by older versions of rome and the other way around. However old version and new version won't interfere with each other.
I will add this to the release notes.
from rome.
@blender @erichoracek I think part of the cache layout was overlooked in my implementation. This doesn't affect whether or not cache layout is interfering with previous versions - that is still fine, but the current build is using
<framework-name>/<platform>/<framework-name-and-version>.[framework|dSYM]
I'm putting together a fix to use git repo name at the top level since it is reasonable to expect different repos may have targets with the same framework names. This shouldn't affect single project caches since the conflict would pose a problem for the consuming project, but if you have different projects with different dependencies using the same bucket, you could see conflicts.
from rome.
How do you envision this working with respect to the list
command?
from rome.
@r-peck how about a switch just like carthage? like list [--platform iOS]
in the Romefile one could specify a default-platform key
from rome.
So for the --missing
/--present
options, should a platform or list of platforms be required? If not, should having a cached version for any platform be counted?
Without the presence option it could probably just list broken out by platform.
from rome.
It feels a little odd since you may have frameworks that don't support a given platform and that you don't care to use there (maybe a framework that's only used in your main application but not in your watch app).
from rome.
I think it's appropriate to consider how carthage handles this.
By default carthage is platform agnostic and will perform the given command for all platforms in sequence unless a --platform [list, of, platforms] option is given to the command.
That said let's imagine how this would work for Rome.
rome upload awesome-cat-names
will upload awesome-cat-names for all platformsrome upload --platform iOS watchOS awesome-cat-names
will upload awesome-cat-names for both iOS and watch.rome download awesome-cat-names
will download awesome-cat-names for all platformsrome download --platform iOS watchOS awesome-cat-names
will download awesome-cat-names for both iOS and watchrome list [--present | missing]
lists frameworks per platform (Note: section dividers and overall output must be easy to parse since other processes might rely on being able to understand the output)rome list [--present | missing] --platform iOS watchOS
lists frameworks restricted to the iOS and watchOS platform (Note: section dividers and overall output must be easy to parse since other processes might rely on being able to understand the output)rome list [--present | missing] --platform tvOS
lists frameworks restricted to the tvOS platform only (Note: Should there be a section header here? If yes overall output must be easy to parse since other processes might rely on being able to understand the output)
Now, let's consider a few examples:
Example 1:
- You have an iOS app with watchOS companion app
- You have
awesome-cat-names
that buildsAwesomeCatNames-iOS.framework
andAwesomeCatNames-watchOS.framework
- In your Romefile RepositoryMap section you have
awesome-cat-names = AwesomeCatNames-iOS, AwesomeCatNames-watchOS
- rome download will try to download both AND macOS, tvOS (this fails for the last two platforms. No big deal)
- rome upload will try to upload both AND macOS, tvOS (this fails for the last two platforms. No big deal)
- rome list --missing --platform iOS watchOS (no frameworks are reported missing)
Example 2:
- You have a macOS app
- You have
awesome-cat-names
that buildsAwesomeCatNames-macOS.framework
- In your Romefile RepositoryMap section you have
awesome-cat-names = AwesomeCatNames-macOS
- rome download will try to download all platforms (this fails for the 3 platforms. No big deal)
- rome upload will try to upload all platforms (this fails for 3 platforms. No big deal)
- rome list --missing --platform macOS (no frameworks are reported missing)
Seems to me that on the contrary of what I initially suggested there is no need for a default platform.
Regarding the cache structure I think it should mirror the way carthage does it with Build/<platform>/Name.framework
@r-peck what do you think?
I will soon merge #41 that should also work transparently with the multi platform feature
from rome.
That mirrors my thinking for the most part. My biggest concern was with changing the listing format:
- Let's say you've got an app with A.framework and B.framework that also embeds a watch app that uses A.framework. B.framework is not available for watchOS.
rome list --missing
might have something likeiOS:;watchOS:B
(not actual suggested format, but just an example of something parseable containing the needed info).- It would be on the user to run with the appropriate platform for the information they are needing (if the expected availability is different per-platform, scripting that uses
list
needs to be done per-platform). - Existing users might have scripts that incorrectly indicate something is missing because they depend on the old output format. This could break things in a quiet way.
- If the command requires the platform to be specified (at least for
--missing
and--present
where the current output is just a list), the output format could change while breaking existing scripts in a more obvious way.
from rome.
I agree that it is upon the user to run the command for the appropriate platform. With the current Romefile format there is no way to specify frameworks per platform. This is doable but it would mean going in a completely different direction.
Wether a platform argument should be specified or not for the list command it will still be a breaking change. I'm afraid that there is not easy way around it. The breaking change should also be indicated with a major change in version number (0.8 to 0.9).
If rome refuses to list without a platform argument it will still output an error (form the CLI command parser) and the error will still be parsed by whatever is next.
Example with required platform (thus bad command):
rome list | build_missing
will still continue to build_missing
even if rome exits with non zero status (which currently is not the case, upon bad command rome still exits with zero). Garbage will be fed to build_missing
and the build will eventually fail.
Example with no required platform:
rome list | build_missing
will still continue to build_missing
feeding something that the script will not parse and the build will eventually fail.
So builds will fail anyways and once the user realises that missing dependencies have not been downloaded it should give them an hint of where things have failed.
For the sake or consistency, I would prefer to have all commands behave the same where possible. There is no "semantic" reason (i.e. the command does not make sense) why list
would require a platform.
@r-peck do you see a better way? Am I missing something?
from rome.
Great! Thanks for helping defining this :)
from rome.
Heads up, #41 was merged and #38 closed. This restructures some stuff.
from rome.
Thanks for moving this proposal forwards guys! It's something we'll be needing soon. A few questions/comments:
With respect to argument style, to match the style of Carthage, I believe that this would be --platform iOS,watchOS
. See Carthage/Carthage#726.
With respect to the missing
/present
behavior, I have a question that I believe will affect the implementation of this. Specifically, can you elaborate on how you will know whether a dependency does not produce a framework for a specific architecture vs. the dependency not being present in the cache?
For example, say that frameworkA
builds for watchOS
and iOS
, while frameworkB
builds for just iOS
. How will rome list --missing
know that frameworkA
is missing for watchOS
while frameworkB
is not missing for watchOS
since it does not produce a framework for it?
As far as I can tell, this would require either:
- Parsing Xcode projects from the
carthage checkouts
(this assumes that the user has a localcarthage checkout
) - Adding all possible target paths to the
Romefile
RepositoryMap
Let me know if I've missed something. Thanks!
from rome.
I think --platform in carthage works with or without the comma, or if it doesn't I never noticed :D
The behaviour with list [--missing | present]
is problematic only in the case of multiple platforms being used in the same project at the same time. This is the case for and iOS & watchOS app or for a frameworks that builds for all platforms.
Assumptions
Let's assume the situation that @erichoracek described:
frameworkA
builds forwatchOS
andiOS
, whileframeworkB
builds for justiOS
- list output is in the format:
<framework-name>: ((+|-)<platform>)?
Solutions
Solution 1 - Ignore the problem
The easiest way to solve this is to ignore the problem just like carthage does.
This means that the --platform
switch an it's argument are really just a tip for where to look in the cache for a certain artefact.
Example:
$ rome list
frameworkB: +iOS, -watchOS
frameworkA: +iOS, +watchOS
$ rome list --platform iOS, watchOS
frameworkB: +iOS, -watchOS
frameworkA: +iOS, +watchOS
$ rome list --missing
frameworkB: -watchOS
$rome list --missing --platform watchOS
frameworkB: -watchOS #this is not really missing see assumptions
In a few cases frameworkB
is a false positive missing which means that if the output is further used to trigger builds for the missing dependencies in the cache it will trigger a build for frameworkB
for platform watchOS
. However In the assumptions frameworkB
does not have a watchOS target in the first place so triggering this build should do nothing. This is the case if you would try to trigger a carthage build frameworkB --platform watchOS
The real issue here is if frameworkB
actually does have a watchOS
target that you're just not using. This will trigger a unnecessary build.
Solution 2 - [IgnoreMap]
per platform
This solutions entails creating an [IgnoreMap] section per platform.
The INI file format does not support nested sections so in a Romefile one would have:
- optional global
[IgnoreMap
] - optional per platform
[IgnoreMap-<platform>]
Example:
#Romefile - other parts omitted
[IgnoreMap-watchOS]
frameworkB = frameworkB
$rome list --missing --platform watchOS
Solution 3 - [IgnoreMap]
and [RepositoyMap]
per platform
This solutions entails creating an [IgnoreMap]
and a [RepositoyMap]
section per platform.
The INI file format does not support nested sections so in a Romefile one would have:
- optional global
[RepositoyMap]
- optional per platform
[RepositoyMap-<platform>]
- optional global
[IgnoreMap]
- optional per platform
[IgnoreMap-<platform>]
#Romefile - other parts omitted
[RepositoyMap]
frameworkA = frameworkA # this is a global dependency
[RepositoryMap-iOS]
frameworkB = frameworkB
or
#Romefile - other parts omitted
[RepositoyMap-iOS]
frameworkB = frameworkB
frameworkA = frameworkA #this is really a global dependency but the user made a mistake
[RepositoyMap-watchOS]
frameworkA = frameworkA #this is really a global dependency but the user made a mistake
or
#Romefile - other parts omitted
[RepositoyMap]
frameworkB = frameworkB # this is NOT a global dependency but the user made a mistake
frameworkA = frameworkA # this is a global dependency
[IgnoreMap-watchOS]
frameworkB = frameworkB
$rome list --missing --platform watchOS
I believe that here there are other scenarios I can't think of right now.
Solution 4 - DependencyMap global and per platform
This solutions introduces a new optional section in which dependencies for a given platform are specified.
This replicates Solution 3 but adds another section leaving [RepositoryMap]
and [IgnoreMap]
as they are instead of abusing them to implicitly specify dependencies. They idea with [RepositoryMap]
is just to provide a name resolution system for repository<->frameworkName(s).
#Romefile - other parts omitted
[RepositoyMap]
frameworkB = frameworkB
frameworkA = frameworkA
[IgnoreMap]
xcconfig = xcconfig
[DependencyMap]
frameworkA = frameworkA #global dependency
[DependencyMap-iOS]
frameworkB = frameworkB
$rome list --missing --platform watchOS
Observations
- Solution 1 is the easiest but might trigger unnecessary builds
- Solution 2 is abuses the IgnoreMap to implicitly specify dependencies
- Solution 3 is exponentially complex and error prone imho
- Solution 4 is clear but you the
[IgnoreMap]
remains global. Maybe it's worth having one per platform ? See solution 2 concerns.
@erichoracek @r-peck what do you guys think?
from rome.
For the platforms delimiter, Carthage documents requiring comma. It will, however, handle spaces as long as the list is treated as a single argument by the shell (enclosed in quotes, for example).
On listing per-platform, Solutions 2-4 are additive to solution 1 since not specifying something in the Romefile would fall back to ignoring. I would lean toward get the functionality in place first (solution 1), then enhance with solution 4.
from rome.
@blender @erichoracek Any strong feelings about the updated list output format, especially regarding sectioning—how are sections delimited, should it nest frameworks per platform or platforms per framework, etc.
from rome.
@r-peck I can get behind Solution 1 which solves the problem for the time being and is also retro-compatible
I suggest the following format that also gets rid of headers completely:
<framework-name>: ((+|-)<platform>)?
Example:
$ rome list
FrameworkA: +iOS, +watchOS
FrameworkB: -iOS, +watchOS
$rome list --present
FrameworkA: +iOS, +watchOS
$rome list --missing
FrameworkB: -iOS
$rome list --present --platform watchOS
FrameworkA: +watchOS
from rome.
From an implementation standpoint, I like the simplicity of that, and I like that it is consistent in all modes (default, missing, present). Would it make sense to include an example in the readme of how to extract a list of frameworks to forward to Carthage for a given platform since this will be less straightforward than with the current list --missing
behavior of just printing the list?
from rome.
Sure or a output formatter flag could be provided like --format=carthage
. I guess that for the sake of a simple first implementation it's easier to provide instructions in the Readme.
from rome.
I have added a ready-to-implement
label that I will add to this issue once we agree on the design of this feature. I would like to wait for @erichoracek comments on both the solution approch and the output format. Then give it a few days to sink in. Hopefully all of this can happen before the weekend.
I would encourage anyone to try to sketch the CLI command and the output in a textual format. I find that it really helps to simulate the usage & desired outcome.
If anyone is interested in implementing this please let me know here now or once the ready-to-implement
label is applied.
from rome.
@blender I had actually taken a stab at the upload/download part of it, but stopped short of listing since that seemed to be the most contentious area. Also, if upload/download does change direction, I'm not worried about the lost effort.
from rome.
That is awesome! I don't think upload / download behavior will change anyhow. It's pretty clear that the only real problem is list and what the output format should look like.
I have also realized that the current output of list also reports the version of the framework. So a more compete format definition would be <framework-name> <version-hash>: ((+|-)<platform>)?
from rome.
Of all of the options, I agree that Solution 1 is the most reasonable in terms of complexity and as a first pass. We may have to update our script that does a carthage build
for each Rome cache miss to run once for each platform (rather than building both at the same time) to prevent unnecessary builds. However, this doesn't seem like too much of an issue for me.
One small comment with respect to output. I feel that if a single platform is specified, platform should not be included after the framework name, as it is redundant with what the user just typed. E.g.:
$rome list --present
FrameworkA: +iOS, +watchOS -tvOS, -macOS
FrameworkB: +iOS, -watchOS -tvOS, -macOS
$rome list --present --platform watchOS
FrameworkA
$rome list --present --platform iOS
FrameworkA
FrameworkB
Something like --format=carthage
feels a bit like overkill until it's explicitly requested. An example of how to strip trailing platforms from the output in the README seems like the way to go.
Thanks for the detailed responses!
from rome.
It seems we have an agreement for Solution 1, a.k.a ignore the problem.
I let this sink in for a bit and these are my comments
Clarification about presence switches --missing
|--present
@erichoracek I think we have a misunderstanding on$rome list --present
from what I see in from your first command. The --present
switch will only report cache hits per framework per platform. So the actual output should be.
$rome list --present
FrameworkA : +iOS, +watchOS
FrameworkB : +iOS
Since FrameworkA is not a cache hit for tvOS nor macOS and FrameworkB is not a cache hit for -watchOS, tvOS and macOS.
Thus the output for --missing
should be
$rome list --missing
FrameworkA : -tvOS, -macOS
FrameworkB : -watchOS, -tvOS, -macOS
If you don't activate any presence switch (--missing
|--present
) then you would get your output
$rome list
FrameworkA : +iOS, +watchOS -tvOS, -macOS
FrameworkB : +iOS, -watchOS -tvOS, -macOS
I think what if the presence switches (--missing
|--present
) don't filter per framework per platform but just per framework then eventually all frameworks will be reported as either missing or present for some platform, thus rendering the filters useless in practice.
Simplifying output when only one platform is present in the --platform
filter parameters
@erichoracek Unfortunately with the introduction of the platforms I think repeating the platform name is necessary simply because if you don't activate a presence filter the output doesn't make sense.
Example:
$rome list --present --platform iOS #note the --present
FrameworkA #makes sense, FrameworkA is present
FrameworkB #makes sense, FrameworkB is present
$rome list --platform watchOS #note the --present or --missing are omitted
FrameworkA #does NOT makes sense, is FrameworkA present or missing?
FrameworkB #does NOT makes sense, is FrameworkB present or missing?
I think a uniform solution solves this elegantly
$rome list --present --platform iOS #note the --present
FrameworkA : +iOS #makes sense, FrameworkA is present
FrameworkB : +iOS #makes sense, FrameworkB is present
$rome list --platform watchOS #note the --present or --missing are omitted
FrameworkA : +watchOS #makes sense, FrameworkA is present
FrameworkB : -watchOS #makes sense, FrameworkB is missing
@r-peck suggested to provide examples of how to parse this in the README so this should ease some pain for the users.
On another note as I mentioned before when listing all the current output also echoes the versions back. I think there is value in this since it does not require whoever/whatever will read the output to
read again the Cartfile.resolved
to figure out the version numbers.
#rome 0.8.0.17
$ rome list
Alamofire 3.5.1 ✔︎
GCDKit 1.3.0 ✔︎
HockeySDK-iOS 3.8.6 ✔︎
KeychainAccess v2.4.0 ✔︎
frameworkA d702e1e120f1f930abc568508386845b470596d2 ✔︎
What do you guys think @r-peck @erichoracek ? Should the version be reported? If yes I would put it after the name for all cases i.e. : <git-repo-name> <version> : ...
Note that the version is not added if you toggle any additional filter at the moment.
Going further I think that in the future it might be valuable to have formatters, for example the list command could output JSON
Proposed format so far
Note that in our previous discussions we have simply referred to the identifier of the framework as <framework-name>
but that is not actually what that is. For sake of simplicity I will just call it <git-repo-name>
.
Format definition and examples
- In plain english:
<The name of the git repository> space <the version hash> space : space <ios-platform-cache-status>, <watchOS-platform-cache-status>, <tvOS-platform-cache-status>, <macOS-platform-cache-status>
<git-repository-name> <version> : ((+|-)<platform-name>){1-4}
. Note the max 4 repetitions
Examples:
$rome list
FrameworkA 3.5.1 : +iOS, +watchOS, -tvOS, -macOS
FrameworkB d702e1e120f1f930abc568508386845b470596d2 : +iOS, -watchOS -tvOS, -macOS
$rome list --missing
FrameworkA 3.5.1 : -tvOS, -macOS
FrameworkB d702e1e120f1f930abc568508386845b470596d2 : -watchOS -tvOS, -macOS
$rome list --platform watchOS
FrameworkA 3.5.1 : +watchOS
FrameworkB d702e1e120f1f930abc568508386845b470596d2 : -watchOS
$rome list --missing --platform watchOS
FrameworkB d702e1e120f1f930abc568508386845b470596d2 : -watchOS
$rome list --missing --platform watchOS, iOS
FrameworkB d702e1e120f1f930abc568508386845b470596d2 : -watchOS
$rome list --missing --platform watchOS, iOS, macOS #note I added a platform where BOTH are missing
FrameworkA 3.5.1 : -macOS
FrameworkB d702e1e120f1f930abc568508386845b470596d2 : -watchOS, -macOS
@erichoracek @r-peck Let's give this a final round and sign it off. I'm very excited to see it happen :)
from rome.
@blender ah yes—I think I left something out in my response. My intent was to say that if a single --platform
argument and either --present
or --missing
were both specified at the same time, (+/-)<platform-name>
could be omitted from the output as it would be redundant. This would be similar to the existing behavior of --present
or --missing
with iOS
as the only available platform (e.g. you don't see checkmarks next to every framework name when you specify --present
).
I don't feel super strongly about this point. Either way seems fine for me, as it's simple to just take the first word of each line when parsing the output. Since it is now a rare case (you need to specify both a single platform as well as a presence flag), filtering out part of the output in that scenario does feel like it may be unexpected for users, especially if they're passing a dynamic number of platform flags to rome
, as they could potentially get differing output depending on the number of flags passed.
from rome.
Yeah, I would be in favor of consistency. having different output formats based on different combinations of otherwise-unrelated options seems like unnecessary complexity. As mentioned by @blender, a formatted could always be added in the future to do this more explicitly.
As far as including version number, no strong feelings from my side.
from rome.
I prefer consistency at this point. I would also include the version number for the reasons I stated above.
We can provide scripts to parse the output for the special case we're talking about.
For me this is ready for implementation. I will add the tag. Nice discussion everyone!
from rome.
Update about --platform
option
The CLI parsing library does not allow options with multiple arguments so it's either --platform
which works in both like the following fashion
$ rome list --platform iOS --platform watchOS
$ rome list --platform iOS,watchOS # no spaces or whatever comes next will be considered a new CLI option
or
$ rome list iOS, watchOS # note no --platform
from rome.
So it is then :)
from rome.
And thanks to @r-peck 's incredible work, we have a pre-release https://github.com/blender/Rome/releases/tag/v0.9.0.18 !
Please test if you can and report any bugs. I will promote it to a proper release if no issues are found in a week or so.
from rome.
Thanks guys, this is awesome! Can this new directory structure coexist in the same bucket that previous versions of rome
have (or will) use? E.g. can we try this out without potentially breaking our bucket's backwards compatibility with previous versions?
from rome.
New pre-release with fix: https://github.com/blender/Rome/releases/tag/v0.9.0.19
from rome.
Hello,
I just want to reiterate that the cache paths have changed from:
<framework-name>/<framework-name-and-version>.[framework|dSYM]
to
<git-repo-name>/<platform>/<framework-name-and-version>.[framework|dSYM]
Example:
Suppose you have the following
Cartfile.resolved:
...
git "ssh://[email protected]:7999/iossdk/savingtheworld-ios-core.git" "v0.8.9"
...
and Romefile:
...
[RepositoryMap]
savingtheworld-ios-core = Kernel
The cache path for rome <= 0.8.x.x was
Kernel/Kernel-v0.8.9.framework
it is now
savingtheworld-ios-core/<platform>/Kernel-v0.8.9.framework
same for the dSYMs
from rome.
I have not heard any complains about this version thus I will close the issue. Thank you all for the discussion and thanks @r-peck for you great contribution.
from rome.
Related Issues (20)
- Xcode version specific upload/download HOT 4
- Concurrent downloads for remote cache HOT 1
- Nexus as example engine HOT 4
- [Question] Framework produces both static and dynamic output HOT 4
- Losing debug capability when using cached builds downloaded by Rome HOT 2
- Static folder isn't uploaded to remote server on Xcode 12 HOT 4
- Rome won't upload all downstream re-built frameworks HOT 3
- Rome sometimes printing partial output HOT 1
- Rome not failing when engine exit code is not zero HOT 1
- Issues running Rome with --no-skip-current HOT 6
- Needs to be updated to support XCFrameworks. HOT 13
- When using currentMap its not possible to reuse the generated binaries HOT 3
- Rome crashes when downloading dependencies HOT 1
- Skip bcsymbolmaps and dsyms when uploading/downloading a framework HOT 4
- Issues with Rome for xcframeworks HOT 24
- Failed to install latest ROME in MAC HOT 1
- using rome download --concurrently with --use-xcframeworks leads to io error: openBinaryFile: resource busy (file is locked)
- Add Rome arm64 release artifact HOT 3
- Error if carthage cache and rome cache have a file and directory with the same name
- Build failure with ghc 9.6.2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rome.