Argument list too long about bmk

[woollybah]

Interesting. I had the same issue whilst testing on AROS, but the Amiga's max command-line length is much shorter than Linux...
Looks like we can use @file, and dump the compiler options into a temp file during the build..

[woollybah]

From @GWRon on January 21, 2016 10:3

Please take into consideration that this "temp file" must use a individual file name, just to avoid problems in a multithreaded compilation process when writing to "args.tmp".

I do not know how fast an md5 hash is (compared to the rest) but maybe hashing the complete content of that tempfile will result in a unique file name. But even then thread 1 might want to remove a file which thread 2 is using. This is of course only possible if there is the possibility of exactly the same arguments (compiling the same file?).

A (super-stricted) Md5-Hash-Code could be found in my framework:
https://github.com/GWRon/Dig/blob/master/base.util.hashes.bmx

Just copy/paste it if you want to use such a thing.

Maybe it is enough to use a tmp file named after the sourcecode filename we compile ... would make it way easier :-) (but allows for clashes when not somehow using the absolute path to that file)

[woollybah]

md5 is too slow. I'll use a system API for this probably.

[woollybah]

From @GWRon on January 21, 2016 10:41

Just as I was interested in measuring it:

SuperStrict
Framework Brl.StandardIO
Import "base.util.hashes.bmx"

Global hash:String
Global hashString:string = ""
For local i:int = 0 to 300
    hashString :+ "Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua." 
Next


Global startTime:int = Millisecs()
For local i:int = 0 to 100
    hash = Hashes.MD5(hashString)
Next
Global endTime:int = Millisecs()

print "length of string: "+hashString.length
print "hash: " + hash
print "time: " + ((endTime - startTime)/100.0) +" ms"

Result:

Executing:md5time
length of string: 46655
hash: 31ec1dc4be15cdeafd394365dc1007b4
time: 0.569999993 ms

I assume the actual compilation takes longer than this hashing process of a 45kb sized string.

[woollybah]

Yeah. I still prefer an API solution. Just because we are programmers, doesn't necessitate us designing new wheels every time ;-) ▪️

[woollybah]

Out of interest, what result do you get from:

getconf ARG_MAX

Thanks!

[woollybah]

From @GWRon on January 21, 2016 15:17

$ getconf ARG_MAX
2097152

Sorry for the delay, just came home to my linux box

[woollybah]

From @GWRon on January 21, 2016 15:47

For Windows the limits are discussed there:
https://blogs.msdn.microsoft.com/oldnewthing/20031210-00/?p=41553

CreateProcess: 32767 characters
CMD.exe: 8192 characters
ShellExecute/Ex: 2048 characters
Environment: 32767 characters
_spawnv(): 1024 bytes

[AlexCeleste]

I feel like this is somehow related to this.

As said over there, I've got a lot of arguably-redundant Import declarations in my project (because of wanting to explicitly import everything relevant rather than trusting hidden details). When I removed a few Import "runtime/Util.bmx from files at the toplevel that should be able to see it indirectly anyway, the "argument list too long" error went away.

Noticed this because util itself was appearing multiple times in the compile list (which could mean nothing, I don't know what that actually represents). Maybe there's something in the way it detects duplicate paths?

[woollybah]

I've reworked the internals a bit, and it's not generating duplicate include paths now...