Fails if vhosts contains more directories about apache2buddy

[richardforth]

Can you please paste in the script output with --verbose option. Thanks

…

On 15 Dec 2016 11:29, "Varyen" ***@***.***> wrote: Hi, I have following structure of my vhost files: /etc/apache2/vhosts/user1/family.user1.com /etc/apache2/vhosts/user1/thai.user1.com /etc/apache2/vhosts/user1/valery.user1.com /etc/apache2/vhosts/user2/user2.ru in expand_included_files you use following command to find files in directory: # ls /etc/apache2/vhosts/* 2> /dev/null /etc/apache2/vhosts/user1:family.user1.com thai.user1.com valery.user1.com /etc/apache2/vhosts/user2:user2.ru Look's like it should be **/* to work as intended # ls /etc/apache2/vhosts/**/* /etc/apache2/vhosts/user1/family.user1.com /etc/apache2/vhosts/user1/valery.user1.com /etc/apache2/vhosts/user2/user2.ru /etc/apache2/vhosts/user1/thai.user1.com I'm running Ubuntu 16.04.1 LTS. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#29>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFpUV8jpp9NA3xnXHUq1AIiIlRhcnrZ3ks5rISSngaJpZM4LN_vg> .

[varyen]

Sure

VERBOSE: Adding glob to /etc/apache2/vhosts/, is a directory
VERBOSE: Adding /etc/apache2/vhosts/user1: to list of files for processing
VERBOSE: Adding family.user1.com to list of files for processing
VERBOSE: Adding thai.user1.com to list of files for processing
VERBOSE: Adding valery.user1.com to list of files for processing
VERBOSE: Adding  to list of files for processing
VERBOSE: Adding /etc/apache2/vhosts/user2: to list of files for processing

................

VERBOSE: Processing /etc/apache2/vhosts/user1:
Uncaught exception from user code:
        Unable to open file: /etc/apache2/vhosts/user1:
        main::find_included_files(ARRAY(0x1830b60), ARRAY(0x18312b0), "/etc/apache2") called at ./apache2buddy.pl line 314
        main::build_config_array("/etc/apache2/apache2.conf", "/etc/apache2") called at ./apache2buddy.pl line 1586
        main::preflight_checks() called at ./apache2buddy.pl line 2032

[richardforth]

Looks like it doesn't like the way you've included the files in your config, can you paste in your include lines? As it's expanding globs and not a hardcoded command I need to see what it's doing I'm not sure if I can fix it but I will have a look. I'm not able to work on this til the new year now due to other commitments. But once I have all the details I'll try to figure out what it doesn't like about your particular setup. Bear in mind that this code has been in use for over 5 years. I'm thinking you're doing something very specific and non standard here.

…

On 15 Dec 2016 11:41, "Varyen" ***@***.***> wrote: Sure VERBOSE: Adding glob to /etc/apache2/vhosts/, is a directory VERBOSE: Adding /etc/apache2/vhosts/user1: to list of files for processing VERBOSE: Adding family.user1.com to list of files for processing VERBOSE: Adding thai.user1.com to list of files for processing VERBOSE: Adding valery.user1.com to list of files for processing VERBOSE: Adding to list of files for processing VERBOSE: Adding /etc/apache2/vhosts/user2: to list of files for processing ................ VERBOSE: Processing /etc/apache2/vhosts/user1: Uncaught exception from user code: Unable to open file: /etc/apache2/vhosts/user1: main::find_included_files(ARRAY(0x1830b60), ARRAY(0x18312b0), "/etc/apache2") called at ./apache2buddy.pl line 314 main::build_config_array("/etc/apache2/apache2.conf", "/etc/apache2") called at ./apache2buddy.pl line 1586 main::preflight_checks() called at ./apache2buddy.pl line 2032 — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#29 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFpUVwPFYLWqdo9Mf7X-f-9FeFlxjrMrks5rISddgaJpZM4LN_vg> .

[richardforth]

Basically it thinks user1 and user2 is a file and isn't able to correctly expand your globs If you paste in your include lines I should be able to run some tests once I get free time.

…

On 15 Dec 2016 11:47, "Richard Forth" ***@***.***> wrote: Looks like it doesn't like the way you've included the files in your config, can you paste in your include lines? As it's expanding globs and not a hardcoded command I need to see what it's doing I'm not sure if I can fix it but I will have a look. I'm not able to work on this til the new year now due to other commitments. But once I have all the details I'll try to figure out what it doesn't like about your particular setup. Bear in mind that this code has been in use for over 5 years. I'm thinking you're doing something very specific and non standard here. On 15 Dec 2016 11:41, "Varyen" ***@***.***> wrote: > Sure > > VERBOSE: Adding glob to /etc/apache2/vhosts/, is a directory > VERBOSE: Adding /etc/apache2/vhosts/user1: to list of files for processing > VERBOSE: Adding family.user1.com to list of files for processing > VERBOSE: Adding thai.user1.com to list of files for processing > VERBOSE: Adding valery.user1.com to list of files for processing > VERBOSE: Adding to list of files for processing > VERBOSE: Adding /etc/apache2/vhosts/user2: to list of files for processing > > ................ > > VERBOSE: Processing /etc/apache2/vhosts/user1: > Uncaught exception from user code: > Unable to open file: /etc/apache2/vhosts/user1: > main::find_included_files(ARRAY(0x1830b60), ARRAY(0x18312b0), "/etc/apache2") called at ./apache2buddy.pl line 314 > main::build_config_array("/etc/apache2/apache2.conf", "/etc/apache2") called at ./apache2buddy.pl line 1586 > main::preflight_checks() called at ./apache2buddy.pl line 2032 > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#29 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AFpUVwPFYLWqdo9Mf7X-f-9FeFlxjrMrks5rISddgaJpZM4LN_vg> > . >

[richardforth]

Ps if you figure it out before me please make a pull request from staging and I'll try and get it merged. Sorry for the delay

…

On 15 Dec 2016 11:49, "Richard Forth" ***@***.***> wrote: Basically it thinks user1 and user2 is a file and isn't able to correctly expand your globs If you paste in your include lines I should be able to run some tests once I get free time. On 15 Dec 2016 11:47, "Richard Forth" ***@***.***> wrote: > Looks like it doesn't like the way you've included the files in your > config, can you paste in your include lines? > > As it's expanding globs and not a hardcoded command I need to see what > it's doing I'm not sure if I can fix it but I will have a look. > > I'm not able to work on this til the new year now due to other > commitments. But once I have all the details I'll try to figure out what it > doesn't like about your particular setup. Bear in mind that this code has > been in use for over 5 years. > > I'm thinking you're doing something very specific and non standard here. > > On 15 Dec 2016 11:41, "Varyen" ***@***.***> wrote: > >> Sure >> >> VERBOSE: Adding glob to /etc/apache2/vhosts/, is a directory >> VERBOSE: Adding /etc/apache2/vhosts/user1: to list of files for processing >> VERBOSE: Adding family.user1.com to list of files for processing >> VERBOSE: Adding thai.user1.com to list of files for processing >> VERBOSE: Adding valery.user1.com to list of files for processing >> VERBOSE: Adding to list of files for processing >> VERBOSE: Adding /etc/apache2/vhosts/user2: to list of files for processing >> >> ................ >> >> VERBOSE: Processing /etc/apache2/vhosts/user1: >> Uncaught exception from user code: >> Unable to open file: /etc/apache2/vhosts/user1: >> main::find_included_files(ARRAY(0x1830b60), ARRAY(0x18312b0), "/etc/apache2") called at ./apache2buddy.pl line 314 >> main::build_config_array("/etc/apache2/apache2.conf", "/etc/apache2") called at ./apache2buddy.pl line 1586 >> main::preflight_checks() called at ./apache2buddy.pl line 2032 >> >> — >> You are receiving this because you commented. >> Reply to this email directly, view it on GitHub >> <#29 (comment)>, >> or mute the thread >> <https://github.com/notifications/unsubscribe-auth/AFpUVwPFYLWqdo9Mf7X-f-9FeFlxjrMrks5rISddgaJpZM4LN_vg> >> . >> >

[richardforth]

Thinking I need to check each expanded file to see if it is either a file or a directory then decide

directory = dont add to array
file = add to array

at the moment:

VERBOSE: Adding /etc/apache2/vhosts/user1: to list of files for processing = WRONG IS DIRECTORY
VERBOSE: Adding family.user1.com to list of files for processing  =  @EXPECTING /etc/apache2/vhosts/user1/family.user1.com
VERBOSE: Adding thai.user1.com to list of files for processing = EXPECTNG /etc/apache2/vhosts/user1/thai.user1.com
VERBOSE: Adding valery.user1.com to list of files for processing = EXPECTNG /etc/apache2/vhosts/user1/valery.user1.com
VERBOSE: Adding  to list of files for processing  - NOT SURE WHY THIS IS EMPTY STRING
VERBOSE: Adding /etc/apache2/vhosts/user2: to list of files for processing = WRONG IS DIRECTORY

[varyen]

Just figured out that **/* is not enough here, if I add another folder inside user1 it won't work. :( And it skips files inside folder we're inspecting, prints only files in subfolders.
I think it's better to use something like find /etc/apache2/vhosts/ instead of ls, it prints all files recursively. Just need to skip all directories.

About errors:
ls dir/* output (":" after directory name):

SUBFOLDER:
file1 file2 file3
(empty string here)
SUBFOLDER2:

P.S.

[varyen]

Include lines are:

Include conf.d/
Include vhosts-default/
Include vhosts/

Then it's user under vhosts and separate config for each vhost inside user folder.

That's the way latest ISPManager panel organizes it's vhosts (pretty nice panel btw).

[richardforth]

Your config is broken then. The recommended way to include configs is with something like Include conf.d/*.conf Include vhosts-default/*.conf Include vhosts/*.conf And basically one Include line for each directory full of ".conf" files such as Include vhosts/user1/*.conf Include vhosts/user2/*.conf and so on Apache2buddy.pl has never had that problem before. My advice is to fix your config file, and ensure that your config files have the correct extension (.conf)

[varyen]

Why is it broken? Apache starts and every vhost works.
https://httpd.apache.org/docs/2.4/mod/core.html#include

Shell-style (fnmatch()) wildcard characters can be used in the filename or directory parts of the path to include several files at once, in alphabetical order. In addition, if Include points to a directory, rather than a file, Apache httpd will read all files in that directory and any subdirectory.

It's not recommended only because of "it is easy to accidentally leave temporary files" and apache won't start at all in this case. But anyway, it's a documented feature.
ls output breaks your code if there're more directories under.

As I said before it's managed by web hosting panel (ISPManager) that way. :) It's pretty popular in Russia and Eastern Europe.

[richardforth]

However, including entire directories is not recommended, because it is easy to accidentally leave temporary files in a directory that can cause httpd <https://httpd.apache.org/docs/2.4/programs/httpd.html> to fail. Instead, we encourage you to use the wildcard syntax shown below, to include files that match a particular pattern, such as *.conf, for example.

…

On 15 Dec 2016 15:22, "Varyen" ***@***.***> wrote: Why is it broken? Apache starts and every vhost works. https://httpd.apache.org/docs/2.4/mod/core.html#include Shell-style (fnmatch()) wildcard characters can be used in the filename or directory parts of the path to include several files at once, in alphabetical order. In addition, if Include points to a directory, rather than a file, *Apache httpd will read all files in that directory and any subdirectory*. It's not recommended only because of "it is easy to accidentally leave temporary files" and apache won't start at all in this case. But anyway, it's a *documented feature*. ls output breaks your code if there're more directories under. As I said before it's managed by web hosting panel (ISPManager) that way. :) It's pretty popular in Russia and Eastern Europe. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#29 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFpUVz4zpS83CarQTnxlqO8qiVUe7ZnEks5rIVsrgaJpZM4LN_vg> .

[varyen]

Still, it's possible, it's documented and it's used that way in many cases.

I understand that for some reason this bug have never been discovered (submitted?) for all project time being. So it's your decision whether to consider this case or not. All this long code with globs with dir/* (not dir/*.conf as recommended) could just be find dir/, it's universal solution.
You you just don't have time I can prepare and send you PR with this change. Is it worth it?

[richardforth]

Yes please if you can PR to staging i can test and play with it. Ideally I would say you're commiting a bad practice and should adjust your config files to be more in line with best practice, but I'm also thinking we may be able to address this edge case by simply querying if each glob is a file or a directory and whether or not to include it in the array, since basically the problem is its thinking a directory is a file and failing to open it. I will have more time in the new year, but yes a PR would be appreciated if you think you can solve it a better way.

…

On 15 December 2016 at 15:51, Varyen ***@***.***> wrote: Still, it's possible, it's documented and it's used that way in many cases. I understand that for some reason this bug have never been discovered (submitted?) for all project time being. So it's your decision whether to consider this case or not. All this long code with globs with dir/* (not dir/*.conf as recommended) could just be find dir/, it's universal solution. You you just don't have time I can prepare and send you PR with this change. Is it worth it? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#29 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFpUVxn4c3wZ2ssR0Qh2QvowXilRf4pXks5rIWHpgaJpZM4LN_vg> .

[richardforth]

Yes please if you can PR to staging i can test and play with it. Ideally I would say you're commiting a bad practice and should adjust your config files to be more in line with best practice, but I'm also thinking we may be able to address this edge case by simply querying if each glob is a file or a directory and whether or not to include it in the array, since basically the problem is its thinking a directory is a file and failing to open it. I will have more time in teh new year, but yes a PR woudl be appreciated

…

On 15 December 2016 at 15:51, Varyen ***@***.***> wrote: Still, it's possible, it's documented and it's used that way in many cases. I understand that for some reason this bug have never been discovered (submitted?) for all project time being. So it's your decision whether to consider this case or not. All this long code with globs with dir/* (not dir/*.conf as recommended) could just be find dir/, it's universal solution. You you just don't have time I can prepare and send you PR with this change. Is it worth it? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#29 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFpUVxn4c3wZ2ssR0Qh2QvowXilRf4pXks5rIWHpgaJpZM4LN_vg> .