consolidation / site-process Goto Github PK

Describe the bug or behavior
When a Drupal site is set up to log to stdout using monolog. When running drush updb, the output in ProcessBase.php:170 includes log messages as well as the process output as the $output value. This then causes json_decode() to return NULL at line 182 as it is invalid json.

This happens when running drush updb and an update hooks triggers code that logs messages.

To Reproduce
Enable monolog module and set to use a stdout handler:

  monolog.handler.stdout:
    class: Monolog\Handler\StreamHandler
    arguments: ['php://stdout']

Set it as the default.
Run drush updb where an update hook triggers a log message (e.g. queue_mail calling module install)

Expected behavior
The log messages are filtered.

Actual behavior

In ProcessBase.php line 184:

  Unable to decode output into JSON.

Workaround
The workaround was to disable logging to stdout for CLI operations, however, we can no longer get the logs from drush cron.

System Configuration

Additional information

Describe the bug or behavior
Running drush drupal:directory always results in an exception:

E:\Web\Ambient.Impact\Web (4.x -> origin) ([email protected])
λ drush drupal:directory ambientimpact_base

In ProcessBase.php line 172:

  Unable to decode output into JSON: Syntax error

  {
      "%paths": {
          "%root": "E:\Web\Ambient.Impact\Web/drupal",
          "%site": "sites/default",
          "%modules": "sites/all/modules",
          "%themes": "sites/all/themes",
          "%config-sync": "E:\Web\Ambient.Impact\Web/drupal/../drupal_config/sync",
          "%files": "sites/default/files",
          "%temp": "C:\Users\i\AppData\Local\Temp",
          "%private": "E:\Web\Ambient.Impact\Web/drupal/../drupal_private_files",
          "%ambientimpact_base": "E:\Web\Ambient.Impact\Web/drupal/themes/ambientimpact/ambientimpact_base"
      }
  }

To Reproduce
See above.

Expected behavior
Not a horrible error. :P

Actual behavior
See output above.

Workaround
After a bit of digging and trial and error, I discovered that commenting out line 162 in ProcessBase.php fixes the issue for some reason, i.e. this line:

$output = preg_replace('#\\\\{2}#', '\\', $output);

Then running the same Drush command completes successfully:

E:\Web\Ambient.Impact\Web (4.x -> origin) ([email protected])
λ drush drupal:directory ambientimpact_base
E:/Web/Ambient.Impact/Web/drupal/themes/ambientimpact/ambientimpact_base

System Configuration

Additional information
Possible related, posted comment: drush-ops/drush#4281

Describe the bug or behavior
When this method is faced with [] as output, it strips the brackets and then gives an exception for invalid JSON. But I'm pretty sure that [] is valid JSON.

System Configuration

Opening this in continuation of drush-ops/drush#4199.

Describe the bug or behavior
On Windows, attempting a drush @alias status (or any command really; @alias is a remote site alias) results in the process running indefinitely with no visible output most of the time. Occasionally, a single line of output appears but the process still doesn't exit.

To Reproduce
drush @alias status

Expected behavior
Process should display output and exit.

Actual behavior
Process doesn't exit, usually no output.

Workaround
Running with -vvv and copying/pasting the ssh command that's generated.

System Configuration

Additional information
After a lot more digging in addition to the linked Drush issue, the exact location that execution is getting stuck is in Symfony\Component\Process\Pipes\AbstractPipes::write() on line 132:

$written = fwrite($stdin, $this->inputBuffer);

On Windows, Symfony\Component\Process\Pipes\WindowsPipes is used, which extends AbstractPipes and uses temporary files as a workaround for various Windows PHP issues. If you open the temporary files that Symfony generates, the remote site output is all there.

With that in mind, it would seem that this would be a Symfony issue, but then if you bypass Drush, creating and running a PHP file (either via drush php:script or directly) like so:

<?php

require '../vendor/autoload.php';

use Symfony\Component\Process\Process;

$process = new Process(<ssh command>);

$process->run();

print $process->getOutput();

the ssh command correctly executes and exits. Replace <ssh command> with the SSH command generated by Drush. This seems to indicate that there's something in Drush or site-process that's doing something a bit differently, possibly in RealtimeOutputHandler?

File system and stream stuff are outside of my area of expertise, so that's as far as I've gotten. I'm not really sure where to go from here, so I'll leave it here for now in case someone else wants to continue looking into this.

Is your feature request related to a problem? Please describe.

We're encountering an infinite loop when running Drush commands on remote site aliases when using Drall: jigarius/drall#77

Describe the solution you'd like

The culprit is this line in Symfony\Component\Process\Process::wait():

$running = '\\' === \DIRECTORY_SEPARATOR ? $this->isRunning() : $this->processPipes->areOpen();

The issue was identified and a solution was suggested in 2020 here: symfony/symfony#21580

To prevent waiting for this upstream change, I propose overriding the checkTimeout() method in SiteProcess:

public function checkTimeout()
    {
        $this->updateStatus(false);
        parent::checkTimeout();
    }

This function is called in the problematic part of the wait() method. updateStatus()is the method that isRunning() uses to determine if the process is still running or not.

Describe alternatives you've considered

• I tried to find a way to modify Drush's injected services, so we can change process.manager to return a class that extends SiteProcess with the code change above. Ideally, this would be done via a command-line option that could be added by Drall.

• readPipes() would be an even better fit, but it's private. 😞

This is a support request.

I know a transport manager was added in #20 but I can't figure out a way to add more transports at runtime (per site). I was looking for something I could add with configuration on a per-site basis or by including a package in composer.json or by some other such mechanism. Something like "contrib modules".

I'm trying to add different ways to invoke drush in containers which are not docker-compose (or some other wrapper). For example, I want to add a site alias that runs drush in an OpenShift cluster for a specific project.

If adding a Transport in this project is the only way, how about adding a custom Transport that lets you wrap drush calls to a different command. So, drush @stage status could become oc rsh ... drush status. This would be very similar to the existing Docker Transport but would work for various wrappers such as oc, kubectl, or even ddev and lando.

I know there's a drush-alias setting but last I checked, there was a limitation that it only runs on a remote-host. That wouldn't make sense in these cases.

Is your feature request related to a problem? Please describe.

For convenience of our customers why heavily depend on drush, we would like to see users be able to utilize their scripts when they start to host on Skpr.

Describe the solution you'd like

We'd like a plugin implementation much like is presented here to map out the command for Skpr.

Describe alternatives you've considered

Simply recommending to not use drush is currently the only thing we can pass on.

Additional context

I have already completed the work, but I would love to know if you're open to such a change before I open a PR.

You can view what we have available on the Skpr command line tool here

Describe the bug or behavior
While installing Drupal from the command line using Drush 9.6.0 the installation fails during the translation import phase. We tracked this issue down to the second preg_replace in ProcessBase::removeNonJsonJunk(). The failure is caused by the fact that the json string contained in $data is about 4 times longer than the pcre.backtrack_limit setting allows.

To Reproduce
set a low pcre.backtrack_limit and input a larger string then pcre.backtrack_limit into $data.
Run the method ProcessBase::removeNonJsonJunk()

Expected behavior
ProcessBase::removeNonJsonJunk() should not return NULL when valid json of any size is passed in.

Actual behavior
ProcessBase::removeNonJsonJunk() returns NULL when a large, but valid json string is entered.

Workaround
set an enormous pcre.backtrack_limit in php.ini

System Configuration

Additional information
I will provide a PR that resolves this issue shortly.

Is your feature request related to a problem? Please describe.
The \InvalidArgumentException 'Unable to decode output into JSON.' is not very helpful.
I have updated drush to 9.6.0 and Drupal core 8.7.0-alpha1 and running the drush updatedb -y exists with following error.

In ProcessBase.php line 184:
                                      
  Unable to decode output into JSON.  
                                      

This is not very helpful. I have to turn the debugger on and put a breakpoint at ProcessBase.php line 181 to see the actual error.

[2019-03-19 12:46:26] menu.ERROR: Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'base_table.revision_id' in 'field list': SELECT base_table.revision_id AS revision_id, base_table.id AS id FROM {menu_link_content} base_table INNER JOIN {menu_link_content_data} menu_link_content_data ON menu_link_content_data.id = base_table.id WHERE menu_link_content_data.rediscover = :db_condition_placeholder_0; Array (     [:db_condition_placeholder_0] => 1 )  in Drupal\menu_link_content\Plugin\Deriver\MenuLinkContentDeriver->getDerivativeDefinitions() (line 63 of /data/app/core/modules/menu_link_content/src/Plugin/Deriver/MenuLinkContentDeriver.php). {"referer":"","ip":"127.0.0.1","request_uri":"http://127.0.0.1/","uid":0,"user":""}
{
    "0": {
        "system": {
            "8701": {
                "results": {
                    "query": null,
                    "success": true
                }
            },
            "add_expand_all_items_key_in_system_menu_block": {
                "results": {
                    "query": null,
                    "success": true
                }
            }
        },
        "content_moderation": {
            "8700": {
                "results": {
                    "query": null,
                    "success": true
                }
            },
            "entity_display_dependencies": {
                "results": {
                    "query": null,
                    "success": true
                }
            },
            "set_default_moderation_state": {
                "results": {
                    "query": null,
                    "success": true
                }
            },
            "set_views_filter_latest_translation_affected_revision": {
                "results": {
                    "query": null,
                    "success": true
                }
            }
        },
        "file": {
            "8700": {
                "results": {
                    "query": null,
                    "success": true
                }
            }
        },
        "media": {
            "8700": {
                "results": {
                    "query": null,
                    "success": true
                }
            },
            "enable_standalone_url": {
                "results": {
                    "query": null,
                    "success": true
                }
            }
        },
        "node": {
            "8700": {
                "results": {
                    "query": null,
                    "success": true
                }
            }
        },
        "taxonomy": {
            "8701": {
                "results": {
                    "query": null,
                    "success": true
                }
            },
            "make_taxonomy_term_revisionable": {
                "results": {
                    "query": "Taxonomy terms have been converted to be revisionable.",
                    "success": true
                }
            },
            "remove_hierarchy_from_vocabularies": {
                "results": {
                    "query": null,
                    "success": true
                }
            }
        },
        "menu_link_content": {
            "make_menu_link_content_revisionable": {
                "results": {
                    "query": "Custom menu links have been converted to be revisionable.",
                    "success": true
                }
            }
        },
        "views": {
            "exposed_filter_blocks_label_display": {
                "results": {
                    "query": null,
                    "success": true
                }
            },
            "make_placeholders_translatable": {
                "results": {
                    "query": null,
                    "success": true
                }
            }
        }
    },
    "drush_batch_process_finished": true
}

Describe the solution you'd like
Throw helpful message with the appropriate info from $output variable.
Describe alternatives you've considered
Using debugger and putting the breakpoint before the exception.

Additional context
This is an exsiting bug in D8 and it is not yet fixed so it can be reproduced now.

• Drush version 9.6.0.
• Update Drupal 8.6.12 site to 8.7.0-alpha1.
• Run drush updatedb -y