berekuk / ubic Goto Github PK

Polymorphic service manager.

Home Page: https://metacpan.org/release/Ubic

Perl 99.78% Shell 0.22%

ubic's Introduction

Ubic

My priorities have changed and I can't promise the proper maintainence of this software anymore. The codebase is solid enough to be used without any significant issues, though. Feel free to ask me for a commit bit if you want to fix something or take over completely.

-- @berekuk

Ubic is a polymorphic service manager.

"Polymorphic" means that Ubic can use various pluggable backends for managing services, for configuring services and even for describing a list of all services. Don't panic, it offers easy-to-use default solutions for the common tasks out-of-the-box too!

1 minute intro

Put this code in file /etc/ubic/service/example.ini:

[options]
bin = sleep 100

Start it:

$ ubic start example
Starting example... started (pid 41209)

Check its status:

$ ubic status
example running (pid 41209)
ubic
    ubic.ping   off
    ubic.update off
    ubic.watchdog   running (pid 93226)

Or:

$ ubic status example
example running (pid 41209)

Now let's see how watchdog works by killing the process (don't forget to change pid with the pid you got in status command above):

$ kill 41209

$ ubic status example
example not running

$ ubic-watchdog
[Thu May 26 20:20:54 2011]  example is broken, restarting

You don't have to run ubic-watchdog manually; it will do its work in background in a minute.

Read Ubic::Manual::Intro and Ubic::Manual::Overview for more.

Installation

Run 'cpan -i Ubic && ubic-admin setup' to install Ubic.

We also provide .deb packages for Debian/Ubuntu. Latest .deb package can be downloaded from ppa:berekuk/ubic (see https://launchpad.net/~berekuk/+archive/ubic for details).

Debian package build can be reproduced with this command: dzil build && cd Ubic* && cp -r ../debian . && debuild

Rpm package can be created with this command: rpmbuild -ba redhat/perl-Ubic.spec

If you'll write an ebuild for Gentoo, please contribute it back :)

Documentation

After installing, you can find documentation for this module using perldoc and man commands.

man ubic
perldoc Ubic

You can also look for information at:

Support

There is also a low-volume mailing list is [email protected]. Send an empty message to [email protected] to subscribe.

Copyright and licence

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See http://www.perl.com/perl/misc/Artistic.html

ubic's People

Contributors

Stargazers

Watchers

ubic's Issues

Installation through CPAN fails due to Win32 dependencies

Win32-pwent-0.100 is marked as a dependency of Ubic 1.25
When installing via cpan / cpanm on a CentOS5 VPS, installation fails as this dependency doesn't work on this OS.

That module requires:
Win32-Registry-0.10
Win32-NetAdmin-0.11
Win32-0.44
Win32

Non of them is relevant on non-windows machines.

Manual installation of Ubic through the Makefile.PL works well enough, but it would be nice it would work through CPAN/M.

Compatibility with redhat init scripts

In Ubic::Run it assumes that $0 will be debian style (/etc/init.d/service) and on RedHat machines it fails because $0 can be /etc/rc3.d/S98service.

Here is a tentative change to handle it:

# before:
# my ($name) = $0 =~ m{^/etc/init\.d/(.+)$} or die "Strange $0";

# after:
my $name;
if ( $0 =~ qr{^/etc/init\.d/(.+)$} ) {
    $name = $1;
} elsif ( $0 =~ qr{^/etc/rc\d\.d/(?:K|S)\d+(.+)$} ) {
    $name = $1;
}

test error with BSD::Resource v1.28: RLIMIT_..., not "RLIMIT_..."

I had 1 fail out of 365 tests. I upgraded BSD::Resource to v1.2904 and everything installed ok.
Using Ubic-v1.45

#   Failed test 'ulimit died (Failed to create daemon: 'pidfile written
# Error: setrlimit: use RLIMIT_..., not "RLIMIT_..." at lib/Ubic/Service/SimpleDaemon.pm line 90.
#
# ' at lib/Ubic/Daemon.pm line 354.)'
#   at t/simple-daemon.t line 185.
#   (in main->ulimit)
# Looks like you failed 1 test of 27.
t/simple-daemon.t ..............

The same error is produced with:

perl -MBSD::Resource -Mwarnings -E 'BSD::Resource::setrlimit("RLIMIT_NOFILE", 100, 100)'
setrlimit: use RLIMIT_..., not "RLIMIT_..." at -e line 1

Perhaps make BSD::Resource version >= 1.2904 a requirement (or skip this test).

Refactor service/daemon code to be usable outside of Ubic framework

One possible plan for refactoring: https://gist.github.com/b7a2ab5ae45df5847682

Alternatively, we can move all Ubic::Daemon code to Daemon::Control as patches and then port Ubic::Daemon to use Daemon::Control.

ubic-install script

Ubic setup process should be automated for non-Debian installations through setup script (which would be bundled in CPAN distribution).
It should create log dir, service dir and ubic dir with appropriate permssions, setup crontab and logrotate and symlink /usr/bin/ubic from /etc/init.d.

Depending on Test::Class

Hi.

I notice that you module is depending on Test::Class.
Test:Class have a high test failure rate, and this means that a lot of the people that want to install you module won't be able to.

Worse, you can't even see this failures in your CPAN testers reports, as dependency fail is not reported.

Can you please fix that?

Thanks,
Shmuel.

Ubic::Daemon - custom graceful stop signal

Ubic::Daemon can stop daemons gracefully by sending them SIGTERM first, and SIGKILL later.
We need to make SIGTERM customizable, some daemons perform graceful stop using other signals.
(Request from https://github.com/diegok).

fix autocompletion behaviour

mongodb01.intrafeeds:# ubic status
mongodb
mongodb.storage27017 running
mongodb01.intrafeeds:# ubic status mongodb (tab-tab)
mongodb

make ubic-guardian optional

Ubic::Daemon should create ubic-guardian process only if ubic_log is set. In other cases, it's unnecessary and just confuses people.

ubic-watchdog should recheck services less frequently after several failures

ubic-watchdog currently rechecks and trying to revive every broken service every minute.
We should remember history of checks for every broken service, and don't try to revive permanently broken services so often.

It would be nice for this feature to be configurable both globally and per-service.

Upload to launchpad.net for Saucy Salamander

The PPA on https://launchpad.net/~berekuk/+archive/ubic doesn't work for Saucy Salamander.

ulimit support

Many people want to set ulimits for their services.
We need to provide some way to set them either in SimpleDaemon or with some wrapper/role for SimpleDaemon

Example of a possible wrapper syntax:

Ubic::Service::WithUlimits->new(
    Ubic::Service::SimpleDaemon->new({ bin => "sleep 1000" }),
    {
        nofile => 4096,
        cpu => 3600,
    }
);

Ulimits themselves can be set with BSD::Resource.

Credentials should work on OSX when user's group is unnamed

Here's what id command shows on my LDAP-enabled Macbook:

uid=123123123(mmcleric) gid=444444444 groups=444444444,402(com.apple.sharepoint.group.1),12(everyone),33(_appstore),62(netaccounts),80(admin),98(_lpadmin),100(_lpoperator),204(_developer),401(com.apple.access_screensharing)

And here's how tests are failing:

mmcleric@macbook:~/coding/Ubic[master]$ prove -Ilib -v t/cmd.t 
t/cmd.t .. 
1..8
Use of uninitialized value $group in string ne at lib/Ubic/Credentials/OS/MacOSX.pm line 171.
Use of uninitialized value $current_group in string ne at lib/Ubic/Credentials/OS/MacOSX.pm line 171.
Use of uninitialized value $group in string ne at lib/Ubic/Credentials/OS/MacOSX.pm line 171.
Use of uninitialized value $current_group in string ne at lib/Ubic/Credentials/OS/MacOSX.pm line 171.
Use of uninitialized value $group in string ne at lib/Ubic/Credentials/OS/MacOSX.pm line 171.
Use of uninitialized value $current_group in string ne at lib/Ubic/Credentials/OS/MacOSX.pm line 171.
Use of uninitialized value $group in getgrnam at lib/Ubic/Credentials/OS/MacOSX.pm line 119.
Use of uninitialized value $group in getgrnam at lib/Ubic/Credentials/OS/MacOSX.pm line 119.
not ok 1 - Ubic::Cmd logged something on start

#   Failed test 'Ubic::Cmd logged something on start'
#   at t/cmd.t line 25.
#                   'Starting sleeping-daemon... Failed to set effective gid to 0: Operation not permitted at lib/Ubic/Credentials/OS/MacOSX.pm line 247.
# '
#     doesn't match '(?^:^Starting\ sleeping\-daemon\.\.\.\ started\ \(pid\ \d+\)\n$)'
Use of uninitialized value $group in string ne at lib/Ubic/Credentials/OS/MacOSX.pm line 171.
Use of uninitialized value $current_group in string ne at lib/Ubic/Credentials/OS/MacOSX.pm line 171.
Use of uninitialized value $group in getgrnam at lib/Ubic/Credentials/OS/MacOSX.pm line 119.
Use of uninitialized value $group in getgrnam at lib/Ubic/Credentials/OS/MacOSX.pm line 119.
Failed to set effective gid to 0: Operation not permitted at lib/Ubic/Credentials/OS/MacOSX.pm line 247.
# Looks like you planned 8 tests but ran 1.
# Looks like you failed 1 test of 1 run.
# Looks like your test exited with 255 just after 1.
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 8/8 subtests 

Test Summary Report
-------------------
t/cmd.t (Wstat: 65280 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 8 tests but ran 1.
Files=1, Tests=1,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.09 cusr  0.02 csys =  0.14 CPU)
Result: FAIL

So, Credentials should handle nameless groups somehow.
Maybe it's also time to refactor all credentials code into the separate distribution (Credentials::Any?)

SimpleDaemon - daemon_user option

User semantics in ubic isn't always what our users want.
Ubic drops privileges before doing anything with service — $service->start() gets called with non-root user, for example.
This feature allows non-root users operate services themselves — if service's user is www-data, then www-data user can start and stop this service.

But some people want the user to remain root and do some stuff using root before starting a daemon in their start method.
And they want to inherit from Ubic::Service::SimpleDaemon at the same time!
So, we need to implement daemon_user option in Ubic::Service::SimpleDaemon for them.

Windows support

See https://github.com/berekuk/Ubic/wiki/Windows for the details.

Test::TCP missing from dependencies

The tests require Test::TCP, but this is not listed as a dependency.

Ubic installation fails when installing under root user

This happened to me on two servers, where I installed Ubic via cpan when under the root user.

test 7 in daemon.t fails as under the root user, it is possible to create "forbidden.log" in "/".

Other tests fail in ubic.t for the same reason.

Installation with tests fail

Trying to install from CPAN (using cpanm), the installation fails because of tests.

This isn't because tests are failing but because the watchdog.t tests are skipping (no reason given) and so the script isn't running any tests (without SKIP), so prove fails.

Here's a hopefully helpful snippet:
t/service.t ............... ok
t/simple-daemon.t ......... ok
t/ubic.t .................. ok
t/watchdog.t .............. # No tests run!
t/watchdog.t .............. skipped: (no reason given)

Test Summary Report
-------------------
t/watchdog.t            (Wstat: 65280 Tests: 0 Failed: 0)
  Non-zero exit status: 255
Files=18, Tests=168, 42 wallclock secs ( 0.10 usr  0.73 sys +  6.15 cusr  9.04 csys = 16.02 CPU)
Result: FAIL
Failed 1/18 test programs. 0/168 subtests failed.
make: *** [test_dynamic] Error 255

implement IPv6 support in ubic.ping

subj.

Implement ubic --fix-permissions command

"ubic --fix-permissions" must re-chown all files in data, status and lock dirs to correct users.

Services with dots or maybe other special symbols are not found

I have a website called showmetheco.de and obviously I've created a showmetheco.de service. But when I tried to start it I got:

Service 'showmetheco.de' not found

'ubic fix-permissions' command

A lot of people are asking why their service breaks when they change its user/group parameters.
The reason is explained in FAQ, but they're still asking, and we can improve their experience.

Ubic->install_service

We should implement Ubic->install_service($name => $content) method, so that those who install services from perl won't have to create files manually.
(Feature request by @alnewkirk)

Place Ubic in the official Ubuntu repository

Ubic is cool software, it belongs there.

Would you take a pull request which allow comments in .json service files?

I would like to have support for:

# some comment
{
  "module": "...",
  "options": {}
}

Are you interested?

my $content = do { local $/; <$fh> };
close $fh or die "Can't close $file: $!";
$content =~ $body =~ s/^#(.*)$//gm;

Typo in documentation

In http://search.cpan.org/~mmcleric/Ubic-1.52/lib/Ubic/Manual/FAQ.pod#Can_I_use_ubic_with_perlbrew?_How? it says "This especially includes the crontab which entry ubic installs for you." - The correct way to express this would be: "This especially includes the crontab entry which ubic installs for you."

"Can't read daemon cmdline" shouldn't be fatal

From Ubic::Daemon:

    # TODO - wrap in eval and return undef if pid2cmd fails?
    my $daemon_cmd = $OS->pid2cmd($piddata->{daemon});

In case of pidfile collisions (which generally gets resolved with guid checking), it's possible that new process reusing old daemon's pid is "weird" somehow and we can't read its cmdline.
check_daemon shouldn't fail in this case, but print unknown daemon or something in place of daemon cmd.

Improve cached/non-cached status checks for non-root users

Currently, you have to be root to be able to check service status, even if service is coded in such fashion that this isn't really necessary.
'ubic status' command hints about this:

Not a root, printing cached statuses

There are several options how to improve current behavior:

allow non-root status command to run $service->status() method, and recommend service authors to make their checks simple enough to run from every user (but this will not be possible in all cases - imagine a service binded on local unix socket with chmod 600); cached status can be keeped as well, via "ubic status --cached", for example;
let service authors to specify whether their status checks require root access, and fallback to a cached status only when it's necessary;
implement some daemon which would run status checks from root point of view; this role could be performed by ubic-ping, but ubic-ping is currently single-threaded, and i'm unsure if additional dependencies to full-blown http frameworks (Plack or something else) are worth it.

Unadvertised crontab entry made

Discovered an entry in my crontab made by Ubic. There's nothing in your documentation that I could find about this only that the watchdog script should be run every minute.

Problem running ubic-admin setup

I get this error trying to run the "setup", right after I approve all the questions:

Can't locate object method "flush" via package "IO::Handle" at /usr/lib/perl5/site_perl/5.8.8/Ubic/AtomicFile.pm line 20, <STDIN> line 9.

I'm trying to install Ubic 1.30 from CPAN using cpanm. I have IO::Handle 1.25 (from IO) which is the latest.
I also reinstalled it and tried again. Still same thing.

Fix FreeBSD credentials

Ubic::Credentials::POSIX doesn't work on FreeBSD. It has something to do with the order of setting effective and real uid (and also gids).

I tried to fix it in 1.39, but turned out perl activates the tainted mode in this case, so since 1.43 release we're back at "works on linux and broken on freebsd".
It works fine on MacOSX, though.

--parallel parameter for start/stop commands

ubic start --parallel A B C should start all services in parallel.

Watchdog doesn't recognize "running (pid xxx)", shows fail

Services return "running (pid xxx)" but watchdog doesn't check for it.

In t/simple-daemon, there's the following line that illustrates this point:
like($service->status, qr/^running (pid \d+)$/, 'start works');

However, when using ubic-watchdog, I get the following:

# ubic-watchdog 
[Tue Sep 14 11:40:06 2010]  <service> is broken, restarting
[Tue Sep 14 11:40:06 2010]  <service> started, but status is still 'running (pid 3336)'

Am I missing something?

running the tests creates a cron job

this is a very serious bug

ubic.ping: http-code checks by service name

ubic.ping supports following urls:

  /ping (self-check)
  /status/port/777 (check by port, return status in body)
  /status/service/foo.bar ( check by name, return status in body)
  /noc/777 (check by name, return status in http code)

This is inconsistent and we need to be able to check status by name and represent result with http code.
Also, none of these are documented (except in tutorials)!

wrong name of ubic.watchdog service in init-script


host# apt-cache policy ubic| head -2
ubic:
  Installed: 1.31

host# ls -l /etc/rc*.d/*20ubic-watchdog
lrwxrwxrwx 1 root root 23 2011-07-05 18:21 /etc/rc0.d/K20ubic-watchdog -> ../init.d/ubic-watchdog
lrwxrwxrwx 1 root root 23 2011-07-05 18:21 /etc/rc1.d/K20ubic-watchdog -> ../init.d/ubic-watchdog
lrwxrwxrwx 1 root root 23 2011-07-05 18:21 /etc/rc2.d/S20ubic-watchdog -> ../init.d/ubic-watchdog
lrwxrwxrwx 1 root root 23 2011-07-05 18:21 /etc/rc3.d/S20ubic-watchdog -> ../init.d/ubic-watchdog
lrwxrwxrwx 1 root root 23 2011-07-05 18:21 /etc/rc4.d/S20ubic-watchdog -> ../init.d/ubic-watchdog
lrwxrwxrwx 1 root root 23 2011-07-05 18:21 /etc/rc5.d/S20ubic-watchdog -> ../init.d/ubic-watchdog
lrwxrwxrwx 1 root root 23 2011-07-05 18:21 /etc/rc6.d/K20ubic-watchdog -> ../init.d/ubic-watchdog

host# cat /etc/init.d/ubic-watchdog 
#!/usr/bin/perl

### BEGIN INIT INFO
# Provides:          ubic-watchdog
# Required-Start:    $all
# Required-Stop:
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: ubic watchdog service
# Description:       ubic.watchdog brings other ubic services to their expected state
### END INIT INFO

use Ubic::Run;

host# cat /etc/ubic/service/ubic/watchdog 
# vim: ft=perl

use Ubic::Service::SimpleDaemon;

Ubic::Service::SimpleDaemon->new(
    bin => 'ubic-periodic --period=60 --stdout=/var/log/ubic/watchdog.log --stderr=/var/log/ubic/watchdog.err.log ubic-watchdog',
);

And as a result:

host# /etc/init.d/ubic-watchdog start
Service 'ubic-watchdog' not found

SimpleDaemon - implement reload_signal option

reload_signal option should send given signal to a daemon on ubic reload.

Options for commands

It would be cool to be able to do:

  ubic dosomething --foo xxx --bar yyy my.service

Ubic should pass --foo xxx --bar yyy as a list or as a hash { foo => 'xxx', bar => 'yyy' } to service's dosomething method.

This feature would be useful for some built-in commands too. Probably.

Crashed service reported as running for non-root user

[root@server ~]# /etc/init.d/service status
service pidfile /var/lib/ubic/simple-daemon/pid/service removed - daemon with cached pid 25348 not found
not running
[root@server ~]# su - user
[user@server ~]$ /etc/init.d/service status
Not a root, printing cached statuses
service running
[user@server ~]$ echo $?
0

This shows a status of running and a good exit code when the service is actually off.
A user should be able to check if the cached PID is running, but doesn't.

Solaris support

Tests are failing on SunOS/Solaris - http://www.cpantesters.org/cpan/report/4782734c-3fa9-11e1-9d6f-f6dbfa7543f5.
It looks like a problem with ps -p $pid output (see Ubic::Daemon::OS::POSIX).
Need to find solaris user or install it in virtual machine and investigate.

Option to change directory when starting

As discussed on IRC, I think it would be useful for Ubic::Service::SimpleDaemon to provide an option to change directory before starting a daemon... for instance:

my $service = Ubic::Service::SimpleDaemon->new(
    bin => "myapp",
    cwd => "/var/myapp",
);

... or something similar.

Exit code 5 on install

I'm running on ubuntu (12.04.3) and have tried this via apt-get and also via cpanm to install ubic. I get the following after running sudo ubic-admin setup (with standard parameters)

Looks like ubic is already configured, do you want to reconfigure? [y/N] y

Ubic can be installed either in your home dir or into standard system paths (/etc, /var).
You need to be root to install it into system.
Would you like to configure as much as possible automatically? [Y/n] 
Installing dirs...
Installing ubic.ping service...
Installing ubic.watchdog service...
Installing ubic.update service...
Installing cron jobs...
Installing /etc/ubic/ubic.cfg...
Starting ubic.watchdog...
Service 'ubic.watchdog' not found
exit code 5 at /usr/share/perl5/Ubic/Admin/Setup.pm line 113.

I first installed it locally and was able to get it running, but can't running as root, although it appears that all of the files are installing in the correct places and setting the correct permissions.

Test failures due to hash randomisation in perl 5.17.6

Since bleadperl v5.17.5-518-g7dc8663 your tests are failing frequently.
That commit introduced hash key randomization and it seems at least the test

t/setup.t

is hit by that. Find a sample fail report at:

http://www.cpantesters.org/cpan/report/fc557bbe-3159-11e2-ba43-888fa290f8f5

You may have to run the test several times until the randomization causes a fail.

HTH&&Thanks&&Regards,

Improve perlbrew and ~/perl5 handling

If you install ubic to ~/ instead of system paths, your services won't start on boot, since current starting method is:

start ubic.watchdog from crontab by calling ubic-watchdog ubic.watchdog;
wait until ubic.watchdog starts everything else.

If ubic-watchdog script is located in ~somebody/perl5/bin/, or PERL5LIB is not set properly, then crontab will silently fail. This is bad.
Ideas on what we can do about it:

check for PATH and PERL5LIB in ubic-admin setup and print some kind of warning;
ask the user if he wants to add PATH and PERL5LIB to his crontab, and generate crontab accordingly;
or think about some other way to start services on boot.

ulimits strange behaviour

Failed to create daemon: 'Error: Failed to set RLIMIT_NOFILE=8192 ulimit at /usr/share/perl5/Ubic/Service/SimpleDaemon.pm line 99.

pidfile written
' at /usr/share/perl5/Ubic/Daemon.pm line 413.

on smaller values, such as 4096 or 1024 it's all ok.

ubic=1.49-1, Ubuntu 12.04 LTS