kjmrknsn / livy-manager

Livy Manager - Web UI for Managing Apache Livy Sessions

Livy Manager

Abstract

Livy Manager is a Web UI for managing Apache Livy sessions.

Issues Livy Manager Tackles with

• It is difficult for non-developer Livy users to monitor or kill their Livy sessions and Spark applications.
  • They have to use an HTTP client tool like curl to call the Livy REST APIs.
  • Additionally, they have to manipulate machines on which a Kerberos client is installed if the Livy service is Kerberized.
  • In some services which use Livy, there's no way to kill a Livy session while a Spark application is running, so non-developer users cannot stop their Spark applications by themselves when they submitted a heavy and long running application accidentally.

Solutions Livy Manager provides

• Non-developer Livy users can see and kill their Livy sessions.
• Optional LDAP authentication and authorization feature is included.
  • Admin users can see and kill all of the Livy sessions.
  • Non-admin users can see and kill only their Livy sessions.
  • This feature works well with Zeppelin with LDAP authentication and the Livy interpreter.

Setup

1. Download an executable binary file from the Releases page and deploy it to your server.
2. Deploy a Livy Manager configuration file to your server. Please see conf/livy-manager.toml.template for its template.
3. Run Livy Manager by executing the following command:

$ /path/to/livy-manager -c /path/to/livy-manager-configuration-file

livy-manager's Issues

Error on running executable: * symbol not found

Hi @kjmrknsn, I tried using the livy-manager on my livy deployment running on a Kubernetes cluster. I'm using Alpine 3.9 and got these:

Error relocating ./livy-manager: sk_value: symbol not found
Error relocating ./livy-manager: OPENSSL_add_all_algorithms_noconf: symbol not found
Error relocating ./livy-manager: SSL_load_error_strings: symbol not found
Error relocating ./livy-manager: CRYPTO_set_id_callback: symbol not found
Error relocating ./livy-manager: CRYPTO_num_locks: symbol not found
Error relocating ./livy-manager: __rawmemchr: symbol not found
Error relocating ./livy-manager: SSLeay: symbol not found
Error relocating ./livy-manager: SSL_CTX_get_ex_new_index: symbol not found
Error relocating ./livy-manager: SSL_get_ex_new_index: symbol not found
Error relocating ./livy-manager: __register_atfork: symbol not found
Error relocating ./livy-manager: CRYPTO_set_locking_callback: symbol not found
Error relocating ./livy-manager: sk_free: symbol not found
Error relocating ./livy-manager: SSLv23_method: symbol not found
Error relocating ./livy-manager: SSL_library_init: symbol not found
Error relocating ./livy-manager: sk_num: symbol not found
Error relocating ./livy-manager: __res_init: symbol not found
Error relocating ./livy-manager: sk_pop: symbol not found
Segmentation fault (core dumped)

I also got errors with libssl and libcrypto packages but managed to get over them by creating symlinks on /usr/lib and /lib. I also have the following packages installed:

• libssl1.1-1.1.1d-r2
• libressl-2.7.5-r0
• libressl-dev-2.7.5-r0

Can you share what OS (and relevant packages) was used in building the application and if you have any leads resolving the issue? Livy manager is super useful and I'd really like to implement it. Appreciate your help, thank you!

apache livy and ajax post requests: Method Not Allowed

Good morning everyone,

can someone help me to see why this jquery code can note create a session to the apache livy? I got these errors :

OPTIONS http://localhost:8998/ 405 (Method Not Allowed) and Failed to load http://localhost:8998/: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8084' is therefore not allowed access. The response had HTTP status code 405.

This is the code I used :

`$.ajax({
type: 'POST',
contentType: "application/json; charset=utf-8",
dataType: 'json',
url: 'http://localhost:8998',
data: {kind: 'spark'},
success: function (resultat) {
document.getElementById(response).innerHTML = resultat;
console.log(resultat);

        }

    });`

Thank you for your help