bluishglc / ranger-emr-cli-installer

This is a powerful cli tool for Apache Ranger and AWS EMR automated installation & integration with OpenLDAP & Windows AD. It supports Open-Source Ranger and EMR-Native Ranger both, supports OpenLDAP & Windows AD both, and works in all AWS regions (also including China regions).

An Installer for Apache Ranger and AWS EMR Automated Installation and Integration with OpenLDAP & Windows AD

This is a powerful cli tool for Apache Ranger and AWS EMR automated installation & integration with OpenLDAP & Windows AD. It supports Open-Source Ranger and EMR-Native Ranger both, supports OpenLDAP & Windows AD both, and works in all AWS regions (also including China regions). Especially, for Open-Source Ranger, it can install ranger on an existing cluster and supports multi-master cluster and single-master cluster both. On each step, this installer always checks connectivity first then decides whether to go for the next steps, this is very helpful to identify network issues or service failure, i.e., when Ranger or OpenLDAP is not up. Finally, the actual installation job is a trial-and-error process. Users always need to try different parameter values to find the one that works in users' environment. The installer allows users to rerun an all-in-one installation anytime without side effects and users can also do a step-by-step run for debugging. The following is a key features summary:

The following is detailed documents for solutions overview and scenario 1,2,3,4. No matter which one you selected, please read solutions overview first so as to get a full picture, then pick one from 4 scenarios according to your environments and requirements.

Solutions Overview

Apache Ranger and AWS EMR Automated Installation Series (1): Solutions Overview

Scenario 1: OpenLDAP + EMR-Native Ranger

Apache Ranger and AWS EMR Automated Installation Series (2): OpenLDAP + EMR-Native Ranger

Scenario 2: Windows AD + EMR-Native Ranger

Apache Ranger and AWS EMR Automated Installation Series (3): Windows AD + EMR-Native Ranger

Scenario 3: OpenLDAP + Open-Source Ranger

Apache Ranger and AWS EMR Automated Installation Series (4): OpenLDAP + Open-Source Ranger

Scenario 4: Windows AD + Open-Source Ranger

Apache Ranger and AWS EMR Automated Installation Series (5): Windows AD + Open-Source Ranger

ranger-emr-cli-installer's Issues

Upload Diagrams of Doc

Solr Permission denied with OpenLDAP + Open-Source Ranger example

Hi Laurence,

Thank you so much for putting together this wonderful repo and the tutorial on DZone. I am following the "Apache Ranger and AWS EMR Automated Installation and Integration Series (4): OpenLDAP + Open-Source Ranger", and I got stuck on step 2.3.3 Install Ranger with permission denied on starting Solr.

Here the log output of what I am seeing now.

======================================================================================    INIT SOLR AS RANGER AUDIT STORE    ======================================================================================

Fri Mar  3 00:13:18 CST 2023|INFO|Solr Ranger Home </opt/solr/ranger_audit_server> exists. Will overwrite configurations
Fri Mar  3 00:13:18 CST 2023|INFO|Configuring standalone instance
Fri Mar  3 00:13:18 CST 2023|INFO|Copying Ranger Audit Server configuration to /opt/solr/ranger_audit_server
Fri Mar  3 00:13:18 CST 2023|INFO|Done configuring Solr for Apache Ranger Audit
Fri Mar  3 00:13:18 CST 2023|INFO|Solr HOME for Ranger Audit is /opt/solr/ranger_audit_server
Fri Mar  3 00:13:18 CST 2023|INFO|Data folder for Audit logs is /opt/solr/ranger_audit_server/data
Fri Mar  3 00:13:18 CST 2023|INFO|To start Solr run /opt/solr/ranger_audit_server/scripts/start_solr.sh
Fri Mar  3 00:13:18 CST 2023|INFO|To stop Solr run /opt/solr/ranger_audit_server/scripts/stop_solr.sh
Fri Mar  3 00:13:18 CST 2023|INFO|After starting Solr for RangerAudit, it will listen at 8983. E.g http://ip-____.ec2.internal:8983
Fri Mar  3 00:13:18 CST 2023|INFO|Configure Ranger to use the following URL http://ip-____.ec2.internal:8983/solr/ranger_audits
Fri Mar  3 00:13:18 CST 2023|INFO| ** NOTE: If Solr is Secured then solrclient JAAS configuration has to be added to Ranger Admin and Ranger Plugin properties
Fri Mar  3 00:13:18 CST 2023|INFO| ** Refer documentation on how to configure Ranger for audit to Secure Solr
########## Done ###################
Created file /opt/solr/ranger_audit_server/install_notes.txt with instructions to start and stop
###################################
No process found for Solr node running on port 8983
/opt/solr/bin/solr: line 2246: /opt/solr/bin/solr-8983.pid: Permission denied
Warning: Available entropy is low. As a result, use of the UUIDField, SSL, or any other features that require
RNG might not work properly. To check for the amount of available entropy, use 'cat /proc/sys/kernel/random/entropy_avail'.

Waiting up to 180 seconds to see Solr running on port 8983 [\]  Still not seeing Solr listening on 8983 after 180 seconds!
tail: cannot open ‘/var/log/solr/ranger_audits/solr.log’ for reading: No such file or directory
    
============================================================================    TEST SOLR CONNECTIVITY    ============================================================================

I was running the following shell command on my Ranger server EC2 instance.

sudo sh ./ranger-emr-cli-installer/bin/setup.sh install-ranger \
    --region "$REGION" \
    --access-key-id "$ACCESS_KEY_ID" \
    --secret-access-key "$SECRET_ACCESS_KEY" \
    --solution 'open-source' \
    --auth-provider 'openldap' \
    --openldap-host "$OPENLDAP_HOST" \
    --openldap-base-dn 'dc=example,dc=com' \
    --ranger-bind-dn 'cn=ranger,ou=services,dc=example,dc=com' \
    --ranger-bind-password 'Admin1234!' \
    --openldap-user-dn-pattern 'uid={0},ou=users,dc=example,dc=com' \
    --openldap-group-search-filter '(member=uid={0},ou=users,dc=example,dc=com)' \
    --openldap-user-object-class 'inetOrgPerson'

My shell script and linux permissions are rusty, so any insights or recommendation on how to solve this problem will be great.

Thank you,
Heng