mapr-emea / mapr-ansible Goto Github PK

implement setup for mapr-tez

If you deploy a secured cluster (or add a new node to a secured cluster) some files (cldb.key, maprserverticket, ssl_keystore, ssl_truststore from /opt/mapr/conf) are copied from the first cldb node to the ansible host. From here those files are deployed to the other nodes in order to configure them properly. After you deployed the keys and configured the nodes those keys should be removed from the ansible host.

The playbook site-cluster failed on the following error:
fatal: [mapr03]: FAILED! => {"changed": true, "cmd": "hadoop fs -mkdir -p /apps", "delta": "0:00:04.294720", "end": "2018-04-01 01:46:55.671684", "msg": "non-zero return code", "rc": 1, "start": "2018-04-01 01:46:51.376964", "stderr": "mkdir: failure to login: Unable to obtain MapR credentials"
Please help me to fix it

Spark Shuffle and Sort for YARN

Add --isvm option and set volumes to replication factor 1 on single node

Implement Custom SSL certificates.

Make log aggregation configurable

Implement HTTPS for Oozie

Feature Request for this Ansible repo to support different Kerberos principals + keytabs for different components such as Yarn, Drill, Hive, HBase, MapR-DB etc rather than just 1 cluster-wide keytab.

Make max container size confgurable for YARN

Hue can only handle one endpoint for moste components.
e.g. httpfs or hive meta store.
when more then one component is installed, the hue setup breaks.

MapR 6.1 MEP 6 add mapr-nfsv4 server

MapR 6.1 MEP 6 add mapr-kafka-ksql

Verify SSL for Spark History Server

MapR 6.1 MEP 6 add mapr-mastgateway

security key files are not download when running site-cluster again and no restart happens

Make it configurable if ansible_fqdn or IP address from inventory file should be taken for configure.sh

my.cnf cannot be found in ubuntu 16.04. (bind-address should be commented in /etc/mysql/mysql.conf.d/mysqld.cnf)

Would it be possible to clarify the License for this project?

Sometimes Hue database is not created in MySQL on Redhat 7.x
Only when ran from Ansible script. Manual execution works.

implement setup for mapr-spark-thriftserver

from sites/mapr-core.yml

• name: Fetch keys, when no CLDB change happened
  hosts: mapr-cldb[0]
  roles:
  • {role: ../roles/mapr-configure-security-fetch-keys,
    when: mapr_security != 'none' and (cldb_changes|int) == 0}

And we have in role: ../roles/mapr-configure-security-fetch-keys, this stuff 👍

• name: Copy keys to local
  fetch: src=/opt/mapr/conf/{{ item }} dest=/tmp/{{ item }} flat=yes
  with_items:
  • maprserverticket
  • cldb.key
  • ssl_keystore
  • ssl_truststore

Then generate an error

TASK [../roles/mapr-configure-security-fetch-keys : Copy keys to local] *****************************************************************************************************
failed: [192.168.2.60] (item=maprserverticket) => {"changed": false, "item": "maprserverticket", "msg": "file not found: /opt/mapr/conf/maprserverticket"}
failed: [192.168.2.60] (item=cldb.key) => {"changed": false, "item": "cldb.key", "msg": "file not found: /opt/mapr/conf/cldb.key"}
failed: [192.168.2.60] (item=ssl_keystore) => {"changed": false, "item": "ssl_keystore", "msg": "file not found: /opt/mapr/conf/ssl_keystore"}
failed: [192.168.2.60] (item=ssl_truststore) => {"changed": false, "item": "ssl_truststore", "msg": "file not found: /opt/mapr/conf/ssl_truststore"}

Cheers

core-site.xml is not rolled out when no aws file is found

Implement mapr-impala setup
Really required?

Disable log aggregation

Make Drill pstore configureable.

MapR 6.1 MEP 6 add mapr-objectstoregateway

Hi,

It would be appreciated to be able to create mirror volumes thanks to your "mapr_volume" library.
Is this feature in your backlog already ?

Hi @devproof

I use this issue to make a request to improve the way issues are used to track PR, could you reference in PR the issue it solves so one can find how a feature was implemented.
example issue #41 the issue is closed but no commit is provided.

Thanks for taking care of my comment,

Regards

Implement option to deploy nm-local-dirs with local volumes

MapR recommends disabling the Transparent Huge Pages (THP) feature in the Linux kernel.
https://mapr.com/docs/home/AdvancedInstallation/PreparingEachNode-infrastructure.html

Implement HTTPS for httpfs

Implement Timeline server

TASK [../roles/mapr-configure-warden-cldb-restart : Wait for CLDB] **********************************************************************************************************
fatal: [192.168.2.60]: FAILED! => {"changed": false, "elapsed": 1800, "msg": "Timeout when waiting for search string CLDB State change \:\ RUNNING|CLDBShutdown\:\ license not found for CLDB HA|Becoming Slave Thread in /opt/mapr/logs/cldb.log"}

Implement SSD support and trim for mfs.conf

Spark package should also work without Hive installation

Following error will take place on Suse (6 nodes example):
TASK [../roles/mapr-core-install : Install mapr-core] ***************************************************************************************************************************************
failed: [ip-172-.ec2.internal] (item=mapr-compat-suse) => {"changed": false, "cmd": ["/usr/bin/zypper", "--quiet", "--non-interactive", "--xmlout", "install", "--type", "package", "--auto-agree-with-licenses", "--no-recommends", "--", "+mapr-compat-suse"], "item": "mapr-compat-suse", "msg": "No provider of '+mapr-compat-suse' found.", "rc": 104, "stderr": "", "stderr_lines": [], "stdout": "\n\n<prompt id="11">\nFile 'repomd.xml' from repository 'maprecosystem' is unsigned, continue?\n<option value="yes" desc=""/>\n<option default="1" value="no" desc=""/>\n\n<message type="error">Repository 'maprecosystem' is invalid.\n[maprecosystem|http://package.mapr.com/releases/MEP/MEP-4.1.0//suse/] Valid metadata not found at specified URL\nPlease check if the URIs defined for this repository are pointing to a valid repository.\n\n<message type="warning">Skipping repository 'maprecosystem' because of the above error.\n<prompt id="11">\nFile 'repomd.xml' from repository 'maprtech' is unsigned, continue?\n<option value="yes" desc=""/>\n<option default="1" value="no" desc=""/>\n\n<message type="error">Repository 'maprtech' is invalid.\n[maprtech|http://package.mapr.com/releases/v6.0.0/suse/] Valid metadata not found at specified URL\nPlease check if the URIs defined for this repository are pointing to a valid repository.\n\n<message type="warning">Skipping repository 'maprtech' because of the above error.\n<message type="error">Some of the repositories have not been refreshed because of an error.\n<message type="error">No provider of '+mapr-compat-suse' found.\n\n", "stdout_lines": ["", "", "<prompt id="11">", "File 'repomd.xml' from repository 'maprecosystem' is unsigned, continue?", "<option value="yes" desc=""/>", "<option default="1" value="no" desc=""/>", "", "<message type="error">Repository 'maprecosystem' is invalid.", "[maprecosystem|http://package.mapr.com/releases/MEP/MEP-4.1.0//suse/] Valid metadata not found at specified URL", "Please check if the URIs defined for this repository are pointing to a valid repository.", "", "<message type="warning">Skipping repository 'maprecosystem' because of the above error.", "<prompt id="11">", "File 'repomd.xml' from repository 'maprtech' is unsigned, continue?", "<option value="yes" desc=""/>", "<option default="1" value="no" desc=""/>", "", "<message type="error">Repository 'maprtech' is invalid.", "[maprtech|http://package.mapr.com/releases/v6.0.0/suse/] Valid metadata not found at specified URL", "Please check if the URIs defined for this repository are pointing to a valid repository.", "", "<message type="warning">Skipping repository 'maprtech' because of the above error.", "<message type="error">Some of the repositories have not been refreshed because of an error.", "<message type="error">No provider of '+mapr-compat-suse' found.", ""]}

root cause: zypper asks user in case of repository unsigned

Implement setup for mapr-filemigrate

Uplift Ansible to MapR 6.1 and MEP 6

https://mapr.com/docs/60/Hive/Config-HiveMetastore-StorageBasedAuth.html?hl=storagebasedauthorizationprovider

Drill memory should be configurable in group_vars/all

https://mapr.com/docs/61/Drill/a-config-drill-memory.html

Make MAPR_SUBNETS optional

Copy conf.new to conf folder during upgrade

Uplift to MEP 4.1

Fix module mapr-volume for MapR 6
module works only with MapR 5, seems that API has changed.

When
PLAY [Uninstall MCS]
PLAY [Uninstall YARN Resource Manager]
PLAY [Uninstall History Server]
PLAY [Uninstall MapR-Fileserver]...

Add a parameter : keep_backup_templates=True/False in group_vars/all