GithubHelp home page GithubHelp logo

ansible-apim's Introduction

WSO2 API Management Ansible scripts

This repository contains the Ansible scripts for installing and configuring WSO2 API Management.

Supported Operating Systems

  • Ubuntu 16.04 or higher
  • CentOS 7

Supported Ansible Versions

  • Ansible 2.5 or higher

Directory Structure

.
├── dev
│   ├── group_vars
│   │   └── apim.yml
│   ├── host_vars
│   │   ├── apim-control-plane_1.yml
│   │   ├── apim-gateway_1.yml
│   │   ├── apim-tm_1.yml
│   │   ├── apim_1.yml
│   └── inventory
├── docs
│   ├── images
│   │   ├── 3-fully-distributed-setup.png
│   │   ├── API-M-single-node-deployment.png
│   │   ├── P-H-2.png
│   │   ├── P-H-3.png
│   │   ├── P-M-1.png
│   │   └── P-S-1.png
│   ├── Pattern_1.md
│   ├── Pattern_2.md
│   ├── Pattern_3.md
│   ├── Pattern_4.md
│   └── Pattern_5.md
├── files
│   ├── lib
│   │   ├── amazon-corretto-11.0.14.1-linux-x64.tar.gz
│   └── packs
│   │   ├── wso2am-4.1.0.zip
│   ├── system
│   │   └── etc
│   │       ├── security
│   │       │   └── limits.conf
│   │       └── sysctl.conf
│   └── misc
├── issue_template.md
├── LICENSE
├── pull_request_template.md
├── README.md
├── roles
│   ├── apim
│   │   ├── tasks
│   │   └── templates
│   ├── apim-control-plane
│   │   ├── tasks
│   │   └── templates
│   ├── apim-tm
│   │   ├── tasks
│   │   └── templates
│   └── common
│       └── tasks
├── scripts
│   ├── update.sh
│   └── update_README.md
└── site.yml

Following instructions can be followed to deploy a all-in-one standard APIM deployment to tryout the product and for demonstrations purposes. If you want to deploy a production ready deployment pattern refer the documentation in the docs directory.

Copying packs locally

Packs could be either copied to a local directory, or downloaded from a remote location.

Copy the following files to files/packs directory.

  1. WSO2 API Manager 4.1.0 package (.zip)

Copy the following files to files/lib directory.

  1. Amazon Corretto for Linux x64 JDK 11 (.tar.gz)

Copy the miscellaneous files to files/misc directory. To enable file copying, uncomment the misc_file_list in the yaml files under group_vars and add the miscellaneous files to the list.

Downloading from remote location

In group_vars, change the values of the following variables in all groups:

  1. The value of pack_location should be changed from "local" to "remote"
  2. The value of remote_jdk should be changed to the URL in which the JDK should be downloaded from, and remove it as a comment.
  3. The value of remote_pack should be changed to the URL in which the package should be downloaded from, and remove it as a comment.

Running WSO2 API Management Ansible scripts

1. Run the existing scripts without customization

The existing Ansible scripts contain the configurations to set-up a single node WSO2 API Manager pattern. In order to deploy the pattern, you need to replace the [ip_address] and [ssh_user] given in the inventory file under dev folder by the IP of the location where you need to host the API Manager. An example is given below.

[apim]
apim_1 ansible_host=172.28.128.4 ansible_user=vagrant

Then, update the hostname in dev/host_vars/apim_1.yml with ansible_host, 172.28.128.4

Run the following command to run the scripts. Use -K to provide a password to become root. Configure become_method: in ansible.cfg when using an alternative to sudo. If your host allows passwordless sudo, then omit the -K argument.

ansible-playbook -K -i dev site.yml

If you need to alter the configurations given, please change the parameterized values in the yaml files under group_vars and host_vars.

NOTE:

If you have mounted the 'persistent artifacts' as guided below, make sure to unmount the artifacts, prior to running the Ansible playbook as the playbook running process has a step to remove the existing setup. After completing the Ansible playbook running process, make sure to remount the artifacts.

If the client-truststore.jks is monuted among the Gateway nodes, then make sure to copy the client-truststore.jks from the mount source to the <ANSIBLE_HOME>/files/security/wso2am/ directory of the Ansible resources, prior to re-running the playbook.

2. Customize the WSO2 Ansible scripts

The templates that are used by the Ansible scripts are in j2 format in-order to enable parameterization.

The deployment.toml.j2 file is added under roles/apim/templates/carbon-home/repository/conf/, in order to enable customizations. You can add any other customizations to custom.yml under tasks of each role as well.

Step 1

Uncomment the following line in main.yml under the role you want to customize.

- import_tasks: custom.yml

Step 2

Add the configurations to the custom.yml. A sample is given below.

- name: "Copy custom file"
  template:
    src: path/to/example/file/example.xml.j2
    dest: destination/example.xml.j2
  when: "(inventory_hostname in groups['am'])"

Follow the steps mentioned under docs directory to customize/create new Ansible scripts and deploy the recommended patterns.

Including custom Keystore and Truststore

If custom keystores and truststores are needed to be added, uncomment the below list in the yml file

# security_file_list:
#   - { src: '{{ security_file_location }}/wso2am-analytics/client-truststore.jks',
#       dest: '{{ carbon_home }}/resources/security/client-truststore.jks' }
#   - { src: '{{ security_file_location }}/wso2am-analytics/wso2carbon.jks',
#       dest: '{{ carbon_home }}/resources/security/wso2carbon.jks' }

Then save the changed file and add the required files under files/security/<product-home>/<path-to-file>

Performance Tuning

System configurations can be changed through Ansible to optimize OS level performance. Performance tuning can be enabled by changing enable_performance_tuning in dev/group_vars/apim.yml to true.

System files that will be updated when performance tuning are enabled is available in files/system. Update the configuration values according to the requirements of your deployment.

Configuration Guide

Refer the below documentation on configuring key-stores for APIM and APIM-Analytics

  1. WSO2 API Manager key-stores configuration guide

Refer the below documentation on configuring persistent artifacts of the servers.

  1. Persistent artifacts of the servers

Refer the below documentation on configuring Load-Balancers for your deoloyment.

  1. Load balancer configurations

Previous versions of Ansible

The master branch of this repository contains the latest product version with the latest Ansible version. The Ansible resources for previous Ansible versions can be found in the branches. The following is an example.

Ansible resources for API Manager 4.0.0

Branch name: 4.0.x

ansible-apim's People

Contributors

ahsancse avatar ayeshw avatar bbaassssiiee avatar chamilad avatar chamindi-a avatar chathuraswso2 avatar ctienshi avatar dilanua avatar hasuniea avatar irunika avatar isurumaduranga avatar maheshika avatar msmshariq avatar nishikadesilva avatar pubudu538 avatar rakhitharr avatar rmsamitha avatar sanethmaduranga avatar savidude avatar senthuja avatar vimukthiperera avatar yasassri avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ansible-apim's Issues

Update pattern 1 doc

Description:
Step 3 in Pattern 1 doc mentions "Customize the roles for API Manager pattern 4". Should be "Customize the roles for API Manager pattern 1".

Error while starting Trafficmanager node "Exception occured while loading the Axis configuration"

Description:

Hi, i am still facing the same issue while trying to run traffic-manager profile through APIM 3.0.0

[2019-12-24 16:11:25,041] INFO - EmbeddedRegistryService Connected to mount at configregistry in 2ms
[2019-12-24 16:11:25,042] INFO - EmbeddedRegistryService Connected to mount at govregistry in 4ms
[2019-12-24 16:11:25,066] INFO - RegistryCoreServiceComponent Registry Mode : READ-WRITE
[2019-12-24 16:11:25,316] INFO - JmxReporterBuilder Creating JMX reporter for Metrics with domain 'org.wso2.carbon.metrics'
[2019-12-24 16:11:25,323] INFO - AbstractReporter Started JMX reporter for Metrics
[2019-12-24 16:11:27,453] INFO - SolrClient Default Embedded Solr Server Initialized
[2019-12-24 16:11:27,935] INFO - UserStoreMgtDSComponent Carbon UserStoreMgtDSComponent activated successfully.
[2019-12-24 16:11:30,733] FATAL - CarbonServerManager WSO2 Carbon initialization Failed
org.apache.axis2.AxisFault: Exception occured while loading the Axis configuration from C:\WSO2\Distributed3.0.0\TF\TrafficManager/repository/conf/axis2/axis2.xml
at org.wso2.carbon.core.CarbonAxisConfigurator.getAxisConfiguration(CarbonAxisConfigurator.java:194) ~[org.wso2.carbon.core_4.5.1.jar:?]
at org.apache.axis2.context.ConfigurationContextFactory.createConfigurationContext(ConfigurationContextFactory.java:64) ~[axis2_1.6.1.wso2v38.jar:?]
at org.wso2.carbon.core.CarbonConfigurationContextFactory.createNewConfigurationContext(CarbonConfigurationContextFactory.java:65) ~[org.wso2.carbon.core_4.5.1.jar:?]
at org.wso2.carbon.core.init.CarbonServerManager.initializeCarbon(CarbonServerManager.java:415) [org.wso2.carbon.core_4.5.1.jar:?]
at org.wso2.carbon.core.init.CarbonServerManager.removePendingItem(CarbonServerManager.java:305) [org.wso2.carbon.core_4.5.1.jar:?]
at org.wso2.carbon.core.init.PreAxis2ConfigItemListener.bundleChanged(PreAxis2ConfigItemListener.java:118) [org.wso2.carbon.core_4.5.1.jar:?]
at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:973) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:234) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:345) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
Caused by: org.apache.axis2.deployment.DeploymentException: org.apache.synapse.transport.passthru.PassThroughHttpSender cannot be found by axis2_1.6.1.wso2v38
at org.apache.axis2.deployment.AxisConfigBuilder.processTransportSenders(AxisConfigBuilder.java:713) ~[axis2_1.6.1.wso2v38.jar:?]
at org.apache.axis2.deployment.AxisConfigBuilder.populateConfig(AxisConfigBuilder.java:125) ~[axis2_1.6.1.wso2v38.jar:?]
at org.wso2.carbon.core.CarbonAxisConfigurator.populateAxisConfiguration(CarbonAxisConfigurator.java:341) ~[org.wso2.carbon.core_4.5.1.jar:?]
at org.wso2.carbon.core.CarbonAxisConfigurator.getAxisConfiguration(CarbonAxisConfigurator.java:192) ~[org.wso2.carbon.core_4.5.1.jar:?]
... 8 more
Caused by: java.lang.ClassNotFoundException: org.apache.synapse.transport.passthru.PassThroughHttpSender cannot be found by axis2_1.6.1.wso2v38
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:512) ~[org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:423) ~[org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:415) ~[org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.internal.loader.ModuleClassLoader.loadClass(ModuleClassLoader.java:155) ~[org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_232]
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_232]
at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_232]
at org.apache.axis2.util.Loader.loadClass(Loader.java:261) ~[axis2_1.6.1.wso2v38.jar:?]
at org.apache.axis2.deployment.AxisConfigBuilder.processTransportSenders(AxisConfigBuilder.java:687) ~[axis2_1.6.1.wso2v38.jar:?]
at org.apache.axis2.deployment.AxisConfigBuilder.populateConfig(AxisConfigBuilder.java:125) ~[axis2_1.6.1.wso2v38.jar:?]
at org.wso2.carbon.core.CarbonAxisConfigurator.populateAxisConfiguration(CarbonAxisConfigurator.java:341) ~[org.wso2.carbon.core_4.5.1.jar:?]
at org.wso2.carbon.core.CarbonAxisConfigurator.getAxisConfiguration(CarbonAxisConfigurator.java:192) ~[org.wso2.carbon.core_4.5.1.jar:?]
... 8 more
Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:
mysql

Steps to reproduce:
run wso2 apim 3.0.0 as traffic manager

Related Issues:
#50

"FAILED! => {"msg": "'performance_tuning_file_list' is undefined"}" when running analytics and IS as KM

Description:
Error message FAILED! => {"msg": "'performance_tuning_file_list' is undefined"} is encountered while running APIM analytics and IS as KM profiles.

Can be fixed by adding the following to apim-analytics.yml and apim-is-as-km.yml

# Performance tuning configurations
enable_performance_tuning: false
performance_tuning_file_list:
  - { src: '{{ product_package_location }}/system/etc/sysctl.conf',
      dest: '/etc/sysctl.conf' }
  - { src: '{{ product_package_location }}/system/etc/security/limits.conf',
      dest: '/etc/security/limits.conf' }

Suggested Labels:
Type/Bug
Priority/High

Suggested Assignees:
@Savidude

File Missing

Description:
apim-gateway.yml file missing from /dev/group_vars
File is referenced for Deployment Pattern 2 but is not
provided in the /dev directory

Suggested Labels:

Suggested Assignees:

Affected Product Version:
3.1.0

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Backup existing APIs before running the playbook

Description:
<CARBON_HOME>/repository/deployment/server directory contains the APIs created by the UI and they need to be backed up before applying the newest changes.

Affected Product Version:
v2.6.0.5

Deprecation warning "evaluating enable_performance_tuning as a bare variable" when running on Ansible 2.8.0

Description:
The following warning is encountered when running the Ansible playbook.

[DEPRECATION WARNING]: evaluating enable_performance_tuning as a bare variable, this behaviour will go away and you might need to add
|bool to the expression in the future. Also see CONDITIONAL_BARE_VARS configuration toggle. This feature will be removed in version
2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

Can be fixed by changing the conditional checking if performance tuning is enabled to a Boolean value.

- name: System performance tuning
  copy:
     src: "{{ item.src }}"
     dest: "{{ item.dest }}"
   loop: "{{ performance_tuning_file_list }}"
   when: enable_performance_tuning|bool

Suggested Labels:
Type/Bug
Priority/Low

Suggested Assignees:
@Savidude

Missing sudo password in the scripts

Description:
if you don't pass the parameter -K to run the playbook it fails with the message: Missing sudo password.

$ ansible-playbook -i dev site.yml
...
TASK [common : Create group] ****************************************************************************
fatal: [apim_1]: FAILED! => {"msg": "Missing sudo password"}
fatal: [apim-is-as-km_1]: FAILED! => {"msg": "Missing sudo password"}
fatal: [apim-analytics-worker_1]: FAILED! => {"msg": "Missing sudo password"}

Suggested Labels:
Documentation

Sample Approaches for Deployment Sync.

Description:
It will be better to have sample deployment Sync approaches also in this repository. Something like a pre-deployment role and a post-deployment role. Where it will remove the NFS mount before deleting the packs and then after the redeployment it will remount the NFS location to the deployment folder. WDYT?

Suggested Labels:
Improvement

Add Ansible scripts for WSO2 API Management 2.6.0

Description:
Add Ansible scripts for WSO2 API Management 2.6.0 with the following groups.

  • apim
  • apim-analytics
  • apim-is-as-km

Suggested Labels:
Type/Improvement
Priority/Normal

Affected Product Version:
API Management 2.6.0
API Management Analytics 2.6.0

OS, DB, other environment details and versions:
Ansible 2.6.2
Ubuntu 16.04 or higher
CentOS 7

[3.2.x] Issue when starting the server as a service in some roles

Description:
The following error log can be seen while starting the server as a service in gateway, km, tm, publisher roles.

JAVA_HOME environment variable is set to /opt/java
CARBON_HOME environment variable is set to /mnt/apim-tm/wso2am-3.2.0
Using Java memory options: -Xms256m -Xmx1024m
[2020-09-15 05:55:40,777]  INFO {org.wso2.config.mapper.ConfigParser} - Initializing configurations with deployment configurations
[2020-09-15 05:55:40,965] ERROR {org.wso2.config.mapper.TomlParser} - user_store previously defined at line 7, column 1 (line 15, column 1)
[2020-09-15 5:55:40,965]  SEVERE {org.wso2.carbon.server.Main handleConfiguration} - Error while performing configuration changes 
org.wso2.config.mapper.ConfigParserException: Error parsing deployment configuration
	at org.wso2.config.mapper.TomlParser.parse(TomlParser.java:140)
	at org.wso2.config.mapper.ConfigParser.parse(ConfigParser.java:257)
	at org.wso2.config.mapper.ConfigParser.deploy(ConfigParser.java:225)
	at org.wso2.config.mapper.ConfigParser.deployAndStoreMetadata(ConfigParser.java:188)
	at org.wso2.config.mapper.ConfigParser.parse(ConfigParser.java:135)
	at org.wso2.carbon.server.Main.handleConfiguration(Main.java:236)
	at org.wso2.carbon.server.Main.main(Main.java:107)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.wso2.carbon.bootstrap.Bootstrap.loadClass(Bootstrap.java:69)
	at org.wso2.carbon.bootstrap.Bootstrap.main(Bootstrap.java:51)

WSO2 3.0.0 Exception occured while loading the Axis configuration from

Description:

While trying to run key-manager profile getting issue below:

2019-12-19 15:31:11,470] INFO - UserStoreMgtDSComponent Carbon UserStoreMgtDSComponent activated successfully.
[2019-12-19 15:31:12,362] FATAL - CarbonServerManager WSO2 Carbon initialization Failed
org.apache.axis2.AxisFault: Exception occured while loading the Axis configuration from C:\WSO2\Distributed3.0.0\KeyManager/repository/conf/axis2/axis2.xml
at org.wso2.carbon.core.CarbonAxisConfigurator.getAxisConfiguration(CarbonAxisConfigurator.java:194) ~[org.wso2.carbon.core_4.5.1.jar:?]
at org.apache.axis2.context.ConfigurationContextFactory.createConfigurationContext(ConfigurationContextFactory.java:64) ~[axis2_1.6.1.wso2v38.jar:?]
at org.wso2.carbon.core.CarbonConfigurationContextFactory.createNewConfigurationContext(CarbonConfigurationContextFactory.java:65) ~[org.wso2.carbon.core_4.5.1.jar:?]
at org.wso2.carbon.core.init.CarbonServerManager.initializeCarbon(CarbonServerManager.java:415) [org.wso2.carbon.core_4.5.1.jar:?]
at org.wso2.carbon.core.init.CarbonServerManager.removePendingItem(CarbonServerManager.java:305) [org.wso2.carbon.core_4.5.1.jar:?]
at org.wso2.carbon.core.init.PreAxis2ConfigItemListener.bundleChanged(PreAxis2ConfigItemListener.java:118) [org.wso2.carbon.core_4.5.1.jar:?]
at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:973) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:234) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:345) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
Caused by: org.apache.axis2.deployment.DeploymentException: org.apache.synapse.transport.passthru.PassThroughHttpSender cannot be found by axis2_1.6.1.wso2v38
at org.apache.axis2.deployment.AxisConfigBuilder.processTransportSenders(AxisConfigBuilder.java:713) ~[axis2_1.6.1.wso2v38.jar:?]
at org.apache.axis2.deployment.AxisConfigBuilder.populateConfig(AxisConfigBuilder.java:125) ~[axis2_1.6.1.wso2v38.jar:?]
at org.wso2.carbon.core.CarbonAxisConfigurator.populateAxisConfiguration(CarbonAxisConfigurator.java:341) ~[org.wso2.carbon.core_4.5.1.jar:?]
at org.wso2.carbon.core.CarbonAxisConfigurator.getAxisConfiguration(CarbonAxisConfigurator.java:192) ~[org.wso2.carbon.core_4.5.1.jar:?]
Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Database driver is not updated in the master-datasource.xml

Description:

When using script for distributed setup database driver did not get set in the master-datasource.xml. It set a empty value.

Suggested Labels:

Suggested Assignees:

Affected Product Version:
3.2.x

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Create IS as Key Manager role in Ansible resources for APIM

Description:
Create IS as Key Manager role in Ansible resources for APIM since IS is used as a key manager instead of the default key manager.

Suggested Labels:
Type/Improvement

Affected Product Version:
APIM 2.6.0
OS, DB, other environment details and versions:
Ansible 2.6.2

Related Issues:
#4

Fix inconsistency in the values of the 'name' in each task

Description:
Some of the tasks contain double quotes for the name and some do not.

- name: Install a .rpm package
yum:
name: "{{ product_path }}/{{ product_archive_rpm }}"
when: ansible_os_family == "RedHat"
- name: "Copy JDBC Driver"
copy:
src: "files/{{ jdbc_driver }}"
dest: "{{ setup_path }}/{{ product_name }}/{{ product_version }}/repository/components/lib"

Suggested Assignees:
@VimukthiPerera

Affected Product Version:
All up to 2.6.0

Ansible scripts for API Manager 2.5.0

Description:
Add Ansible scripts to deploy WSO2 API Manager 2.5.0

Suggested Labels:
Type/Improvement

Affected Product Version:
API Manager 2.5.0

OS, DB, other environment details and versions:
Ubuntu 16.04 or higher
Ansible 2.6.2

Optimize Ansible scripts by removing repeatable tasks

Description:

Ansible scripts for Identity Server use a repeatable task to copy every single configuration file as below.

      template:
        src: "carbon-home/repository/conf/datasources/master-datasources.xml.j2"
        dest: "{{ setup_path }}/{{ product_name }}/{{ product_version }}/repository/conf/datasources/master-datasources.xml"

    - name: "Copy identity.xml"
      template:
        src: "carbon-home/repository/conf/identity/identity.xml.j2"
        dest: "{{ setup_path }}/{{ product_name }}/{{ product_version }}/repository/conf/identity/identity.xml"

    - name: "Copy carbon.xml"
      template:
        src: "carbon-home/repository/conf/carbon.xml.j2"
        dest: "{{ setup_path }}/{{ product_name }}/{{ product_version }}/repository/conf/carbon.xml"

    - name: "Copy registry.xml"
      template:
        src: "carbon-home/repository/conf/registry.xml.j2"
        dest: "{{ setup_path }}/{{ product_name }}/{{ product_version }}/repository/conf/registry.xml"

    - name: "Copy axis2.xml"
      template:
        src: "carbon-home/repository/conf/axis2/axis2.xml.j2"
        dest: "{{ setup_path }}/{{ product_name }}/{{ product_version }}/repository/conf/axis2/axis2.xml"

But there is a optimal way to achieve same as follows:

TASKS:

- name: copy wso2 conf files
  template:
    src:   "{{ carbon_home }}/repository/conf/{{ item.source }}"
    dest: "{{ setup_path }}/{{ product_name }}/{{ product_version }}/repository/conf/{{ item.dest }}"
    owner: "{{ wso2_user }}"
    group: "{{ wso2_group }}"
  with_items: "{{ config_files }}"

VARS:

config_files:

  - source: 'datasources/master-datasources.xml.j2'
    dest: 'datasources/master-datasources.xml'

  - source: 'identity/identity.xml.j2'
    dest: 'identity/identity.xml'

  - source: 'carbon.xml.j2'
    dest: 'carbon.xml'

  - source: 'registry.xml.j2'
    dest: 'registry.xml'

  - source: 'axis2/axis2.xml.j2'
    dest: 'axis2/axis2.xml' 

Affected Product Version:
v2.6.0.1

Remove task copying the JDBC Driver

Description:
This could remain in the documentation. However, is not required by default. As a result, this task could be left commented out.

Suggested Labels:
Type/Improvement
Priority/Normal

Suggested Assignees:
@Savidude

Error while starting Trafficmanager node "Exception occured while loading the Axis configuration"

Description:
The following error is encountered while starting a TrafficManager node

TID: [-1234] [] [2019-07-12 10:25:15,936] FATAL {org.wso2.carbon.core.init.CarbonServerManager} -  WSO2 Carbon initialization Failed {org.wso2.carbon.core.init.CarbonServerManager}
org.apache.axis2.AxisFault: Exception occured while loading the Axis configuration from /Users/savidude/.wum3/products/wso2am/2.6.0/wso2am-2.6.0/repository/conf/axis2/axis2.xml
	at org.wso2.carbon.core.CarbonAxisConfigurator.getAxisConfiguration(CarbonAxisConfigurator.java:194)
	at org.apache.axis2.context.ConfigurationContextFactory.createConfigurationContext(ConfigurationContextFactory.java:64)
	at org.wso2.carbon.core.CarbonConfigurationContextFactory.createNewConfigurationContext(CarbonConfigurationContextFactory.java:65)
	at org.wso2.carbon.core.init.CarbonServerManager.initializeCarbon(CarbonServerManager.java:399)
	at org.wso2.carbon.core.init.CarbonServerManager.start(CarbonServerManager.java:220)
	at org.wso2.carbon.core.internal.CarbonCoreServiceComponent.activate(CarbonCoreServiceComponent.java:105)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
	at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
	at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
	at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
	at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
	at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
	at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
	at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
	at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
	at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
	at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
	at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
	at org.eclipse.equinox.http.servlet.internal.Activator.registerHttpService(Activator.java:81)
	at org.eclipse.equinox.http.servlet.internal.Activator.addProxyServlet(Activator.java:60)
	at org.eclipse.equinox.http.servlet.internal.ProxyServlet.init(ProxyServlet.java:40)
	at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.init(DelegationServlet.java:38)
	at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1230)
	at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1174)
	at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1066)
	at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5370)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5668)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1700)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1690)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.axis2.deployment.DeploymentException: org.apache.synapse.transport.passthru.PassThroughHttpSender cannot be found by axis2_1.6.1.wso2v28
	at org.apache.axis2.deployment.AxisConfigBuilder.processTransportSenders(AxisConfigBuilder.java:712)
	at org.apache.axis2.deployment.AxisConfigBuilder.populateConfig(AxisConfigBuilder.java:124)
	at org.wso2.carbon.core.CarbonAxisConfigurator.populateAxisConfiguration(CarbonAxisConfigurator.java:341)
	at org.wso2.carbon.core.CarbonAxisConfigurator.getAxisConfiguration(CarbonAxisConfigurator.java:192)
	... 41 more

This can be fixed by renaming the repository/conf/axis2/axis2_TM.xml to repository/conf/axis2/axis2.xml

Suggested Labels:
Type/Bug
Priority/High

Suggested Assignees:
@Savidude

XSS vulnerability

Description:

In XSSValue which is enabled to mitigate cross-site scripting attack [1], verification is done against the patterns configured in xss-patterns.properties [2]. Since there is a pattern as "alert(.*?)", this case fails. Validation will fail for all actions that have the word alert.

[1] https://docs.wso2.com/display/ADMIN44x/Mitigating+Cross+Site+Scripting+Attacks
[2] https://github.com/wso2-support/carbon4-kernel/blob/support-4.4.35/core/org.wso2.carbon.ui/src/main/java/org/wso2/carbon/ui/valve/XSSValve.java#L107

Fix

In the /repository/conf/security/xss-patterns.properties file, there is a pattern defined as pattern14=alert(.?)*.

Replace it with the following pattern

pattern14=alert\((.*?)\)
Above changes will fix the XSS issue which AAA life is facing when they have enabled the XSS configuration.

To enable the XSS configuration, do the following in the carbon.xml file.

  1. Need to add the following value under the tag of the configuration under the section.
    commonauth
    After the modification configuration will be as follows.
true allow commonauth 2. Need to add the following configuration within the element of the /repository/conf/tomcat/catalina-server.xml file.

Problem with the sample given in the script

Description:
The site.yml playbook at the end of the file contains a sample that shows how to combine two different roles at a single node. The sample describes how to start both API Manager default profile and Identity Server as Key Manager in the same apim_1 node.

But this causes an error that wso2is-km requested service could not be found.

Suggested Labels:
Documentation

JDK unzipping failure

Description:
Sometimes the JDK archive name is different from the extracted folder name. Current Ansible scripts expect them to be the same.

Possible Fix

- name: Unzip JDK
  unarchive:
    src: /path/to/jdk.tar.gz
    dest: /path/to/extract/{{ jdk_name }}
    extra_opts: [--strip-components=1]

APIM Gateway Profile is not deploying

Description:
APIM Gateway Profile is not deploying . When analyse the carbon logs its noticed that this is a toml parser issue in deployment.toml.j2 template of gateway role due to some duplicate tags

Suggested Labels:

Suggested Assignees:

Affected Product Version:
3.2

OS, DB, other environment details and versions:

Steps to reproduce:
Run ansible-playbook with gateway profile information in the site.yml

Related Issues:

Parameterize the validation query in the masterdatasource.xml

Description:
The current validation query is set to "SELECT 1". This changes according the type of the database. Therefore the query should be parameterized similar to the DriverClassName.

For eg - Oracle DB the validation query is "SELECT 1 FROM DUAL"

WSO2 Analytics with ansible in a distributed environment

Please

I'm currently working in a distributed API Manager and Analytics environment to be set through ansible scripts

I need to run analytics balanced through 2 IPs

In my project structure I have the following:
ansible-apim/roles/apim-analytics-worker/templates/carbon-home/conf/dashboard/deployment.yaml.j2

and in this yaml the following content:
baseUrl: https://localhost:9643

When I try to connect in Analytics :
https://:9643/analytics-dashboard

It tries to redirect it to localhost

I have read the documentation:

https://apim.docs.wso2.com/en/3.0.0/learn/analytics/configuring-apim-analytics/#step-5-configure-apim-idp-client

How can I change the baseUrl so that when I ran the ansible script it reflects properly in both distributed environment IPs ?

Thks!!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.