GithubHelp home page GithubHelp logo

meltmedia / jgroups-aws Goto Github PK

View Code? Open in Web Editor NEW
45.0 69.0 25.0 984 KB

An AWS Discovery Component for JGroups

Home Page: http://meltmedia.github.com/jgroups-aws

License: Apache License 2.0

Java 100.00%
ec2 jgroups java auto-discovery

jgroups-aws's Introduction

AWS Auto Discovery for JGroups

Overview

This package provides auto discovery for other cluster members on AWS using both tag matching and filters. It is a drop in replacement for TCPPING, allowing you to remove the definition of your initial members from your configuration file.

Usage

To use AWS auto discovery, you need to add a dependency to this package in your pom:

    <dependency>
      <groupId>com.meltmedia.jgroups</groupId>
      <artifactId>jgroups-aws</artifactId>
      <version>1.6.1</version>
    </dependency>

and then replace TCPPING in your stack with com.meltmedia.jgroups.aws.AWS_PING:

    <com.meltmedia.jgroups.aws.AWS_PING
         port_number="7800"
         tags="TAG1,TAG2"
         filters="NAME1=VALUE1,VALUE2;NAME2=VALUE3"
         access_key="AWS_ACCESS_KEY"
         secret_key="AWS_SECRET_KEY"/>

see the configuration section for information. You can find an example stack in conf/aws_ping.xml.

This implementation will only work from inside EC2, since it uses environment information to auto wire itself. See the Setting Up EC2 section for more information.

SNAPSHOTs of the project are located in the Sonatype Nexus Snapshots repository. You can use SNAPSHOTs by adding the following repository to your project:

    <repository>
      <releases>
        <enabled>false</enabled>
      </releases>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
      <id>sonatype-nexus-snapshots</id>
      <name>Sonatype Nexus Snapshots</name>
      <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    </repository>

And then update your dependency to the current SNAPSHOT version.

Configuration Options

  • port_number - the port number that the nodes will communicate over. This needs to be the same on all nodes. The default is 7800.
  • port_range - the number of additional ports to be probed for membership. A port_range of 0 does not probe additional ports. Example: initial_hosts=A[7800] port_range=0 probes A:7800, port_range=1 probes A:7800 and A:7801. The default is 50.
  • tags - A comma delimited list of EC2 node tag names. The current nodes values are matched against other nodes to find cluster members.
  • filters - A colon delimited list of filters. Each filter defines a name and a comma delimited list of possible values. All filters must match a node for it to be a cluster member.
  • access_key and secret_key - the access key and secret key for an AWS user with permission to the "ec2:Describe*" action. If both of these attributes are not specified, then the instance profile for the EC2 instance will be used (Since version 1.1).
  • credentials_provider_class - the fully qualified name of the com.amazonaws.auth.AWSCredentialsProvider to use (Since version 1.3). This option can only be used when the access_key and secret_key options are not provided.

Setting Up EC2

You will need to setup the following in EC2, before using this package:

  • The EC2 instances will need permission to the "ec2:Describe*" action. You can either create an IAM user with this permission and pass the users credentials with the access_key and secret_key attributes or associate an instance profile with that permission to the instances and not specify the access_key and secret_key attributes.
  • In the EC2 console, you will need to create a security group for your instances. This security group will need a TCP_ALL rule, with itself as the source (put the security group's name in the source field.) This will allow all of the nodes in that security group to communicate with each other.
  • Create two EC2 nodes, making sure to include the security group granting TCP communication.
  • If you are going to use the tag matching feature, then define a few tags on the nodes with matching values.

Setting up JGroups Chat Demo

The JGroups project provides a chat application that is great for testing your configuration. To set up the chat application, first create two EC2 nodes, following the Setting Up EC2 instructions. Once the nodes are created, SSH into each machine and install the java 6 JDK, Maven 3, and Git.

sudo apt-get install openjdk-6-jdk
wget http://www.carfab.com/apachesoftware/maven/binaries/apache-maven-3.0.4-bin.tar.gz
sudo tar xzf apache-maven-3.0.4-bin.tar.gz /opt
ln -s /opt/maven /opt/apache-maven-3.0.4
echo "export PATH=/opt/maven/bin:$PATH" >> .profile
. ~/.profile
sudo apt-get install git
mkdir ~/git

Now your machine is ready to compile maven projects.

Next, clone and build this project.

cd ~/git
git clone git://github.com/meltmedia/jgroups-aws.git
cd jgroups-aws
mvn clean install

Finally, it is time to run the project. You will need to edit the configuration in conf/aws_ping.xml. Add values for the tags, access_key and secret_key attributes. Remove the filters attribute. Then execute the following:

mvn exec:java -Dexec.mainClass="org.jgroups.demos.Chat" -Dexec.args="-props conf/aws_ping.xml" -Djava.net.preferIPv4Stack=true

Request timeout bug

If you are seeing the following error in you logs. You may wish to upgrade to the 1.2.0 version, as this seems to fix the problem:

Status Code: 400, AWS Service: AmazonEC2, AWS Request ID: 6ec551f5-7f56-493f-a213-b8c5cb3e856d, AWS Error Code: RequestExpired, AWS Error Message: Request has expired.

jgroups-aws's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

jgroups-aws's Issues

Unable to connect to jgroup cluster using AWS_PING

Hi,

We are working on an application that requires AWS Auto Discovery of the nodes. We have the setup using jgroups ready.

We have tested by running the same application in AWS EC2 with preconfigured IP addresses and Cluster is getting built.

This is our AWS_PING line in the TCP.xml. We are using TCP unicast for communication.
<com.meltmedia.jgroups.aws.AWS_PING timeout="3000" port_number="7900"
filters="InstanceType=t2.micro;KeyName=XXXXXXXX" access_key="XXXXXXXXX" secret_key="XXXXXXXXXXXXXXX"/>

The version of jgroups aws getting used is **1.3.1 _. Jgroups version is _3.6.8.Final

We are using IPV4 stack, All Traffic is enabled on Ec2 instances and ec2-describe* command too works which gives details of other EC2 instances.


But, when we use AWS_PING Protocol, we get an exception saying:

**22-Apr-2016 00:07:16.766 WARNING [localhost-startStop-1] org.jgroups.stack.Confi gurator.resolveAndAssignField JGRP000014: BasicTCP.use_send_queues has been depr ecated: will be removed in 4.0
22-Apr-2016 00:07:16.777 WARNING [localhost-startStop-1] org.jgroups.stack.Confi gurator.resolveAndAssignField JGRP000014: TP.loopback has been deprecated: enabl ed by default
22-Apr-2016 00:07:16.777 WARNING [localhost-startStop-1] org.jgroups.stack.Confi gurator.resolveAndAssignField JGRP000014: TP.discard_incompatible_packets has be en deprecated: incompatible packets are discarded anyway
22-Apr-2016 00:07:16.778 WARNING [localhost-startStop-1] org.jgroups.stack.Confi gurator.resolveAndAssignField JGRP000014: TP.enable_bundling has been deprecated : will be ignored as bundling is on by default
22-Apr-2016 00:07:16.786 WARNING [localhost-startStop-1] org.jgroups.stack.Confi gurator.resolveAndAssignField JGRP000014: Discovery.timeout has been deprecated: GMS.join_timeout should be used instead
22-Apr-2016 00:07:17.628 INFO [localhost-startStop-1] org.jgroups.protocols.UFC. init UFC is not needed (and can be removed) as we're running on a TCP transport


GMS: address=ip-172-31-54-146-23940, cluster=usermap, physical address=172.31.54 .146:7900

java.lang.Exception: connecting to channel "usermap" failed
at org.jgroups.JChannel._connect(JChannel.java:570)
at org.jgroups.JChannel.connect(JChannel.java:294)
at org.jgroups.JChannel.connect(JChannel.java:279)
at org.jgroups.blocks.ReplCache.start(ReplCache.java:278)
at com.cloud.jgroups.NodeInitializer.contextInitialized(NodeInitializer. java:107)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContex t.java:4811)
at org.apache.catalina.core.StandardContext.startInternal(StandardContex t.java:5251)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase .java:725)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:70 1)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:940)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java: 1816)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:51 1)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor .java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AbstractMethodError: org.jgroups.protocols.Discovery.findMe mbers(Ljava/util/List;ZLorg/jgroups/util/Responses;)V
at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244)
at org.jgroups.protocols.Discovery.down(Discovery.java:385)
at org.jgroups.protocols.MERGE3.down(MERGE3.java:254)
at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:371)
at org.jgroups.protocols.FD.down(FD.java:318)
at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:92)
at org.jgroups.protocols.BARRIER.down(BARRIER.java:132)
at org.jgroups.protocols.pbcast.NAKACK2.down(NAKACK2.java:582)
at org.jgroups.protocols.UNICAST3.down(UNICAST3.java:669)
at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:347)
at org.jgroups.protocols.pbcast.ClientGmsImpl.joinInternal(ClientGmsImpl .java:76)
at org.jgroups.protocols.pbcast.ClientGmsImpl.join(ClientGmsImpl.java:41 )
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:1085)
at org.jgroups.protocols.FlowControl.down(FlowControl.java:351)
at org.jgroups.protocols.FlowControl.down(FlowControl.java:351)
at org.jgroups.protocols.FRAG2.down(FRAG2.java:136)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java: 202)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:1039)
at org.jgroups.JChannel.down(JChannel.java:790)
at org.jgroups.JChannel._connect(JChannel.java:564)
... 17 more
**

Keycloak jgroups-aws configuration

Hi,
I'm trying config keycloak on ha mode with jgroups-aws without success.
I have compiled keycloak with maven adding the dependency of jgroups-aws, but I don't know if I have to do any other config before or after. (compilation results seems correct, but I don't know the correct steps for add this function)
What's xml section of configuration I must replace TCPPING? on keycloak on this case, I only view this section for replace it with :

 <subsystem xmlns="urn:jboss:domain:jgroups:6.0">
<stack name="tcp">
                    <transport type="TCP" socket-binding="jgroups-tcp"/>
                    <socket-protocol type="MPING" socket-binding="jgroups-mping"/>
                    <protocol type="MERGE3"/>
                    <protocol type="FD_SOCK"/>
                    <protocol type="FD_ALL"/>
                    <protocol type="VERIFY_SUSPECT"/>
                    <protocol type="pbcast.NAKACK2"/>
                    <protocol type="UNICAST3"/>
                    <protocol type="pbcast.STABLE"/>
                    <protocol type="pbcast.GMS"/>
                    <protocol type="MFC"/>
                    <protocol type="FRAG3"/>
                </stack>
            </stacks>
        </subsystem>

but the error says:


> OPVDX001: Validation error in standalone-ha.xml --------------------------------
> |
> |  339:                     <com.meltmedia.jgroups.aws.AWS_PING
> |                           ^^^^ 'com.meltmedia.jgroups.aws.AWS_PING' isn't an allowed element here
> |
> |                                Elements allowed here are:
> |                                  auth-protocol              relay
> |                                  encrypt-protocol           socket-discovery-protocol
> |                                  jdbc-protocol              socket-protocol
> |                                  protocol                   transport

The aws credentials must set on this section?

Thanks

Support MULTI_PING

MULTI_PING allows for multiple discovery protocols to be used in a configuration.
I have an application that needs to support multiple providers: Data Center (TCPPING\ MPING), AWS, GOOGLE_PING, and AZURE_PING Per the original use case of MULTI_PING: "Allow for multiple discovery protocols. This would allow us to use a single configuration which includes discovery protocols for multiple cloud providers." Also see: https://github.com/belaban/JGroups/blob/master/doc/manual/protocols.adoc#multiple-discovery-protocols-in-the-same-stack

I attempted to use it with AWS_PING. Unfortunately at initialization AWS_PING attempts to discover the EC2 Instance's identity and throws an exception if it cannot be found. This does not allow me to utilize MULTI_PING.
Catching and suppressing the exception would allow for MULTI_PING to be used.

Example stack:

<stack name="tcp">
    <transport type="TCP" socket-binding="jgroups-tcp"/>
    <socket-protocol type="MPING" socket-binding="jgroups-mping" />
    <protocol type="TCPPING">
        <property name="initial_hosts">host1[7600],host2[7600],host3[7600]</property>
    </protocol>
    <protocol type="com.meltmedia.jgroups.aws.AWS_PING" module="com.meltmedia.jgroups.jgroups-aws">
        <property name="port_number">7600</property>
        <property name="port_range">0</property>
        <property name="tags">stack_name,role</property>
        <property name="filters">instance-state-name=running</property>
    </protocol>
    <protocol type="MULTI_PING">
        <property name="async_discovery">true</property>
    </protocol>
    <protocol type="MERGE3"/>
    <socket-protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
    <protocol type="FD_ALL"/>
    <protocol type="VERIFY_SUSPECT"/>
    <protocol type="pbcast.NAKACK2"/>
    <protocol type="UNICAST3"/>
    <protocol type="pbcast.STABLE"/>
    <protocol type="pbcast.GMS"/>
    <protocol type="MFC"/>
    <protocol type="FRAG3"/>
</stack>

Expected behavior: If not in AWS MPING or TCPPING would be used in the Data Center.
Actual behavior: When not in AWS, AWS_PING throws an exception and Wildfly fails to use the other JGroups discovery protocols

Running Chat demo doesn't seem to work if you don't specify -Djava.net.preferIPv4Stack=true

The Readme has the following command:
mvn exec:java -Dexec.mainClass="org.jgroups.demos.Chat" -Dexec.args="-props conf/aws_ping.xml"

But when i ran it, the 2 EC2 instances running the Chat demo couldn't seem to talk to each other. After adding -Djava.net.preferIPv4Stack=true and running it again it work after a while.

So using the following command:
mvn exec:java -Dexec.mainClass="org.jgroups.demos.Chat" -Dexec.args="-props conf/aws_ping.xml" -Djava.net.preferIPv4Stack=true

I was getting the following warning sometimes which stopped the other EC2 instances message:
Jan 23, 2015 4:27:33 AM org.jgroups.logging.JDKLogImpl warn
WARNING: ip-172-31-11-65-19587: dropped message 12 from ip-172-31-0-254-8352 (sender not in table [ip-172-31-11-65-19587]), view=[ip-172-31-11-65-19587|0] [ip-172-31-11-65-19587]

1.6.2 release?

Any chance we could get a release? The hardcoded INSTANCE_IDENTITY_URL is eating my lunch.

Thanks.

Discovery of nodes on a single EC2 host

At some point between releases 1.4.2 → 1.6.1, I seem to have lost the ability to discover JGroups stacks running on the same host (in different JVMs). This coincided with the change from JGroups 3.6.8 → 4.0.5. My initial thought was misconfiguration, but I have tried a lot of adjustments, and I'm not seeing it. To be clear, what I am describing is:

  • JGroups running on separate hosts: pbcast.GMS reports first as co-ordinator, merges in second.
  • JGroups running in different JVMs, different ports, same host: pbcast.GMS reports first as co-ordinator, then second as co-ordinator, and they don't see each other.

Has anyone seen this and/or can anyone offer any configuration advice? This is using configuration as supplied with this project.

Cannot get instances to form a cluster

I am following your instructions but cannot get my instances to join together. Maybe I'm forgetting something?

I'm using JGroups to distribute EhCache 2.6.8 http://ehcache.org/documentation/2.6/replication/jgroups-replicated-caching. My ehcache.xml file looks like:

<cacheManagerPeerProviderFactory
    class="net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProviderFactory"
    properties="connect=TCP(bind_port=7800):
       com.meltmedia.jgroups.aws.AWS_PING(timeout=3000;port_number=7800;tags=Env,Role;log_aws_error_messages=true):         
       MERGE2(min_interval=10000;max_interval=30000):
       FD_SOCK:
       FD(timeout=3000;max_tries=3):
       VERIFY_SUSPECT(timeout=1500):
       pbcast.NAKACK(retransmit_timeout=3000;use_mcast_xmit=false):
       UNICAST:
       pbcast.STABLE(stability_delay=1000;desired_avg_gossip=50000;max_bytes=4M):
       pbcast.GMS(join_timeout=5000;print_local_addr=true;view_bundling=true):
       pbcast.STATE_TRANSFER"
    propertySeparator="::" />

In the logs, we see this upon startup:

2015-07-01 00:58:57.889 DEBUG [localhost-startStop-1] [com.meltmedia.jgroups.aws.AWS_PING]

Configured with tags [[Env, Role]]

2015-07-01 00:58:58.999 DEBUG [localhost-startStop-1] [com.meltmedia.jgroups.aws.AWS_PING] Describing AWS instances with the following filters [[{Name: tag:Rol
e,Values: [API]}, {Name: tag:Env,Values: [PROD]}]]
2015-07-01 00:58:58.999 DEBUG [localhost-startStop-1] [com.meltmedia.jgroups.aws.AWS_PING] Making AWS Request {{InstanceIds: [],Filters: [{Name: tag:Role,Value
s: [API]}, {Name: tag:Env,Values: [PROD]}],}}
2015-07-01 00:58:59.111 DEBUG [localhost-startStop-1] [com.meltmedia.jgroups.aws.AWS_PING] Instances found [[10.0.2.233, 10.0.3.188]]
2015-07-01 00:59:02.147 DEBUG [localhost-startStop-1] [org.jgroups.protocols.pbcast.NAKACK]
[prod-tomcat-api-2b-1-53235 setDigest()]
existing digest: []
new digest: prod-tomcat-api-2b-1-53235: [0 (0)]
resulting digest: prod-tomcat-api-2b-1-53235: [0 (0)]
2015-07-01 00:59:02.148 DEBUG [localhost-startStop-1] [org.jgroups.protocols.pbcast.GMS] prod-tomcat-api-2b-1-53235: installing view [prod-tomcat-api-2b-1-53235|0] [prod-tomcat-api-2b-1-53235]
2015-07-01 00:59:02.154 DEBUG [localhost-startStop-1] [org.jgroups.protocols.pbcast.STABLE] resuming message garbage collection
2015-07-01 00:59:02.154 DEBUG [localhost-startStop-1] [org.jgroups.protocols.FD_SOCK] VIEW_CHANGE received: [prod-tomcat-api-2b-1-53235]
2015-07-01 00:59:02.190 DEBUG [localhost-startStop-1] [org.jgroups.protocols.pbcast.STABLE] [ergonomics] setting max_bytes to 4MB (1 members)
2015-07-01 00:59:02.191 DEBUG [localhost-startStop-1] [org.jgroups.protocols.pbcast.STABLE] resuming message garbage collection
2015-07-01 00:59:02.196 DEBUG [localhost-startStop-1] [org.jgroups.protocols.pbcast.GMS] created group (first member). My view is [prod-tomcat-api-2b-1-53235|0], impl is org.jgroups.protocols.pbcast.CoordGmsImpl
2015-07-01 00:59:02.336 INFO [localhost-startStop-1] [net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProvider] JGroups Replication started for 'hatch-api'. JChannel: local_addr=prod-tomcat-api-2b-1-53235
cluster_name=hatch-api
my_view=[prod-tomcat-api-2b-1-53235|0] [prod-tomcat-api-2b-1-53235]
connected=true
closed=false
discard_own_messages=true
state_transfer_supported=true
props=TCP(discard_incompatible_packets=true;thread_pool_max_threads=10;ergonomics=true;enable_unicast_bundling=false;port_range=50;conn_expire_time=0;recv_buf_size=150000;thread_naming_pattern=cl;send_buf_size=150000;tcp_nodelay=true;stats=true;oob_thread_pool_enabled=true;oob_thread_pool_rejection_policy=discard;id=22;thread_pool_rejection_policy=Discard;logical_addr_cache_max_size=500;use_send_queues=true;loopback=true;timer_rejection_policy=run;sock_conn_timeout=2000;oob_thread_pool_min_threads=2;send_queue_size=10000;max_bundle_timeout=20;enable_diagnostics=true;external_port=0;oob_thread_pool_max_threads=10;log_discard_msgs=true;name=TCP;oob_thread_pool_keep_alive_time=30000;bind_addr=/fe80:0:0:0:9d:eeff:fee5:c543%eth0;wheel_size=200;bundler_capacity=200000;tick_time=50;timer_max_threads=10;thread_pool_queue_enabled=true;enable_bundling=true;oob_thread_pool_queue_enabled=true;thread_pool_keep_alive_time=30000;bind_port=7800;thread_pool_min_threads=2;reaper_interval=0;bind_interface_str=;diagnostics_ttl=8;logical_addr_cache_expiration=120000;peer_addr_read_timeout=1000;oob_thread_pool_queue_max_size=500;linger=-1;diagnostics_addr=/ff0e:0:0:0:0:0:75:75;receive_on_all_interfaces=false;timer_queue_max_size=500;thread_pool_queue_max_size=500;max_bundle_size=64000;timer_min_threads=4;thread_pool_enabled=true;bundler_type=new;timer_keep_alive_time=5000;timer_type=new;diagnostics_port=7500)
:com.meltmedia.jgroups.aws.AWS_PING(credentials_provider_class=com.amazonaws.auth.DefaultAWSCredentialsProviderChain;ergonomics=true;port_range=50;stagger_timeout=0;timeout=3000;tags=Env,Role;log_aws_error_messages=true;num_initial_srv_members=0;break_on_coord_rsp=true;stats=true;port_number=7800;num_initial_members=10;name=AWS_PING;id=600;return_entire_cache=false)
:MERGE2(merge_fast=true;merge_fast_delay=1000;stats=true;min_interval=10000;ergonomics=true;name=MERGE2;id=0;max_interval=30000;inconsistent_view_threshold=1)
:FD_SOCK(get_cache_timeout=1000;sock_conn_timeout=1000;client_bind_port=0;ergonomics=true;start_port=0;port_range=50;suspect_msg_interval=5000;num_tries=3;bind_interface_str=;stats=true;external_port=0;name=FD_SOCK;bind_addr=/fe80:0:0:0:9d:eeff:fee5:c543%eth0;keep_alive=true;id=3)
:FD(stats=true;ergonomics=true;name=FD;max_tries=3;id=2;timeout=3000)
:VERIFY_SUSPECT(num_msgs=1;bind_interface_str=;stats=true;ergonomics=true;name=VERIFY_SUSPECT;bind_addr=/fe80:0:0:0:9d:eeff:fee5:c543%eth0;id=13;timeout=1500;use_icmp=false)
:pbcast.NAKACK(use_mcast_xmit=false;ergonomics=true;xmit_table_msgs_per_row=10000;xmit_table_max_compaction_time=600000;xmit_stagger_timeout=200;print_stability_history_on_failed_xmit=false;retransmit_timeouts=3000;discard_delivered_msgs=true;max_msg_batch_size=100;exponential_backoff=300;xmit_table_num_rows=5;stats=true;xmit_from_random_member=false;log_discard_msgs=true;log_not_found_msgs=true;xmit_table_resize_factor=1.2;name=NAKACK;id=15;max_rebroadcast_timeout=2000;use_mcast_xmit_req=false;use_range_based_retransmitter=true)
:UNICAST(ergonomics=true;xmit_table_msgs_per_row=1000;xmit_table_max_compaction_time=600000;max_retransmit_time=60000;xmit_interval=2000;timeout=400,800,1600,3200;max_msg_batch_size=500;conn_expiry_timeout=60000;xmit_table_num_rows=100;stats=true;xmit_table_resize_factor=1.2;name=UNICAST;id=12;segment_capacity=1000)
:pbcast.STABLE(cap=0.1;stability_delay=1000;stats=true;ergonomics=true;name=STABLE;desired_avg_gossip=50000;max_bytes=4000000;id=16)
:pbcast.GMS(max_join_attempts=0;print_local_addr=true;handle_concurrent_startup=true;view_bundling=true;leave_timeout=1000;log_view_warnings=true;ergonomics=true;resume_task_timeout=20000;use_flush_if_present=true;print_physical_addrs=true;join_timeout=5000;view_ack_collection_timeout=2000;stats=true;num_prev_views=20;merge_timeout=5000;max_bundling_time=50;name=GMS;num_prev_mbrs=50;id=14;log_collect_msgs=true)
:pbcast.STATE_TRANSFER(stats=true;ergonomics=true;name=STATE_TRANSFER;id=17)

My security group has all ports open to itself over TCP.

As you can see, the call to AWS succeeds and finds the two servers. But the server identities somehow don't seem to make it over to JGroups.

Any ideas?

Support cross-region node discovery

I am running the demo on 2 nodes in different regions with a user with full EC2 access and security group with a configuration of all inbound and outbound traffic. Also, I have given the nodes tags with the same key.

<com.meltmedia.jgroups.aws.AWS_PING
     port_number="7800"
     tags = "Application"
     access_key="**"
     secret_key="****"/>

The tag has the same value on both the nodes. Is this a problem with the chat demo or is this a problem with the AWS discovery?

screen shot 2017-12-23 at 10 28 29 pm

Readme has incorrect xml filename 'aws-ping.xml' in mvn java command, should be 'aws_ping.xml'

When i ran the following command:
mvn exec:java -Dexec.mainClass="org.jgroups.demos.Chat" -Dexec.args="-props conf/aws-ping.xml"
I got the following error:
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:java (default-cli) on project jgroups-aws: An exception occured while executing the Java class. null: InvocationTargetException: unable to load class for protocol conf (either as an absolute - conf - or relative - org.jgroups.protocols.conf - package name)! -> [Help 1]

After playing around with for a while, i realised the filename was wrong. Perhaps it got renamed in the repo but the readme never got updated.

The following command does work fine:
mvn exec:java -Dexec.mainClass="org.jgroups.demos.Chat" -Dexec.args="-props conf/aws_ping.xml"

Would a failure in AWS_PING.findMembers() be terminal for a node?

Hello,

This isn't really a code issue, just a question about it, and I'm not sure there's a more appropriate forum. Also, I'm still using 1.4.2, though the question should be general enough to be applicable to the latest release.

I have a web application running on several instances. One of them saw this stack trace a few days ago:

Aug 26, 2018 11:11:19 AM org.jgroups.protocols.MERGE2$FindSubgroupsTask findAndNotify
SEVERE: JGRP000242: FindSubgroupsTask failed
com.amazonaws.services.ec2.model.AmazonEC2Exception: AWS was not able to validate the provided access credentials (Service: AmazonEC2; Status Code: 401; Error Code:
 AuthFailure; Request ID: c172da23-4bbb-44b2-9531-bb55b607d5d3)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1630)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1302)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
    at com.amazonaws.services.ec2.AmazonEC2Client.doInvoke(AmazonEC2Client.java:16126)
    at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:16102)
    at com.amazonaws.services.ec2.AmazonEC2Client.executeDescribeInstances(AmazonEC2Client.java:7846)
    at com.amazonaws.services.ec2.AmazonEC2Client.describeInstances(AmazonEC2Client.java:7821)
    at com.meltmedia.jgroups.aws.AWS_PING.getInstanceTags(AWS_PING.java:519)
    at com.meltmedia.jgroups.aws.AWS_PING.getPrivateIpAddresses(AWS_PING.java:378)
    at com.meltmedia.jgroups.aws.AWS_PING.findMembers(AWS_PING.java:328)
    at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244)
    at org.jgroups.protocols.Discovery.down(Discovery.java:388)
    at org.jgroups.protocols.MERGE2$FindSubgroupsTask.fetchViews(MERGE2.java:446)
    at org.jgroups.protocols.MERGE2$FindSubgroupsTask._findAndNotify(MERGE2.java:374)
    at org.jgroups.protocols.MERGE2$FindSubgroupsTask.findAndNotify(MERGE2.java:359)
    at org.jgroups.protocols.MERGE2$FindSubgroupsTask$1.run(MERGE2.java:330)
    at org.jgroups.util.TimeScheduler2$RecurringTask.run(TimeScheduler2.java:563)
    at org.jgroups.util.TimeScheduler2$MyTask.run(TimeScheduler2.java:495)
    at org.jgroups.util.TimeScheduler2$Entry.execute(TimeScheduler2.java:400)
    at org.jgroups.util.TimeScheduler2$1.run(TimeScheduler2.java:260)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

My question is quite simple: was that a terminal failure for that node, or would JGroups have tried again and presumably re-joined the cluster? (The "authentication failure" was due to something transient AWS-side—the credentials in use are fine.)

(I don't have logging for JGroups turned up high enough to see what happened next, but there was some application-level weirdness that might be explained if JGroups had died on that node.)

Configure cluster with Tomcat 9

Hi, how can I configure Tomcat 9 to use cluster/session replication with jgroups-aws?
I could not find how to configure tomcat server.xml.
Thanks

Make access_key and secret_key optional

Access to the AWS APIs can be controlled by creating a role with ec2:Describe* and adding that role to the EC2 instance. In this case, the access_key and secret_key would not be required. They should either both be present or both absent.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.