GithubHelp home page GithubHelp logo

consuldotnet's Introduction

Consul.NET

Build Status

  • Consul API: v0.7.2
  • .NET: >= 4.5 - .NET Core: >= 1.0.0

Consul.NET is a .NET port of the Go Consul API, but reworked to use .NET idioms such as Tasks/CancellationTokens instead of Goroutines/Channels. The majority of the calls directly track the HTTP API, but this API does have additional functionality that is provided in the Go API, like Locks and Semaphores.

Example

You'll need a running Consul Server on your local machine, or a Consul Agent connected to a Consul Server cluster. To run a local server:

  1. Download a copy of the latest Windows version and unzip it into the Consul.Test folder.
  2. Open a command prompt and cd to the Consul.Test folder.
  3. Run .\consul.exe agent -dev -config-file test_config.json

This creates a 1-server cluster that operates in "dev" mode (does not write data to disk) and listens on 127.0.0.1:8500.

Once Consul is running (you'll see something like consul: cluster leadership acquired) in your command prompt, then do the following steps in your project.

Add a reference to the Consul library and add a using statement:

using Consul;

Write a function to talk to the KV store:

public static async Task<string> HelloConsul()
{
    using (var client = new ConsulClient())
    {
        var putPair = new KVPair("hello")
        {
            Value = Encoding.UTF8.GetBytes("Hello Consul")
        };

        var putAttempt = await client.KV.Put(putPair);

        if (putAttempt.Response)
        {
            var getPair = await client.KV.Get("hello");
            return Encoding.UTF8.GetString(getPair.Response.Value, 0,
                getPair.Response.Value.Length);
        }
        return "";
    }
}

And call it:

Console.WriteLine(HelloConsul().GetAwaiter().GetResult());

You should see Hello Consul in the output of your program. You should also see the following lines in your command prompt, if you're running a local Consul server:

[DEBUG] http: Request /v1/kv/hello (6.0039ms)
[DEBUG] http: Request /v1/kv/hello (0)

The API just went out to Consul, wrote "Hello Consul" under the key "hello", then fetched the data back out and wrote it to your prompt.

Usage

All operations are done using a ConsulClient object. First, instantiate a ConsulClient object, which connects to localhost:8500, the default Consul HTTP API port. Once you've got a ConsulClient object, various functionality is exposed as properties under the ConsulClient.

All responses are wrapped in QueryResponse and WriteResponse classes, which provide metadata about the request, like how long it took and the monotonic Consul index when the operation occured.

This API also assumes some knowledge of Consul, including things like blocking queries and consistency modes

ACL

The ACL endpoints are used to create, update, destroy, and query ACL tokens.

Agent

The Agent endpoints are used to interact with the local Consul agent. Usually, services and checks are registered with an agent which then takes on the burden of keeping that data synchronized with the cluster. For example, the agent registers services and checks with the Catalog and performs anti-entropy to recover from outages.

Catalog

The Catalog is the endpoint used to register and deregister nodes, services, and checks. It also provides query endpoints.

Event

The Event endpoints are used to fire new events and to query the available events.

Health

The Health endpoints are used to query health-related information. They are provided separately from the Catalog since users may prefer not to use the optional health checking mechanisms. Additionally, some of the query results from the Health endpoints are filtered while the Catalog endpoints provide the raw entries.

KV

The KV endpoint is used to access Consul's simple key/value store, useful for storing service configuration or other metadata.

Query

The Prepared Query endpoints are used to create, update, destroy, and execute prepared queries. Prepared queries allow you to register a complex service query and then execute it later via its ID or name to get a set of healthy nodes that provide a given service.

Session

The Session endpoints are used to create, destroy, and query sessions.

Status

The Status endpoints are used to get information about the status of the Consul cluster. This information is generally very low level and not often useful for clients.

Additional Functions

Functionality based on the Consul guides using the available primitives has been implemented as well, just like the Go API.

Lock

Lock is used to implement client-side leader election for a distributed lock. It is an implementation of the Consul Leader Election guide.

Semaphore

Semaphore is used to implement a distributed semaphore using the Consul KV primitives. It is an implementation of the Consul Semaphore guide.

Using with .NET Core and Mono

Both .NET 4.5+ and .NET Core 1.0+ are fully supported. Mono is supported on a best-effort basis. It should compile and run happily on Mono but this is not as heavily tested as Microsoft .NET stacks. If you have any issues using the Nuget package or compiling this code with .NET, .NET Core, or Mono, please file a Github issue with details of the problem.

consuldotnet's People

Contributors

akatz0813 avatar anwright-ms avatar banshooter avatar driis avatar eugenyshchelkanov avatar evgenyk avatar freein avatar grounded042 avatar highlyunavailable avatar hungwunfai avatar latop2604 avatar lockwobr avatar mjgoethe avatar normanhh3 avatar pliner avatar shmutalov avatar vchekan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

consuldotnet's Issues

DNS Lookup Service

Currently, I use Consul for service discovery in ASP.NET Application. But I don't know how to use DNS lookup in Consul Client .NET

Expose configuration options interface?

What are your thoughts on making a public read-only interface of the configuration options?

We had some tests confirming that configuration options were passed into the client correctly from a wrapper class somewhat similar to https://github.com/PlayFab/consuldotnet/blob/master/Consul.Test/ClientTest.cs#L28

Now that the Config object is internal those tests do not compile obviously. These tests are obviously redundant to your own tests, although still somewhat comforting to confirm our wrapper is working correctly.

I guess since its internal, i think actively debugging it would be able to display how the client is configured, which is probably enough for 90% of uses. But exposing the options in a read-only way might be nice for logging weird cases.

The type or namespace name 'PushStreamContent' could not be found

Severity Code Description Project File Line Suppression State
Error CS0246 The type or namespace name 'PushStreamContent' could not be found (are you missing a using directive or an assembly reference?) Consul C:\Users\xxx\Desktop\consuldotnet-master\consuldotnet-master\Consul\Client.cs 949 Active

Create an interface for the consul client class.

Hi,

It would be nice to have an interface for the consul client class so that dependency injection/unit testing with mocks could be done properly.
I'm happy to do this and submit a pull request.

Don't (a)wait for me

Hi,

I'm thinking about start working with Consul in my .Net environment, I saw your cool client and it's seems one of the best tools to use for .Net developers.

However, when I dive into the code I saw a lot (really a lot) of places you're using the async await pattern but you're waiting for nothing. for example:

      /// <summary>
        /// Checks returns the locally registered checks
        /// </summary>
        /// <returns>A map of the registered check names and check data</returns>
        public async Task<QueryResult<Dictionary<string, AgentCheck>>> Checks()
        {
            return await _client.Get<Dictionary<string, AgentCheck>>("/v1/agent/checks").Execute().ConfigureAwait(false);
        }

I don't understand why there is await over there, nothing happens afterwards, why not just return the Task from the Execute method and let the user decide if he wish to await for that or not.

As far as I know there is a little bit of overhead when using IAsyncStateMachine.

So to my final questions:

  1. What do I miss? Is there a meaning behind the awaits? (Just before returning the tasks).
  2. This tool seem cool and I want to help, do you have any thoughts about the future?

Thanks,

Question about service discovery a la Microphone

Hello,

I came across this project after researching microservices, specially service discovery in C#, which led me to the Microphone project and from there to here.

My ultimate aim is to use something like Owin-self-hosting, TopShelf, ILRepack and Consul to build small console exes that behave as RESTful microservices and that can simply be deployed onto boxes by copying one exe and have it "just work" - configuration and discovery being automatic.. It would be great for scalability and reliability too.

Microphone looked like a great start, but then I discovered that it doesn't actually use your Consul.Net library. That is not a show-stopper in itself, but that project also seems to be heading towards being agnostic about its back end and providing lots of options (Nancy, ETCD etc.) whereas my feeling is, if you're going to pick Consul, use it, and build something small and targeted at Consul. Too many options just makes for developer confusion and extra maintenance burden. Plus, it seems to be written using DotNetCore which means it pulls in about 40 dependencies and is hard to ILRepack.

So I was wondering about taking the principles and design expressed in Microphone and and just creating (let's say) Consul.Net.ServiceDiscovery.

I have had a look over the Microphone source and there is not a lot of code there, it doesn't feel like a massive job, though I'll need to get my employer to let me do it on their time.

I noted your question here, to which you never got a reply: rogeralsing/Microphone#21

Have you thought about this at all, and do you have any particular direction in mind for Consul.Net?

dotnet client has a really bad performance

i create a dev server on 192.168.15.235 and my c# client is on 192.168.15.24.

the simple sample cost about 150ms every request KV.Get. and i used wireshark to test the real latency. and the one single request cost about 1ms when the response came back.

50  16:22:26.949815 192.168.15.24   192.168.15.235  HTTP    154 GET /v1/kv/hello HTTP/1.1 
51  16:22:26.950450 192.168.15.235  192.168.15.24   HTTP    337 HTTP/1.1 200 OK  (application/json)

Set Proxy and Timeouts

What is the best method to set proxy settings and timeouts for the underlying httpclient used by consuldotnet?

Cannot reuse ConsulClientConfiguration for multiple clients

The following throws an exception of Cannot access a disposed object.Object name: 'System.Net.Http.WebRequestHandler'.

This is because ConsoleClient.Dispose also calls down to Config.Handler.Dispose() and Config only creates the Handler in the constructor.

void Main()
{
    var config = new ConsulClientConfiguration();

    using (var client = new ConsulClient(config))
    {
        Console.WriteLine(client.Catalog.Services().Result.StatusCode);
    }
    using (var client = new ConsulClient(config))
    {
        Console.WriteLine(client.Catalog.Services().Result.StatusCode);
    }
}

I would expect the Configuration settings should not be a holder for a WebRequestHandler, or should rebuild it if the config object is reused. OR the constructor of the client should take in a Action<ConsulClientConfiguration> to show it can't simply be reused.

Unable to alter Consul server endpoint from localhost:8500 using ConsulClient(Action<...>) approach

Hi there,

I would to change the Consule server endpoint from localhost:8500 to something else. I tried to instantiate the client object using the following code:

using (var client = new ConsulClient(x => new ConsulClientConfiguration() 
{ Address = new Uri("http://consul.local:8500") }))
            {
....

}

However when I run the code, the result looks like it's still pointing to localhost:8500. Am I doing the configuration wrong?

Thank you for your help.

ILRepack Newtonsoft.Net

Hi guys!

Thank you for the library 👍

But we have one small trouble using your library - Newtonsoft.Json dependency. Our code base depends on 6.0.8 version and it's very difficult to update it to latest.

What do you think about ilmerging this dependency?

Cannot register service without a port

I've been using the following code to register services:

var serviceRegistration = new AgentServiceRegistration
{
    ...
};

Client.Agent.ServiceRegister(serviceRegistration);

It appears there is no way of not setting a port. Am I wrong?

Port is an non-nullable int, which means it is always set, despite the attribute declaring nulls are ignored.

Depends on .NET 4.5.1 in 0.7.0

I have large suolution with 300+ projects targeting .NET 4.5 and I have no chance to target .NET Framework 4.5.1 in next 6 month. Last NuGet package targeting this .NET is v0.6.4.7.

So what can I do now to use Consul v.0.7?

Consul.Test doesn't like Mono

"Core" Consul version 0.6.3.3 (bleeding edge as I write this) loads into MonoDevelop ok (which is just a heuristic for getting along with Mono on my Xubuntu / mono 4.2.2 machine). However, Consul.Test is unhappy with more than just the xunit.runner.visualstudio.2.1.0 Import element declarations in Consul.Test.csproj.
I'd like to potentially contribute to this project, but I'm not able to resolve the above "safely" for users on Windows. If this were resolvable, I'd be in a position to run the tests....
Kind regards

LockTryOnce behaviour

Hi,
When specifying LockWaitTime in LockOptions, if LockTryOnce is set, the expected behavior is to block for the lock acquisition for a max of LockWaitTime just once,
The blocking GET call that happens for the lock doesn't block until the wait time is coupled with index value. But the index value isn't obtained until after the first call is made(which ignores the wait time since index to wait on isn't supplied) and this first call fulfills the LockTryOnce criteria and throws an exception after that. So the blocking wait never happens in case LockTryOnce is set.
Can this be fixed so that even in case the requested lock key is already acquired, the wait happens for the specified wait time once?

Accepting Timeout value from the Configuration

Hi,

In my application I try to join to couple of server nodes (so if the one hard coded server is temporary down, I'm covered :))

However I've seen in Client.cs that you initializing the ConsulClient.HttpClient.Timeout with a hard coded value of 15 minutes:
HttpClient.Timeout = TimeSpan.FromMinutes(15);

(Which mean for me if one Consul node is down or have difficulties responding I need to wait 15 minutes for the result)

I was wondering why this is the value and if I can refactor this to be a configurable value from the ConsulClientConfiguration

By the way, I saw there is a Wait property in the ConsulClientCofiguration is it for consul usage?

Thank you very much.

Publishing service state via API sends incorrect note string

Method Agent.UpdateTTL has parameter note which value gets encoded via Uri.EscapeDataString(note) before adding to web request parameters dictionary. It is incorrect because in method ConsulRequest.BuildConsulUri all uri parameter values gets encoded via Uri.EscapeDataString. So when we publish some note with service state, its value gets encoded 2 times.

Correct standard service check (proves that it is not consul core bug):
image

Incorrect service check:
image

I can send PR with bug fix and tests.

Make tests for deserialization

Make a generic way to determine whether deserialization "worked" (that is, all JSON message fields were consumed to set a field) for all consul message. Brought up by @lockwobr in #23 .

Serialization/Dependency Questions

Hello - would you entertain accepting a pull request that would break the dependency of your project on Json.Net? This could be achieved by exposing a serialization interface and moving the Json.net specific code into a separate nuget package as an adapter.

Connect to remote server

Sorry about the question but the documentation doesn't not having enough info.
Can this library connect to remote Consul server (not localhost and default port) to register itself as a service for specific data center !

Add abortable lock

If a lock is lost, add an abort of the action if it's not completed within the TTL.

Support for TPL & async await

I would like to use your library to add support for Consul to Microsoft Orleans (https://github.com/dotnet/orleans) but the lack of async support is a big issue for that project.

I see there is a PR which has switched the client code to use HttpClient, do you intend to build on this and move to supporting async requests?

Thanks for the work so far!

Accept `CancellationToken` in higher-level APIs

The higher-level APIs (eg, ConsulClient.Catalog.Service()) do not accept CancellationToken arguments. It would be nice if they did.

It appears that the methods they call into (eg, Consult.GetRequest<T>.Execute()) do accept a CancellationToken, so it should just be a matter of adding method overloads and passing the argument through.

Write better docs

Write docs about how to, from a C# perspective

  • Construct and use the ConsulClient class
  • Do blocking queries/"watches"
  • Wrap distributed locks around critical sections
  • Use query metadata

Status Code always 0

Checking the status code of the k/v get response always return 0.

    var getPair = await client.KV.Get(key, setting);
    if (getPair.StatusCode == System.Net.HttpStatusCode.OK) // Alaways fail 
    {
       var value = Encoding.UTF8.GetString(getPair.Response.Value, 0, getPair.Response.Value.Length));
         }

Specifying a WaitTime using ConsulClientConfiguration.WaitTime results in a "bad request"

Hi,

If I set ConsulClientConfiguration.WaitTime to a value (I've tried 10 seconds and 5 minutes, so sane values) then when I use ICatalogEndpoint.Service it fails with the error message:

Unexpected response, status code BadRequest: Invalid wait time

If I use QueryOptions instead then it works.

I've not tried any other calls so this might not affect just ICatalogEndpoint.Service.

Don't finalize me, just dispose

Hi,

I've seen that you use Finalizer in your code,

    ~ConsulClient()
        {
            if (HttpClient != null)
            {
                HttpClient.Dispose();
            }
        }

Is there any reason why the ConsulClient does not implement the IDisposable interface and you just add a Dispose method? (Finalizer are not so common in the .Net framework and should be used when there is absolutely no other way to achieve the goal. You're in the Managed world, Enjoy it my friend :))

As I see it the class should like like this:

public partial class ConsulClient : IDisposable

/* Some awesome code (without finalizers */

public Dispose()
{
 if (HttpClient != null)
            {
                HttpClient.Dispose();
            }
} 

What do you think?

Make Consul.Net more unit-test friendly

Hi,

I'm not able to create unit tests that handle the response from ICatalogEndpoint.Service (and similar calls) because QueryResult<>.Response has an internal setter. Could the setter be made public, please? I'm guessing that's the least disruptive means of solving this problem.

Thanks.

KV operations in .net core platform are very slow

I added "Consul": "0.7.0.4" to my .net core 1.0 project via nuget in vs2015. The problem was all KV operations on Consul took 5000ms+ while no more than 10ms in .net framework 4.5 platform. Consul version is 0.7.1 in centos7. Here is my test code.

    static async Task TestConsulKV(string flag)
    {
        Stopwatch _watcher = new Stopwatch();
        Stopwatch _subWatcher = new Stopwatch();
        _watcher.Start();
        _subWatcher.Start();
        var _count = 0;
        var _consul = new ConsulClient(config => config.Address = new Uri("http://172.18.70.71:8500"));
        for (; _count < 5; _count++)
        {
            var _key = $"schema/test/{_count}";
            var _existsPair = await _consul.KV.Get(_key);
            if (_existsPair.Response != null)
            {
                Console.WriteLine(Encoding.UTF8.GetString(_existsPair.Response.Value));
            }
            var _kvPair = new KVPair(_key);
            _kvPair.Value = Encoding.UTF8.GetBytes($"{flag ?? "default"}{_count}");
            _consul.KV.Put(_kvPair).Wait();

            //if (_count % 100 == 0)
            //{
            Console.WriteLine($"finished {_count} in {_subWatcher.ElapsedMilliseconds}ms");
            _subWatcher.Restart();
            //}
        }
        Console.WriteLine($"all finished in {_watcher.ElapsedMilliseconds}ms");
    }

.net core result:
netcore

netframework4.5 result:
netframework451

Strong Named Project

Hi,

Would you be able to release a strongly named version of this library to Nuget?

I have a project which requires strong naming in assemblies and this failed the build. I was able to pull down the source and build myself but that's not as ideal as nuget as I will miss updates and such.

Thanks,

Mark

compile error

when i compile, display “无法在 finally 子句体中等待” in await _client.Session.Destroy(id).ConfigureAwait(false);

Is ConsulClient thread safe?

Hello.
I want to use one ConsulClient for application - is it thread safe? Or I should just create it (in using) wherever I need cause the creation has minimal overhead and internal connection pooling is used?

Support for other Watch types?

Hello
I see that consultdotnet supports Watches on KV pairs.
Is there support for other Watch types (services, nodes...)? Or, can this be implemented in some way (i.e. I missed something)?
Kind regards
Edward

Session no longer exists errors need to be handled

From @rmul:

I have an issue and would like to check if you think this is related:
When a semaphore has been aqcuired and the session gets lost (for instance use the api to destroy the session) the RenewPeriodic method throws a "Session no longer exists" exception but catches that also and a cycle later a "Session timed out" exception is thrown which is not handled by the semaphore.acquire method.
If it is not related I am more than willing to create a new issue and help out with it.


This is a broader error in that both locks and semaphores need to be looked at.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.