kimcuhoang / dapr-tye-simple-microservices

An example of building .NET Core microservices with Dapr and Tye

simple-microservices

An example of building .NET Core microservices with Dapr and Tye

Overview

• There are 3 services in this repo products-api, inventories-api, graphql-api run as Dapr clients which can be started by dapr run or tye run.
• Observability
  • Distributed tracing: zipkin
  • Distributed logging: seq

Dapr Building Block

1. Services Communication

   • Pub/Sub : Redis
     • Create a product at products-api then publish an event to inventories-api to create a product as well.
   • Service Invocation (aka Service Discovery)
     • Listing products at products-api within inventories information from inventories-api

2. Observability

   • Distributed Tracing with zipkin
   • Metrics

3. Ingress

Prerequisites

1. .NET Core 3.1
2. Docker for desktop
3. Install Dapr 0.9
4. Install Tye 0.4

Running Locally

There are 2 options to start services

1. Starts Services with Tye which is default mode

   • In this mode, all of services has been started by Tye. Also, we utilize the extensions which are supported out-of-the-box.

2. Starts Services with Darp

   • By running this mode, we use Tye as docker-compose to start the infrastructure, i.e. seq and sqlserver. Then, all services will be started manually with dapr run command.

Why do we need an option to start services with Dapr while Tye can start with only one command?

Perhaps, this question maybe rise up when every one touch this repository. It just because this repo aims to apply purely Dapr before apply Tye in order to understand

1. How Dapr works
2. Without Tye
   • We have to plug serilog and its extension for seq to implement Distributed Logging
   • The services need to be predefined with specific port number
3. If we apply Tye, we do not need to do the above stuffs, because it simplify microservices development by making it easy to:

   • Run many services with one command
   • Use dependencies in containers
   • Discover addresses of other services using simple conventions

Deploy to Kubernetes

After experience on locally and see how Tye is useful, then we may want to step up by deploy to Kubernetes, of course with Tye as well. This guide is step-by-step of:

• Install infrastructure via Helm: SqlServer, Redis, Seq, Zipkin and even Dapr
• Using tye deploy to deploy our micro-services to Kubernetes
• Using NGINX Ingress Controller to avoid kubectl port-forward
• Monitoring with Grafana and Prometheus

Resources

• HttpClientFactory .NET Core 2.1
• Issue: Globalization Invariant Mode is not supported while using EntityFramework Core with dotnet core alpine images
• Github Actions Documentation
• Dapr
  • Darp Doc
• Serilog Best Practices
• 5 ways to set the URLs for an ASP.NET Core app
• Introduction Project Tye
• MediatR Pipeline Behaviour in ASP.NET Core – Logging and Validation

Give a Star! ⭐

If you liked this project or if it helped you, please give a star ⭐ for this repository. Thank you!!!

dapr-tye-simple-microservices's Issues

Utilize Dapr .Net Core SDK

For purpose:

• PubSub
• Invoke remote services (aka Service Discovery or Service invocation)

TODO:

PubSub Scenario

• Remove CloudNative.CloudEvents.AspNetCore
• Install Dapr.AspNetCore from Nuget
• Add [Topic("ProductCreated")] for subscribe action which is ProductController > CreateProduct

        [Topic("ProductCreated")]
        [HttpPost("ProductCreated")]
        public async Task<IActionResult> CreateProduct([FromBody]CreateProductRequest request)

• Modify Startup.cs > Configure

            app.UseCloudEvents();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapGet("/", context =>
                {
                    context.Response.Redirect("/playground");
                    return Task.CompletedTask;
                });
                endpoints.MapSubscribeHandler();
                endpoints.MapControllers();
            });

Invoke Method Scenario

• Currently, product-catalog-api retrieves inventory info of products via api/inventories. We're going to use v1.0/invoke/inventories/method/get-list instead
• Inventories Api
  • Controllers > InventoriesController

  [HttpPost("get-list")]
  public async Task<IActionResult> GetInventories([FromBody] GetInventoriesRequest request)

Resources:

• Exploring Dapr: Pub/Sub – Part 2: ASP.NET Core integration
• Invoke remote services

Apply project tye

What have we done before using tye?

• By utilizing dapr, we've developed a microservices application. In which, we enable the following capabilities:
  • Pub/Sub mechanism
  • Service Discovery; also known as Service Invocation which is a term in dapr.
• Observability
  • Distributed Tracing: we can easily integrate with Zipkin while using dapr since it supports out-of-the-box, here is the document
  • Distributed Logging: by utilizing serilog within seq

Limitation or inconvenience

• We all know the service once started with dapr should be assigned a random port number. It's convenience until we started using GraphQL and stiched schema functionality. Since the main GraphQL API need to know all of the other services; and it does not know exactly what the port number of other services at runtime. Therefore, we have to define all of the port numbers in services.json; and in the implementation, we have to make each service starts with the corresponding pre-defined port number.
• We have to start the zipkin and seq containers before starting services
• We start the services separately

What is tye?

Tye is a tool that makes developing, testing, and deploying microservices and distributed applications easier. Project Tye includes a local orchestrator to make developing microservices easier and the ability to deploy microservices to Kubernetes with minimal configuration.

What can tye do?

• Simplify microservices development by making it easy to:
  • Run many services with one command
  • Use dependencies in containers
  • Discover addresses of other services using simple conventions
• Deploy .NET applications to Kubernetes by:
  • Automatically containerizing .NET applications
  • Generating Kubernetes manifests with minimal knowledge or configuration
  • Using the same conventions as development to keep it consistent

Time to apply tye

• Install tye
• Initialize by run the command tye init; It will create the tye.yaml file which includes our services here

Enable zipkin

• By adding zipkin extension to tye.yaml - refer the document

extensions:
 - name: zipkin

• In the implementation, we have to install Microsoft.Tye.Extensions.Configuration and use the below code to retrieve the address of service

configuration.GetServiceUri("service_name");

Enable dapr

• By adding dapr extension to tye.yaml - refer the document

extensions:
 - name: dapr
   log-level: debug
   config: tracing

Enable seq logging

• By adding seq extension to tye.yaml - refer document

extensions:
  - name: seq
    logPath: ./.logs

Resources

https://devblogs.microsoft.com/aspnet/introducing-project-tye/

Deploy to Kubernetes with Tye

Prerequitsites

• Docker for Desktop and enabled Kubernetes

Steps

• Install the following charts via Helm

  • SqlServer
  • Redis which serves for Dapr
  • Seq which serves for Logging
  • Zipkin which serves for Tracing
  • NGINX ingress controller which serves for Seq and Tye's ingress

• Install Dapr with Kubernetes

• Install components for Dapr: pubsub, statestore, zipkin

• Install configuration for Dapr's tracing

• Deploy our services with Tye

Notes:

• When using tye deploy with already defined ingress; they just help us to install NGINX Ingress Controller; we have to define Ingress for us

ProductCatalog and Inventories integration

Requirement:

• Create Product in ProductCatalog context should publish a notification in order to create Product in Inventory context automatically
• Get Product from ProductCatalog context should include its assignments in Inventories context

Format Json for Strongly-Typed Id

Problem:

[
	{
		"productId": {
			"id": "8300d0f7-c54c-420a-b9da-afb1213fef79"
		},
		"name": "Product-3"
	},
	{
		"productId": {
			"id": "43860c5d-4244-4c8e-b455-be8c58a67d95"
		},
		"name": "Product-1"
	},
	{
		"productId": {
			"id": "9d90c1b8-623a-4d29-85b5-d24b9ec61d91"
		},
		"name": "Product-2"
	},
	{
		"productId": {
			"id": "2dd7b97e-01bc-426e-8bac-d591fd218e52"
		},
		"name": "Kim"
	}
]

Expected:

[
	{
		"productId": "8300d0f7-c54c-420a-b9da-afb1213fef79",
		"name": "Product-3"
	},
	{
		"productId": "43860c5d-4244-4c8e-b455-be8c58a67d95",
		"name": "Product-1"
	},
	{
		"productId": "9d90c1b8-623a-4d29-85b5-d24b9ec61d91",
		"name": "Product-2"
	},
	{
		"productId": "2dd7b97e-01bc-426e-8bac-d591fd218e52",
		"name": "Kim"
	}
]

Solutions:

1. Define the custom of JsonConverter
2. It also need to create a custom of TypeConverter in order to create a specific instance of IdentityBase (i.e. ProductId). It happens in the below flow

• User post a request which is for example

{
	"productId": "8300d0f7-c54c-420a-b9da-afb1213fef79",
	"name": "Product-3"
}

• Then, our code will convert to ProductId within value 8300d0f7-c54c-420a-b9da-afb1213fef79

Resources:

1. https://andrewlock.net/using-strongly-typed-entity-ids-to-avoid-primitive-obsession-part-2/
2. https://weblogs.thinktecture.com/pawel/2019/10/aspnet-core-3-0-custom-jsonconverter-for-the-new-system_text_json.html

Add Inventory Service

Domain

• Inventory : Aggregate

  • Id
  • Location
  • Name
  • Products : List

• Product : Aggregate

  • Id
  • Code
  • Inventories: List

• ProductInventory : Entity

  • ProductInventoryId
  • Product
  • Inventory
  • Quantity
  • CanPurchase

Persistence

• EntityFrameworkCore
• Use-cases
  • Get Inventories
  • Create Inventories
  • Create Product
  • Get Products
  • Add or Update ProductInventory

Api

• Enable GraphQL

Integrate with GraphQL Api

Support Docker

Requirement

• Ability to deploy services in docker

Solution:

1. Write Dockerfile for each service

• Use mcr.microsoft.com/dotnet/core/sdk:3.1.201-alpine3.11 for build stage
• Use mcr.microsoft.com/dotnet/core/aspnet:3.1.3-alpine3.11 for runtime

2. Define services in docker-compose.yaml
3. Define overrides configurations for services in docker-compose-deploy.yaml
4. Define docker-compose-dev.yaml for working in development mode

• only use sqlserver & redis

Add GraphQL with hotchocolate

Resources:

1. https://hotchocolate.io/docs/introduction.html
2. https://github.com/ChilliCream/hotchocolate

Suppose that the following use-cases has been developed

• Get all products
• Create new product
• Update existing product

ProductCatalogApi

• Add package HotChocolate.AspNetCore (contains the ASP.NET core middleware for hot chocolate and also includes the Hot Chocolate query engine as a dependency)
• Add package HotChocolate.AspNetCore.Playground (in order to write & execute query)
• Define the following objects
  • Query which contains the method GetProducts, it sends the GetProductsRequest via IMediator then receive GetProductsResponse. It's Query in term of CQRS
  • Mutation similar to Query but execute CUD actions (aka Create, Update, Delete). It's also Command in term of CQRS
• Define ObjectType classes:
  • ProductType which represents for Product in GraphQL
  • GetProductsRequestType which represents for GetProductsRequest in GraphQL
  • GetProductsResponseType which represents for GetProductsResponse in GraphQL
  • QueryType which represents for Query in GraphQL
  • MutationType which represents for Mutation in GraphQL
• Define InputObject classes
  • CreateProductsRequestType which represents for CreateProductsRequest in GrapQL
  • UpdateProductRequestType which represents for UpdateProductRequest in GrapQL
• Register for GraphQL in Startup.cs

services
                .AddGraphQL(sp => Schema.Create(cfg =>
                {
                    cfg.RegisterServiceProvider(sp);
                    cfg.RegisterQueryType<QueryType>();
                    cfg.RegisterMutationType<MutationType>();
                }), new QueryExecutionOptions
                {
                    IncludeExceptionDetails = true,
                    TracingPreference = TracingPreference.Always
                });

//In order to run our server we now just have to add the middleware.
            app.UseGraphQL("/graphql");

            //In order to write queries and execute them it would be practical if our server also serves up Playground
            app.UsePlayground(new PlaygroundOptions
            {
                QueryPath = "/graphql",
                Path = "/playground",
            });

            app.UseRouting();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapGet("/", context =>
                {
                    context.Response.Redirect("/playground");
                    return Task.CompletedTask;
                });
            });

Applying Dapr Pub-Sub

Resources

• Dapr
• Documents

Applying Pub-Sub

1. Concepts of Publish/Subscribe Messaging
2. Configure Redis
3. Publish Topic
4. Consume Topic
5. Examples

TODO

ProductCatalog

• Use HttpClient to implement the publisher mechanism by making a POST request to http://localhost:{DAPR_HTTP_PORT}/v1.0/publish/ProductCreated. In which, DAPR_HTTP_PORT must be retrieved from Environment => DaprPublisher class
• After created a new Product, consume DaprPublisher above to publish the information of created product

Inventories

• Remove HostedService that listen on Redis which was implemented in #10 before
• Add CloudNative.CloudEvents.AspNetCore from nuget; since Dapr use CloudEvent format to communicate between Publisher and Subscribers
• Define the endpoint which will
  • Use ProductCreated as route template
  • Accept POST vertb
  • Receive CloudEvent as argument
• Under Config method of Startup, should define the endpoint dapr/subscribe which will be return the array that contains only ProductCreated string as Json

Others

• Remove SimpleStore.Infra.RedisBus

Add Log to track every request

Problem:
There is lack of logging for every request, hence, if any issue happen it's not easy to trace the flow

Solution:

• Add Serilog.AspNetCore

Resource:

• https://github.com/serilog/serilog-aspnetcore
• https://nblumhardt.com/2019/10/serilog-in-aspnetcore-3/

Support CI with Github Actions

Resources:

• https://help.github.com/en/actions/getting-started-with-github-actions
• https://hjerpbakk.com/blog/2019/10/03/asp-net-core-and-github-actions
• https://rajbos.github.io/blog/2019/10/26/Deploy-dotnetcore-webapp-with-GitHub-Actions

TODO

1. Create folders .github\workflows
2. Define ci-simple-microservices.yml which is the definition of CI process, in which we do the following steps

• Watch on push and pull_request to master branch
  • Ignore the following path
    • README.md
    • assets/**
    • Helm/**
• Checkout the new commits
• Install .NET Core with version 3.1.201
• Install the dependencies via dotnet restore command
• Build ProductCatalog Api
• Build Inventories Api
• Build GraphQL Api

3. Open README.md and add the badge icon

![ci](https://github.com/kimcu-on-thenet/simple-microservices/workflows/ci-simple-microservices/badge.svg)

Notes:

• Since we're using the workflow name ci-simple-microservices which was declared in ci-simple-microservices.yml; therefore, in the badge's url, we must use ci-simple-microservices
• Otherwise, we can use the path of yml file instead of workflow's name, i.e. .github/workflows/ci-simple-microservices.yml

Ability to deploy on Kubernetes (k8s)

TODO

1- Define Helm chart for each Api
2- Practical install Dapr to Kubernetes

• dapr init --kubernetes
• install and configure Redis for Statetore and PubSub

Resources:

1. Install Dapr on Kubernetes cluster
2. Install Helm
3. Getting Started with Helm
4. Template everything with Helm