How to use url_pattern ? about spring-cloud-zuul-ratelimit

[github-actions]

Hello @denapucaco, thanks for getting in touch, we will get back to you asap! If you have issues in the 1.x.x.RELEASE line we recommend you to update to the latest version, unfortunately this line is not supported anymore.

[marcosbarbero]

The configuration you added to the description looks odd to me. Are you using it exactly the way you described? I'm talking about the full configuration not just the url_pattern

[denapucaco]

@marcosbarbero Thanks for your reply.

Below is the zuul config for service we are trying to add rate limiting. Please let me know if I am missing anything for url-pattern.

zuul:
  routes:
    audit-api:
      path: /gw/audit/**
      serviceId: audit-api-ms
      stripPrefix: true
      retryable: true
  ratelimit:
    enabled: true
    repository: JPA
    policy-list:                # Zuul rate limit policy list
      audit-api:                # custom policy name
        - limit: 2              # request number limit per refresh interval window
          refresh-interval: 60  # in seconds
          type:
           - user
           - url_pattern=/api/v1.0.0/audit/*

audit-api-ms:
  ribbon:
    listOfServers: ${SERVER_LIST_AUDIT}
    ServerListRefreshInterval: 3600000

When I just use the url without value then rate table has below rows after I try few endpoints, so it rate limiting is working for all the urls if no url or url_pattern is mentioned in type. But we want it for just few urls

mi-gateway:audit-api:lossmgr:/api/v1.0.0/audit/statuscode/IPG -1 57011 2021-08-16 16:03:57
mi-gateway:audit-api:lossmgr:/api/v1.0.0/audit/statuscode/COM -1 57419 2021-08-16 16:04:06
mi-gateway:audit-api:lossmgr:/api/v1.0.0/audit/statuscode/WAT -1 30756 2021-08-16 16:03:22

Thanks you

[marcosbarbero]

I see, I'm not on my computer now but I suspect the serviceId being different than the uri is causing the problem.

Can you try to use audit-api-ms on the policy-list?

If it works then we have a bug 😬

[denapucaco]

Sure, I'll try and get back to you.

Thanks
Prashant

[denapucaco]

@marcosbarbero As per your suggestion tried with audit-api-ms in the policy-list. This didn't solve the issue.

I tried to debug the code to understand what could be causing the issue with url_pattern and found that -

com.marcosbarbero.cloud.autoconfigure.zuul.ratelimit.filters.AbstractRateLimitFilter Class ; policy method

        policies = properties.getPolicies(routeId).stream()
                .filter(policy -> applyPolicy(request, route, policy))
                .collect(Collectors.toList());

When used the url_pattern the above code returns the empty list. I think because of this rate limiting is not working with url_pattern

I tried with url again and this time I could make it work with below config

  ratelimit:
    enabled: true
    repository: JPA
    policy-list:                # Zuul rate limit policy list
      audit-api:                # custom policy name
        - limit: 2              # request number limit per refresh interval window
          refresh-interval: 60  # in seconds
          type:
           - user
           - url=/api/v1.0.0/audit/statuscode/INP
        - limit: 1              # request number limit per refresh interval window
          refresh-interval: 60  # in seconds
          type:
           - user
           - url=/api/v1.0.0/audit/statuscode/COM
        - limit: 3              # request number limit per refresh interval window
          refresh-interval: 60  # in seconds
          type:
           - user
           - url=/api/v1.0.0/audit/statuscode/WAT

This created below rows in rate table. The URL is added twice in the rate_key. Is this a normal behavior?

mi-gateway:audit-api:lossmmgr:/api/v1.0.0/audit/statuscode/INP:/api/v1.0.0/audit/statuscode/INP -1 48423 2021-08-17 16:51:25
mi-gateway:audit-api:lossmmgr:/api/v1.0.0/audit/statuscode/COM:/api/v1.0.0/audit/statuscode/COM -1 43007 2021-08-17 16:51:42
mi-gateway:audit-api:lossmmgr:/api/v1.0.0/audit/statuscode/WAT:/api/v1.0.0/audit/statuscode/WAT -1 55993 2021-08-17 16:54:41

For now we want to use rate limiting with few URLs but we would like to make the url_pattern work. If you have any suggestions please let me know

Thanks,
Prashant

[marcosbarbero]

Thanks for the detailed response.

Can you try using ** instead of *?

[denapucaco]

I think I have tried it already but will try it one more time tomorrow and will update you.

[denapucaco]

@marcosbarbero Thank you for your help, I am able to make it work with url_pattern

I had to debug the code to understand the issue.

The request.getRequestURI() in the apply method of URL_PATTERN in RateLimitType class contains the URI with the path /gw/audit/. Since we are using the stripPrefix: true I was not adding this path in the url_pattern. ( url worked fine without this path )

Along with the path addition had to use ** as the AntPathMatcher uses them. I have added the config that worked for me below.

    /**
     * Rate limit policy considering an URL Pattern
     */
    URL_PATTERN {
        @Override
        public boolean apply(HttpServletRequest request, Route route, RateLimitUtils rateLimitUtils, String matcher) {
            return new AntPathMatcher().match(matcher.toLowerCase(), request.getRequestURI().toLowerCase());
        }

Working Config

zuul:
  routes:
    audit-api:
      path: /gw/audit/**
      serviceId: audit-api-ms
      stripPrefix: true
      retryable: true
  ratelimit:
    enabled: true
    repository: JPA
    policy-list:                # Zuul rate limit policy list
      audit-api:                # custom policy name
        - limit: 2              # request number limit per refresh interval window
          refresh-interval: 60  # in seconds
          type:
           - user
           - url_pattern=/gw/audit/api/v1.0.0/audit/**
           - http_method=get