whargrove / qlog Goto Github PK

Hi @whargrove this is Franco from Cribl, again :).

Please see the behavior when querying empty files.

• Request issued is:
  http://localhost:8080/queryLog?relativePath=empty
• Process goes up to +250% in CPU utilization.
  
• Request doesn't complete:
  `franco@franco:~$ curl -v "http://localhost:8080/queryLog?relativePath=empty"

• Trying 127.0.0.1:8080...
• Connected to localhost (127.0.0.1) port 8080 (#0)

GET /queryLog?relativePath=empty HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.81.0
Accept: /

`
Environment:

Hi @whargrove this is Franco from Cribl. Please see the scenario below. I'm seeing the application consuming +250% of CPU on my laptop when:

• specifying a term that can't be found easily (i.e. at the end of the file) AND/OR
  curl -v "http://localhost:8080/queryLog?relativePath=large.csv&count=1&filter=cribl
• when asking the service for too many lines:
  curl -v "http://localhost:8080/queryLog?relativePath=large.csv&count=999999999

*Where large.csv is a 4.3GB file with 28,741,648 lines that doesn't contain the searched term cribl.

Can you please take a look at each case and see whether this is expected behavior or there's something you can do to improve perf?

Thanks!

Environment:

The service consumes a lot of CPU, and will hang if the filter query is not found.

To reproduce:

• echo "line one" > /var/log/smallfile
• curl -Ss "http://localhost:8080/queryLog?relativePath=smallfile&filter=WILLNOTFIND"