mtth / tracing

Distributed tracing

Tracing

An OpenTracing-compliant, simple, and extensible distributed tracing library.

• Simple: add a single MonadTrace constraint to start tracing, without making your code harder to test!
• Extensible: use the built-in Zipkin backend or hook in your own trace publication logic.

import Monitor.Tracing

-- A traced action with its root span and two children.
run :: MonadTrace m => m ()
run = rootSpan alwaysSampled "parent" $ do
  childSpan "child-a" runA
  childSpan "child-b" runB

To learn more, hop on over to Monitor.Tracing, or take a look at examples in the examples/ folder.

tracing's Issues

Allow 16 lower-hex character TraceId's

The spec seems to allow 16 lower-hex character TraceId's:
https://github.com/openzipkin/b3-propagation#traceid-2

However the parser checks explicit for a length of 16 bytes (32 hex character):

tracing/src/Control/Monad/Trace/Internal.hs

Line 45 in 82bdea2

Just bs | BS.length bs == 16 -> Just $ TraceID bs

The workaround/solution is simple - just add 16 times '0' if the TraceId string is shorter than expected; however this is something that the library could handle.

Discarding of HTTP result might hide failures.

tracing/src/Monitor/Tracing/Zipkin.hs

Line 123 in aa0cc70

void $ HTTP.httpLbs req' mgr

Do we need to check the HTTP status?
I'm unsure about the impact - but in my case a unreachable address resulted in a 502 Bad Gateway status.

Maybe we should check for 2XX results (statusIsSuccessful)? But what should we do in other cases? Just output a warning?

Background periodic publisher doesn't recover from a network exception

If a trace collector is temporarily down, a background thread that tries to reach it is expected to survive flushSpans throwing ConnectionFailure:

HttpExceptionRequest Request {
  host                 = "localhost"
  port                 = 9411
  secure               = False
  requestHeaders       = [("content-type","application/json")]
  path                 = "/api/v2/spans"
  queryString          = ""
  method               = "POST"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 (ConnectionFailure Network.Socket.connect: <socket: 54>: does not exist (Connection refused))

How to run instrumented code without span collection?

If I have instrumented code and want to disable tracing, how do I run that? I have no Zipkin-Endpoint at hand and collectSpanSamples will fill up all my memory.

So I wonder if something more intelligent than the following should be provided by the library to actually have something like a "no-operation-trace-runner".

discardSpanSamples :: MonadUnliftIO m => TraceT m a -> m a
discardSpanSamples actn = do
  tracer <- newTracer
  _ <- liftIO $ forkIO $ fix $ \loop -> do
    _ <- atomically $ readTChan (spanSamples tracer)
    loop
  runTraceT actn tracer

Provide better support for a "root-producer-span"?

If I have some software that's main purpose is to generate and ingest events, I believe something like a "root-producer-span" is useful. However implementing this is already possible with the current state of the library:

TC.rootSpanWith (ZPK.addTag "z.k" "PRODUCER") TC.alwaysSampled name $ do
      spn <- TC.activeSpan
      error "use B3 here" $ fmap b3FromSpan spn

The downside is that you need to copy b3FromSpan (also implies parentID) and use the 'magic' values "z.k" and "PRODUCER". So I wonder if it is worth to provide a convenience function in the library? Your thoughts? Could an additional function be an improvement? Or maybe it's better to just expose more internals like b3FromSpan so that client code is empowered to directly implement such things?

Provide better support for a "root-consumer-span"?

Similar to #11 - should we strive for a similar API extension that was done to support root-producers? Or will this clutter the API?

As always implementing this is already possible:

TC.rootSpanWith (ZPK.addInheritedTag (cs "service.instance.id") serviceInstance . \builder -> builder { TC.builderTags = M.insert (cs "z.k") (JSON.toJSON "CONSUMER") (TC.builderTags builder) })

Might be a rather rare scenario - in my case we have a simple MQTT3 producer which isn't able to create/transfer traces on it's own, so our consumer should create the root span.

Message Tracing span ID's

To be honest I'm still a newbie to all the details of span propagation, but what caught my intention was the fact that in this framework PRODUCER/CONSUMER spans act the same way as CLIENT/SERVER spans.

But as far as I understood https://zipkin.io/pages/instrumenting.html ("Message Tracing is different than RPC tracing because the producer and consumer don’t share span IDs."). Unfortunately the specification how things should behave seems to be inexistent - at least I couldn't find a document that describes the correct behaviour.

Please let me know if this is really an issue - my guess is that someone who wrote such a library must be an expert to these topics 😉 Thanks in advance!

Fails to build with `base16-bytestring-1.0.0.0`

Hi!

I tried downloading and building this library, but ran into a compile error! I noticed that the cabal file specifies a dependency on base16-bytestring >= 0.1, but in 1.0.0.0, there was a breaking API change. In particular, I ran into this error:

src/Control/Monad/Trace/Internal.hs:135:3: error:
    • Couldn't match expected type ‘Either String ByteString’
                  with actual type ‘(ByteString, ByteString)’
    • In the pattern: (bs, trail)
      In a case alternative: (bs, trail) | BS.Char8.null trail -> Just bs
      In the expression:
        case Base16.decode $ BS.Char8.pack $ T.unpack t of
          (bs, trail) | BS.Char8.null trail -> Just bs
          _ -> Nothing
    |
135 |   (bs, trail) | BS.null trail -> Just bs
    |   ^^^^^^^^^^^

I would love to help out making a patch to fix this, but I'm not sure what the preferred solution is. Would it be better to make a patch that restricts the version bounds to >= 0.1 && < 1.0.0.0, or would it be better to make a patch that lets the project compile against base16-bytestring 1.0.0.0?

Wrong documentation - and somehow misleading API

Guess this should be equivalent (or it least was once intended) , but it is not since tag adds the publicKeyPrefix while addTag doesn't.

tracing/src/Monitor/Tracing/Zipkin.hs

Lines 168 to 173 in 82bdea2

	-- > childSpanWith (addTag "key" "value") "run" $ action
	--
	-- Note that there is no difference with adding the tag after the span. So the above code is
	-- equivalent to:
	--
	-- > childSpan "run" $ tag "key" "value" >> action

Since the user of the library shouldn't now about the internal prefixes addTag is somehow useless (or at least a very low-level) function. So maybe it just should not be exported? Or it should be splited into an internal version that doesn't add a prefix and an exported version that also adds publicKeyPrefix?