polysemy-research / polysemy Goto Github PK

What would be the proper way to write the following so the (Connection -> IO a) callbacks are an abstract monad?

{-# LANGUAGE DataKinds #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE FunctionalDependencies #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE PolyKinds #-}
{-# LANGUAGE RankNTypes #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TemplateHaskell #-}
{-# LANGUAGE TypeApplications #-}
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE TypeOperators #-}

module Main where
  
-------------------------------------------------------------------------------
import Control.Monad.IO.Class (liftIO)
import Data.Pool
import Data.Time.Clock (NominalDiffTime)
import Data.Word (Word16)
import Database.PostgreSQL.Simple (ConnectInfo(..), Connection, close, connect)
import Database.PostgreSQL.Simple (withTransaction)
import Lens.Micro
import Lens.Micro.Extras (view)
import Lens.Micro.TH
import Polysemy
import Polysemy.Resource
-------------------------------------------------------------------------------

data PgPoolConfig
  = PgPoolConfig
  { pgPoolConfigStripes :: Int
  , pgPoolConfigResourceTime :: NominalDiffTime
  , pgPoolConfigKillTime :: Int
  }
makeFields ''PgPoolConfig

data PgConfig
 = PgConfig
 { pgConfigConnectInfo :: ConnectInfo
 , pgConfigPgPoolConfig :: PgPoolConfig
 }
makeFields  ''PgConfig

-- | Smart constuctor for 'ConnectInfo' from postgresql-simple
mkConnectInfo :: String -> Word16 -> String -> String -> String -> ConnectInfo
mkConnectInfo = ConnectInfo

data Postgres m a where
  CreatePostgresPool :: PgConfig -> Postgres m (Pool Connection)
  RunPostgres :: Pool Connection -> (Connection -> IO a) -> Postgres m a
  RunPostgresTrans :: Pool Connection -> (Connection -> IO a) -> Postgres m a
makeSemantic ''Postgres

runPostgresIO
  :: ( Member (Lift IO) r
     , Member Resource r
     )
  => Semantic (Postgres ': r) a
  -> Semantic r a
runPostgresIO = interpret $ \case
  CreatePostgresPool cfg ->
    sendM $ createPool (connect i) close s t k
    where
      s = view (pgPoolConfig . stripes) cfg
      t = view (pgPoolConfig . resourceTime) cfg
      k = view (pgPoolConfig . killTime) cfg
      i = view connectInfo cfg
  RunPostgres p f ->
    bracket (sendM $ takeResource p)
            (sendM . uncurry (flip putResource))
            (\(c, _) -> sendM $ f c)
  RunPostgresTrans p f ->
    bracket (sendM $ takeResource p)
            (sendM . uncurry (flip putResource)) $ \(c, _) ->
              sendM $ withTransaction c (f c)

Following up on error I mentioned in #65, I can get an error like:

solveSimpleWanteds: too many iterations (limit = 4)
  Set limit with -fconstraint-solver-iterations=n; n=0 for no limit
  Simples = {[WD] $d(%,,%)_akon {0}:: Member
                                        (Lift m0) r0 (CNonCanonical),
             [WD] hole{co_akoo} {0}:: String ~ x (CNonCanonical)}
...

to happen in two ways with the Teletype interpreter in the readme, i.e starting with:

runTeletypeIO :: Member (Lift IO) r => Sem (Teletype ': r) a -> Sem r a
runTeletypeIO = interpret $ \case
  ReadTTY      -> sendM getLine
  WriteTTY msg -> sendM $ putStrLn msg

1. Change interpret to reinterpret. I'd expect it to give an error similar to the interpret/interpretH one advertised in the readme.
2. Leave a function not fully applied. For instance, this happens if I forgot to put msg as the input to putStrLn and instead wrote sendM putStrLn. The solution is pretty obvious here since putStrLn is so common but it can take a while to hunt this down in other scenarios.

Then we can get rid of the interpretInStateT, and stateful and friends --- a nice simplification of Polysemy.Internal.Combinators

This should be pretty easy: runState s = flip S.runStateT s . hoistStateIntoStateT, and then eliminate dead code from there.

I've got a logging effect https://github.com/adamConnerSax/knit-haskell/blob/master/src/Knit/Effects/Logger.hs. I'd like to provide something like (where log :: Member (Logger a) effs => a -> Semantic effs ())

import Control.Monad.Log
import Polysemy
instance (Member (Logger a) effs) => MonadLog a (Semantic r) where
  logMessageFree inj = mapM_ log (inj $ pure @[])

(mind you, I've no idea of that's a correct instance but that's neither here nor there for this issue) and I get an error because of the functional dependency in MonadLog:

• Illegal instance declaration for ‘ML.MonadLog a (Semantic effs)’
        The liberal coverage condition fails in class ‘ML.MonadLog’
          for functional dependency: ‘m -> message’
        Reason: lhs type ‘Semantic effs’ does not determine rhs type ‘a’
        Un-determined variable: a
    • In the instance declaration for ‘ML.MonadLog a (Semantic effs)’

Which is fair since I could have mutiple logger instances, etc. So I can't write this instance, right?
But then I have a problem, or might. Because what should people do with an existing function which has that MonadLog constraint? I tried also to write an instance like

instance (MonadLog a m, Member (Lift m) effs) => MonadLog a (Semantic effs) where
  logMessageFree inj = sendM @m $ logMessageFree inj

but that has the same issue, plus a new, inference-related one (even with the TypeApplication):

 • Ambiguous use of effect 'P.Lift'
      Possible fix:
        add (Member (P.Lift m0) effs) to the context of 
          the type signature
      If you already have the constraint you want, instead
        add a type application to specify
          'm0' directly
    • In the ambiguity check for an instance declaration
      To defer the ambiguity check to use sites, enable AllowAmbiguousTypes
      In the instance declaration for ‘ML.MonadLog a (Semantic effs)’

I guess I can give this particular issue up but I am wondering a bit about the general interoperability story when it comes to supporting effects that are baked into existing code via a typeclass constraint where that class has a fundep.

As I've been thinking about common effects and compatibility with existing libraries, I started thinking about an interpreter zoo.

The main thing that comes to mind is the pile of logging abstractions and exception handlers floating around Hackage.

It makes some kind of sense for common effects (Input, Output, State, etc.) to live in the polysemy "core" since they can be defined without any dependencies, but interpreters potentially require arbitrary dependencies. If there's an identifiable common API between these log libraries, for instance, the effect could be written once in polysemy, then a user could use polysemy-co-log or polysemy-monad-logger or whatever rather than writing their own interpreters.

To this end, it'd be interesting to work on an ecosystem of interpreters for a variety of primitive effects. Mainly because there are some non-trivial optimizations that need to happen (late specialization, INLINE pragmas, your compiler plugin, etc.) to make things really fast, and paying that cost the minimum number of times would be nice.

Is this sensible? Or is the vision for the library that it ought to be easy enough to write these interpreters that having "blessed" configurations would be unnecessary? On the other hand, are interpreters so application-specific as to make sharing them sort of pointless?

#36 talks about interop with mtl, but could we also support MonadUnliftIO? I haven't tried anything at all yet, just raising this from the lack of instance MonadUnliftIO (Sem r). I mostly need this from the context of writing interpret with something that uses MonadUnliftIO. Fortunately at the moment I can actually use liftIO to use MonadUnliftIO IO, but that won't always work.

Hey, I have an application that throws various errors:

main :: IO (Either S3.S3Error (Either Dynamo.DBError (Either CouldntDecodeThing [Message])))
  application
    & ...
    & runError      @CouldntDecodeThing
    & runError      @Dynamo.DBError
    & runError      @S3.S3Error
    & runM

Those nested eithers aren't nice, so I wanted to aggregate the errors into a single data type:

data AggregateError
  = AggregateDBError Dynamo.DBError
  | AggregateS3Error S3.S3Error
  | AggregateSchemeError String
  deriving (Show)

aggregateDBError
  :: ( Member (Error AggregateError) r )
  => Sem (Error Dynamo.DBError ': r) a
  -> Sem r a
aggregateDBError = interpretH $ \case
  Throw dbError -> throw $ AggregateDBError dbError
  Catch action handler -> do
    a  <- runT action
    h  <- bindT handler

I got this far but I'm not really sure how to implement the Catch interpretation. Does anyone have any thoughts?

The th-abstraction package gives a significantly nicer view of GADTs. This would make it much easier to fix #48 and any other hidden TH bugs that nobody has run into yet.

Consider the following code as discussed in #53 :

deleteKV :: forall k v r. Member (KVStore k v) r => k -> Sem r ()
deleteKV k = updateKV k (Nothing @v)

based off of polysemy-zoo's KVStore effect.

This thing is type-ambiguous in v, even though it's not effect-ambiguous. But when -XNoAllowAmbiguousTypes is set, this will complain about an ambiguous usage of KVStore k v :(

Consider this:

    wv     <- input
    h      <- bindT hook
    istate <- getInitialStateT

    void $ sendM $ after wv #keyPressEvent $ \kp -> do
      kv   <- get kp #keyval
      void $ lower .@ runNavigation $ h $ chr (fromIntegral kv) <$ istate
      pure False

even though hook returns a Bool, h returns an f Bool. But an f Bool is no good because I need a regular bool for the line below!

DANG DOES THIS EVER SUCK.

The problem is that we don't know which higher-order interpreters may have already run that have thrown off our chances for static evacuation of f. Ok, but the interpreters themselves know. What if we added a constructor to Yo:

evacuate :: (forall x. f x -> Maybe x)

Effects like state could say "yeah no problem bruh." Effects like NonDet or Error, nope, no can do. But I'm reasonably sure the only Yos you have to deal with are the ones that have already run above you, so as long as you keep your Errors low in the stack, it should be fine?

If evacuate is feasible, we can stick it into Tactics and then in some cases call the callback above. Is this a good idea? I don't know, it's 2am.

https://github.com/isovector/polysemy/blob/630b27d026dd9f749756c2f986d8f83113ba0177/src/Polysemy/Internal/Combinators.hs#L222

Should "reinterpret2" be "reinterpret3"?

https://github.com/isovector/polysemy/blob/ff222d826a91b5c7cabad1a5dc9858c55afbca64/polysemy-plugin/test/InlineRecursiveCallsSpec.hs#L24

This should use the (===) operator, but fails unless I use (==-). But the good news is that this is a bug in inspection-testing and should be fixed as soon as nomeata/inspection-testing#34 is merged.

Trying to understand the code and have a couple of questions about the names of the data types.

Sem, short for Semantics, seems anything but. To me, the data type represents a DSL program short of semantics, which are provided to it in the form of the function (∀ x. Union r (Sem r) x -> m x). Perhaps a better name for it would be Free(r), Eff, Syntax, or Dsl. In the talk, it was named Freer.

What is the inspiration for the name: Yo?

For the Union data type, does union there pertains to the effect types?

Kind (* -> *) ->* should have an extra -> *

Require '[] around Lift m

I have the following:

{-# language BlockArguments #-}
{-# language DataKinds #-}
{-# language FlexibleContexts #-}
{-# language GADTs #-}
{-# language RankNTypes #-}
{-# language ScopedTypeVariables #-}
{-# language TemplateHaskell #-}
{-# language TypeOperators #-}

module Polysemy.Async where

import Control.Concurrent.Async ( Async, async )
import Control.Monad.IO.Class
import Polysemy
import Prelude


data Concurrent m a where
  Fork :: m a -> Concurrent m ( Async a )


makeSem ''Concurrent


runWithConcurrency
  :: forall r a.
     Member ( Lift IO ) r
  => ( forall x. Sem r x -> IO x )
  -> Sem ( Concurrent ': r ) a
  -> Sem r a
runWithConcurrency toIO =
  interpretH \( Fork m ) -> do
    fm <- runT m

    let run :: forall a. Sem ( Concurrent ': r ) a -> IO a
        run = toIO .@ runWithConcurrency
    
    fmap _ ( liftIO ( async ( run fm ) ) )

But I'm stuck at the final hole:

Found hole: _ :: Async (f a1) -> f (Async a1)

I need to distribute Async over f, but I can't see how to do that.

Been playing around with and enjoying the library but think I could benefit from some general "understanding" help!

What I wanted to do was implement the KVStore in polysemy-zoo interpreter for Elasticsearch (ES), using Bloodhound (see here for relevant docs).

My working code that I have so far:

esIndex = IndexName "index1"
esMapping = MappingName "mapping"

data ElasticsearchKVError = 
    ResponseParseError Text
  | KnownEsError Text

runKVStoreBloodhound :: FromJSON v
  => Members '[Reader (BHEnv), Lift IO, Error ElasticsearchKVError] r
  => Sem (KVStore k v ': r) a 
  -> Sem (Error ElasticsearchKVError ': r) a
runKVStoreBloodhound = reinterpret $ \case
  LookupKV k    -> do
    bhEnv <- ask 
    reply <- runBH bhEnv $ getDocument esIndex esMapping (DocId "12345")
    case eitherDecode (responseBody reply) of 
      Left err          -> throw $ ResponseParseError (pack err)
      Right esResponse  -> return $ Just esResponse
  UpdateKV k v  -> undefined

Note that the above doesn't do /exactly/ what would be expected but I left the rest out since it's unimportant for this question. In any case, I think the signature describes it pretty well: I want to have some way of getting BHEnv so that I can run ES actions, which will run in IO, and want some way to throw errors specific to this domain when things go wrong (and leave that for some upstream interpreter to handle).

The part I'd like to understand more is the ask/runBH part. I had originally written this outside of the do declaration, like: LookupKV k -> ask >>= flip runBH do ... but realized that didn't work when I put in the throw statements. I'm assuming this is because by wrapping more (than just the getDocument call) with runBH, I'm now running in MonadBH and have no access to my member constraint methods via Sem.

I'm assuming there's some way around this; ideally I would like to define ask >>= flip runBH or similar at the top level of this function, so that inside I have access to all the ES functions as well as my member constraint methods.

Is this possible? I tried to get this working myself with some combination of raise/lift/etc but could never get it to typecheck. Also I'd be interested in hearing from folks on whether this is a good approach in the first place, or if there's some better way to mix free effects with others, preferably without manually re-writing all the library functions in terms of Sem.

The TH currently spits out smart constructors like this:

data Foo e m a where
  FooC :: Foo e m ()

fooC :: forall e r. Member (Foo e) r => Sem r ()
fooC = send FooC

Uh oh! The usage of FooC here is ambiguous! The plugin will correctly determine what it should be (send $ FooC @e), but it'd be nicer if the TH just did the right thing to begin with.

We want to generate this:

fooC :: forall e r. Member (Foo e) r => Sem r ()
fooC = send (FooC @e)

Since it has an external dependency, let's move it out of the core package.

https://github.com/isovector/polysemy-zoo

They're used in Effect right now, but I'm pretty sure they're not necessary. If they can be removed, we can target older GHC versions.

I'm not really sure how to title this issue better, but anyway, I've been playing with polysemy and trying to make it work with Reflex, which is where I do most of my development nowadays.

I managed to succeed in writing a fixpoint effect (below), but writing an interpret requires either AllowAmbiguousTypes or a Proxy argument, which isn't the end of the world, though I expected more from the typechecker...

data Store s t (m :: * -> *) a where
  Get :: Store s t m (Dynamic t s)
  Put :: Event t s -> Store s t m ()

runStore ::
     forall m s t r a.
     ( Typeable s
     , Typeable t
     , Reflex t
     , MonadHold t m
     , Member Fixpoint r
     , Member (Lift m) r
     )
  => s
  -> Semantic (Store s t ': r) a
  -> Semantic r a

Where I ran into trouble was in the next step, when trying to simultaneously use the effect and the GUI elements from Reflex-DOM. Despite having only one (Lift m) effect member, I've needed to annotate all calls to sendM with @m. As before, this isn't completely unusable but it's pretty annoying.

widget :: forall t m. (Typeable t, MonadWidget t m) => m ()
widget = runM $ runFixpointM runM $ runStore @m @Text "" $ do
  x <- get @Text @t
  dynText x
  dText <- sendM @m $ textInput def
  eSave <- sendM @m $ button "Save"
  put (current (value dText) <@ eSave)

Trying a third time, I thought that I could write a Semantic instance for the necessary typeclasses so that I could use the functions directly, as all Reflex functions are parametric in the monad, but that attempt ended abruptly after running into:

instance (Reflex t, DomBuilder t m, Member (Lift m) r) => DomBuilder t (Semantic r)

-- Illegal instance declaration for ‘DomBuilder t (Semantic r)’
--         The liberal coverage condition fails in class ‘DomBuilder’
--           for functional dependency: ‘m -> t’
--         Reason: lhs type ‘Semantic r’ does not determine rhs type ‘t’
--         Un-determined variable: t

When writing up this issue, I thought of introducing a newtype wrapper for Semantic with another type parameter, say

newtype SemR t r a = SemR { unSemR :: forall m. Member (Lift m) r => Semantic r a }

but quickly trying that, the definition doesn't even typecheck due to Ambiguous use of effect 'Lift'.

By now I've long ran out of my type-level programming knowledge and, more importantly, the time I've set aside for exploring this, so while I'd love to continue, it'll have to wait some two weeks.

This issue is not meant to report a problem with the library, it is more of a request for advice, but I'd say it's closely related to #15 with problems with the base monad...

First of all great job on the library, I'm having a great time playing around with it!

My issue now: when using an editor like HIE on VSCode to play with polysemy, it fails to compile in-editor on pieces that require the plugin, I suppose since HIE doesn't get started with the plugin. I tried working around this by putting
{-# OPTIONS_GHC -fplugin=Polysemy.Plugin #-}
at the top of my module but that didn't seem to force it through HIE.

This probably isn't really specific to this plugin, but it would be nice to know if there's a way to force HIE to load a plugin (I tried searching and couldn't find anything) or otherwise get this working.

Barring that, it might be helpful to provide more examples of helping out the type checker without the plugin? I know I can fix this by writing it "correctly" in the first place, but am having a hard time even figuring that out. For what it's worth, the snippet I'm currently trying to get working without the plugin is

deleteKV :: Member (KVStore k v) r => k -> Sem r ()
deleteKV k = updateKV k Nothing

from Polysemy.KVStore in polysemy-zoo.

I am trying to create a custom effect which handles going to a database. Here is a minimal example:

-- select returning an 'a'
data Select a = Select a

data DBAction whichDb m a where
  DoSelect :: Select a -> DBAction whichDb m (Maybe a)

makeSem ''DBAction

The idea is that one effect interpreter will maintain a pool of connections and these actions will be reduced to actual IO actions against them. However, in a program that works with multiple databases there needs to be some way to disambiguate which pool of connections to use for each query. I added the whichDb type variable to describe this and get this error:

    • Ambiguous use of effect 'DBAction'
      Possible fix:
        add (Member (DBAction whichDb0) r) to the context of 
          the type signature
      If you already have the constraint you want, instead
        add a type application to specify
          'whichDb0' directly
    • In the ambiguity check for ‘doSelect’
      To defer the ambiguity check to use sites, enable AllowAmbiguousTypes
      In the type signature:
        doSelect :: forall a_Xfbf (whichDb_afbf :: k_afpj).
                    forall r_afXj.
                    Member (DBAction whichDb_afbf) r_afXj =>
                    Select a_Xfbf -> Sem r_afXj (Maybe a_Xfbf)
   |
84 | makeSem ''DBAction
   | ^^^^^^^^^^^^^^^^^^

Turning on polysemy-plugin makes that go away, but then a similar message pops up at the use sites, even with a TypeApplication. I can make it go away as well by putting whichDb as an argument to the data constructor, which is probably OK because it'll mean no more work for users (I say users, but it's just me) of doSelect than adding a type application.

I'm wondering if I've organized the effect wrong (so this is just PEBKAC) or if there's something to be done with polysemy. I am willing to help, would probably just need a little hand-holding - I am only just learning the type-level stuff.

Somewhat related, maybe it would be useful to add an explanation in the haddocks for exactly what code makeSem generates and why. Again, I am willing to help as best I can - thank you for making a very usable library!

I've been trying to verify that Polysemy.Resource works correctly with Polysemy.Error, but I can't even work out how to compose the two interpreters. I've tried:

runM . runErrorInIO runM . runResource (runM . runErrorInIO runM)

But the thing given to runResource isn't forall x. m x -> IO x, because runErrorInIO produces Either e x. What should I do?

Hmm. Trying to write slightly more general interpreters for running things that might be sitting "on top of" other monads where all we have are constraints for them. Should that work? I am getting

 • Overlapping instances for Polysemy.Internal.Union.Find
                                  '[P.Lift IO, P.Lift m] (P.Lift m)
        arising from a use of ‘KPM.runPandoc’
      Matching instances:
        instance forall k (z :: [k]) (t :: k) (_1 :: k).
                 (Polysemy.Internal.Union.Find z t,
                  Polysemy.Internal.Union.Found (_1 : z) t
                  ~ 'Polysemy.Internal.Union.S
                      (Polysemy.Internal.Union.Found z t)) =>
                 Polysemy.Internal.Union.Find (_1 : z) t
          -- Defined in ‘Polysemy.Internal.Union’
        instance [overlapping] forall k (t :: k) (z :: [k]).
                               Polysemy.Internal.Union.Find (t : z) t
          -- Defined in ‘Polysemy.Internal.Union’
      (The choice depends on the instantiation of ‘m’
       To pick the first instance above, use IncoherentInstances
       when compiling the other instance declarations)
    • In the expression: KPM.runPandoc @m
      In an equation for ‘runPandoc’: runPandoc = KPM.runPandoc @m
      In the expression:
        do let runPandoc ::
                 forall a.
                 Semantic '[Pandoc,
                            KLog.Logger KLog.LogEntry,
                            KLog.PrefixLog,
                            PE.Error KPM.PandocError,
                            P.Lift IO,
                            P.Lift m] a
                 -> Semantic '[KLog.Logger KLog.LogEntry,
                               KLog.PrefixLog,
                               PE.Error KPM.PandocError,
                               P.Lift IO,
                               P.Lift m] a
               runPandoc = KPM.runPandoc @m
...

I think the Alternative is False, True, but the NonDet interpretation is True, False.

I wrote some code like this:

type S3BucketName = Text
type S3Key = Text

fn
  :: ( Member (Input S3BucketName) r
     , Member (Input S3Key) r
     )
  => Sem r a
fn = do
    bucketName <- input @S3BucketName
    key        <- input @S3Key
...

And expected that I could provide the correct values to each input like so:

fn
  & runConstInput     @S3BucketName "bucket"
  & runConstInput     @S3Key "key"

But this just applies "bucket" to both arguments. In hindsight, this is obvious because S3BucketName and S3Key are just type aliases for Text, making them newtypes instead worked:

newtype S3BucketName = S3BucketName Text
newtype S3Key = S3Key Text

I know this seems a silly thing to point out (it's expected behavior for type aliases), but it's not something that I expected. I thought it'd be worth pointing out, just in case it can be resolved or documented.

The following program crashes the plugin:

{-# LANGUAGE TemplateHaskell #-}

module Polysemy.KVStore where

import Polysemy

data KVStore k v m a where
  LookupKV :: k -> KVStore k v m (Maybe v)
  UpdateKV :: k -> Maybe v -> KVStore k v m ()

makeSem ''KVStore

foo :: Member (KVStore k v) r => Sem r (Maybe Bool)
foo = do
  lookupKV "hello"

instead, it should be rejected saying couldnt prove k ~ String or v ~ Bool

#46 points out that the example code doesn't work when built with stack. The stackage guys are mad at me because it's broken there too (commercialhaskell/stackage#4572)

In my tests, downgrading polysemy-plugin to 0.1.0.0 seems to fix the problem.

I couldn'ŧ wait for the release so I tried to see whether I could use this package in GHCJS (where I do most of my development). My build system (nix) however sees that there's no cabal file and tries to compile and use GHCJS's hpack, which is not possible as the package 'yaml' doesn't work there.

Do you think you could include the cabal file? I think stack regenerates it automatically whenever package.yaml changes, so it shouldn't add any more work...

After #63 @isovector took the time to engage me on a video call and explained how Tactical mechanics work. They appear to be something you wouldn't commonly use, but may run into edge cases where they are necessary.

What (I think) I learned

• When you have some outside m context (something in IO, a mtl-style call, etc) that needs to be ran, you can't incorporate it into your polysemy effects, but you need some way to run it. This is where Tactics come into play
• pureT, runT, and bindT put your external monadic function into the current Sem context as some opaque f (Sandy said this could be thought of as a monad transformer), which you execute with the (forall x. Sem r x -> IO x) runner passed in and composed with your interpreter, like f' <- bindT f; r .@ runMyInterpreterIO $ f'
• Sadly you can't treat these things as monads, so no standard combinators available

Things that may be useful for Sandy to explain

• What problem is being solved with Tactics?
• What is the opaque f and why is it synonymous with a monad transformer?
• Why do we have to fmap values passed into bound functions? boundFunc' (param <$ initialstate)
• What does returning getInitialStateT achieve when using an async function?
• Why do we have to call the runner over the same interpreter on an external effect?

I'm trying to understand how to use this library properly, and I've hit a couple of scenarios that I'm uncertain of.

Given this example effect type:

{-# LANGUAGE TemplateHaskell #-}

module Test where

import Polysemy
import Polysemy.Trace
import System.Directory

data Test m a where
  DoTest :: Test m ()

makeSemantic ''Test

1. If I have a "runE" function that needs to use some other effect, is it normal to lift that other effect into the resulting Semantic that gets produced?

ex:

runTestIO :: ( Member (Lift IO) r
             , Member Trace r
             )
          => Semantic (Test ': r) a -> Semantic r a
runTestIO =
  interpret $ \case
    DoTest -> do
      trace $ "This is a test"
      sendM $ copyFile "C:\\test.txt" "D:\\test.txt"

Should Trace show up here in the type signature? Or is it expected to runTraceIO in my handler and only get down to IO?

2. If it is OK to add effects in these run methods, how can I properly run them whenever I want to get to IO ?

Given the following code:

test1 :: (Member (Lift IO) r, Member Trace r)
      => Semantic (Test ': r) a -> Semantic r a
test1 x = runTestIO x

test2 :: Member (Lift IO) r
      => Semantic (Trace ': r) a -> Semantic r a
test2 x = runTraceIO x

testToIO :: Semantic (Test ': Trace ': Lift IO ': r) a
            -> IO a
testToIO x = do
  let a = test1 x
  let b = test2 a
  runM b -- This doesn't compile

I'm trying to break apart the nested effects step by step here to understand them, but I'm coming up a bit short. b should have the type Member (Lift IO) r => Semantic r a, but that doesn't seem to satisfy the input to runM, which is Monad m => Semantic '[Lift m] a

3. How does Semantic '[Lift m] a differ from Member (Lift m) r => Semantic r a ? Are there certain situations in which one form is more appropriate than the other? I'm fairly new to Haskell, and this syntax isn't something that I've encountered before :)

Thanks

I don't think I have a concrete issue. Just an observation:

I have some functions which returned the freer-simple Eff effs x monad, defined in a let statement in do block, returning Eff effs' x where all the constraints on effs are also constraints satisfied by effs'. These functions did not have type signatures. E.g. (very schematic example! The real case has much more...noise between the definition of myOtherFunc and where/how it is used)

f :: Int -> (Member MyEff effs) => Eff effs ()
f = ...

myFunc :: (Member MyEff effs) => Eff effs ()
my = do
  let myOtherFunc = f 2
  myOtherFunc

With freer-simple, these compiled. After switching to polysemy, some of them do not, basically because they fail to unify the effs in the do block and effs being returned from the function? I think? The error message is about a missing constraint of the form (Member MyEff effs0) on the at-that-point ambiguous effs0 at the line where the let is.

What's notable, is that if I enable PartialTypeSignatures and then put a type signature on each function of the form let myOtherFunc :: _, many of these now compile fine. Which is confusing. What does PartialTypeSignatures know that is a secret from the rest of ghc?

I don't think I can cook up an example because I don't really know why it happens sometimes and not others and the code where it happens is way too messy and full of other things to be of any use figuring this out, I think. So feel free to close this on account of it's not specific enough. But I thought you might find the information helpful!

Consider the very reasonable effect:

data Delayed m a where
  Delay :: Int -> m () -> Delayed m ()
  Cancel :: Delayed m ()


runDelayed
    :: (forall x. r@> x -> IO x)
    -> Delayed :r@> a
    -> IO ~@r@> a
runDelayed lower m = do
  ref <- sendM $ newIORef []
  runDelayed' ref lower m

-- not exported
runDelayed'
    :: IORef ([A.Async ()])
    -> (forall x. r@> x -> IO x)
    -> Delayed :r@> a
    -> IO ~@r@> a
runDelayed' ref lower = interpretH \case
  Cancel -> do
    sendM $ do
      mfuture <- readIORef ref
      for_ mfuture $ \e -> do
        A.cancel e
      modifyIORef ref $ drop 1
    getInitialStateT

  Delay time m -> do
    m' <- runT m
    sendM $ do
      a <- A.async $ do
        threadDelay time
        void $ lower .@ runDelayed' ref $ m'
        modifyIORef ref $ drop 1
      modifyIORef ref $ (++ [a])
    getInitialStateT

(that is, it's reasonable besides the fact that it races and should give you back a handle to cancel rather than just canceling whatever it wants.)

Okay, great, but now what happens if you use it like this:

runM .@ runDelayed .@ runSomethingElse

where runSomethingElse needs to lower directly into IO?

Well, what happens is that the newIORef in runDelayed gets duplicated for each lowering into IO. While that's to be expected, but it's certainly not what was intended. So that sucks.

Especially because if we generated the IORef ourselves in the containing scope, and then called runDelayed' ref directly, this function would behave as intended. But no, the hiding of the implementation details in runDelayed explicitly causes bad behavior.

This is aggressively bad semantics and is in desperate need of a solution.

Maybe a blog post?

Similar to last time I still feel like I am not grokking how to use Tactics (or how to track the proper monadic state in my head, which seems like an unnecessary burden).

Playing with the hedis library:

data MessageReader m a where
  WaitMessage :: Integer -> ByteString -> (ByteString -> m ()) -> MessageReader m ()

runMessageReaderIO
  :: forall r a.
     Member (Lift IO) r
  => Connection
  -> (forall x. Sem r x -> IO x)
  -> Sem (MessageReader ': r) a
  -> Sem r a
runMessageReaderIO conn r = interpretH $ \case
  WaitMessage mid dom f -> do
    is <- getInitialStateT
    f' <- bindT f
    sendM $ runRedis conn $ pubSub (subscribe [ mkIdKey dom mid ]) $ \msg -> do
      r .@ runMessageReaderIO conn $ f' (msgMessage msg <$ is)
      pure (unsubscribe [ msgChannel msg ])

src/Service/Redis.hs:142:5: error:
    • Couldn't match type ‘()’ with ‘f ()’
      Expected type: Sem (WithTactics MessageReader f m r) (f x)
        Actual type: Sem (WithTactics MessageReader f m r) ()
    • In a stmt of a 'do' block:
        sendM
          $ runRedis conn
              $ pubSub (subscribe [mkIdKey dom mid])
                  $ \ msg
                      -> do r .@ runMessageReaderIO conn $ f' (msgMessage msg <$ is)
                            pure (unsubscribe [...])
      In the expression:
        do is <- getInitialStateT
           f' <- bindT f
           sendM
             $ runRedis conn
                 $ pubSub (subscribe [mkIdKey dom mid]) $ \ msg -> do ...
      In a case alternative:
          WaitMessage mid dom f
            -> do is <- getInitialStateT
                  f' <- bindT f
                  sendM
                    $ runRedis conn
                        $ pubSub (subscribe [mkIdKey dom mid]) $ \ msg -> ...
    • Relevant bindings include
        f' :: f ByteString -> Sem (MessageReader : r) (f ())
          (bound at src/Service/Redis.hs:141:5)
        is :: f () (bound at src/Service/Redis.hs:140:5)
    |
142 |     sendM $ runRedis conn $ pubSub (subscribe [ mkIdKey dom mid ]) $ \msg -> do
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^...

You have a stack.yaml file, so I would have thought that reproducible builds would be guaranteed. Alas:

$ stack build
Building all executables for `too-fast-too-free' once. After a successful build of all of them, only specified executables will be rebuilt.
too-fast-too-free-0.1.0.0: build (lib + exe)
Preprocessing library for too-fast-too-free-0.1.0.0..
Building library for too-fast-too-free-0.1.0.0..
[5 of 7] Compiling Lib              ( src/Lib.hs, .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/Lib.o )

./src/Lib.hs:53:12: error:
    • Occurs check: cannot construct the infinite type: a ~ (a, s)
      Expected type: s -> Eff (State s : r) a -> Eff r a
        Actual type: s -> Eff (State s : r) a -> Eff r (a, s)
    • In the expression:
        stateful
          $ \case
              Get -> S.get
              Put s' -> S.put s'
      In an equation for ‘runState’:
          runState
            = stateful
                $ \case
                    Get -> S.get
                    Put s' -> S.put s'
    • Relevant bindings include
        runState :: s -> Eff (State s : r) a -> Eff r a
          (bound at src/Lib.hs:53:1)
   |
53 | runState = stateful $ \case
   |            ^^^^^^^^^^^^^^^^...

--  While building package too-fast-too-free-0.1.0.0 using:
      ~/.stack/setup-exe-cache/x86_64-osx/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.2 --builddir=.stack-work/dist/x86_64-osx/Cabal-2.4.0.1 build lib:too-fast-too-free exe:too-fast-too-free-exe --ghc-options " -ddump-hi -ddump-to-file"
    Process exited with code: ExitFailure 1

There seems to be a bug with higher order effects. Here is an example with the current version of polysemy on hackage (0.1.2.1) and ghc-8.6.4.

main :: IO ()
main = runM . runReader 4 $ test

test :: Sem '[Reader Int, Lift IO] ()
test = do
  ask @Int >>= liftIO . print
  local (+(4 :: Int)) $ do
    ask @Int >>= liftIO . print

Executing the main function, I would expect:
4
8

What's actually happening is:
4
4

I'm writing a set of effects that sit on top of PandocIO and thus get their MonadIO via that. But the MonadIO instance for Semantic requires that the union have Lift IO, rather than a MonadIO "LastMember"-type-thing (like freer-simple, which I was using previously). So I had to write:

runIOInPandocIO :: P.Member (P.Lift PA.PandocIO) effs => P.Semantic (P.Lift IO ': effs) x -> P.Semantic effs x
runIOInPandocIO = P.interpret ((P.sendM @PA.PandocIO) . liftIO . unLift)

then add runIOInPandocIO before runM in the function that runs the combined effects and then I have a Lift IO in the union for everything else that needs it (in this case just the Log.filteredLogEntriesToIO):

runPandocAndLoggingToIO
  :: [Log.LogSeverity]
  -> P.Semantic
       '[Pandoc, Log.Logger Log.LogEntry, Log.PrefixLog, P.Error
         PA.PandocError, P.Lift IO, P.Lift PA.PandocIO]
       a
  -> IO (Either PA.PandocError a)
runPandocAndLoggingToIO lss =
  fmap mergeEithers
    . PA.runIO    
    . P.runM
    . runIOInPandocIO
    . P.runError
    . Log.filteredLogEntriesToIO lss
    . runPandoc @PA.PandocIO

Is there a more idiomatic/better way to do this? If not, should it be documented someplace? It took me a while to figure that out!

Edit: For context, if it matters, the rest of the code is here

Water under the bridge now, but was removing the export of "Typeable" consistent with only bumping from 0.1.1.0 to 0.1.2.0?

I ask because I'm always assuming that I can safely upper bound things at < A.(B+1).0.0, but in this case that broke because I was using the Polysemy export of Typeable to add it. The constraint is harmless now that it's not needed, so removing it is backward-compatible, but now my code fails to compile because Typeable is no longer exported for me to use.

The documentation on using polysemy-plugin doesn't mention it needs to exist in your package yaml dependencies.

@adamConnerSax on #34 wrote:

A question, slightly off-topic maybe.
I feel like there is something similar about this hoist function and the MonadRandom thing we have been talking about over in polysemy-zoo.
All of them are ways of trying to accommodate a different version of an effect that polysemy supports.
So, two questions:

• Maybe these should be in a separate library, polysemy-mtl or polysemy-interop? That would get rid of the dependencies for users who don't care about mtl.
• What is the ideal form for such a thing? I can imagine three versions, though maybe not all would work equally well for all effects.

1. Something like hoist: hoistX:: Sem (X ': r) a -> XT (Sem r) a
2. Some kind of orphan instance: instance (Member X r) => MonadX (Sem r) where ...
3. A function--like liftIO---which discharges the constraint within Sem : liftX :: (MonadX m, Member X r) => m a -> Sem r a.
   I'm not quite sure how that third version would work, but it seems like it keeps most of the convenience of instances without the issue of actually having them.

I'm going to think about how to do that for MonadRandom in RandomFu.

If I have a Sem r a, is there any way that I can inspect r to see what effects are in there?

Ideally I would like to be able to dynamically determine which interpreters to run while there are remaining effects, other than Lift IO, in my Sem

Something like:

case inspectEffects sem of
  Effect1':rs -> runEffect1 sem
  Effect2':rs -> runEffect2 sem

Enabling the -fplugin=Polysemy.Plugin ghc flag will solve 90%+ of ambiguous effect errors; the custom error messages should mention this!

For new projects that use stack and wish to build on a foundation of polysemy it could be helpful to have a template which enables the necessary extensions/ dependencies.

Setting up polysemy isn't that much effort, but it could be a way to get folks up and running quickly, even just to experiment.

I'm wondering if anyone sees value in this idea.

As /u/yakrar points out on reddit, this seems to be really similar to Control.Monad.Free.Church from free, judging from the instances. Just FYI.

Maybe you can inspire @ekmett to work out the connection to his blog post series? :) http://comonad.com/reader/2011/free-monads-for-less-2/

Subj.

Doesn't it requre all eliminators (runX, runY, runZ) to be applied sequentally and independently to each argument of send? With your encoding, putting all eliminators in an array looks to me as more natural strategy. You loose efficient top-effect-removal, but running becomes as simple as (arr ! index) ... effect. At least, this array would be present in the cache all the time.

This can also spare you from having your output monad m being the last element of the effect stack - so you can now convert to any monad, not just the last one. Without some IO at the bottom you can declare that the code can't launch rockets, unless Launching effect is provided.

This is helpful - the user of the framework might want to not have IO monad necessarily stuck in her code, because some testing frameworks refuse to shrink or whatnot in IO cases.

Another advantage (or disadvantage)... Ok, lets call this a property: in array-of-dispatchers case, the order of effects is delayed until transformation into working monad, instead of being peeled in some predefined order. The order becomes fixed at the point you run the Effect effs () program.

Also, in what cases does the user want to eliminate effects one-by-one? This is not a question against that style of running, I'm just curious. Technically, you can do this in array case as well, by transforming to (pseudocode) Effects '(tail effs). I didn't try it, because I prefer the effect list to have "final" encoding, but it should be possible.

See the discussion here: #45 (comment)

Unfortunately, this issue will come up whenever there is a phantom effect parameter :|

Both these results seem wrong to me:

ghci> runM .@ runResource $ runState 0 $ bracket
        (pure ())
        (const (sendM . print =<< get @Integer))
        (const (put 2))
0
(2,())
ghci> runM .@ runResource $ runState 0 $ bracket
        (pure ())
        (const (put 2))
        (const (pure ()))
(0,())

It seems like it ought to be possible to fix this by using a custom version of bracket inside runResource that threads the state through appropriately, but if that’s necessary, then I suppose I’m a little confused about what makes the approach polysemy takes any better than MonadBaseControl with respect to dropping state.

I just got:

    • Ambiguous use of effect 'State'
      Possible fix:
        add (Member (State FeederId) Error
                                       InsertTraydatasAndTraytypesFailure
                                       : r1) to the context of 
          the type signature

I think this is actually telling me to add ( Member ( State FeederId ) r, Member ( Error InsertTraydatasAndTraytypesFailure ) r)