Set up factories for our tests to DRY them up about papercups

[yakryder]

Yep!

[MuhammadMullah]

Hey @MuhammadMullah! Hope you're well. Just checking if you are still looking at this.

Hey @Sherspock, I was but I got so busy. But yeah sure you can go ahead and work on it. Because I am moving slowly because of a lot of work I have currently

[yakryder]

@reichert621 Sounds great on both counts!

[yakryder]

@MuhammadMullah Sounds good! Very understanding of you.

@reichert621 @MuhammadMullah
Sorry for the radio silence. I didn't have as much time as I thought I would to dig in this weekend, but I was still able to do a fair bit of thinking on the subject and dig into some writing today.

Long story short, I might stick with the ecosystem standard for the first iteration of this functionality.

In somewhat more detail:

Departing from the ecosystem standard is a little scary if you don't feel your alternative is rock solid or liable to get there soon, and/or you can't find an alternate pattern readily available.

I thought I'd be able to find another pattern in-ecosystem that addressed the concerns I outlined earlier in the thread, but the research I did came up empty. I'm sure it's out there but I didn't find it this time.

Failing that, I thought I'd be able to pull over an alternate factory pattern I'm familiar with from Ruby, but it too proved to be less cut-and-dry than I had hoped.

There's a lot of different pieces to how fixtures live in the test suite right now -- i.e. the valid and invalid attrs, the fixtures themselves (sometimes pattern-matched, sometimes differently named), the test helpers -- one thing that this and ongoing work in a similar vein should help continue to consolidate and standardize. That together with being new to the codebase and not an experienced hand at Elixir also took some time.

As such, I've been writing today to the standard outlined in the Ecto docs at the top of the issue. I think I should have time to finish an all-knowing factory pattern for the base builder function and slack conversation threads, along with switching the fixture references in the corresponding test files, tomorrow!

[yakryder]

@reichert621 @m1ome
I had a "Eureka!" moment on this last week, my enthusiasm for which I hope is warranted. It might not have a place in papercups, but it's definitely something I want to get down on paper. I will probably offer it to the Ecto docs as a possible alternative as well assuming I haven't overlooked any glaring issues.

Basically and in brief, what I'm thinking of now for a factory implementation are modules containing two or three of the following sets of members:

1. A set of sample data-providing functions that return potentially persistible structs, possibly implementing a BlueprintStore protocol that has at least two callbacks: simple_valid/1 which returns a struct that will save, and simple_invalid/0 for one that won't.

This would make it possible to see all the data-providing functions on the builder when dotting the module, in addition to putting doc tags on each of these functions to provide hoverable additional details of what they are.

2. A set of fluent builder functions imported from a BlueprintEditor module, containing two functions: revise/2 to return a struct with an added or edited key/value pair, and remove/2 to return a struct with an excised key/value pair.

In my opinion and limited and fallible experience, fluent builders can -- when reasonably implemented and used -- make instantiation and altering easier to read and write. Like the factory implementation proposed in the Ecto docs, these would call struct! under the hood, thus offloading the responsibility of struct-ual integrity (dad joke) to the data structure being built.

3. A create/1 and/or create! function that wraps the persistence layer call for the appropriate context. This might also be left off, giving the builder no persistence responsibilities, in which case we would just call the appropriate create function from the tests. Either way we use the same pathway for persisting entities in our tests that we do in the production code.

I'd be inclined to name an exemplar of this factory implementation XBuilder if persistence is one of its responsibilities and XBlueprintMaker if it isn't.

As to where these would live -- maybe in test/support/builders under a Builders namespace?

Calling one of these modules an XFactory wouldn't be my first choice -- I think calling it an XBuilder or an XBlueprintMaker manages to dodge some of the baggage-by-association of other factory implementations that seem to me to be less optimal, in Elixir's own ecosystem and sister ecosystem of Ruby.

The advantages of this over a factory_bot-esque implementation seem clear:

1. Each persistible gets its own factory, improving modularity

2. The different persistible types provided by each factory are also made into first-class citizens. And we can potentially refer to them as 'blueprint types', which makes a whole heck of a lot more sense than 'factory types' to refer to what are essentially data patterns, not factories.

3. The fluent-builder code provides a nice and pipeable API for changing things, while remaining extremely simple -- we neither have to write nor generate with_connection_id, with_account_id, etc., but we still get targeted validation with nice failure messages on the passed key/value pairs by calling struct!

Hope that wasn't too long an explanation 😄 -- I would have been briefer, but I see the ex_machina issue may have made some progress. I wanted to make sure to check in with some broad strokes of this proposed implementation and open it up to feedback before I spend more time on this

[MuhammadMullah]

@reichert621 I'd like to attempt getting this one done. Is there anything I would need to get started?

[reichert621]

@MuhammadMullah that sounds great! nothing you should need to get started, other than setting up your development environment: https://github.com/papercups-io/papercups/wiki/Development-Setup#getting-started

basically we have a lot of repeated boilerplate code in our tests and would like to clean it up with factories, so just make sure mix test still passes before opening a PR 😄

[MuhammadMullah]

great

[MuhammadMullah]

@reichert621 working on this isssue but I am a bit unsure whether this has been done already. Just want to confirm if I can still continue with this

[reichert621]

hey @MuhammadMullah! a bunch of the fixtures were refactored already, but I still think it would be really useful to set up factories (https://hexdocs.pm/ecto/test-factories.html) for a bunch of our models (e.g. messages, conversations, accounts, etc.)

so there is still some work to do here if you want to pick it up!

[yakryder]

Hey @MuhammadMullah! Hope you're well. Just checking if you are still looking at this.

[reichert621]

@Sherspock given how long it's been, I'm happy to assign this to you if you're interested!

[yakryder]

@reichert621 I am!

Forgive me if I'm overcomplicating this, but there's two things I wanted to get your input about before starting anything:

1. Could we split this work into multiple work items?

2. Could we have a separate module for each factory?

The way the Ecto documentation does factories is reminiscent of how factories are done with factory_bot and Laravel -- it lumps everything onto an all-knowing factory. This is nice in that you can find all your factories in one file, and less nice when your one file gets cluttered or you'd like Intellisense to tell you what factory definitions are available for a given model. And if in the all-knowing factory case, you start pulling your factory definitions into separate files to better organize them, I feel like you have now reached the worst of both worlds -- you have neither Intellisense (at least not without writing or using an extra plugin that understands these definitions) nor separation of concerns, but your file organization could easily lead an unsuspecting dev to expect both.

[reichert621]

@Sherspock sure!

1. happy to split this up -- I'll probably hold off on creating a bunch of tickets for every model, but feel free to tackle just a chunk of this first to demonstrate a good example of it :)
2. yes, I like the idea of separating the factories rather than lumping them all into one (I imagine this will also make it easier to split up the work if anyone else ends up wanting to help out with this)

going to assign this to you if that's ok!

[reichert621]

thanks for the update @Sherspock! curious if you have any thoughts on using ex_machina? (from #316)

[yakryder]

I thought about it but didn't look at it as a contender for the current context -- can't remember why. I've worked on a project that uses it before.

I'll give it a look!

[reichert621]

makes sense! happy to skip it for now, just wanted to make a note of it since @m1ome brought it up :)

[yakryder]

@reichert621
Sure! I took a quick look.

Looking at it again, t looks like an Elixir port of factory_bot. Maybe I've just never fully understood what factory_bot gets you, but I don't understand the appeal. It doesn't seem to solve the problem of the all-knowing factory, and having a library telling you to do it that way in the codebase makes it easy for that way of thinking to get entrenched on the testing side.

@m1ome
If you've had great experiences with ex_machina I'd certainly be happy to hear them

[yakryder]

@reichert621
Hey sorry this is taking longer than I thought. I quickly realized following the Ecto factory implementation was going to trace a different path to the db than the current persistence layer, and that's not something I was willing to see have a chance at being merged even provisionally.

From there I revisited the idea of factories in separate modules with Intellisense completion of factory types, and am certainly guilty of trying to make them too nice prior to getting them functional enough to PR.

I took this issue because I thought it was a clear win for the project, simple, and within my capabilities to tackle quickly. At the very least it is not the latter.

I don't know that it makes sense for me to hang onto it if there's another implementation of a fix in flight.

@MuhammadMullah
All yours if you want to take back over

[reichert621]

@Sherspock tbh i don't think another implementation is in flight at the moment, so feel free to play around with this!

(but if you've lost interest, that's fine too)

[m1ome]

I can do this along with #316. It should be done at a same time.

[yakryder]

@reichert621
Haven't lost interest! Far from it, I see a good test-data abstraction layer as essential to maximizing happiness and productivity in a test suite. So I see this work as valuable and [relatively] low-hanging fruit.

Having said that, making headway has been a struggle and I think part of it is just not having enough Elixir hours under my belt.

I might be being overly rigid, but I'm not willing to do the Ecto docs implementation and I don't think bringing in ex_machina is a good idea. Based on my constraints -- which I'll be the first to admit are not the same as the constraints of the issue -- I could theoretically sit down to try and finish a home-rolled POC and be done in an hour, but based on experience so far it's more likely to be a slog.

@m1ome
Should it prove helpful to you if you go forward with ex_machina, this is the ex_machina factory module for a project I did some work on. It's more simplistic than what is called for here but it would be another datapoint in addition to the docs.

[yakryder]

Ah, I forgot at least one thing, part number 4 -- a unique_token/0 function (possibly imported from a newly defined UniqueTokenGenerator module) that would replace the counter/0 function in the test_helpers and be called everywhere we need to prevent data collisions.

[reichert621]

sounds interesting @Sherspock! would you be able to set up a bite-sized example of what this might look like in code?

[flmel]

Me