http4s / rho Goto Github PK

A self documenting DSL built on http4s

License: Other

Scala 99.99% Shell 0.01%

rho's Introduction

ρ: A DSL for building HTTP services with http4s

val httpService = new RhoRoutes[IO] {
   GET / "hello" / pv"world" +? param[Int]("fav") |>> { (world: String, fav: Int) => 
     Ok(s"Received $fav, $world") 
   }
}

See the tutorial, wiki and the tests for more examples.

Get more from your route definitions

The primary goal of ρ is to provide an easy to use AST with which to build HTTP services which can be inspected to extract a variety of information including:

Swagger documentation
HAL documentation

Get ρ

Rho artifacts are available at Maven Central and snapshots are available from the Sonatype repositories.

Read the Rho Scaladocs

resolvers += Resolver.sonatypeRepo("snapshots")  // Only if you are using a -snapshot version

libraryDependencies += "org.http4s" %% "rho-swagger" % version

Stability

ρ remains a work in progress. However, it is now approaching a point where things are beginning to stabilize. Going forward changes to the api should will come more slowly and have deprecation period.

Contributing

Contributions of all kinds are welcome! Documentation contributions are especially useful for others who are also just learning to use ρ. The wiki and the tests are the primary source of documentation. Pull requests are greatly appreciated from their original authors, licensed to the http4s project under the project's open source license.

License

ρ is licensed under the terms of the Apache 2.0 license. See the license file in the base directory for more information.

rho's People

Contributors

Stargazers

Watchers

rho's Issues

Make 'comment metadata' more intuitive

Right now the syntax for adding comments to the self documentation are terrible. It would be nice if the comments felt more natural, almost like a Scala comment.

"This is the description" ~
GET / foo / bar !> { (foo: String, bar: String) => ??? }

would feel nice. (I'm not tied to the ~ operator, but we do need one that will take precedence over the compile operator !> operator which disqualifies operators starting with | ^ & < >)

Keep on the lookout for swagger-core 2.0 artifacts

The swagger 2.0 spec is out, and the jars we need to integrate it should be close behind. We need to keep on the lookout, I want to get this done as soon as possible.

Replace usage of Manifest

Should we replace Manifest by TypeTag?

Provide URI and URI template of a route definition

Specifications that can be used to self describe (e.g. Swagger, JSON HAL) web services use URI RFC3986 and URI Template RFC6570 often.

It would be helpful be able to transform a route definition into a URI or URI Template. For example, this would lower the amount of redundant code when working with JSON HAL to describe references to other resources within a specific resource.

object MyWebService {

  val startParam = query[Int]("start", 0)
  val limitParam = query[Int]("limit", 0)
  val orders = GET / "orders"
  val ordersSearch = GET / "orders" +? query[String]("searchterm")
  val ordersPaginated = orders +? startParam & limitParam
  val order = orders / pathVar[Int]
  val orderItems = orders / pathVar[Int] / "items" +? startParam & limitParam

  // what we can do today
  val service1 = new RhoService {
    ordersPaginated |>> { (start: Int, limit: Int) =>
      "return page of orders"
    }
    order |>> { id: Int =>
      s"return order with id $id"
    }
    orderItems |>> { (id: Int, start: Int, limit: Int) =>
      s"return page of items from order with id $id"
    }
  }

  // what we need for JSON HAL
  val service2 = new RhoService {
    ordersPaginated |>> { (start: Int, limit: Int) =>
      ResourceObject[Map[String, Any]](
        Vector(
          "self" -> Single(LinkObject(self.toUri)), // self should be implicitly available
          "next" -> Single(LinkObject(orders.toUri +? startParam(start + limit))),
          "search" -> Single(LinkObject(ordersSearch.toUriTemplate))),
          Vector.empty,
          Some(Map("orders" -> List(1, 2, 3, 4, 5, 6, 7, 8, 9)))))
    }
    order |>> { id: Int =>
      ResourceObject[Map[String, Any]](
        Vector(
          "self" -> Single(LinkObject(self.toUri)), // self should be implicitly available
          "items" -> Single(LinkObject(orderItems(id).toUri))),
          Vector.empty,
          Some(Map("items" -> List(1, 2, 3, 4, 5, 6, 7, 8, 9))))
    }
    orderItems |>> { (id: Int, start: Int, limit: Int) =>
      ResourceObject[Map[String, Any]](
        Vector(
          "self" -> Single(LinkObject(orders.toUri)),
          "next" -> Single(LinkObject(orders.toUri +? startParam(start + limit)))),
          Vector.empty,
          Some(Map("items" -> List("concrete item 1", "concrete item 2" ... ))))
    }
  }

}

Explore CRUD patterns

CRUD patterns are common and clearly have easy to exploit type information that would fit really well into rho. Perhaps we can provide some abstractions to ease their use.

Considerations

How will rho deal with the multiple methods that CRUD needs without redefining them? (GET, PUT, etc.)
CRUD resources are exactly the types of things JSON HAL is good for, so how do we make sure they are fully exposed through those models?
Not all CRUD operations will have the same needs. For example, a list operation probably doesn't want to pull a million records (at least by default, if ever). How can we provide this flexibility through the abstraction?

Refactor imports to mirror those of Http4s

The import style of Http4s has been changing, and I think rho should follow suit. Here is what I propose:

import org.http4s.rho._              // all important types should be in scope
import org.http4s.rho.Rho._          // all batteries included

That would duplicate the Http4s imports, but we may deviate some. First, our route building is based on a trait, so we could include many of those imports in that trait automatically.

Provide extension point to add custom data type transformations for Swagger

After juggling around with JSON HAL and Swagger I think we need an extension point in SwaggerSupport to be able to add custom transformations for data types. The signature could be A <: AnyRef => ModelProperty or PartialFunction[A <: AnyRef, ModelProperty].

I think it should basically work as the serializers in Json4s. Maybe we can borrow some code there. Opinions?

help me organize a documentation feature list?

Stuff I've noticed in rho:

route description
path parameter name
query parameter name
result codes
(somehow) response model for json response
required headers

Stuff I see in Swagger UI but haven't spotted in rho

path parameter description
query parameter description
required header description

Stuff I can think of but didn't see in Swagger or in rho

custom reasons for error result codes
response model with field descriptions for json response (maybe I can help here)

(I haven't read the swagger 2.0 spec)

What am I missing?

Regard a context path definition when routing requests

For example, if a custom RhoService mounted on /ws all requests should be filtered so that the path does not starts with /ws anymore.

Documentation for `requireThatR` and `paramR`

The two aforementioned methods allow one to define a route requirement that can return an arbitrary response. An example is thus:

val requireCookie = requireThatR(headers.Cookie){ cookie =>
    cookie.values.toList.find(c => c.name == "Foo" && c.content == "bar") match {
      case Some(_) => None   // Cookie found, good to go.
      case None => // Didn't find cookie
        Some(TemporaryRedirect(uri("/addcookie")))
    }
}

I'm not sure the semantics are perfect: its counter intuitive to interpret a None as a 'success'.

consider HTTP 400 over 404 for pathVar parse failures

Given: https://gist.github.com/refried/0eb77004d2f41c470e71

curl -v localhost:8081/puppy/bySSN/foo returns 404 with the body /puppy/bySSN/foo not found, which made me think something was wrong with my route, when it was really something wrong with my request (parse failure).

I think there's an argument to be made for 404, or a different code, or a different body, or maybe the answer is documentation on pathVar. What do you think?

java.util.NoSuchElementException: head of empty list

> rho-examples[ERROR] java.util.NoSuchElementException: head of empty list
rho-examples[ERROR]     at scala.collection.immutable.Nil$.head(List.scala:420)
rho-examples[ERROR]     at scala.collection.immutable.Nil$.head(List.scala:417)
rho-examples[ERROR]     at org.http4s.rho.swagger.SwaggerModelsBuilder.mkCollectionProperty$1(SwaggerModelsBuilder.scala:244)
rho-examples[ERROR]     at org.http4s.rho.swagger.SwaggerModelsBuilder.org$http4s$rho$swagger$SwaggerModelsBuilder$$typeToProp$1(SwaggerModelsBuilder.scala:229)
rho-examples[ERROR]     at org.http4s.rho.swagger.SwaggerModelsBuilder$$anonfun$mkResponse$1.apply(SwaggerModelsBuilder.scala:256)
rho-examples[ERROR]     at org.http4s.rho.swagger.SwaggerModelsBuilder$$anonfun$mkResponse$1.apply(SwaggerModelsBuilder.scala:256)
rho-examples[ERROR]     at scala.Option.map(Option.scala:146)
...

Evaluate the usefullness of the static RouteExecutor

The static RouteExecutor was nice to bootstrap the project, but I'm not certain it still has a use case. It is used in some tests, but I think those should probably also be changed.

suppose I want to compose two action-do-thingies

In Play framework (and probably others?) it's common to have a higher-order action that does some processing, and conditionally delegates to another action.

pseudo-code examples:

// reads some parameters from the request and call a delegate
//   with derived parameters or with derived + original parameters
def secure(d: Request => Response): Request => Response
def withUser(d: User => Response): Request => Response
def withUser2(d: User => Request => Response): Request => Response
def withUserOrElse(d: User => Request => Response, orElse: => Response): Request => Response

which check some properties on the Request and either continue with f or return some error.
(Related to #51)

// extracts two parameters from the request and passes them to a delegate, 
//   which may use other parameters from the request
def paged[A:EncodeJson](d: Request => (Offset,MaxResults) => A): Request => Response =
  req => d(req)(req.queryParams[Offset]("paging_offset"), 
                req.queryParams[MaxResults]("paging_max"),
               ).asJson

From what I can tell, in rho, the F needs to accept all of the parameters on the HList stack, making it difficult to modularize things like this.

Can I write an F that pushes / pops some arguments from the stack, and pass the remainder to another F? Or how can I get some kind of modularity here?

Uncouple the pom from http4s

With http4s gaining momentum and a lot of bug fix releases on the horizon, it should start maintaining binary compatibility between bugfix releases. That means that if rho wants to get those fixes 'for free' and not need to do a lockstep release with every http4s bugfix, we should think about decoupling the pom from http4s. That will mean that users will need to specify http4s explicitly, but thats already a given considering they will need a backend.

How to exclude resources from SwaggerSupport

I wanna exclude some resources from the API listing when using SwaggerSupport. How should we mark a route definition to be excluded?

How do JSON result bodies become Swagger model documentation?

In the examples, I see the tag trait AutoSerializable and the encoder

implicit def jsonWritable[A <: AutoSerializable]: EntityEncoder[A]

which uses jackson's write to produce JSON, but that isn't part of the process for generating the Swagger model, is it?

Routing requests with parameters

Differ between optional and required parameters when routing a request.

val service = new RhoService {
  // optional parameter
  GET +? param[Option[String]]("bar") |>> { bar: String => "root with " + bar }
  // requires parameter
  GET +? param("foo") |>> { foo: String => "root with " + foo }
  // requires parameters
  GET +? param("foo") & param[Option[String]]("bar") |>> { (foo: String, bar: Option[String]) => "root with " + foo + " " + bar }
}

Passing GET / to this routing table the first definition should match
Passing GET /?bar=1 also the first definition should match
Passing GET /?foo=2 the second definition should match
Passing GET /?foo=2&bar=1 the third definition should match

Does it make sense to build a tree of actions and determine the ordering?

Should SwaggerSupport be an abstract class?

Right now in the topic/swagger branch SwaggerSupport is a trait. Swagger needs some info such as the api version, api-path, etc, which is a bit ugly in a trait as they must be provided by a def or even worse, a val. With an abstract class, that info could be provided as constructor params, but limits its ability to be stacked, resulting in needing a new way to "squash" different API's together.
Is this something we want to explore?

Issues around body parameters

Specifying both

(POST / "login" decoding(EntityDecoder.text)) |>> {p: String) => Ok("success")}

(POST / "login" decoding(ArgonautJSONDecoder)) |>> {p: Login) => Ok("success")}

in the HttpService fail with the server reporting Invalid Content-Type header

Ehen running through the swagger-ui, in both cases, the browser sent:
Content-Type: "text/*; charset=UTF-8"

Swagger is generating invalid docs

As noted in #46, the swagger validator has alerted us that the docs generated by the http4s demo are invalid.

Add JSON HAL example

We should provide a simple example how to create a HTTP service that provides JSON HAL conform content.

is it possible to compose Rho services?

I naively tried to use .toService.orElse, but I suppose append or __tree need to come into play.

I'm guessing this isn't currently supported (after all, we're only at 0.4), but that it could be without too much trouble. Is there anything you can think of that I should specifically avoid if I wanted to implement this?

Allow failed validation to yield a Response instead of just a message

It may be nice to yield a 401 Unauthorized or some other type of status code. It should be easy to implement by changing the ParserResult types to either have ValidationFailure contain a Response or add a new type with the same purpose.

Explore scalaz Validation

I think there is a lot of potential for integrating the scalaz Validation type into rho. It should be explored.

Explore the idea of a 'parsing failure strategy'

@before had the good idea of allowing for different strategies with regard to parser failure.
Questions that should be addressed:

What is the scope of the strategy? Eg, is this limited to the query.
What types should this deal in? The scalaz Validation comes to mind, but I don't know if it fits well.
How is the strategy specified? Eg, is it captured as an implicit, a member function of the RhoService?

Authorization

It would be nice to support authorizations directly in rho. I'm not sure of HAL would benefit from it, but swagger would.
Prior Art:

Scalatra (suggested to be not so good by @casualjim)
By proxy, Warden

Enforcement of path construction grammer

I made the DSL relatively strict on the ordering with which things can occur. What I mean is that you can't add a query string requirement in the middle of the path requirements. Is this something we want to enforce?

How to combine rho services

It seems like it would be rather nice to smash services together at the rho level, not just the http4s level. That way we can preserve information such as swagger data, etc. However, we need a clean way to do this. As it is, combining a RhoService with SwaggerSupport with a standard RhoService would be an operation that doesn't commute as the standard RhoService is not aware if what SwaggerSupport offers.

Possibility to define a root path

Currently it is not possible to define the root path with the DSL.

  val service = new RhoService {
    GET / "" |>> { () => "root" }
  }

  "RhoService" should {
    "handle path to root" in {
      val req = Request(GET, Uri(path = "/"))
      service(r).run.status must equalTo(Ok)
      service(r).run.body must equalTo("root")
    }
  }

Provide rule when a default parameter value should be used

Lets say we have the following query parameters

val start = param[Int]("start", 0)
val limit = param[Int]("limit", 10)

and we want to apply the default values always if a given predicate evaluates true.

val start = param[Int]("start", 0, _ < 0)
val limit = param[Int]("limit", 10, _ < 1)

Possibility to define a path with a query without HTTP method specification

It would be helpful to be able to define a path with a query without the need to specify a HTTP method.

// current approach is limited
val methodPlusPathPlusQuery = GET / "hello" / "world" +? query[String]("username")

// enhancement would allow us to be more flexibile
val pathPlusQuery = "hello" / "world" +? query[String]("username")

General cleanup

We've come a long ways with rho, and I think its time to take a small step back and do some house cleaning. I don't have anything in particular in mind, but taking a bit if time to clean up the work site seems like a good idea. Additionally, we need to decide what to do next.

Things that need cleanup

Naming. Some things have silly names. Validatable? When did I have that idea?
Swagger api generation is building invalid specs according to the swagger validator. I'll open a bug.

Topics to Explore

CRUD operations
Body parsing and validation (will likely be handled by http4s proper)

Last but not least, once we get a release in maven that doesn't have broken decoder support, we should advertise and maybe others will get interested in the project.

Custom field serializers will not be regarded

import scala.reflect.runtime.universe._

import org.specs2.mutable.Specification

import com.wordnik.swagger.model.Model
import com.wordnik.swagger.model.ModelProperty

package object model {
  sealed trait Fruit
  case object Apple extends Fruit
  case object Pear extends Fruit
  case object Cherry extends Fruit
  case class FruitBox(fruits: Seq[Fruit])
}

class SwaggerFormatsSpec extends Specification {
  import model._

  "SwaggerFormats" should {

    "withFieldSerializers" in {
      val swaggerFormats = DefaultSwaggerFormats.withFieldSerializers(typeOf[Fruit], Set(ModelProperty("List", "array«Fruit»")))
      def models[T](t: TypeTag[T]): Set[Model] = TypeBuilder.collectModels(t.tpe, Set.empty, swaggerFormats)
      models(typeTag[FruitBox]).nonEmpty must_== true
      models(typeTag[FruitBox]).head.properties.head._1 must_== "fruits"

      // following assertion fails
      models(typeTag[FruitBox]).head.properties.head._2 must_== ModelProperty("List", "array«Fruit»")
    }

  }

}

The issue is that SwaggerFormats.customFieldSerializers is not in use by TypeBuilder.

Swagger does not show the model for status OK when having mixed result types

When having mixed result types Swagger does not render the model in OK route.

case ModelA(name: String, color: Int)
case ModelB(name: String, id: Long)

route ||> { () =>
  if(true) Ok(ModelA("test ok", 1))
  else NotFound(ModelB("test not found", 234))
}

ModelB is documented for NotFound as expected. For status Ok the referenced model is ModelA but it is not present in the list of models (only ModelB exists).

Decoders are ugly

I think the decoders need some help. The syntax is ugly, and we should watch how the body decoders of http4s evolve so we can reuse as much of that as possible.

Require a route to respond with a status

Right now we can return any T for which exists in scope a Writable[T]. This is provided using a chain of implicits starting with HListToFunc -> ObjToResponse -> Writable.
It has become apparent that this is a brittle endevor, and It might be better to enforce a bit of order much like the http4s dsl has, namely a series of operators such as OK etc. This will cut out the ObjToResponse at the cost of losing flexibility in the route definition.
_Any comments?_
I will likely start a branch to see how this change feels.

Decoding syntax sucks

Stuff like

"Get form data from the user" **
    POST / "foopath" decoding(formEncoded) |>> { formData: Map[String, Seq[String]] =>
      Ok("Foo")
    }

doesn't work because of operator precedence, so instead, I'm forced to do

(("Get form data from the user" **
    POST / "foopath") decoding(formEncoded)) |>> { formData: Map[String, Seq[String]] =>
      Ok("Foo")
    }

or some other nonsense.

This should be fixed.

Improve API to make the work with different response types easier

  PUT / "update" |>> { (request: Request, id: Int) =>
    text(request).flatMap { text =>
      services.updateText(id, text) match {
        case 0 => NotFound(s"text $id not found")
        case 1 => Ok("updated")
      }
    }
  }

This example leads to the following compiler error

could not find implicit value for parameter hf: org.http4s.rho.bits.HListToFunc[shapeless.::[Int,shapeless.HNil],(org.http4s.Request, Int) => scalaz.concurrent.Task[org.http4s.rho.Result[_ >: org.http4s.Status.NotFound.type with org.http4s.Status.Ok.type <: org.http4s.Status, String]]]

From a usability perspective this should be possible to write without struggling with typing issues.

We should think about if it simplifies our code base by using Swagger Annotations to describe which response codes are in use and what model is in the response body. I would say the EntityResponseGenerator needs to be reworked to get more flexibility into Rho.

Upgrade to swagger-2.0

Swagger artifacts are out (in milestone form, but quite usable) now here. The most useful are swagger-core and swagger-models.

The documentation building needs to be nearly complete reworked, but this isn't as bad as it seems as the new swagger spec is much simpler in some ways. There is already a topic/swagger2.0 that compiles, but the generated specs are probably/certainly wrong and type models don't build at all.

I'm planning on dumping swagger 1.2 and focusing all work here so this is the place to contribute.

Cannot add multiple query requirements

This will require a new operator which would logically be &, but that will cause operator precedence issues with the compile operator, !>.

Get SNAPSHOTs working

I suspect this is my problem.

Resource definitions in Swagger

In case I wanna define a resource in more detail how would I do that?

Lets say I have the following routing.

object MyService extends RhoService with SwaggerSupport {
  GET / "pet" / "findByTags" +? param[String]("tags") |>> { (tags: String) => "use a service to find pets by tags" }
}

And I want to create a definition that looks like this http://petstore.swagger.wordnik.com/#!/pet/findPetsByTags.

How would I provide a Data Type definition to a resource?

Body parameters are not being generated

#50 describes thrown exceptions when attempting to build HttpServices with body parameters.

Currently it is possible to build them, although the generated swagger api does not correctly expose them.

split swagger support into a module

Swagger and the future HAL support probably belong in modules: they may not be needed by everyone and swagger brings json dependencies.

how does rho handle websocket

or I guess in more general how does it handle Task[Response] ? As far as I can tell, it cannot compile if Task[Response] is provided directly to the dsl, e.g.,

GET / "foo" +? param[Option[Int]]("bar") |>> { (bar: Option[Int]) => WS(Exchange(Process.constant(bar.getOrElse(-1)).map(b => Text(b.toString())).take(5), Process.halt)) }

could not find implicit value for parameter hf: org.http4s.rho.bits.HListToFunc[shapeless.::[Int,shapeless.HNil],Int => scalaz.concurrent.Task[org.http4s.Response]]

mixing path params and `EntityDecoder` ?

Is it possible to do something like this?

"Update person info" **
  PUT / apiv1 / "person" / 'id ^ jsonOf[PersonInfo] |>> { 
    (person: PersonInfo, id: String)  => s"updating person $id"
  }

I get an error like the following, regardless of the order of the F's args:

could not find implicit value for parameter hf: org.http4s.rho.bits.HListToFunc[shapeless.::[PersonInfo,shapeless.::[String,shapeless.HNil]],(PersonInfo, String) => String]
[error]       PUT / apiv1 / "person" / 'id ^ jsonOf[PersonInfo] |>> { (person: PersonInfo, id: String)  => s"updating person $id"}
[error]                                                         ^

  val service = new RhoService {
    GET / "error" |>> { () => throw new Error("an error") }
  }

  "RhoService" should {
    "handle internal server errors" in {
      val req = Request(GET, Uri(path = "/error"))
      service(r).run.status must equalTo(InternalServerError)
    }
  }