swift-server / http Goto Github PK
View Code? Open in Web Editor NEW:warning: Historical HTTP API - please use https://github.com/swift-server/async-http-client instead
License: Apache License 2.0
:warning: Historical HTTP API - please use https://github.com/swift-server/async-http-client instead
License: Apache License 2.0
The current implementation doesn't honor the stop
flag (the inout Bool
) in HTTPBodyHandler
. Need to fix that and write a test for it.
Determine the lifecycle of the HTTP server.
Refer to the discussions from #82 for history.
An initial idea discussed in that PR is:
- init: sets up the server only
- suspend: continues to accept work but doesn't process (socket would still be open, bound and listening, but not calling accept(). That will work until the kernel listen queue is full.)
- resume: resumes from suspend, starts the server if not already started.
- cancel: completely shutdown and destroy the server
@helje5 raised the point of:
For iOS, apps really need to close when going bg and completely reopen listen socks when going fg (unless you have special VoIP like blessing, in that mode the kernel will ensure the listen sockets stay alive - which may or may not be another thing we would want to support).
No instructions about installation are provided. I can't install it via Swift Package Manager.
The following code causes a crash at the specified line because parsedHTTPVersion
is nil
try! server.start(port: port) { req, resp in
DispatchQueue.main.async {
resp.writeHeader(status: .ok)
resp.done()
}
return .discardBody
}
http/Sources/HTTP/HTTPStreamingParser.swift
Line 401 in 825b680
It seems .discardBody
causes resp.done()
to be called, although writing a response after the request has been processed seems like a use case that should be supported.
I'm pulling comment from @helje5 from #28 over into this issue, because that issue is getting cluttered.
The worse aspect is that this calls the done-cb before the queued write has actually finished! This needs to be passed along the stream, that's the whole point.
My reason for not doing this was two-fold:
write()
completes. But, as the documentation for write(2)
explicitly says "A successful return from write()
does not make any guarantee that data has been committed," holding onto the CB that long doesn't provide any more reliability or information that if it was called where it is now.There's no guarantee the connection won't be closed before the bytes that were queued will be written to the socket, but there's no guarantee they're going to make it to the far end of the TCP connection either.
So why does it matter whether the callback happens after the bytes are queued for write()
or once write()
had completed? The guarantee (or lack thereof) is the same either way. By Occam's Razor then, I think the simpler design is better.
I might well be missing something here. If so, I'd love to know what it is.
Thanks
I don't know where I can submit proposals for api changes so I opened this issue.
server port is determined in start(port:, handler:)
method, while I think a server can start once with a one specified port. I could find a port
property in HTTPServer
class while with this design it have to be ports
array. I propose relocating port
argument from start()
to class init()
method.
Personally I prefer something like response.header.write()
instead of response.writeHeader()
, response.body.write()
instead of response.writeBody()
and response.trailer.write()
instead of response.writeTrailer()
. Also we can keep room to extend header
, body
and trailer
properties later.
HTTP versions are limited, thus having HTTPVersion.v1_1
and HTTPVersion.v2_0
static vars might be fair.
HTTPHeaders
HTTP headers are simply defined as a pair of strings. But for many of headers there have a set of defined values or a structured value. This implementation is prone to typo and human errors. So I propose to have some setters for frequent ones.
// To set 'Range'
let range = 100... // half open or closed
headers.set(range: range) // result: 'Range: bytes 100-'
// To set 'Accept-Encoding'
headers.set(acceptEncodings: [.deflate, .gzip, .brotli, .identity])
// or
headers.set(acceptEncoding: .deflate, quality: 1.0)
headers.add(acceptEncoding: .gzip, quality: 0.8) // Note add instead of set
headers.add(acceptEncoding: .identity, quality: 0.5)
// values are defined in a enum/struct
// To set 'Cookie'
let cookie = HTTPCookie(properties: [.name : "name", .expires: Date(timeIntervalSinceNow: 3600)])
headers.set(cookie: cookie)
...
headers.add(cookie: cookie2)
// To set 'Accept-Language'
let locale = Locale.current
headers.set(acceptLanguages: [locale])
// also 'add(acceptLanguage:)' method to add progressively like Accept-Encoding
// To set 'Accept-Charset'
headers.set(acceptCharset: String.Encoding.utf8)
For Content-Type, we would define this struct in HTTPHeaders
(should be extended with more types):
extension HTTPHeaders {
struct MIMEType: RawRepresentable {
public var rawValue: String
public typealias RawValue = String
public init(rawValue: String) {
self.rawValue = rawValue
}
static let javascript = MIMEType(rawValue: "application/javascript")
static let json = MIMEType(rawValue: "application/json")
static let pdf = MIMEType(rawValue: "application/pdf")
static let stream = MIMEType(rawValue: "application/octet-stream")
static let zip = MIMEType(rawValue: "application/zip")
// Texts
static let css = MIMEType(rawValue: "text/css")
static let html = MIMEType(rawValue: "text/html")
static let plainText = MIMEType(rawValue: "text/plain")
static let xml = MIMEType(rawValue: "text/xml")
// Images
static let gif = MIMEType(rawValue: "image/gif")
static let jpeg = MIMEType(rawValue: "image/jpeg")
static let png = MIMEType(rawValue: "image/png")
}
}
And then we can set MIMEs this way:
headers.set(accept: .plainText)
headers.set(contentType: .json, encoding: .utf8)
// method converts 'NSStringEncoding' to IANA by 'CFStringConvertEncodingToIANACharSetName()' method
// Also similar approach for 'Pragma', 'TE' , etc...
headers.set(pragma: .noCache)
headers.set(transferEncodings: [.trailers, .deflate])
For some headers that accept date or integer or url, we implement methods accordingly:
// to set 'Content-Length'
headers.set(contentLength: 100)
// to set 'Date', 'If-Modified-Since', etc...
let fileModifiedDate = file.modifiedDate // or Date(timeIntervalSinceReferenceDate: 1000000)
headers.set(ifModifiedSince: fileModifiedDate)
// to set 'Referer' or 'Origin'
headers.set(referer: URL(string: "https://www.apple.com")!)
About Authorization
header, we can have something like that, though I think it should be more refined and must be evaluated either we need them or not:
headers.set(authorizationType: .basic, user: "example", password: "pass")
Obviously, we must have some methods to fetch values. For example headers.contentLength
will return a Int64?
value, if it's defined by user, or headers.cookies
will return a [HTTPCookie]
array, headers.contentType
will return a HTTPHeaders.MIMEType?
optional, and so on and so forth.
I hope I can participate in implementing some parts if proposals are accepted
Best wish
With the last few swift-4.0 development snapshots, all but one of the XCTests
with names that end with EndToEnd
fail on Linux with timeouts. These are the tests that start a server on a random port and create a URLSession
to send requests to it Those tests pass fine on Darwin (and on Linux under Swift 3.1.1).
If you comment all but one of the EndToEnd tests out, then the one you leave in (which ever one you pick) will pass. With two EndToEnd tests in, regardless of the order you choose, the first one will complete, and the second one will time out.
I'm guessing something isn't being cleaned up correctly, but I'm not sure what. Will update as I learn more.
It looks like the default branch has been deleted so the repository checks out empty.
It's easily fixed personally, but it would be nice if it could be fixed globally.
https://help.github.com/articles/error-remote-head-refers-to-nonexistent-ref-unable-to-checkout/
I started Swift 4 support (https://github.com/carlbrown/http/tree/swift4), but got blocked on #22. It doesn't look like that's going to get fixed, so I'll work around it, starting with #39
Once that gets merged, I'll push a new swift-tools-version = 4.0 Package.swift
file and associated fixes to either develop
or a swift4 specific branch, whichever we think makes the most sense.
The code uses POSIXError
which I think is not an actual thing an implementation can create, right? It is just a namespace (at least on macOS, maybe that is different on Linux?).
Is that meant to be POSIXErrorCode
?
https://github.com/swift-server/http/blob/develop/Sources/SwiftServerHttp/HTTPCommon.swift#L72
Note: This is reported as an Issue, because the error part of the API cannot be implemented. (My suggestion would be to not expose 'POSIX' at all but instead use an own HTTPError type which wraps the underlying socket error)
In theory with current implementation of HTTPStreamingParser
it is possible to run out of the stack. This problem can occur if a long chain of callback receiving methods (i.e. writeBody
) is called. To solve this completion callback should never be called before method returns.
Because of this problem it looks like two different APIs for sync/async should exist. Right now I see no solution to perform sync operations via async API.
In #42 there was some discussion on the use of htons()
. Moving that to this issue so that we can get that PR merged (which fixes port setting not being honoured).
Docs are exist in two incarnations: in the root of gh-pages
branch and in docs
directory of develop
. Let's remove docs
dir from develop
branch. Why this should be done? Because it is very hard to read diffs of PRs containing docs update now.
In working on writing a framework on top of this one, I've discovered that I quite often want to send a payload that has a fixed length (e.g. a JSON
Data
produced by Codable
), and the code paths to do that (e.g. in HTTPStreamingParser.swift
when isChunked == false
) are hard to use and poorly tested.
I need to fix that.
From @helje5:
#81 (comment)
An open question is, do we want to support multiple binds? I tend to say yes. This would modify the structure of the server a little.
(the other option is to use multiple servers if you want to listen on multiple addresses, but that would be kinda wasteful).
When explicitly declaring the Connection: close
header on a client request, the server successfully completes the request, but fails to properly close the socket.
Looking deeper into this, it appears due to the below code block in HTTPStreamingParser
in Done()
. Here, self.parserConnector?.closeWriter()
is called instead of self.parserConnector?.responseComplete()
, which does not set self?.responseCompleted = true;
thus, preventing PoCSocketConnectionListener.close()
from passing the preconditions required to close the socket.
This only occurs on linux, which I think is due to URLSession replacing the explicit Connection: close
header with a Connection: keep-alive
header, but I'm not positive about that.
public func done(completion: @escaping (Result) -> Void) {
...
let closeAfter = {
if self.clientRequestedKeepAlive {
self.keepAliveUntil = Date(timeIntervalSinceNow:
StreamingParser.keepAliveTimeout).timeIntervalSinceReferenceDate
self.parserConnector?.responseComplete()
} else {
self.parserConnector?.closeWriter()
}
}
...
}
func close() {
self.shouldShutdown = true
// Always returns since responseComplete is false and no error has occurred
if !self.responseCompleted && !self.errorOccurred {
return
}
...
}
Below is a simple example. You'll notice in the terminal Closing idle socket (#)
is continuously printed until the expectation timeout and the serverConnection.count
is not decremented
func testCloseConnections() {
let expectation = self.expectation(description: "0 Open Connection")
let server = HTTPServer()
do {
try server.start(port: 0, handler: OkHandler().handle)
let session = URLSession(configuration: URLSessionConfiguration.default)
let url1 = URL(string: "http://localhost:\(server.port)/echo")!
var request = URLRequest(url: url1)
request.httpMethod = "POST"
request.setValue("close", forHTTPHeaderField: "Connection")
let dataTask1 = session.dataTask(with: request) { (responseBody, rawResponse, error) in
XCTAssertNil(error, "\(error!.localizedDescription)")
DispatchQueue.main.asyncAfter(deadline: .now() + 25) {
XCTAssertEqual(server.connectionCount, 0)
expectation.fulfill()
}
}
dataTask1.resume()
self.waitForExpectations(timeout: 30) { (error) in
if let error = error {
XCTFail("\(error)")
}
}
server.stop()
} catch {
XCTFail("Error listening on port \(0): \(error). Use server.failed(callback:) to handle")
}
}
I'm pulling comment from @helje5 from #28 over into this issue, because that issue is getting cluttered.
I don't remember what we decided on here, IMO we should use DispatchData to pass buffers around as this supports iovecs (and AFAIK Data doesn't, right?).
A DispatchData doesn't require copying but supports custom deallocators, that is a key part of its design. (I'm not denying that there might be a better alternative, but DispatchData is much better than Data for this specific purpose (which presumably is the reason why it still exists ;-) )).
For what it's worth, I started with DispatchData
in the APIs, and I wanted to keep it that way. The problem is that, at every turn, I found myself having to convert from Data
/NSData
to DispatchData
and back. For example:
BlueSocket
(that I really want to replace with something much simpler) explicitly reads/writes all socket data in NSData
format."OK\r\n"
, that string easily converts to a Data
, but getting it into a DispatchData
is more of a pain in the ass, and when you have several things (like (String($0.count, radix: 16) + "\r\n").data(using: .utf8)
) that you are sticking into the same packet, you end up doing that conversion over and over.String
(Because the programmer typed it in to be turned into a response), or encoded as a Data
(Because it's the contents of a file pulled off disk or it's the result of a JSONEncoder
call). In both of those cases, it was easier not to have to do a bunch of Data
<->DispatchData
conversions before handing off to http
.So you tell me - What do you think the right answer is here?
[Just in case there's any confusion as to my tone here - I'm not being defensive - I honestly want to know. I'm not thrilled with the compromise I made, and I'd love for there to be an obvious or a consensus choice.]
Noticed that we don't seem to have an official style guide that we adhere to. I think it would be nice if we could pick one as we could encode at least parts of it into linting rules and ideally wouldn't have to look for examples in other Swift projects.
Based on comments from @helje5:
Don't do port: 0, should raise an error.
Do port: nil for kernel assigned, or a real port
I've been looking through example code for the HTTP library and the use of a associated value closure for the HTTPBodyProcessing.processBody
case feels kind of usual. I've never seen such a pattern used in the Standard Library. And if this library's future is to be integrated into the Standard Library or shipped with Swift, I think it would be worth looking into keeping as close as possible to the API style of the Standard Library.
Any ideas?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.