apple / swift-nio-imap

A Swift project that provides an implementation of the IMAP4rev1 protocol, built upon SwiftNIO.

swift-nio-imap

A Swift project that provides an implementation of the IMAP4rev1 protocol, built upon SwiftNIO.

This project implements:

• Parsing IMAPv4 wire-format to type-safe Swift data structures
• Encoding these Swift data structures to the IMAPv4 wire format
• Extensive support for common IMAP extensions
• Integration with SwiftNIO

For a list of IMAP extensions, see EXTENSIONS.md.

⚠️ Note: This library is still in development phases and not ready to be used in production system yet. ⚠️

Introduction and Usage

swift-nio-imap implements the IMAP4rev1 protocol described in RFC 3501 and related RFCs. It is intended as a building block to build mail clients and/or servers. It is built upon SwiftNIO v2.x.

To use the framework use import NIOIMAP. NIOIMAP supports a variety of IMAP extensions, check EXTENSIONS.md for full details.

Example Exchange

As a quick example, here’s part of the the exchange listed in RFC 3501 section 8, where lines starting with S: and C: are from the server and client respectively:

S: * OK IMAP4rev1 Service Ready
C: a001 login mrc secret
S: a001 OK LOGIN completed
C: a002 select inbox
S: * 18 EXISTS
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * 2 RECENT
S: * OK [UNSEEN 17] Message 17 is the first unseen message
S: * OK [UIDVALIDITY 3857529045] UIDs valid
S: a002 OK [READ-WRITE] SELECT completed```

The first 3 lines would correspond to the following in SwiftNIO IMAP:

Response.untagged(.conditionalState(.ok(ResponseText(text: "IMAP4rev1 Service Ready"))))
CommandStreamPart.tagged(TaggedCommand(tag: "a001", command: .login(username: "mrc", password: "secret")))
Response.tagged(.init(tag: "a001", state: .ok(ResponseText(text: "LOGIN completed"))))

Next, up is the SELECT command and its responses, which are more interesting:

CommandStreamPart.tagged(TaggedCommand(tag: "a002", command: .select(MailboxName("box1"), [])))
Response.untagged(.mailboxData(.exists(18)))
Response.untagged(.mailboxData(.flags([.answered, .flagged, .deleted, .seen, .draft])))
Response.untagged(.mailboxData(.recent(2)))
Response.untagged(.conditionalState(.ok(ResponseText(code: .unseen(17), text: "Message 17 is the first unseen message"))))
Response.untagged(.conditionalState(.ok(ResponseText(code: .uidValidity(3857529045), text: "UIDs valid"))))
Response.tagged(.init(tag: "a002", state: .ok(ResponseText(code: .readWrite, text: "SELECT completed"))))

There’s more going on here than this example shows. But this gives a general idea of how the types look and feel.

Integration with SwiftNIO

SwiftNIO IMAP provides a pair of ChannelHandler objects that can be integrated into a SwiftNIO ChannelPipeline. This allows sending IMAP commands using NIO Channels.

The two handlers SwiftNIO IMAP provides are IMAPClientHandler and IMAPServerHandler. Each of these can be inserted into the ChannelPipeline. They can then be used to encode and decode messages. For example:

let group = MultiThreadedEventLoopGroup(numberOfThreads: 1)
let channel = try await ClientBootstrap(group).channelInitializer { channel in
    channel.pipeline.addHandler(IMAPClientHandler())
}.connect(host: example.com, port: 143).get()

try await channel.writeAndFlush(CommandStreamPart.tagged(TaggedCommand(tag: "a001", command: .login(username: "mrc", password: "secret"))), promise: nil)

swift-nio-imap's Issues

remove nested `NIOIMAP` enum

There's a NIOIMAP enum which was meant to namespace IMAP stuff but we have NIOIMAP(Core) modules for that.

This extra namespace needs to go.

Rename / flatten RFC 4466 related extension types

See https://tools.ietf.org/html/rfc4466

These are attributes that can be specified with

• CREATE
• SELECT / EXAMINE
• RENAME
• FETCH / UID FETCH
• STORE / UID STORE
• SEARCH

Types

The RFC 4466 calls these optional parameters, and as such, the types could be renamed to

public struct CreateParameter: Equatable {
    public var name: String
    public var value: ParameterValue?
}

with (renaming TaggedExtensionValue and flattening away TaggedExtensionSimple)

public struct ParameterValue: Equatable {
    case sequenceSet(SequenceSet)
    case number(Int)
    case parenthesized([String])
}

Enclosing ()

Note: the parenthesized([String]) should probably wrap a more generic concept such that it can contain strings and numbers (and nested arrays). This is similar to what the extended data of a body structure supports. But it may be ok with not providing full fidelity here.

if a parameter has a mandatory value, which can always be
represented as a number or a sequence-set, the parameter value does
not need the enclosing ().

As such sending

SELECT INBOX (BLURDYBLOOP 5)

and

SELECT INBOX (BLURDYBLOOP (5))

should be interpreted the same by a server, and it may make sense to add a convenience helpers like

extension ParameterValue {
    func readSingleNumber() throws -> Int
    func readSingleSequenceSet() throws -> SequenceSet
}

those would throw if there’s more than one value or if the value is of the wrong type.

fix synchronising literals for Command encoding

We need to wait for 'continue request' before sending the data.

Un-nest MailboxName.List and rename -> MailboxInfo

extension NIOIMAP {
    /// Information returned as part of an untagged `LIST` or `LSUB` response.
    /// IMAPv4 `mailbox-list`
    public struct MailboxInfo: Equatable {
        public var attributes: [Attribute]
        public var pathSeparator: UInt8?
        public var mailbox: MailboxName
        public var extensions: [ListExtendedItem]
    }
}

— the path separator should be a UInt8 since it should be used to split the MailboxName’s raw bytes into parts — which can then be turned into user-readable String.

Note that \Noselect etc. are called attributes in RFC 3501 section 7.2.2.

We’d eventually want something like

extension MailboxName {
    func splitPathComponents(separator: UInt8) -> [String] { … }
}

extension MailboxInfo {
    func pathComponents() -> [String] {
        guard s = pathSeparator else { … }
        mailbox.splitPathComponents(separator: s)
    }
}

Move `ContinueRequest` case up a level into `Response`

Child of #49

Flatten `UID` commands

Flatten the UIDCommandType into the Command enum, i.e.

public enum CommandType: Equatable {
    ...
    case copy([NIOIMAP.SequenceRange], Mailbox)
    case uidCopy([NIOIMAP.SequenceRange], Mailbox)
    case fetch([NIOIMAP.SequenceRange], FetchType, [FetchModifier])
    case uidFetch([NIOIMAP.SequenceRange], FetchType, [FetchModifier])
    ...
}

audit all (and remove most) `public enum`s

public enums are a real issue for a few reasons:

• in many cases, the generated Equatable conformance is wrong. Many of the current public enums have wrong Equatable conformances
• we cannot audit what the user constructs us with because enums "reveal their guts"
• we can't change them without breaking public API

`BODYSTRUCTURE` Types

The types related to the bodystructure are essential to code trying do things such as

• decide which parts of messages to download (as opposed to downloading the complete message including attachments)
• display message previews and/or display message without MIME parsing

The body structure is very complex and recursive. Here’s an attempt at making it a bit more approachable.

Note: I’m being a bit lax about the use of String vs. NString here — need to revise that.

Structure

As noted in #75 the top level structure should be something like

    public indirect enum BodyStructure: Equatable {
        case singlePart(BodyStructure.SinglePart)
        case multiPart(BodyStructure.MultiPart)
    }

Then, generally, nest body structure related types inside BodyStructure.

Single Part

A single part would look like this:

extension BodyStructure {
    public struct SinglePart: Equatable {
        public var bodyFields: Fields
        public var kind: Kind
        public var extension: BodyStructure.SinglePart.ExtensionData?
        public var mediaType: MediaType { get } // derived from `kind`
 
        public enum Kind: Equatable {
            /// A RFC 822 message, i.e. `"MESSAGE" "RFC822"`
            case rfc822Message(BodyStructure .RFC822Message)
            /// A message that is text, i.e. `"TEXT" "<subtype>"`
            case text(BodyStructure.Text)
            /// A message that is not an RFC 822 message, i.e. `"<type>" "<subtype>"`
            /// where `<subtype>` is not `RFC822`.
            case basic(BodyStructure.MediaType)
        }
    }
}

#### Single Part — Additional Types

This one is similar to existing`Body.Fields`:
```swift
extension BodyStructure {
    public struct Fields: Equatable {
        public var parameters: [FieldParameterPair]
        public var identifier: NString
        public var description: NString
        public var encoding: Encoding
        public var byteCount: UInt32
    }
}

with

extension BodyStructure {
    public struct FieldParameterPair: Equatable {
        public var field: String
        public var value: String
    }
}

See Body.FieldEncoding:

extension BodyStructure {
    public enum Encoding: Equatable {
        case sevenBit
        case eightBit
        case binary
        case base64
        case quotedPrintable
        case other(EncodedString)
    }
}

extension BodyStructure {
    public struct RFC822Message: Equatable {
        public var envelope: Envelope
        public var body: BodyStructure
        public var lineCount: Int
        public var mediaType: MediaType { get }
    }
}

extension BodyStructure {
    public struct TextMessage: Equatable {
        public var mediaSubtype: NString
        public var lineCount: Int
        public var mediaType: MediaType { get }
    }
}

public enum MediaType: Equatable {
    case text(subtype: EncodedString)
    case rfc822Message
    case application(subtype: EncodedString)
    case audio(subtype: EncodedString)
    case image(subtype: EncodedString)
    case message(subtype: EncodedString)
    case video(subtype: EncodedString)
    case other(type: EncodedString, subtype: EncodedString)
}

**Note: ** This may have to be a struct for API stability.

extension BodyStructure.SinglePart {
    /// Extension data for a single part message.
    ///
    /// This is never returned with the `BODY` fetch,
    /// but can be returned with a `BODYSTRUCTURE` fetch.
    struct ExtensionData {
            /// MD5 of the body
            public var bodyDigest: EncodedString?
            /// RFC 2183 style content disposition
            public var disposition: MessageData.Disposition?
            /// A string or parenthesized list giving the body language
            /// value as defined in RFC 3066
            public var language: MessageData.LanguageIdentifier?
            /// A string list giving the body content URI as defined in RFC 2557
            public var location: MessageData.ContentLocation?
            public var extension: BodyStructure.Extension?
    }
}

The BodyStructure.Extension needs to represent body-extension — which is a string / number / (nested) array of string or number. I don’t know if this is actually used by anyone.

Multi Part

extension BodyStructure {
    public struct MultiPart: Equatable {
        public var parts: [BodyStructure]
        public var mediaSubtype: MediaSubtype
    }
}

extension BodyStructure.MultiPart {
    public enum MediaSubtype: Equatable {
        case alternative
        case related
        case mixed
        case other(String)
    }
}

extension BodyStructure.MultiPart {
    /// Extension data for a single part message.
    ///
    /// This is never returned with the `BODY` fetch,
    /// but can be returned with a `BODYSTRUCTURE` fetch.
    struct ExtensionData {
            public var parameters: [FieldParameterPair]
            /// RFC 2183 style content disposition
            public var disposition: MessageData.Disposition?
            /// A string or parenthesized list giving the body language
            /// value as defined in RFC 3066
            public var language: MessageData.LanguageIdentifier?
            /// A string list giving the body content URI as defined in RFC 2557
            public var location: MessageData.ContentLocation?
            public var extension: BodyStructure.Extension?
    }
}

Rename Extension*part to *.Extension

Child of #76

Remove typealiases where it makes sense to

This means all of the CoreTypealiases file, and most other types that are dotted around.

Rename and simplify body types

Child of #75

Back `Flag` by a private enum

Better for passing around. We'll provide a .other case also for when non-standard flags arrive. API will be stable as the top-level type will be a struct.

Support Swift 5.3, drop earlier versions

We're going to make Swift 5.3 a minimum requirement for running IMAP.

Simplify Date.DateTime

Can we change the format of Date.DateTime to be something like

struct DateTime {
    init?(day: Int, month: Int, year: Int, hour: Int, minute: Int, seconds: Int, zone: Int)
    var day: Int { get }
    var month: Int { get }
    var year: Int { get }
    var hour: Int { get }
    var minute: Int { get }
    var seconds: Int { get }
    var zone: Int { get }
}

i.e. flatten it and simplify using the components?

Maybe move it out of Date such that it’s simply DateTime? IIRC this is only used for INTERNALDATE and if so, maybe rename it to InternalDate?

The storage can be way more compact that 7 Int, it can in fact fit into a single 64 bit value.

We used to use this code:

    init?(day: Int, month: Int, year: Int, hour: Int, minute: Int, seconds: Int, zone: Int) {
        guard
            (1...31).contains(day),
            (1...12).contains(month),
            (0...24).contains(hour),
            (0...60).contains(minute),
            (0...60).contains(seconds),
            ((-24 * 60)...(24 * 60)).contains(zone),
            (1...Int(UInt16.max)).contains(year)
            else { return nil }
        let zoneValue = UInt16(abs(zone))
        let zoneIsNegative = (zone < 0) ? 1 as UInt8 : 0

        var rawValue = 0 as UInt64

        func store<A: UnsignedInteger>(_ value: A, _ a: A) {
            rawValue *= UInt64(a + 1)
            rawValue += UInt64(value)
        }

        store(UInt16(year), 1)
        store(UInt8(zoneIsNegative), 2)
        store(UInt16(zoneValue), 24 * 60)
        store(UInt8(seconds), 60)
        store(UInt8(minute), 60)
        store(UInt8(hour), 60)
        store(UInt8(month), 12)
        store(UInt8(day), 31)

        self.rawValue = rawValue
    }

and

        var remainder = rawValue

        func take<A: UnsignedInteger>(_ a: A) -> A {
            let r = remainder % UInt64(a + 1)
            remainder /= UInt64(a + 1)
            return A(r)
        }

        let day: UInt8 = take(31)
        let month: UInt8 = take(12)
        let hour: UInt8 = take(60)
        let minute: UInt8 = take(60)
        let seconds: UInt8 = take(60)
        let zoneValue: UInt16 = take(24 * 60)
        let zoneIsNegative: UInt8 = take(2)
        let year: UInt16 = take(UInt16.max - 1)
        let zoneMinutes = Int(zoneValue) * ((zoneIsNegative == 0) ? 1 : -1)

to convert between the day, month, year, hour, minute, seconds, zone set and a single UInt64 for compact storage. But we could probably use something better / more performant.

Rename and flatten ResponsePayload

public enum ResponsePayload: Equatable {
    case conditionalState(ResponseConditionalState)
    case flags([NIOIMAP.Flag])
    case list(NIOIMAP.Mailbox.List)
    case listSubscriptions(NIOIMAP.Mailbox.List)
    case search(NIOIMAP.ESearchResponse)
    case status(NIOIMAP.Mailbox, [NIOIMAP.StatusAttributeValue])
    case exists(Int)
    case namespace(NIOIMAP.NamespaceResponse)
    case expunge(Int)
    case fetch(Int)
    case capability([Capability])
    case enable([NIOIMAP.Capability])
    case id([IDParameter])
}

i.e. move BYE into ResponseConditionalState and flatten the rest to this level.

Move common `SinglepartType` fields up a level

Child of #76

Nest fetch related `Response` cases

    public enum Response: Equatable {
        case greeting(Greeting)
        case untaggedResponse(ResponsePayload)
        case fetch(FetchResponseStream)
        case taggedResponse(TaggedResponse)
        case fatalResponse(ResponseText)
        case continuationRequest(ContinueRequest)
    }

with

    public enum FetchResponseStream: Equatable {
        case start
        case simpleAttribute(MessageAttributeType)
        case attributeBegin(MessageAttributesStatic)
        case attributeBytes(ByteBuffer)
        case attributeEnd
        case finish
    }

— and maybe choose a better name for FetchResponseStream?

Not sure why / how .attributeBegin needs a full MessageAttributesStatic?

Finally: flatten MessageAttributeType into

    public enum MessageAttributeType: Equatable {
        case flags([NIOIMAP.Flag])
        case envelope(Envelope)
        case internalDate(Date.DateTime)
        case rfc822(RFC822Reduced?, NIOIMAP.NString)
        case rfc822Size(Int)
        case body(Body, structure: Bool)
        case bodySection(SectionSpec?, Int?, NString)
        case bodySectionText(Int?, Int) // used when streaming the body, send the literal header
        case uid(Int)
        case binaryString(section: [Int], string: NString)
        case binaryLiteral(section: [Int], size: Int)
        case binarySize(section: [Int], number: Int)
    }

Split NIOIMAP into two modules, inner one only using ByteBuffer

An “inner” one which doesn’t use ChannelHandlerContext & al. (only ByteBuffer) and an “outer” one which properly integrates with NIO.

The inner one should only use import NIO.ByteBuffer.

Pass along `IMAP4rev1` capability

When parsing

S: * CAPABILITY IMAP4rev1 STARTTLS AUTH=GSSAPI

the parser should pass along a NIOIMAP.Capability.IMAP4rev1.

Alternatively, the parser should throw an error such as “invalid IMAP version”.

See RFC 3501 section 6.1.1 “CAPABILITY Command” for details:

The server MUST send a single untagged CAPABILITY response with IMAP4rev1 as one of the listed capabilities […]

ParsingError.incompleteMessage should be internal (return `nil` instead)

ParsingError.incompleteMessage must be internal. It's an implementation detail of the parser that must never leave its module.

write test that shows that synchronising literals don't work correctly yet.

We need a test that shows the following things:

• When sending synchronising literals, we don't wait (yet) for the continue request before sending the data [as a client], issue filed as #22
• (this will only be possible with autoSendContinuations: false because we actually implemented this functionality (in #48 which partially fixes #21)) Show that we don't send continue requests when receiving a synchronising literal [as a server]

make our encoders CAPABILITY aware

We should probably automatically issue the best kind of literal given the CAPABILITies.

Purge static initialisers (where it makes sense to)

Originally created as an attempt at some form of syntactic sugar, instead we should consider if our types are "there" yet.

Refactor and make `encoding` type safe

Convert encoding to be a struct to make API safe and remove the opportunity for error. E.g. .7bit = sevenBit, and .7bit != .other("sevenBit")

Restructure / rename `SectionSpec` and nested/related types

What we’ve used so far, looks (something) like this:

public struct SectionSpecifier: Hashable {
    /// Specifies a particular body section.
    ///
    /// This corresponds to the part number mentioned in RFC 3501 section 6.4.5.
    ///
    /// Examples are `1`, `4.1`, and `4.2.2.1`.
    public struct Numbers: RawRepresentable, Hashable {
        public var rawValue: [Int]
        public init(rawValue: [Int]) {
            self.rawValue = rawValue
        }
    }

    /// The last part of a body section sepcifier if it’s not a part number.
    public enum Kind: Hashable {
        /// The entire section, corresponding to a section specifier that ends in a part number, e.g. `4.2.2.1`
        case complete
        /// All header fields, corresponding to e.g. `4.2.HEADER`.
        case header
        /// The specified fields, corresponding to e.g. `4.2.HEADER.FIELDS (SUBJECT)`.
        case headerFields([HeaderFieldName])
        /// All except the specified fields, corresponding to e.g. `4.2.HEADER.FIELDS.NOT (SUBJECT)`.
        case headerFieldsNot([HeaderFieldName])
        /// MIME IMB header, corresponding to e.g. `4.2.MIME`.
        case MIMEHeader
        /// Text body without header, corresponding to e.g. `4.2.TEXT`.
        case text
    }

    public var numbers: Numbers
    public var kind: Kind
}

the main difference to the existing NIOIMAP.SectionSpec is that we have a proper type for the Numbers -- which turns out to be useful for certain parts of the API that expect such a value, e.g. 4.2.2.1.

Convenience

We also have

extension SectionSpecifier {
    public init(_ numbers: SectionSpecifier.Numbers) {
        self.numbers = numbers
        self.kind = .complete
    }

    public static let header = SectionSpecifier(numbers: SectionSpecifier.Numbers(rawValue: []), kind: .header)
    public static let text = SectionSpecifier(numbers: SectionSpecifier.Numbers(rawValue: []), kind: .text)
    public static let complete = SectionSpecifier(numbers: SectionSpecifier.Numbers(rawValue: []), kind: .complete)
}

extension BodySectionSpecifier.Numbers: ExpressibleByArrayLiteral {
    public init(arrayLiteral numbers: Int...) {
        self.init(rawValue: numbers)
    }
}

which makes using this more convenient.

See also: #77

simplify and clarify streaming rules

Currently, streaming requires intricate knowledge about both IMAP and this implementation to know what to do when. We need do simplify and clarify the rules.

We probably want something akin to

enum IMAPResponsePart {
    // 1
    case greeting(GInfo)
    
    // N
    case responseBegin(BeingInfo)  // <--------------------+
       case simpleAttribute(SIInfo) // 1
       case attributeBegin(ABInfo) // <-----------+        |
           case attributeBytes(ByteBuffer // 0..K | 1..M   | 1..N
       case attributeEnd(AEInfo)  // -------------+        |
    case responseEnd(EndInfo)  //  ------------------------+
}

Make `BodyStructure` a `RandomAccessCollection`

Note: This will require some more thought and discussion. But here are some thoughts.

When receiving BodyStructure data, a client will often want to iterate over this structure in order to find the parts that are of interest.

What we’ve found to work well, is to make it conform to RandomAccessCollection with an opaque Index type.

Then add an initialiser on SectionSpec / SectionSpecifier.Numbers that takes one of these opaque Index. That way, given such an Index the client can create a FETCH command to get the data for that particular part.

The RandomAccessCollection also allows the client to look up the particular part (given a SectionSpec) inside a BodyStructure.

The problem with RandomAccessCollection is that the subscript returns a non-Optional and thus will assert that the index exists — which may or may not be what we want.

extension MessageData.BodyStructure: RandomAccessCollection {
    public typealias Element = MessageData.BodyStructure
    public typealias SubSequence = Slice<MessageData.BodyStructure>
    
    /// Opaque index into a body structure.
    public struct Index: Comparable {
    }

    public subscript(position: BodyStructure.Index) -> BodyStructure {
    }

    ...
}

See also: #78 about SectionSpec / SectionSpecifier.Numbers renaming / restructure.

Rename & restructure ResponseStream

Renaming / Restructure

It would be good to change the ResponseStream type and its nested types -- both names and structure.

The types are currently very closely aligned with the Formal Syntax definitions from RFC 3501 — and that makes sense from a parsing perspective.

But (for a reason unknown to me) those names and that structure does not align very will with the rest of the text of RFCC 3501 and not with how one would typically use / consume these types.

Responses

At a high level there are really three kinds of server responses

• untagged
• tagged
• continuation request
  and I think it makes sense to reflect this in ResponseStream. Special casing the greeting makes sense, even though it’s really just a special untagged response.

I’m not sure if the ResponseStream name itself has special meaning in NIO, but I’d also suggest renaming it itself, since this type is really a response rather than a stream. With this it’d be something like

public enum Response: Equatable {
    case greeting(UntaggedResponse)
    case untaggedResponse(UntaggedResponse)
    case statusResponse(StatusResponses)
    case commandContinuationRequest(ContinuationRequest)
}

Note the renamed, nested types, too:

• UntaggedResponse
• TaggedResponse
• ContinuationRequest

Untagged

The untagged responses are (ignoring associated types here -- for brevity):

enum UntaggedResponse {
  // MARK: Status Responses
  
  case ok
  case no
  case bad
  case preAuth
  case bye
  
  // MARK: Server and Mailbox Status
  
  case capability
  case list
  case subscriptionList
  case status
  case search
  case flags
  case exists
  case recent
  
  // MARK: Message Status
  
  case expunge
  case fetch
}

(Tagged) Status Responses

The tagged status response would look like:

struct StatusResponses {
    var tag: Tag
    var status: Status
    
    enum Status {
        case ok(ResponseCode?)
        case no(ResponseCode?)
        case bad(ResponseCode?)
    }
}

Continuation Request

Finally, the continuation request could look like

struct ContinuationRequest {
    var text: ByteBuffer
}

Rename confusing command types

Convert Command to TaggedCommand and CommandType to Command.

Simplify `CommandType.list`

case list([ListSelection], Mailbox, MailboxPatterns, [ListReturnOption])

with

    public enum ListSelectOption: Equatable {
        case remote
        case subscribed
        case recursiveMatch
        case other(ByteBuffer)
        case vendor(vendor: ByteBuffer, ByteBuffer)
    }

    public enum ListReturnOption: Equatable {
        case subscribed
        case children
        case status([StatusAttribute])
        case other(ByteBuffer, [StatusAttribute])
    }

(I think that’s what they RFC supports, but the formal Syntax specification is super complex.)

If we don’t need the other — since no RFCs specify it, I’d drop it. It’s extremely unlikely that there will be any new extensions to IMAP. But it would take some digging through the RFCs to see what’s in there.

Rename and refactor `SinglepartType`

Child of #76

Rename `Mailbox` → `MailboxName`; change underlying storage

This is really the name of a mailbox:

public struct MailboxName: RawRepresentable, Equatable {
    public var rawValue: String
    
    public static let inbox = Self("inbox")
    
    public static func other(_ name: String) -> Self {
        return Self(name)
    }

    public init(_ name: String) {
        if name.lowercased() == "inbox" {
            self.name = "INBOX"
        } else {
            self.name = name
        }
    }

    public init?(rawValue: String) {
        // Do some sanity check here?!?
        self.init(rawValue)
    }
}

Storage

The storage for this should also not be String but either ByteBuffe or just a plain [UInt8]. Mailbox names can either be UTF-7 (that’s what they’re supposed to be) or UTF-8 (what they often are in the wild), and hence a mapping to a user-friendly String is lossy, but the raw bytes of what we got from the server needs to be preserved, since we need to send this exact series of bytes back to the server in order to select the mailbox (or any other operation on it).

The type should probably also have

    var isInbox: Bool { get }   

and conform to CustomStringConvertible — the latter could do the UTF-7 or UTF-8 decoding. It’d be great to have that logic inside NIOIMAP, but that’s probably best to track in another issue.

`MessageAttributesStatic` / `FETCH` response clarifications

MessageAttribute Rename

Rename MessageAttributesStatic → MessageAttribute (note the singular) or MessageData or FetchResponseAttribute would be easier to understand.

MessageAttribute Structure

RFC822 / RFC822.HEADER / RFC822.TEXT

Split these out into their own cases and remove the RFC822Reduced type:

/// `RFC822` response — equivalent to `BODY[]`
case rfc822(NIOIMAP.NString)
/// `RFC822.HEADER` response — equivalent to `BODY[HEADER]`
case rfc822Header(NIOIMAP.NString)
/// `RFC822.TEXT` response — equivalent to `BODY[TEXT]`
case rfc822Text(NIOIMAP.NString)

INTERNALDATE

case internalDate(Date.DateTime)

See #74 about potentially using a specific InternalDate type for this.

BODY / BODYSTRUCTURE

This part of IMAP is very confusing. They are both the same, but BODY does not have any extension data. And it’s very different from BODY[<section>]<<origin octet>>.

I’d suggest

/// A `BODY` response — a form of `BODYSTRUCTURE` without extension data.
case bodyStructure(Body)
/// A `BODYSTRUCTURE` reponse that describes the MIME body structure of a message.
case bodyStructureWithExtensionData(Body)

FLAGS

See also #69

This should just be a case alongside the other ones.

/// `FLAGS` — flags that are set for this message
case flags([Flag])

What are these?

        case binaryString(section: [Int], string: NString)
        case binaryLiteral(section: [Int], size: Int)
        case binarySize(section: [Int], number: Int)

I think these can be removed?!?

Body Rename

BodyStructure would be less confusing. Rename

• TypeSinglepart → BodyStructure.SinglePart
• TypeMultipart → BodyStructure.MultiPart

    public indirect enum BodyStructure: Equatable {
        case singlePart(BodyStructure.SinglePart)
        case multiPart(BodyStructure.MultiPart)
    }

See #76 for more details about this.

Actually encode data as Base64

The current method to write Base64 assumes that the passed data is already Base64, and then just writes it to a buffer. Let's handle actually encoding Base64.

Add ByteBuffer protocol

Please add a protocol that covers ByteBuffer to ease bring-your-own-I/O.

Add commonly announced Capabilities to `NIOIMAP.Capability`

This would reduce the need to use the .other case.

Here’s a list of commonly used capabilities:

ACL
ANNOTATE-EXPERIMENT-1
AUTH=ATOKEN
AUTH=PLAIN
AUTH=PTOKEN
AUTH=WETOKEN
AUTH=WSTOKEN
BINARY
CATENATE
CHILDREN
CONDSTORE
CONTEXT=SEARCH
CONTEXT=SORT
CREATE-SPECIAL-USE
ENABLE
ESEARCH
ESORT
ID
IDLE
IMAP4rev1
LANGUAGE
LIST-STATUS
LITERAL+
LOGIN-REFERRALS
METADATA
MULTISEARCH
NAMESPACE
QRESYNC
QUOTA
RIGHTS=tekx
SASL-IR
SEARCHRES
SORT
SORT=DISPLAY
SPECIAL-USE
STATUS=SIZE
THREAD=ORDEREDSUBJECT
THREAD=REFERENCES
UIDPLUS
UNSELECT
URL-PARTIAL
URLAUTH
UTF8=ACCEPT
WITHIN
XAPPLEPUSHSERVICE
XLIST
XSNIPPET=FUZZY
XUM1

Additionally, it’d be helpful to add an enum for common sub-values, e.g.

extension NIOIMAP {
    public enum Capability: Equatable {
        case auth(Auth)
        …

        enum Auth: Equatable {
            case aToken
            case plain
            case pToken
            case weToken
            case wsToken
            case other(String)
        }
    }
}

B2MV coverage for all IMAP commands/responses

We should have a B2MV test for pretty much every IMAP command/response.

Note to whoever will implement this: Everything that's being streamed needs to be exactly 1 byte long, otherwise the parser is (deliberately) not deterministic as it just hands through the raw data chunks as they come off the network.

understand and test IMAP String encoding

Currently, we’re just handing through the raw bytes but IMAP does have some charset support and also there’s some UTF-7 (not a typo) stuff in the spec.

We need to understand which things in IMAP are fundamentally Strings (and therefore need decoding with the correct charset) and which things are raw bytes.

Rename `ResponseTagged` to `TaggedResponse`

Parent issue: #49

Extensibility for Command

Commands need to be extensible so they can't be a big enum.

My proposal

public struct RenameCommand {
    public var from: Mailbox
    public var to: Mailbox
    // ...
}
public struct SelectCommand {
    public var mailbox: Mailbox
    // ...
}
public struct Command {
    enum Commands {
        case rename(RenameCommand)
        case select(SelectCommand)
    }
    var command: Commands
    public static func rename(_ command: RenameCommand) -> Command {
        return Command(command: .rename(command))
    }
    public var asRename: RenameCommand? {
        switch self.command {
        case .rename(let command):
            return command
        default:
            return nil
        }
    }
    public var asSelect: SelectCommand? {
        switch self.command {
        case .select(let command):
            return command
        default:
            return nil
        }
    }
    public static func select(_ command: SelectCommand) -> Command {
        return Command(command: .select(command))
    }
}
func switchCommand(_ command: Command) {
    if let rename = command.asRename {
        // handle rename
        print(rename)
    } else if let select = command.asSelect {
        // handle select
        print(select)
    } else {
        // send BAD command
    }
}

integrate synchronising literal parsing and backpressure for Command parsing

We need to fix synchronising literals for Command parsing. Currently, we'll just hang.

This also should work in unison with ChannelOutboundHandler.read. Ie. we shouldn't send a 'continue request' unless we're issuing a read, possibly we can issue the 'continue request' straight from read which would be pretty awesome as we'd build on IMAP's built-in backpressure if we're a server.

Flatten message attribute `rfc822` cases into `MessageAttribute`

Extracted from parent issue #75

Refactor the decoder & encoder into one `ChannelHandler` that properly wires everything up

In many protocols, the decoder and the encoder can be split but that is not the case for IMAP. Reasons:

• synchronising literal parsing needs to emit continue requests
• emitting continue requests should be wired up with backpressure (read events)
• encoder cannot always write, needs to sometimes wait for a continue request to come in

So unfortunately, IMAP isn't just parseable and encodable separately, there needs to be some (very little but non-zero) shared state.

We need to acknowledge that and get rid of the ByteToMessageDecoder and MessageToByteEncoder and make an IMAPCodecHandler: ChannelDuplexHandler out of that.

To not lose the B2MDV tests, we should keep an internal (in Tests/*) version of the decoder that just doesn't do any emitting of continue requests and the like. This ByteToMessageDecoder should then never be used, literally just there to allow us to use B2MDV.

Add common Flag.Keyword instances

public enum Keyword: Equatable {
    /// `$Forwarded`
    case forwarded
    /// `$Junk`
    case junk
    /// `$NotJunk`
    case notJunk
    /// `Redirected`
    case unregistered_redirected
    /// `Forwarded`
    case unregistered_forwarded
    /// `Junk`
    case unregistered_junk
    /// `NotJunk`
    case unregistered_notJunk
    /// `$MailFlagBit0`
    case colorBit0
    /// `$MailFlagBit1`
    case colorBit1
    /// `$MailFlagBit2`
    case colorBit2
    /// `$MDNSent`
    case mdnSent
    case other(String)
}

Support encoding Commands with continuations & options

There are different ways to encode IMAP commands, some of which don’t allow for writting an entire command into a ByteBuffer, but require parts of it to be written later (continuations).

Renaming

(Not strictly part of this, but…)

The type name CommandType is a bit odd. Rename:

• enum CommandType -> Command
• struct Command -> TaggedCommand

Encoding Options

In order to specify how to encode a command, the caller should pass in Command.EncodingOptions:

extension Command {
    public struct EncodingOptions {
        public enum Literal {
            /// Normal RFC 3501
            case standard
            /// Non-synchronizing Literals
            /// `LITERAL+` https://tools.ietf.org/html/rfc2088
            case plus
        }

        public enum MessageBodyEncoding {
            /// Normal RFC 3501
            case standard
            /// `BINARY` https://tools.ietf.org/html/rfc3516
            case binary
        }

        public enum Authentication {
            /// Normal RFC 3501
            case standard
            /// `SASL-IR` https://tools.ietf.org/html/rfc4959
            case SASLIR
        }

        public var literal = Literal.standard
        public var messageBodyEncoding = MessageBodyEncoding.standard
        public var authentication = Authentication.standard

        /// The options defined in RFC 3501, i.e. the default / standard options.
        public static let rfc3501 = EncodingOptions()
    }
}

— this is not a complete list, but a good starting point.

Encoding

When part of a command needs to be sent as a continuation, the writeCommand() method needs to be able to support this.

A simple approach would be something like

extension ByteBuffer {
    @discardableResult
    public mutating func writeCommand(_ command: TaggedCommand, pendingContinuations: inout [ByteBuffer], options: Command.EncodingOptions) -> Int { … }
}

where the pending continuations are appended to a given byte buffer. This may be inefficient, though.

It may make sense to have a dedicated PendingContinuations type backed by a ByteBuffer:

public struct PendingContinuations {
    /// Creates an empty instance.
    public init() { … }
    /// Are there pending continuations?
    public var isEmpty: Bool
    /// Number of pending continuations.
    public var count: Int
    // throws if there are no continuations.
    public mutating func popAndWriteNextContinuation(to buffer: ByteBuffer) throws { … }
}

extension ByteBuffer {
    @discardableResult
    public mutating func writeCommand(_ command: TaggedCommand, pendingContinuations: inout PendingContinuations, options: Command.EncodingOptions) -> Int { … }
}

Rename `ResponseStream` to `Response`

Child of #49

Simplify `LIST` / Mailbox Attributes

See also: #112

mbx-list-flags  = *(mbx-list-oflag SP) mbx-list-sflag
                  *(SP mbx-list-oflag) /
                  mbx-list-oflag *(SP mbx-list-oflag)

mbx-list-oflag  = "\Noinferiors" / flag-extension
                    ; Other flags; multiple possible per LIST response

mbx-list-sflag  = "\Noselect" / "\Marked" / "\Unmarked"
                    ; Selectability flags; only one per LIST response

This is a very complicated way of saying that you should only have one of

• \Noselect
• \Marked
• \Unmarked
  but the ordering does not matter — and you can have any kind of attribute.

But the RFC also says that the server SHOULD do this — it’s not strict. Hence I’d opt to not enforce this and keep things simple.

Rename `StatusAttributeValue` -> `MailboxStatusValue`

This should probably also have a case for PERMANENTFLAGS — not sure where that is right now.

Remove parseCommandStream hack

this needs to be implemented

        public mutating func parseCommandStream(buffer: inout ByteBuffer) throws -> NIOIMAP.CommandStream? {
            // TODO: SynchronisingLiteralParser should be added here but currently we don't have a place to return
            // the necessary continuations.

The right return type gives us the following information:

• CommandStream?: a parsed command if enough data
• number of continue requests to send

Kill XCommand support

We don't know what to do with it, so lets wait to support until someone complains.