GithubHelp home page GithubHelp logo

fabriziobrancati / queuer Goto Github PK

View Code? Open in Web Editor NEW
1.1K 21.0 49.0 909 KB

Queuer is a queue manager, built on top of OperationQueue and Dispatch (aka GCD).

Home Page: https://swiftpackageindex.com/FabrizioBrancati/Queuer

License: MIT License

Swift 100.00%
queuer swift linux operationqueue carthage semaphore dispatch asynchronous-tasks macos tvos

queuer's Introduction

Queuer

GitHub Release Swift Versions Swift Platforms

Features

Queuer is a queue manager, built on top of OperationQueue and Dispatch (aka GCD). It allows you to create any asynchronous and synchronous task easily, all managed by a queue, with just a few lines.

Here is the list of all the features:

  • Works on all Swift compatible platforms (even Linux)
  • Easy to use
  • Well documented (100% documented)
  • Well tested (100% of code coverage)
  • Create an operation block
  • Create a single operation
  • Create chained operations
  • Manage a centralized queue
  • Create unlimited queue
  • Declare how many concurrent operation a queue can handle
  • Create semaphores
  • Create and handle schedules
  • Automatically or manually retry an operation
  • Throttling between each automatic operation retry

Requirements

Swift Queuer iOS macOS macCatalyst tvOS watchOS visionOS Linux
3.1...3.2 1.0.0...1.1.0 8.0+ 10.10+ 9.0+ 2.0+
4.0 1.3.0 8.0+ 10.10+ 9.0+ 2.0+
4.1 1.3.1...1.3.2 8.0+ 10.10+ 9.0+ 2.0+
4.2 2.0.0...2.0.1 8.0+ 10.10+ 9.0+ 3.0+
5.0...5.10 2.1.0...2.2.0 8.0+ 10.10+ 9.0+ 3.0+
5.9...5.10 3.0.0 12.0+ 10.13+ 13.0+ 12.0+ 4.0+ 1.0+

Installing

See Requirements section to check Swift, Queuer, and OS versions.

In your Package.swift Swift Package Manager manifest, add the following dependency to your dependencies argument:

.package(url: "https://github.com/FabrizioBrancati/Queuer.git", from: "3.0.0"),

Add the dependency to any targets you've declared in your manifest:

.target(
    name: "MyTarget", 
    dependencies: [
        .product(name: "Queuer", package: "Queuer"),
    ]
),

Usage

Shared Queuer

Queuer offers a shared instance that you can use to add operations to a centralized queue:

Queuer.shared.addOperation(operation)

Custom Queue

You can also create a custom queue:

let queue = Queuer(name: "MyCustomQueue")

You can even create a queue by defining the maxConcurrentOperationCount and the qualityOfService properties:

let queue = Queuer(name: "MyCustomQueue", maxConcurrentOperationCount: Int.max, qualityOfService: .default)

Create an Operation Block

You have three methods to add an Operation block.

  1. Directly on the queue(or Queuer.shared):

    queue.addOperation {
        /// Your task here
    }
  2. Creating a ConcurrentOperation with a block:

    let concurrentOperation = ConcurrentOperation { _ in
        /// Your task here
    }
    queue.addOperation(concurrentOperation)

Note

We will see how ConcurrentOperation works later.

Chained Operations

Chained Operations are Operations that add a dependency each other.

They follow the given array order, for example: [A, B, C] = A -> B -> C -> completionBlock.

let concurrentOperationA = ConcurrentOperation { _ in
    /// Your task A here
}
let concurrentOperationB = ConcurrentOperation { _ in
    /// Your task B here
}
queue.addChainedOperations([concurrentOperationA, concurrentOperationB]) {
    /// Your completion task here
}

You can also add a completionHandler after the queue creation with:

queue.addCompletionHandler {
    /// Your completion task here
}

Group Operations

Group Operations are Operations that handles a group of Operations with a completion handler.

Allows the execution of a block of Operations with a completion handler that will be called once all the operations are finished, for example: [A -> [[B & C & D] -> completionHandler] -> E] -> completionHandler. It should usually be used with a Chained Opetation.

let groupOperationA = GroupOperation(
    [
        ConcurrentOperation { _ in
            /// Your task A here
        },
        ConcurrentOperation { _ in
            /// Your task B here
        }
    ]
)

let concurrentOperationC = ConcurrentOperation { _ in
    /// Your task C here
}

queue.addChainedOperations([groupOperationA, concurrentOperationC]) {
    /// Your completion task here
}

In this case the output will be the following one: [[A & B -> completionHandler] -> C] -> completionHandler.

Queue States

There are a few method to handle the queue states.

  1. Cancel all Operations in a queue:

    queue.cancelAll()
  2. Pause a queue:

    queue.pause()

Warning

By calling pause() you will not be sure that every Operation will be paused. If the Operation is already started it will not be on pause until it's a custom Operation that overrides pause() function.

  1. Resume a queue:

    queue.resume()

Warning

To have a complete pause and resume states you must create a custom Operation that overrides pause() and resume() function.

  1. Wait until all Operations are finished:

    queue.waitUntilAllOperationsAreFinished()

Important

This function means that the queue will blocks the current thread until all Operations are finished.

Synchronous Queue

Setting the maxConcurrentOperationCount property of a queue to 1 will make you sure that only one task at a time will be executed.

Asynchronous Operation

ConcurrentOperation is a class created to be subclassed. It allows synchronous and asynchronous tasks, has a pause and resume states, can be easily added to a queue and can be created with a block.

You can create your custom ConcurrentOperation by subclassing it.

You must override execute() function and call the finish(success:) function inside it, when the task has finished its job to notify the queue.

For convenience it has an init function with a completion block:

let concurrentOperation = ConcurrentOperation { _ in
    /// Your task here
}
concurrentOperation.addToQueue(queue)

Automatically Retry an Operation

An Operation is passed to every closure, with it you can set and handle the retry feature.

By default the retry feature is disabled, to enable it simply set the success property to false. With success to false the Operation will retry until reaches maximumRetries property value. To let the Operation know when everything is ok, you must set success to true.

With currentAttempt you can know at which attempt the Operation is.

let concurrentOperation = ConcurrentOperation { operation in
    /// Your task here
    if /* Successful */ {
      operation.success = true
    } else {
      operation.success = false
    }
}

Manually Retry an Operation

You can manually retry an Operation when you think that the execution will be successful.

An Operation is passed to every closure, with it you can set and handle the retry feature.

By default the manual retry feature is disabled, to enable it simply set the manualRetry property to true, you must do this outside of the execution closure. You must also set success to true or false to let the Operation know when is everything ok, like the automatic retry feature.

To let the Operation retry your execution closure, you have to call the retry() function. Be sure to call it at least maximumRetries times, it is not a problem if you call retry() more times than is needed, your execution closure will not be executed more times than the maximumRetries value.

let concurrentOperation = ConcurrentOperation { operation in
    /// Your task here
    if /* Successful */ {
      operation.success = true
    } else {
      operation.success = false
    }
}
concurrentOperation.manualRetry = true

/// Later on your code
concurrentOperation.retry()

Caution

If the retry() function is not called, you may block the entire queue.

Manually Finish an Operation

By default the Operation will finish its job when the execution closure is completed.

This means, if you have an asynchronous task insiede the closure, the Operation will finish before the task is completed.

If you want to finish it manually, you can set the manualFinish property to true and call the finish(success:) function when the task is completed.

Call the finish(success:) function with success parameter to true or false to let the Operation know if the task was successful or not. If you don't explicitly set the success parameter, it will be set to true.

let concurrentOperation = ConcurrentOperation { operation in
    /// Your asynchonous task here
}
concurrentOperation.manualFinish = true

/// Later on your code in your asynchronous task
concurrentOperation.finish(success: true)

Caution

If you don't call the finish(success:) function, your queue will be blocked and it will never ends.

Async Task in an Operation

If you want to use async/await tasks, you need to set manualFinish to true in order to be fully in control of the Operation lifecycle.

Note

Read more about manually finish an Operation here.

let concurrentOperation = ConcurrentOperation { operation in
    Task {
        /// Your asynchonous task here
        operation.finish(success: true) // or false
    }
    /// Your asynchonous task here
}
concurrentOperation.manualFinish = true

Caution

If you don't set manualFinish to true, your Operation will finish before the async task is completed.

Scheduler

A Scheduler is a struct that uses the GDC's DispatchSourceTimer to create a timer that can execute functions with a specified interval and quality of service.

let schedule = Scheduler(deadline: .now(), repeating: .seconds(1)) {
    /// Your task here
}

You can even create a Scheduler without the handler and set it later:

var schedule = Scheduler(deadline: .now(), repeating: .seconds(1))
schedule.setHandler {
    /// Your task here
}

With timer property you can access to all DispatchSourceTimer properties and functions, like cancel():

schedule.timer.cancel()

Semaphore

A Semaphore is a struct that uses the GCD's DispatchSemaphore to create a semaphore on the function and wait until it finish its job.

Tip

Is recommend to use a defer { semaphore.continue() } right after the Semaphore creation and wait() call.

let semaphore = Semaphore()
semaphore.wait()
defer { semaphore.continue() }
/// Your task here

You can even set a custom timeout, default is .distantFuture:

semaphore.wait(DispatchTime(uptimeNanoseconds: 1_000_000_000))

It's more useful if used inside an asynchronous task:

let concurrentOperation = ConcurrentOperation {
    /// Your task here
    semaphore.continue()
}
concurrentOperation.addToQueue(queue)
semaphore.wait()

Changelog

To see what has changed in recent versions of Queuer, see the CHANGELOG.md file.

Communication

  • If you need help, open an issue.
  • If you found a bug, open an issue.
  • If you have a feature request, open an issue.
  • If you want to contribute, see Contributing section.

Contributing

See CONTRIBUTING.md file.

License

Queuer is available under the MIT license. See the LICENSE file for more info.

queuer's People

Contributors

fabriziobrancati avatar thatjuan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

queuer's Issues

How to bring persistence?

I have scenario where i need to resume or retry certain operation after killing and reopening app.

Ultimately how to bring persistence with (SQLite, Realm or UserDefaults) in this.

The Queuer watch framework deployment target should be 3.0 or later

When uploading an iOS / watchOS app bundle via fastlane to iTunes Connect I get the following error message from iTunes Connect:

[17:30:23]: ERROR ITMS-90081: "This bundle is invalid. Applications built for more than one architecture require an iOS Deployment Target of 3.0 or later."
Return status of iTunes Transporter was 1: ERROR ITMS-90081: "This bundle is invalid. Applications built for more than one architecture require an iOS Deployment Target of 3.0 or later."
The call to the iTMSTransporter completed with a non-zero exit status: 1. This indicates a failure.

Changing the WATCHOS_DEPLOYMENT_TARGET of the watch target of the Queuer framework from 2.0 to 3.0 solves this issue.

Queue on Linux never gets longer than 1

Hi

I have been experimenting with this library for a tool I am porting from macOS to Linux and I seem to have a strange problem that I am unsure if is in this library or underlying in dispatch.

I am trying to add work to a queue on Linux to parallelise it, but it looks like the queue is never longer than one item and therefore not doing anything in parallel.

A minimum example:

import Queuer
import Foundation
import Dispatch


let queue = Queuer(
        name: "write", maxConcurrentOperationCount: Int.max, qualityOfService: .background)

for i in 1 ..< 10 {
        let op = ConcurrentOperation { _ in
                Thread.sleep(forTimeInterval: 0.2)
                print(i)
        }
        // queue.addOperation(op)
        op.addToQueue(queue)
        print("Queue length: ", queue.operationCount)
}

print("Queue length: ", queue.operationCount)
queue.waitUntilAllOperationsAreFinished()
print("Queue length: ", queue.operationCount)

Resulting in (where everything always print in order, one at the time):

Queue length:  1
Queue length:  1
Queue length:  1
Queue length:  1
Queue length:  1
Queue length:  1
Queue length:  1
Queue length:  1
Queue length:  1
Queue length:  1
1
2
3
4
5
6
7
8
9
Queue length:  0

I might be doing something silly, but I am struggling to see something obvious.

EDIT:

Essential information I forgot:

I have tested this with Queuer 2.1.1 and Swift 5.2.4 on Ubuntu 20.04 and 20.10 Beta. (and macOS, but that works as intended)

Queuer on Linux Ubuntu 16.04

$ cat /etc/lsb-release 
DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"
$ swift --version
Swift version 4.0 (swift-4.0-RELEASE)
Target: x86_64-unknown-linux-gnu
$ cat Package.swift 
import PackageDescription

let package = Package(
    name: "testSwift",
    products: [
        .executable(name: "testSwift", targets: ["testSwift"])
    ],
    dependencies: [
	.package(url: "https://github.com/FabrizioBrancati/Queuer.git", .upToNextMajor(from: "1.0.0"))
    ],
    targets: [
        .target(
            name: "testSwift", dependencies: ["BFKit"]),
    ]
)
$ swift build
error: product dependency 'BFKit' not found

What I am doing incorrect?

Serial task

The call of interface A returns the result and interface B is called, interface B finishes calling C, and finally synthesizes and renders. This kind of blocking doesn't seem to work.

Let's muse about the semantics of the new `func finish(_ hasFailed: Bool)` call of v2.0 of Queuer

I really love simplicity of this framework and the way it is heading. However, I'm not entirely sure I like the semantics of the new func finish(_ hasFailed: Bool) call of version 2.0 of Queuer.

At the call side a successful finish of an operation now looks like this:

operation.finish(false)

Since the parameter name is not exposed, my mind reads "operation not finished". If I'm the only person seeing a potential pitfall here, please close the issue and let's forget about it. Otherwise, I suggest to either exposed the parameter name to make obvious what is happening:

operation.finish(hasFailed: false)

Or, what I think would be a more natural approach, change the semantics to func finish(success: Bool):

operation.finish(success: true)

Or maybe event to func finish(success: Bool = true) :

operation.finish()

I think the last option could have some benefits:

  • The most common task of just finishing an operation is the simplest option: operation.finish()
  • If a user needs a more fine grained behavior, she can add details: operation.finish(success: false)
  • A user new to the framework needs not to know, that a retry-logic event exists
  • This approach is compatible to the previous version of the framework

Is this the expected output?

For the following code, is the output after that the expected result, please?

class ViewController: UIViewController {
	var i:Int = 0
	override func viewDidLoad () {
		super.viewDidLoad()
		let queue = Queuer( name: "mike" )
		print( "a \(i)" )
		queue.addChainedOperations( [
			ConcurrentOperation{ self.i = 7 },
			ConcurrentOperation{ self.i = 5 },
			ConcurrentOperation{ self.i = 3 } ]
		) { print( "complete" ) }
		print( "b \(i)" )
		print( "hey 1" )
		print( "hey 2" )
		print( "hey 3" )
	}
}

a 0
b 3
complete
hey 1
hey 2
hey 3

SPM Warning

The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.4.99.

Possible to create a unviersal queue for alamofire requests?

Just wondered if it was possible to use this Queuer to push all alamofire requests into it with different priority scales? I think i read somewhere in an issue it wasn't possible with Alamofire or something along those lines but saw a new version was brought out.

Unable to subclass due to Public Access Control restrictions

Installed with CococaPods and running Xcode 8.3 / Swift 3.1.

Subclassing ConcurrentOperation in my own project with import Queuer as per the README.

On compile, getting the error:

Cannot inherit from non-open class ConcurrentOperation outside of its defining module.

and

Overriding non-open instance method outside of its defining module.

First error fixed by changing ConcurrentOperation from public to open. Must also change the overrides in the class to open to match Access Controls of Operation.

Second error fixed by making -ConcurrentOperation.execute() open as well.

finish(success: Bool) does nothing to success

Finishing a ConcurrentOperation with operation.finish(success: false) does nothing to its success property. I would expect success to be false if I finish the operation that way 🤔

Chained operation results

Hello,

I am discovering this project which looks super interesting. However, I have a question about the Queuer class and the addChainedOperations method. Is it possible for the queued operations to pass the result of their process to the next one?

For example:
Operation1 -> Copies a file in a temporary directory and returns an URL.
Operation2 -> Takes an URL as input and uploads the corresponding file on a server.

Is it possible for Operation2 to get the input from Operation1 if they are queued with addChainedOperations?

Best regards,
Nicolas

Operations are `isFinished=true` without me explicitly calling `operation.finish()`

Steps to reproduce

  1. Add some ConcurrentOperations to a queue
  2. Add a completion handler to the same queue.

Expected behavior:
Completion handler gets called after the last operation in queue calls finish().

Actual behavior:
Completion handler gets called regardless if the queue operations are still running (finish() hasn't been called yet)

Handle Retry

Can Queuer handle failed task executes and retry accordingly ? Say the task failed due to no network and can retry when network is up.

Queuer by CocoaPod

Hi Fabrizio,

I've imported your Queuer framework by CocoaPod (pod Queuer) in my own Framework that it's used in my app. When I try to do the Archive Xcode shows me this error:
bitcode bundle could not be generated because '/MyFramework../Build/Products/Release-iphoneos/Queuer/Queuer.framework/Queuer' was built without full bitcode. All frameworks and dylibs for bitcode must be generated from Xcode Archive or Install build file '/MyFramework.../Build/Products/Release-iphoneos/Queuer/Queuer.framework/Queuer' for architecture armv7.

I've tried several options searched by google but all have failed. I've downloaded the Queuer framework project and I've added to my framework and it works (without changing any parameters).

Am I doing something wrong?
It's a shame I can't use it with CocoaPods.

Regards,

It is support NSURLUploadTask?

Is it work in NSURLUploadTask in background/suspended state of app.

I want to upload task execute in synchronize manner.

Let's say 10 task in queue.
It should execute in queue one by one like... 1, 2,3..etc

Need to upload thousands of images in background.

Please advise and suggestions are greatly appreciated.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.