GithubHelp home page GithubHelp logo

activejob's Introduction

Welcome to Rails

What's Rails?

Rails is a web-application framework that includes everything needed to create database-backed web applications according to the Model-View-Controller (MVC) pattern.

Understanding the MVC pattern is key to understanding Rails. MVC divides your application into three layers: Model, View, and Controller, each with a specific responsibility.

Model layer

The Model layer represents the domain model (such as Account, Product, Person, Post, etc.) and encapsulates the business logic specific to your application. In Rails, database-backed model classes are derived from ActiveRecord::Base. Active Record allows you to present the data from database rows as objects and embellish these data objects with business logic methods. Although most Rails models are backed by a database, models can also be ordinary Ruby classes, or Ruby classes that implement a set of interfaces as provided by the Active Model module.

View layer

The View layer is composed of "templates" that are responsible for providing appropriate representations of your application's resources. Templates can come in a variety of formats, but most view templates are HTML with embedded Ruby code (ERB files). Views are typically rendered to generate a controller response or to generate the body of an email. In Rails, View generation is handled by Action View.

Controller layer

The Controller layer is responsible for handling incoming HTTP requests and providing a suitable response. Usually, this means returning HTML, but Rails controllers can also generate XML, JSON, PDFs, mobile-specific views, and more. Controllers load and manipulate models, and render view templates in order to generate the appropriate HTTP response. In Rails, incoming requests are routed by Action Dispatch to an appropriate controller, and controller classes are derived from ActionController::Base. Action Dispatch and Action Controller are bundled together in Action Pack.

Frameworks and libraries

Active Record, Active Model, Action Pack, and Action View can each be used independently outside Rails.

In addition to that, Rails also comes with:

  • Action Mailer, a library to generate and send emails
  • Action Mailbox, a library to receive emails within a Rails application
  • Active Job, a framework for declaring jobs and making them run on a variety of queuing backends
  • Action Cable, a framework to integrate WebSockets with a Rails application
  • Active Storage, a library to attach cloud and local files to Rails applications
  • Action Text, a library to handle rich text content
  • Active Support, a collection of utility classes and standard library extensions that are useful for Rails, and may also be used independently outside Rails

Getting Started

  1. Install Rails at the command prompt if you haven't yet:

    $ gem install rails
  2. At the command prompt, create a new Rails application:

    $ rails new myapp

    where "myapp" is the application name.

  3. Change directory to myapp and start the web server:

    $ cd myapp
    $ bin/rails server

    Run with --help or -h for options.

  4. Go to http://localhost:3000 and you'll see the Rails bootscreen with your Rails and Ruby versions.

  5. Follow the guidelines to start developing your application. You may find the following resources handy:

Contributing

We encourage you to contribute to Ruby on Rails! Please check out the Contributing to Ruby on Rails guide for guidelines about how to proceed. Join us!

Trying to report a possible security vulnerability in Rails? Please check out our security policy for guidelines about how to proceed.

Everyone interacting in Rails and its sub-projects' codebases, issue trackers, chat rooms, and mailing lists is expected to follow the Rails code of conduct.

License

Ruby on Rails is released under the MIT License.

activejob's People

Contributors

aesthetikx avatar chancancode avatar cristianbica avatar dhh avatar douwem avatar googya avatar guilleiguaran avatar haileys avatar itolosa avatar jeremy avatar kalmanh avatar larrylv avatar miyagawa avatar mperham avatar mytrile avatar packagethief avatar rafaelfranca avatar seuros avatar timriley avatar zhouguangming avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

activejob's Issues

Girl_friday adapter?

Should we create an adapter for girl_friday? I see that the gem is kind of deprecated to sucker punch but from the rubygems stats they have pretty big user base (300K downloads on latest version)

/cc @mperham

4.2.0.rc2 NoMethodError Exception: undefined method `deliver_now' for #<Mail::Message:0xfbcfef8>

Hello, when I create a #Mail::Message in my Mailer class, and call "deliver_now!" outside of the class the mail is sent, but when I have a method in my Mailer class to send emails I have an error message:

class Mailer < ActionMailer::Base
def send_email
..
mail(from:..).deliver_now!
end
end

=> NoMethodError Exception: undefined method `deliver_now' for #Mail::Message:0xfbcfef8

Is these an include possible in my Mailer class to allow it to send emails with deliver_now/deliver_later?

If not I can send my mails outside the class, but it used to work for my Mailer before ActiveJobs

Thanks

Add new inline-like built-in adapter to process it in a thread

For development purposes, specially when you're working on improving your actions performances, it's useful to see how much time your action takes to process, considering your jobs are not running inline.

For such cases, it would be useful to have some "threaded" adapter that would work just like enqueue_at(job, Time.now) from the "inline" built-in adapter. Having this other adapter as built-in would be ideal. This would avoid the need to run Sidekiq, for instance, in the development environment.

I'm not sure if there's some mailing list for this project, so I'm creating an issue for the feature request. Please let me know if there are better places to do so.

Congrats for the initiative with this project by the way :)

Support multiple adapters, map adapters to queues

It looks like there is a single global rails config to set the job adapter.

One thing I have now in activemessaging is configuring different queues to different messaging backends.

For example I may have some queues that I use to communicate with an external system using SQS, and other queues that I use to run my own background jobs that go to redis (I actually do something like this now).

Add AWS SWF Adapter?

Hi,

I'd love to see also support for Amazons Simple Workflow Service (SWF).

Best,

Thomas

Job and Queue Configuration

Figured I'd open up an issue to discuss this since it's pretty important to tweak various options that the underlying job systems provide. What should go where?

A very simple initial proposal:

class SomeJob < ActiveJob::Base
  job_options retry: false

  def perform(args)
  end
end

The options hash can then be accessed by the adapter along with the arguments.

Support enqueue_in/enqueue_at?

Some of the adapters (sidekiq, resque, delayed_job) supports running jobs in the future. I think activejob should support this also.

Something to consider: Fail silently when global model location fails

When a model passed as an argument is destroyed after queuing but before the job is actually dequeued and performed, ActiveModel::GlobalLocator.locate calls Model.find which will result in a NotFound error being raised. This means the worker will error out and be put on a secondary "retry" queue where it will just error out and be queued again ad infinitum.

If the model in question no longer exists at perform-time, I think we can reasonably infer that the job doesn't need to be performed anymore. In the case of ActiveJob, we could interpret this in two different ways:

  • we pass not-found model arguments to Job#perform as nil and leave returning early to the job implementation; or
  • we could not call #perform at all and silently regard the job as completed.

The first is less magical but requires that the user is aware of this problem and is always be on the lookout for possible nils, which could be annoying. So I would probably prefer the latter implementation.

I'm curious whether you agree this is a problem that needs to be worked around, and if so, what implementation you would prefer.

If a consensus is reached, I would happily implement this.

Add Sidekiq adapter/wrapper?

Haven't looked into Sidekiq closely, but in case we need something else than just Resque to be compatible, let's add that too.

ActiveJob with multiple databases

Hi there,

I have multiple databases that are switched to based on subdomain in my application controller, using:

Mongoid.override_database(database_name)

In my controller, I am running:

UserMailer.password_reset(user, timeline_name).deliver_later

But I am getting the following exception, which is understandable, as I cannot see a way to let ActiveJob know which database to use:

2015-09-03T13:48:31.159Z 11467 TID-18to6c WARN: /var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/criteria.rb:508:in `check_for_missing_documents!'
/var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/criteria/findable.rb:20:in `execute_or_raise'
/var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/criteria/findable.rb:40:in `find'
/var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/findable.rb:93:in `find'
/var/lib/gems/2.2.0/gems/globalid-0.3.6/lib/global_id/locator.rb:132:in `locate'
/var/lib/gems/2.2.0/gems/globalid-0.3.6/lib/global_id/locator.rb:158:in `block in locate'
/var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/scopable.rb:193:in `block in unscoped'
/var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/scopable.rb:245:in `without_default_scope'
/var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/scopable.rb:192:in `unscoped'
/var/lib/gems/2.2.0/gems/globalid-0.3.6/lib/global_id/locator.rb:158:in `locate'
/var/lib/gems/2.2.0/gems/globalid-0.3.6/lib/global_id/locator.rb:17:in `locate'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/arguments.rb:97:in `deserialize_global_id'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/arguments.rb:83:in `deserialize_argument'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/arguments.rb:40:in `block in deserialize'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/arguments.rb:40:in `map'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/arguments.rb:40:in `deserialize'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/core.rb:90:in `deserialize_arguments'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/core.rb:80:in `deserialize_arguments_if_needed'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/execution.rb:30:in `perform_now'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/execution.rb:21:in `execute'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/processor.rb:75:in `execute_job'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/processor.rb:52:in `block (2 levels) in process'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:127:in `block in invoke'
/var/lib/gems/2.2.0/gems/newrelic_rpm-3.13.0.299/lib/new_relic/agent/instrumentation/sidekiq.rb:33:in `block in call'
/var/lib/gems/2.2.0/gems/newrelic_rpm-3.13.0.299/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:362:in `perform_action_with_newrelic_trace'
/var/lib/gems/2.2.0/gems/newrelic_rpm-3.13.0.299/lib/new_relic/agent/instrumentation/sidekiq.rb:29:in `call'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
/var/lib/gems/2.2.0/gems/sidekiq-unique-jobs-3.0.14/lib/sidekiq_unique_jobs/middleware/server/unique_jobs.rb:16:in `call'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/server/retry_jobs.rb:74:in `call'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/server/logging.rb:11:in `block in call'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/logging.rb:30:in `with_context'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/server/logging.rb:7:in `call'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:132:in `call'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:132:in `invoke'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/processor.rb:51:in `block in process'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/processor.rb:98:in `stats'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/processor.rb:50:in `process'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/calls.rb:28:in `public_send'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/calls.rb:28:in `dispatch'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/call/async.rb:7:in `dispatch'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/cell.rb:50:in `block in dispatch'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/cell.rb:76:in `block in task'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/actor.rb:339:in `block in task'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/task.rb:44:in `block in initialize'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/task/fibered.rb:14:in `block in create'
2015-09-03T13:48:52.729Z 11467 TID-18ft2k ActionMailer::DeliveryJob JID-309e70aff24990115bb17718 INFO: start
2015-09-03T13:48:52.738Z 11467 TID-18ft2k ActionMailer::DeliveryJob JID-309e70aff24990115bb17718 INFO: fail: 0.009 sec
2015-09-03T13:48:52.741Z 11467 TID-18ft2k WARN: {"class"=>"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper", "wrapped"=>"ActionMailer::DeliveryJob", "queue"=>"mailers", "args"=>[{"job_class"=>"ActionMailer::DeliveryJob", "job_id"=>"6f95a56e-4a39-424d-bfc6-82a474774cfb", "queue_name"=>"mailers", "arguments"=>["UserMailer", "password_reset", "deliver_now", {"_aj_globalid"=>"gid://timeline/User/55230a4278616e6fbc010000"}, "monitor"], "locale"=>"en"}], "retry"=>true, "jid"=>"309e70aff24990115bb17718", "created_at"=>1441287513.5178096, "enqueued_at"=>1441288132.7244759, "error_message"=>"Error while trying to deserialize arguments: \nmessage:\n  Document(s) not found for class User with id(s) 55230a4278616e6fbc010000.\nsummary:\n  When calling User.find with an id or array of ids, each parameter must match a document in the database or this error will be raised. The search was for the id(s): 55230a4278616e6fbc010000 ... (1 total) and the following ids were not found: 55230a4278616e6fbc010000.\nresolution:\n  Search for an id that is in the database or set the Mongoid.raise_not_found_error configuration option to false, which will cause a nil to be returned instead of raising this error when searching for a single id, or only the matched documents when searching for multiples.", "error_class"=>"ActiveJob::DeserializationError", "failed_at"=>1441288079.3246949, "retry_count"=>1, "retried_at"=>1441288132.7376645}
2015-09-03T13:48:52.741Z 11467 TID-18ft2k WARN: ActiveJob::DeserializationError: Error while trying to deserialize arguments: 
message:
  Document(s) not found for class User with id(s) 55230a4278616e6fbc010000.
summary:
  When calling User.find with an id or array of ids, each parameter must match a document in the database or this error will be raised. The search was for the id(s): 55230a4278616e6fbc010000 ... (1 total) and the following ids were not found: 55230a4278616e6fbc010000.
resolution:
  Search for an id that is in the database or set the Mongoid.raise_not_found_error configuration option to false, which will cause a nil to be returned instead of raising this error when searching for a single id, or only the matched documents when searching for multiples.

I can see a way to specify a queue, but how do I specify a database?

Can I stack resque gem with plugin resque-history and ActiveJob?

I use ActiveJob with resque_history plugin for monitoring done tasks.

Firstly I include this string require _'resque-history/server'_ into routes file, and then I see new history tab in dashboard.

This is some code in _/app/jobs/welcome_email_job.rb_

require 'resque-history'

class WelcomeEmailJob < ActiveJob::Base
    extend Resque::Plugins::History
    @max_history = 200
    @queue = :email

    def perform(user)
         UserMailer.welcome_email(user).deliver_now
         puts "I delivered mail to #{user.login} now, sincerly yours Resque <3"
    end
end

When job was done, I see in stats tab how many jobs was processed, but history tab empty, just only table head. Can I resolve this trouble?

Add actionmailer-deliver_later gem

Until this is merged into rails/master, I'd like to have an actionmailer-deliver_later gem that basically just wraps this pattern:

class DeliverLaterJob
  queue_as :emails

  def perform(mailer, delivery, *parameters)
    mailer.constantize.send(delivery, *parameters).deliver
  end
end

class ApplicationMailer < ActionMailer::Base
  def deliver_later(delivery, *parameters)
    DeliverLaterJob.enqueue self.class, delivery, *parameters
  end
end

# Usage
SubscriptionMailer.deliver_later :new_message, subscription, subscriber

Anyone want to take a stab at that?

MyJob.enqueue Syntax

Hi there,

I believe, to get a more consistent Rails-like syntax, that enqueue should have an options hash opposed to enqueue_in and enqueue_at methods.

So these would be valid method calls:

MyJob.enqueue record, :at => Date.tomorrow.now
MyJob.enqueue record, :in => 1.week

Since the given method definition is valid Ruby:

def enqueue(*args, options={})
  # ...
end

it seems to me that it's a most concise approach given Rails method definitions.

What do you think?

Add tagged logging

I tried to add tagged logging such that each job execution would be stamped with [ActiveJob] [] [] in the logs. But there's some weird thread interaction screwing it up. If anyone wants to take a stab, please do.

Should we add a delayed mixin so users can delay any method?

One common feature when using background tasks is to have the ability to delay any method from any class/instance in your app. There two variants:

1 - Declare the method as "delayed" (aka delayed_job's handle_asynchronously):

class SomeClasses
  def self.a_method(*)
  end
  delay :a_method
end

So if you do SomeClasses.a_method(1,2,3) it will actually schedule a job. Here's an implementation I used on a project https://gist.github.com/cristianbica/7706419.

2 - Have a delay method for each object which will schedule anything after it:

class SomeClass
  include ActiveJob::Delayable
  def self.method1(*)
  end

  def method2(*)
  end
end

SomeClass.delay.method1(1,2,3)
SomeClass.delay(queue: :lowprio).method1(1,2,3)
SomeClass.delay_until(5.minutes.from_now, queue: :lowprio).method1(1,2,3)
SomeClass.new.delay.method2(1,2,3)

Which variant do you prefer? Should we add this feature in AJ or as a gem?

@mperham what do you think? I "suspect" that you rejected this kind of stuff from core sidekiq, right?

/cc @dhh, @seuros, @DouweM, @rafaelfranca

PS: I'm for variant 2 and not sure if in AJ or gem (I'm slightly inclined to a gem and if we see massive adoption we can merge it inside AJ/rails)

Wrappers hide underlying job type

Here's three jobs pushed by AJ:

screen shot 2014-05-19 at 4 54 51 am

Here's three jobs pushed with Sidekiq's native API:

screen shot 2014-05-19 at 4 55 08 am

The issue is that the native tooling doesn't know the actual job type anymore so functionality like searching or filtering based on job type won't work. This is a fundamental limitation to the wrapper pattern.

Sidekiq can almost do without the wrapper except for the parameter deserializing needed for GlobalID:

module ActiveJob
  module QueueAdapters
    class SidekiqAdapter
      class << self
        def queue(job, *args)
          Sidekiq::Client.push 'class' => job, 'queue' => job.queue_name, 'args' => args
        end
      end

      class JobWrapper
        include Sidekiq::Worker

        def perform(job_name, *args)
          job_name.constantize.new.perform *Parameters.deserialize(args)
        end
      end
    end
  end
end

class ActiveJob::Base
  include Sidekiq::Worker
end

If we can count on AJ::Parameters.deserialize as the official way to prepare the job data, I can bake that directly into Sidekiq so the wrapper is not necessary and we won't lose native functionality. WDYT?

Provide more solid default adapter

While responding to #76 I was thinking that we should provide a better default adapter (ActiveJob::QueueAdapters::DefaultAdapter :) ). Most of rails apps just need to send emails async and AJ will greatly improve rails apps speed by providing that. So my proposal is:

  • have a simple adapter as the default adapter
  • the adapter should have a class attribute inline
  • .enqueue should just run the job if inline==true
  • .enqueue should start a new thread and run the job if inline==false
  • .enqueue_at/enqueue_in should raise NotImplementedError (most probably future jobs will be lost on process restart so implementing future jobs will cause trouble)
  • on worker restart we should find a way to give any jobs thread a few seconds to finish and if not finished kill the thread and log that

PS: If we want to go further and protect the users agains opening many threads (example: sending a newsletter to 500 people will open 500 threads for each email and will probably cause all kind of troubles - CPU, RAM, networking) we could implement the some like in the ActiveSupport::Queue and have just one thread per server process that will process jobs as they arrive
PS2: To make testing easier we could rename the current adapter as TestAdapter so people can use it in their tests

/cc @dhh, @rafaelfranca, @seuros, @DouweM, @mperham

Add Resque 2 adapter

The API changed for Resque 2. Add adapter/wrapper to works with that as Resque2Adapter. Figure out a way to run tests with both Resque 1 and 2.

Switch perform to be an instance method

Right now ActiveJobs look like this:

class MyJob < ActiveJob::Base
  def self.perform(*args)
  end
end

This is quite different from ActiveRecord, ActiveController and other Rails APIs. To parallel those APIs, an ActiveJob subclass should be a type of job and an instance of that class should be an individual job, exactly how an ActiveRecord class is a table and an ActiveRecord instance is a row in that table. An ActionController class represents a set of requests which can be handled and an AC instance is an actual request, etc.

class MyJob < ActiveJob::Base
  def perform(*args)
  end
end

This also makes Jobs easier to test in that dependencies can be injected per-instance:

job = MyJob.new
job.web_client = NoopClient.new
job.perform("something")

In additional to the API design, there is a major thread-safety limitation with using a static method: one cannot use instance variables. I've reviewed lots of Sidekiq apps and people do use them all the time. ActiveJobs would have to document the fact that the user cannot use instance variables if they want their jobs to work reliably in Sidekiq or sucker_punch.

Add deep serialization

The following should work:

class RefreshTaggablesJob < ActiveJob::Base
  def perform(taggables)
    taggables.each(&:refresh)
  end
end

RefreshTaggablesJob.enqueue [ post, comment, person ]

In other words, ActiveJob::Arguments.serialize needs to deep dive into arrays and hashes.

resque job serialization fails under Rails

When you enqueue and perform activejob tasks under Rails environment, i.e. with "active_support/all" being loaded, job serialization fails for GlobalID objects with Resque adapter.

Try adding require 'active_support/all' to test/helper.rb and run rake test_resque:

# Running:

.........................................E

Finished in 0.327346s, 128.3046 runs/s, 268.8287 assertions/s.

  1) Error:
JobSerializationTest#test_serialize_job_with_gid:
NoMethodError: undefined method `id' for {"gid"=>#<Person:0x007fa9345f34a0 @id="5">}:ActiveSupport::HashWithIndifferentAccess
    /Users/miyagawa/src/github.com/rails/activejob/test/jobs/gid_job.rb:3:in `perform'
    /Users/miyagawa/src/github.com/rails/activejob/lib/active_job/execution.rb:17:in `block in execute'
    /Users/miyagawa/.rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:113:in `call'
    /Users/miyagawa/.rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:113:in `call'
    /Users/miyagawa/.rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/activesupport-4.1.1/lib/active_support/callbacks.rb:329:in `block (2 levels) in simple'

I suspect that this is due to Resque's own serializer (MultiJson?) tries to encode ActiveModel::GlobalID object as {"gid" => "GlobalID-Person-1"} because of Object#as_json where activejob expects it to be encoded as a plain string.

Job IDs?

Is this something that should/can be baked into AJ? Rails has Request ID middleware and all ActiveRecords have IDs. Sidekiq jobs all have a JID so the user can follow the path of execution in logs.

As a side note, this is one area where the native API leaks out of the AJ API: the return value of enqueue.

> MyJob.enqueue(1)
=> "59c2c1997c3193b3c950cea8"

perform() argument types

The ActiveJob API design has one final point that can cause user confusion and generate support issues: passing complex Ruby objects to perform. Right now Resque and Sidekiq force all perform() arguments to be basic JSON types only: int, float, string, boolean, etc. However delayed_job supports complex objects via YAML. Switching the adapter layer to Resque, e.g., would not work if the user has existing DJ jobs that pass complex objects.

Should AJ restrict argument types or is this more of a documentation issue? See Sidekiq's best practices for what I did.

I'm happy to create a wiki page for it, I just want to open a conversation so we can decide what to do, if anything.

Lazy-load adapters

Using this API: ActiveJob::Base.adapter = :sidekiq. All adapters should be lazy-loaded.

Are multiple queues even possible as written?

Consider the following:

class MailerJob < ActiveJob::Base
  queue_as :mailer_queue

  def perform
    sleep 1
  end
end

class OtherJob < ActiveJob::Base
  queue_as :other_queue

  def perform
    sleep 1
  end
end

I've modified activejob's logger to also report the queue name:

[2] pry(main)> MailerJob.enqueue
Enqueued MailerJob to Inline using active_jobs_other_queue

Note that this was added to "other_queue" rather than "mailer_queue" since was defined later. Am I missing something?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.