customink / mysql2-lambda

Precompiled Mysql2 Gem for AWS Lambda

⚠️ Given Lambda Containers are a thing now... and coupled with the fact that there are no AWS provided Amazon Linux 2 / Lambda containers higher than Ruby 2.7, I'd like to consider this project deprecated in favor of using custom images with the RIC installed. This is really easy to do and I outlined it here for those curious. rails-lambda/lamby-cookiecutter#21

Mysql2 Lambda Gem

Very simple Mysql2 gem precompiled for Amazon Linux 1 & 2 (Ruby 2.5 & 2.7) with statically linked libmysqlclient ready for any AWS Lambda usage, including Rails. Add it to your Gemfile:

# When using Rails.
gem 'mysql2-lambda'

# When NOT using Rails.
gem 'mysql2-lambda', require: 'mysql2'

Part of a suite of open source projects from Lamby: Simple Rails & AWS Lambda Integration using Rack. Details include:

• Tested on Ruby 2.5.x & 2.7.x
• Small Size! Only 5 Megabytes. No Lambda Layers needed!
• Works with any MySQL connection. Including RDS Proxy.

Methodology

We used the amazon/aws-sam-cli-build-image-ruby2.5 Docker image from the SAM CLI project to build the MySQL Connector/C's libmysqlclient. From there we ensure the Mysql2 gem statically includes that library. The resulting packaged gem and mysql2.so file looks something like this.

$ ldd mysql2.so
  linux-vdso.so.1 =>  (0x00007ffcb15d4000)
  libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007fa8ebd21000)
  librt.so.1 => /lib64/librt.so.1 (0x00007fa8ebb19000)
  libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa8eb8fd000)
  libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007fa8eb687000)
  libdl.so.2 => /lib64/libdl.so.2 (0x00007fa8eb483000)
  libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007fa8eb24c000)
  libm.so.6 => /lib64/libm.so.6 (0x00007fa8eaf4a000)
  libc.so.6 => /lib64/libc.so.6 (0x00007fa8eab7c000)
  libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fa8ea966000)
  /lib64/ld-linux-x86-64.so.2 (0x00007fa8ec72e000)
  libfreebl3.so => /lib64/libfreebl3.so (0x00007fa8ea764000)

Development

Clone or fork this repository, make sure you have Docker installed, then run the following commands.

./bin/setup
./bin/build

You will now have a packaged gem in the root of your project. Simple tests to make sure it works.

./bin/test 2.5
./bin/test 2.7

License

Thanks to all the contributors of the Mysql2 (https://github.com/brianmario/mysql2) gem. The license here is the same as that. MIT.

Code of Conduct

Everyone interacting in the Mysql2 Lambda project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.

mysql2-lambda's Issues

require: 'mysql2' is NOT working in bundler

Receiving the following error in my files when executing or running tests:

cannot load such file -- mysql2/mysql2.

Ruby 2.7.5

The gem doesn't seem to work on ruby 2.7.5

Bundle installs all the gems, but when I try to require the gem, I get this error:

irb(main):002:0> require 'mysql2'
Traceback (most recent call last):
       10: from /var/lang/bin/irb:23:in `<main>'
        9: from /var/lang/bin/irb:23:in `load'
        8: from /var/lang/lib/ruby/gems/2.7.0/gems/irb-1.2.6/exe/irb:11:in `<top (required)>'
        7: from (irb):2
        6: from /var/lang/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:149:in `require'
        5: from /var/lang/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:160:in `rescue in require'
        4: from /var/lang/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:160:in `require'
        3: from /var/task/vendor/bundle/gems/mysql2-lambda-0.5.3.1/lib/mysql2.rb:36:in `<top (required)>'
        2: from /var/lang/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
        1: from /var/lang/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
LoadError (/var/task/vendor/bundle/gems/mysql2-lambda-0.5.3.1/lib/mysql2/mysql2.so: cannot open shared object file: No such file or directory - /var/task/vendor/bundle/gems/mysql2-lambda-0.5.3.1/lib/mysql2/mysql2.so)

It worked with ruby 2.7.4, but AWS updated the base image with ruby 2.7.5

Here is my Dockerfile:

FROM public.ecr.aws/lambda/ruby:2.7

# Copy dependency management file
COPY lambda/Gemfile ${LAMBDA_TASK_ROOT}

# Install dependencies under LAMBDA_TASK_ROOT
ENV GEM_HOME=${LAMBDA_TASK_ROOT}/vendor/bundle
RUN bundle install
# Copy function code
COPY lambda ${LAMBDA_TASK_ROOT}

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "lambda_function.LambdaFunction::Handler.process" ]

The file is there

sh-4.2# ls -l /var/task/vendor/bundle/gems/mysql2-lambda-0.5.3.1/lib/mysql2/mysql2.so
-rwxr-xr-x 1 root root 5363112 Dec  1 15:08 /var/task/vendor/bundle/gems/mysql2-lambda-0.5.3.1/lib/mysql2/mysql2.so

but it doesn't seem to be a dynamic exec

sh-4.2# ldd /var/task/vendor/bundle/gems/mysql2-lambda-0.5.3.1/lib/mysql2/mysql2.so
        not a dynamic executable

SSL Support?

I am trying to use a PlanetScale database with my Ruby Lambda, and while I can run it locally and in docker, I get "SSL connection error: unknown error number" when running in Lambda.

  require 'active_record'
  require 'mysql2-lambda'
  require 'mysql2'
  ActiveRecord::Base.establish_connection(
    adapter: 'mysql2',
    database: 'xxx',
    username: 'xxx',
    host: 'xxx',
    password: 'xxx,
    sslverify: true,
    # ssl_mode: :verify_identity,
    # sslca: '/etc/ssl/cert.pem',
  )

    "errorMessage": "SSL connection error: unknown error number",
    "errorType": "Function<ActiveRecord::ConnectionNotEstablished>",
    "stackTrace": [
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/mysql2_adapter.rb:45:in `rescue in new_client'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/mysql2_adapter.rb:39:in `new_client'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/mysql2_adapter.rb:23:in `mysql2_connection'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:882:in `public_send'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:882:in `new_connection'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:926:in `checkout_new_connection'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:905:in `try_to_checkout_new_connection'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:866:in `acquire_connection'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:588:in `checkout'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:428:in `connection'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:1128:in `retrieve_connection'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_handling.rb:327:in `retrieve_connection'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/connection_handling.rb:283:in `connection'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/model_schema.rb:553:in `load_schema!'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/attributes.rb:250:in `load_schema!'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/model_schema.rb:539:in `block in load_schema'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/model_schema.rb:536:in `synchronize'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/model_schema.rb:536:in `load_schema'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/model_schema.rb:392:in `columns_hash'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation/query_methods.rb:1313:in `arel_column'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation/calculations.rb:280:in `aggregate_column'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation/calculations.rb:299:in `execute_simple_calculation'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation/calculations.rb:269:in `perform_calculation'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation/calculations.rb:144:in `calculate'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/relation/calculations.rb:51:in `count'",
        "/var/task/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.4.1/lib/active_record/querying.rb:22:in `count'",
        "/var/task/router.rb:8:in `router'"
    ]

external dependency on downloads.mysql.com

If https://downloads.mysql.com/archives/ goes away, the curl in the Dockerfile will fail, presenting a rather critical external dependency. Is this at all a concern?

Unable to get it working with serverless-ruby-layer

I am trying to use it with serverless-ruby-layer, ruby 7.2

getting following error

Error loading the 'mysql2' Active Record adapter. Missing a gem it depends on? Could not find 'mysql2' (~> 0.5) among 379 total gem(s)\nChecked in 'GEM_PATH=/opt/ruby/2.7.0:/var/runtime' , execute `gem env` for more information"