GithubHelp home page GithubHelp logo

arkadiyt / ssrf_filter Goto Github PK

View Code? Open in Web Editor NEW
77.0 5.0 31.0 75 KB

A ruby gem for defending against Server Side Request Forgery (SSRF) attacks

Home Page: https://rubygems.org/gems/ssrf_filter

License: MIT License

Ruby 99.00% Dockerfile 0.61% Makefile 0.39%
ruby ssrf gem server-side-request-forgery

ssrf_filter's Introduction

ssrf_filter Gem Tests Coverage Status Downloads License

Table of Contents

What's it for

ssrf_filter makes it easy to defend against server side request forgery (SSRF) attacks. SSRF vulnerabilities happen when you accept URLs as user input and fetch them on your server (for instance, when a user enters a link into a Twitter/Facebook status update and a content preview is generated).

Users can pass in URLs or IPs such that your server will make requests to the internal network. For example if you're hosted on AWS they can request the instance metadata endpoint http://169.254.169.254/latest/meta-data/ and get your IAM credentials.

Attempts to guard against this are often implemented incorrectly, by blocking all ip addresses, not handling IPv6 or http redirects correctly, or having TOCTTOU bugs and other issues.

This gem provides a safe and easy way to fetch content from user-submitted urls. It:

  • handles URIs/IPv4/IPv6, redirects, DNS, etc, correctly
  • has 0 runtime dependencies
  • has a comprehensive test suite (100% code coverage)
  • is tested against ruby 2.6, 2.7, 3.0, 3.1, 3.2, and ruby-head

Quick start

  1. Add the gem to your Gemfile:
gem 'ssrf_filter', '~> 1.1.2'
  1. In your code:
require 'ssrf_filter'
response = SsrfFilter.get(params[:url]) # throws an exception for unsafe fetches
response.code
=> "200"
response.body
=> "<!doctype html>\n<html>\n<head>\n..."

API reference

SsrfFilter.get/.put/.post/.delete/.head/.patch(url, options = {}, &block)

Fetches the requested url using a get/put/post/delete/head/patch request, respectively.

Params:

  • url — the url to fetch.
  • options — options hash (described below).
  • block — a block that will receive the HTTPRequest object before it's sent, if you need to do any pre-processing on it (see examples below).

Options hash:

  • :scheme_whitelist — an array of schemes to allow. Defaults to %w[http https].
  • :resolver — a proc that receives a hostname string and returns an array of IPAddr objects. Defaults to resolving with Ruby's Resolv. See examples below for a custom resolver.
  • :max_redirects — Maximum number of redirects to follow. Defaults to 10.
  • :params — Hash of params to send with the request.
  • :headers — Hash of headers to send with the request.
  • :body — Body to send with the request.
  • :http_options – Options to pass to Net::HTTP.start. Use this to set custom timeouts or SSL options.
  • :request_proc - a proc that receives the request object, for custom modifications before sending the request.
  • :allow_unfollowed_redirects - If true and your request hits the maximum number of redirects, the last response will be returned instead of raising an error. Defaults to false.

Returns:

An HTTPResponse object if the url was fetched safely, or throws an exception if it was unsafe. All exceptions inherit from SsrfFilter::Error.

Examples:

# GET www.example.com
SsrfFilter.get('https://www.example.com')

# Pass params - these are equivalent
SsrfFilter.get('https://www.example.com?param=value')
SsrfFilter.get('https://www.example.com', params: {'param' => 'value'})

# POST, send custom header, and don't follow redirects
begin
  SsrfFilter.post('https://www.example.com', max_redirects: 0,
    headers: {'content-type' => 'application/json'})
rescue SsrfFilter::Error => e
  # Got an unsafe url
end

# Custom DNS resolution and request processing
resolver = proc do |hostname|
  [IPAddr.new('2001:500:8f::53')] # Static resolver
end
# Do some extra processing on the request
request_proc = proc do |request|
  request['content-type'] = 'application/json'
  request.basic_auth('username', 'password')
end
SsrfFilter.get('https://www.example.com', resolver: resolver, request_proc: request_proc)

# Stream response
SsrfFilter.get('https://www.example.com') do |response|
  response.read_body do |chunk|
    puts chunk
  end
end

Changelog

Please see CHANGELOG.md. This project follows semantic versioning.

Contributing

Please see CONTRIBUTING.md.

ssrf_filter's People

Contributors

arkadiyt avatar artfuldodger avatar elliterate avatar jakeyheath avatar maxburkhardt avatar mshibuya avatar petergoldstein avatar vaski avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

ssrf_filter's Issues

Treat the odd case where redirect has no Location header

At version 1.0.7, calling SsrfFilter.get('http://www.macys.com') returns:

NoMethodError: undefined method `start_with?' for nil:NilClass

From lib/ssrf_filter/ssrf_filter.rb:133, which is

129:         case response
130:         when ::Net::HTTPRedirection then
131:           url = response['location']
132:           # Handle relative redirects
133>>>         url = "#{uri.scheme}://#{hostname}:#{uri.port}#{url}" if url.start_with?('/')
134:         else
135:           return response
136:         end

In this case, response['location'] is nil, which is unusual, but is permitted.

As macys.com should be resolved, is there a way to make this request pass? From the lib standpoint, perhaps it'd be interesting to raise an exception in these situations.

1.1.1 version breaks Resolv gem IPv4

The monkeypatch for the Resolv gem is actually breaking the regex class for IPv4.

https://github.com/ruby/resolv/blob/993a1a374fcb2a91b9d26a2215ddbb116db43788/lib/resolv.rb#L2349

As far as I can tell, it's due to the code found here:

        if (0..255) === (a = $1.to_i) &&
           (0..255) === (b = $2.to_i) &&
           (0..255) === (c = $3.to_i) &&
           (0..255) === (d = $4.to_i)

It expects global variables $1 to $4 to be set, with values matching each of the ipv4 groups. The overriding === method does not set these global variables in the monkey patched implementation, so one ends up with an IPv4 object with address 0.0.0.0, because the globals are nil and nil.to_i is 0.

An example of an IPv4 dns that breaks is: Resolv.getname('167.89.116.35')

This works before the Resolv patch is applied, but breaks once Resolve has been patched.

Add way to globally disable filter

When using test request mocks (like webmocks gem), the SSRF filter makes it impossible to mock the requests, because the DNS is resolved before making the request. That is:

stub_request(:get, 'https://example.com/file.pdf').to_return(something)

This will fail because the request will be to whatever example.com happens to resolve to.

For testing, it would be useful to be able to globally disable the filtering, and go directly to the underlying request library.

Add option to disable unsafe IP protection

Use Case

I want to add a unit test in a project to catch the exception caused by #56. In it, I'm forced to make a remote request because ssrf_filter doesn't allow local IPs when caling SsrfFilter.get('http://api.localhost/X.json').

I am unsure of the utility of this, due to my lack of knowledge in the gem, thus I opened this issue for discussion. Note that my test does catch the exception with a local request when I manually disable the protection inside the gem.

Relevant Code

The protection is at lib/ssrf_filter/ssrf_filter.rb:129:

public_addresses = ip_addresses.reject(&method(:unsafe_ip_address?))

The option would disable this reject.

Allowing for Proxies

Hey Arkadiy!

We're looking to send requests through a proxy and it looks like if we want to continue using this library we'll need to add proxy support. Not sure if you had any insight on this or if there's a difficult approach. Thanks!

error: wrong version number (OpenSSL::SSL::SSLError)

When trying to download certain URLs:

SsrfFilter.get("https://images.cobot.me/uploads/plan/photo/aadce8899da1936c599bdcc49385c3b3/default_9622d00c-62ef-459c-9239-29d53ade828d.jpg")

The following error is displayed:

/Users/***/.asdf/installs/ruby/3.0.3/lib/ruby/gems/3.0.0/gems/net-protocol-0.1.3/lib/net/protocol.rb:46:in `connect_nonblock': SSL_connect returned=1 errno=0 state=error: wrong version number (OpenSSL::SSL::SSLError)

Not sure if this is the cause but the url above uses TLS 1.3.

I've narrowed it down to the fetch_once method. Here is its code in a simplified version:

url = "https://images.cobot.me/uploads/plan/photo/aadce8899da1936c599bdcc49385c3b3/default_9622d00c-62ef-459c-9239-29d53ade828d.jpg"
uri = URI.parse(url)

hostname = uri.hostname
ip_addresses = ::Resolv.getaddresses(hostname).map { |ip| ::IPAddr.new(ip) }

# culprit
uri.hostname = ip_addresses.sample.to_s

request = ::Net::HTTP::Get.new(uri)
request['host'] = "#{hostname}:443"

http_options = {}
http_options[:use_ssl] = (uri.scheme == 'https')

r = ::Net::HTTP.start(uri.hostname, uri.port, http_options) do |http|
  http.request(request)
end

Running the code above produces the error. Commenting out the line under # culprit causes the error to go away.

Weird timeout in specific domain

We're having some trouble timing out in the .get method from the gem in a specific domain: https://www.npr.org/. Every request from this one is timing out in the get method.
If you guys already got some issue like that with other domains it'd be great for us to know some kind of worakround.

version:
'ssrf_filter', '~> 1.0'

Breaking change introduced in 1.1.0

The pull request #51 changed the usage of the block on calling SsrfFilter methods. Previously it received a request object as an argument but now it receives a response object.
https://github.com/arkadiyt/ssrf_filter/pull/51/files#diff-95bca3ab492cb05975c93fdbbd14a18288350481b2f06307f2699fdd3fdb17d6L192-R184
Because of this, many CarrierWave users are facing issues.
carrierwaveuploader/carrierwave#2625

But this was a minor version bump. Since the project states SemVer compliance, it mustn't break backward compatibility of the public API.
So my proposal is to treat the block given as a request_proc, and add a new argument response_proc which receives a response object as chunked-processing goes by.
Another option will be making it a major version bump. If you consider that public API change crucial, this is the way to go.

I need @arkadiyt 's attention for this. Thanks.

`OpenSSL::SSL::SSLError` when accessing `google.com` domain

ssrf_filter version 1.0.8

When I try to SsrfFilter.get google related domains (google.com, *.googleapis.com) I get the following error.

SsrfFilter.get('https://www.google.com')
Traceback (most recent call last):
        2: from (irb):36
        1: from (irb):37:in `rescue in irb_binding'
OpenSSL::SSL::SSLError (SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate))

I can confirm that Net::HTTP.get('https://google.com') works if I don't use the ip address.

I encountered this error while using carrierwave with downloading google cloud storage hosted images.
From my investigation, this only happens to certificates created by google and I guess maybe it's something related to this patch? but not sure.
https://github.com/arkadiyt/ssrf_filter/blob/main/lib/ssrf_filter/patch/ssl_socket.rb

Drop support for old ruby versions

Thanks for an awesome gem! 💎

At some point it'll probably make sense to drop support for old ruby versions, to make this easier to maintain.

Just wanted to raise the issue, feel free to close this right away if you don't think the time is right to drop any of the older ones just yet.

Add PATCH verb

Any interest in adding additional verbs to the verb list? It would be nice to utilize the PATCH verb. If you agree, I would be happy to open a PR to contribute. Thanks!

Support OpenURI?

Ruby's OpenURI provides a simple way to open the contents of a URI as a file.

URI("https://somedomain/cool.jpg").open
# => <File:/var/folders/buncha_gibberish/open-uri_more_gibberish>

Ruby's own docs recommend using this over NetHTTP (which it wraps) for simple GET requests, and there's a jump in complexity migrating from OpenURI to NetHTTP responses. It would be fantastic to use this simple API while getting the safety benefits of this library. It would expand the scope, but in a limited way since OpenURI is intentionally restricted to basic GET requests.

Would something like this be a welcome addition?

# Calls `URI("https://somedomain/cool.jpg").open` after validating safety.
SsrfFilter.open("https://somedomain/cool.jpg")
# => <File:/var/folders/buncha_gibberish/open-uri_more_gibberish>

EOFError (end of file reached) for some sites

I'm getting the "EOFError (end of file reached)" for some sites, for example

Doing some research on the internet I found out this issue is caused by the user agent having the "ruby" word on the user agent.

Links:
https://stackoverflow.com/questions/50272370/rails-http-request-raises-eoferror-end-of-file-reached-after-some-time
jnunemaker/httparty#594

I tested the potential fix locally and it works well for me. I will create a PR with it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.