GithubHelp home page GithubHelp logo

safe_yaml's Introduction

SafeYAML

Build Status Gem Version

The SafeYAML gem provides an alternative implementation of YAML.load suitable for accepting user input in Ruby applications. Unlike Ruby's built-in implementation of YAML.load, SafeYAML's version will not expose apps to arbitrary code execution exploits (such as the ones discovered in Rails in early 2013).

If you encounter any issues with SafeYAML, check out the 'Common Issues' section below. If you don't see anything that addresses the problem you're experiencing, by all means, create an issue!

Installation

Add this line to your application's Gemfile:

gem "safe_yaml"

Configuration

If all you do is add SafeYAML to your project, then YAML.load will operate in "safe" mode, which means it won't deserialize arbitrary objects. However, it will issue a warning the first time you call it because you haven't explicitly specified whether you want safe or unsafe behavior by default. To specify this behavior (e.g., in a Rails initializer):

SafeYAML::OPTIONS[:default_mode] = :safe # or :unsafe

Another important option you might want to specify on startup is whether or not to allow symbols to be deserialized. The default setting is false, since symbols are not garbage collected in Ruby and so deserializing them from YAML may render your application vulnerable to a DOS (denial of service) attack. To allow symbol deserialization by default:

SafeYAML::OPTIONS[:deserialize_symbols] = true

For more information on these and other options, see the "Usage" section down below.

What is this gem for, exactly?

Suppose your application were to use a popular open source library which contained code like this:

class ClassBuilder
  def []=(key, value)
    @class ||= Class.new

    @class.class_eval <<-EOS
      def #{key}
        #{value}
      end
    EOS
  end

  def create
    @class.new
  end
end

Now, if you were to use YAML.load on user input anywhere in your application without the SafeYAML gem installed, an attacker who suspected you were using this library could send a request with a carefully-crafted YAML string to execute arbitrary code (yes, including system("unix command")) on your servers.

This simple example demonstrates the vulnerability:

yaml = <<-EOYAML
--- !ruby/hash:ClassBuilder
"foo; end; puts %(I'm in yr system!); def bar": "baz"
EOYAML
> YAML.load(yaml)
I'm in yr system!
=> #<ClassBuilder:0x007fdbbe2e25d8 @class=#<Class:0x007fdbbe2e2510>>

With SafeYAML, the same attacker would be thwarted:

> require "safe_yaml"
=> true
> YAML.load(yaml, :safe => true)
=> {"foo; end; puts %(I'm in yr system!); def bar"=>"baz"}

Usage

When you require the safe_yaml gem in your project, YAML.load is patched to accept one additional (optional) options parameter. This changes the method signature as follows:

  • for Syck and Psych prior to Ruby 1.9.3: YAML.load(yaml, options={})
  • for Psych in 1.9.3 and later: YAML.load(yaml, filename=nil, options={})

The most important option is the :safe option (default: true), which controls whether or not to deserialize arbitrary objects when parsing a YAML document. The other options, along with explanations, are as follows.

  • :deserialize_symbols (default: false): Controls whether or not YAML will deserialize symbols. It is probably best to only enable this option where necessary, e.g. to make trusted libraries work. Symbols receive special treatment in Ruby and are not garbage collected, which means deserializing them indiscriminately may render your site vulnerable to a DOS attack.

  • :whitelisted_tags: Accepts an array of YAML tags that designate trusted types, e.g., ones that can be deserialized without worrying about any resulting security vulnerabilities. When any of the given tags are encountered in a YAML document, the associated data will be parsed by the underlying YAML engine (Syck or Psych) for the version of Ruby you are using. See the "Whitelisting Trusted Types" section below for more information.

  • :custom_initializers: Similar to the :whitelisted_tags option, but allows you to provide your own initializers for specified tags rather than using Syck or Psyck. Accepts a hash with string tags for keys and lambdas for values.

  • :raise_on_unknown_tag (default: false): Represents the highest possible level of paranoia. If the YAML engine encounters any tag other than ones that are automatically trusted by SafeYAML or that you've explicitly whitelisted, it will raise an exception. This may be a good choice if you expect to always be dealing with perfectly safe YAML and want your application to fail loudly upon encountering questionable data.

All of the above options can be set at the global level via SafeYAML::OPTIONS. You can also set each one individually per call to YAML.load; an option explicitly passed to load will take precedence over an option specified globally.

What if I don't want to patch YAML?

Excellent question! You can also get the methods SafeYAML.load and SafeYAML.load_file without touching the YAML module at all like this:

require "safe_yaml/load" # instead of require "safe_yaml"

This way, you can use SafeYAML.load to parse YAML that you don't trust, without affecting the rest of an application (if you're developing a library, for example).

Supported Types

The way that SafeYAML works is by restricting the kinds of objects that can be deserialized via YAML.load. More specifically, only the following types of objects can be deserialized by default:

  • Hashes
  • Arrays
  • Strings
  • Numbers
  • Dates
  • Times
  • Booleans
  • Nils

Again, deserialization of symbols can be enabled globally by setting SafeYAML::OPTIONS[:deserialize_symbols] = true, or in a specific call to YAML.load([some yaml], :deserialize_symbols => true).

Whitelisting Trusted Types

SafeYAML supports whitelisting certain YAML tags for trusted types. This is handy when your application uses YAML to serialize and deserialize certain types not listed above, which you know to be free of any deserialization-related vulnerabilities.

The easiest way to whitelist types is by calling SafeYAML.whitelist!, which can accept a variable number of safe types, e.g.:

SafeYAML.whitelist!(Foo, Bar)

You can also whitelist YAML tags via the :whitelisted_tags option:

# Using Syck
SafeYAML::OPTIONS[:whitelisted_tags] = ["tag:ruby.yaml.org,2002:object:OpenStruct"]

# Using Psych
SafeYAML::OPTIONS[:whitelisted_tags] = ["!ruby/object:OpenStruct"]

And in case you were wondering: no, this feature will not allow would-be attackers to embed untrusted types within trusted types:

yaml = <<-EOYAML
--- !ruby/object:OpenStruct 
table: 
  :backdoor: !ruby/hash:ClassBuilder 
    "foo; end; puts %(I'm in yr system!); def bar": "baz"
EOYAML
> YAML.safe_load(yaml)
=> #<OpenStruct :backdoor={"foo; end; puts %(I'm in yr system!); def bar"=>"baz"}>

Known Issues

If you add SafeYAML to your project and start seeing any errors about missing keys, or you notice mysterious strings that look like ":foo" (i.e., start with a colon), it's likely you're seeing errors from symbols being saved in YAML format. If you are able to modify the offending code, you might want to consider changing your YAML content to use plain vanilla strings instead of symbols. If not, you may need to set the :deserialize_symbols option to true, either in calls to YAML.load or---as a last resort---globally, with SafeYAML::OPTIONS[:deserialize_symbols].

Also be aware that some Ruby libraries, particularly those requiring inter-process communication, leverage YAML's object deserialization functionality and therefore may break or otherwise be impacted by SafeYAML. The following list includes known instances of SafeYAML's interaction with other Ruby gems:

  • ActiveRecord: uses YAML to control serialization of model objects using the serialize class method. If you find that accessing serialized properties on your ActiveRecord models is causing errors, chances are you may need to:
    1. set the :deserialize_symbols option to true,
    2. whitelist some of the types in your serialized data via SafeYAML.whitelist! or the :whitelisted_tags option, or
    3. both
  • delayed_job: Uses YAML to serialize the objects on which delayed methods are invoked (with delay). The safest solution in this case is to use SafeYAML.whitelist! to whitelist the types you need to serialize.
  • Guard: Uses YAML as a serialization format for notifications. The data serialized uses symbolic keys, so setting SafeYAML::OPTIONS[:deserialize_symbols] = true is necessary to allow Guard to work.
  • sidekiq: Uses a YAML configiuration file with symbolic keys, so setting SafeYAML::OPTIONS[:deserialize_symbols] = true should allow it to work.

The above list will grow over time, as more issues are discovered.

Versioning

SafeYAML will follow semantic versioning so any updates to the first major version will maintain backwards compatability. So expect primarily bug fixes and feature enhancements (if anything!) from here on out... unless it makes sense to break the interface at some point and introduce a version 2.0, which I honestly think is unlikely.

Requirements

SafeYAML requires Ruby 1.8.7 or newer and works with both Syck and Psych.

If you are using a version of Ruby where Psych is the default YAML engine (e.g., 1.9.3) but you want to use Syck, be sure to set YAML::ENGINE.yamler = "syck" before requiring the safe_yaml gem.

safe_yaml's People

Contributors

apepper avatar blackwinter avatar btoews avatar dtao avatar eagletmt avatar elifoster avatar gjtorikian avatar jackbot avatar jsl avatar kachick avatar ntalbott avatar parndt avatar rolandu avatar sarahhodne avatar steved avatar til avatar yaauie avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

safe_yaml's Issues

Licensing?

Hi there!

I'd like to use this Gem in a project of mine, but I'm a little paranoid about licensing, and the project doesn't explicitly state which license (if any) it is released under. Could you give me guidance on this, or state what the terms of use are in the documentation?

Apologies if I've just read past this!

Calum.

PsychShield

have you seen https://github.com/rapid7/psych_shield ?

By default, Psych Shield allows the following types of objects:

Hash Array String Range
Numeric Fixnum Integer Bignum Float Rational Complex
Time DateTime
NilClass TrueClass FalseClass
To enable additional classes, add the stringified form using the "add" method:

PsychShield.add('MyClass::IsAwesome::And::Safe')
To disable all classes (even the defaults), use the clear method:

PsychShield.clear

doc: add aws-sdk to known issues

I just spent two hours wondering why the bloody thing didn't work at all :)
if you're using aws-sdk gem, you need to set

SafeYAML::OPTIONS[:deserialize_symbols] = true

keywords: aws-sdk gem, define_client_methods

safe_yaml misparses unquoted strings that look like octals

The same thing happens with Ruby 1.8.7, 1.9.3 or 2.0.0.

$ irb -ryaml -rrubygems
irb(main):001:0> YAML.load('08')
=> "08"
irb(main):002:0> require 'safe_yaml'
=> true
irb(main):003:0> YAML.load('08')
Called 'load' without the :safe option -- defaulting to safe mode.
=> 8

invalid value for Float(): "."

I tried to install the last version of tolk which uses safe_yaml 0.8.5 and I got the error below. After downgrading to 0.8.4 everything worked fine.

Extract from this issue : tolk/tolk#43

Backtrace:

rake aborted!
invalid value for Float(): "."
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/transform/to_float.rb:22:in Float' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/transform/to_float.rb:22:intransform?'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/transform.rb:19:in block in to_guessed_type' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/transform.rb:18:ineach'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/transform.rb:18:in to_guessed_type' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/transform.rb:34:into_proper_type'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:62:in resolve_scalar' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:22:inresolve_node'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:46:in block in resolve_map' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:45:ineach'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:45:in resolve_map' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:18:inresolve_node'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:46:in block in resolve_map' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:45:ineach'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:45:in resolve_map' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:18:inresolve_node'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:46:in block in resolve_map' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:45:ineach'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:45:in resolve_map' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:18:inresolve_node'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:46:in block in resolve_map' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:45:ineach'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:45:in resolve_map' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:18:inresolve_node'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:46:in block in resolve_map' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:45:ineach'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:45:in resolve_map' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:18:inresolve_node'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:58:in block in resolve_seq' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:58:ineach'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:58:in inject' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:58:inresolve_seq'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/psych_resolver.rb:17:in resolve_root' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml/resolver.rb:16:inresolve_node'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml.rb:84:in safe_load' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml.rb:88:inblock in safe_load_file'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml.rb:88:in open' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml.rb:88:insafe_load_file'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/safe_yaml-0.8.5/lib/safe_yaml.rb:71:in load_file_with_options' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/i18n-0.6.1/lib/i18n/backend/base.rb:172:inload_yml'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/i18n-0.6.1/lib/i18n/backend/base.rb:157:in load_file' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/i18n-0.6.1/lib/i18n/backend/base.rb:15:inblock in load_translations'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/i18n-0.6.1/lib/i18n/backend/base.rb:15:in each' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/i18n-0.6.1/lib/i18n/backend/base.rb:15:inload_translations'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/i18n-0.6.1/lib/i18n/backend/simple.rb:57:in init_translations' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/tolk-1.3.7/lib/tolk/sync.rb:13:inload_translations'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/tolk-1.3.7/lib/tolk/sync.rb:9:in sync!' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/tolk-1.3.7/lib/tasks/tolk_tasks.rake:19:inblock (2 levels) in <top (required)>'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/gems/tolk-1.3.7/lib/tasks/tolk_tasks.rake:13:in block (2 levels) in <top (required)>' /home/skelz0r/.rvm/gems/ruby-1.9.3-p362/bin/ruby_noexec_wrapper:14:ineval'
/home/skelz0r/.rvm/gems/ruby-1.9.3-p362/bin/ruby_noexec_wrapper:14:in `

'

Weird error with date_select

To reproduce

rails new date_select
rails generate scaffold Post title published_at:datetime

add safe_yaml to the gemfile

Error select_:year undefined

Support Syck with 1.9.2+ as well

Ruby 1.9.2 and above default to Psych, but you can force Syck usage like this:

YAML::ENGINE.yamler = 'syck' if defined?(YAML::ENGINE)

Perhaps safe_yaml should look at YAML::ENGINE.yamler as well as ruby version when deciding which handler to include?

(Another problem is that YAML::ENGINE.yamler may be changed after safe_yaml is loaded, but that can be worked around in the app with some careful ordering.)

NoMethodError: undefined method `tagged_classes' for Psych:Module on Ruby 2.2.0-preview1

irb(main):001:0> RUBY_VERSION
=> "2.2.0"
irb(main):002:0> require 'safe_yaml'
NoMethodError: undefined method `tagged_classes' for Psych:Module
    from /Users/sferik/.rbenv/versions/2.2.0-preview1/lib/ruby/gems/2.2.0/gems/safe_yaml-1.0.3/lib/safe_yaml/load.rb:43:in `<module:SafeYAML>'
    from /Users/sferik/.rbenv/versions/2.2.0-preview1/lib/ruby/gems/2.2.0/gems/safe_yaml-1.0.3/lib/safe_yaml/load.rb:26:in `<top (required)>'
    from /Users/sferik/.rbenv/versions/2.2.0-preview1/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from /Users/sferik/.rbenv/versions/2.2.0-preview1/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from /Users/sferik/.rbenv/versions/2.2.0-preview1/lib/ruby/gems/2.2.0/gems/safe_yaml-1.0.3/lib/safe_yaml.rb:1:in `<top (required)>'
    from /Users/sferik/.rbenv/versions/2.2.0-preview1/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:128:in `require'
    from /Users/sferik/.rbenv/versions/2.2.0-preview1/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:128:in `rescue in require'
    from /Users/sferik/.rbenv/versions/2.2.0-preview1/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:39:in `require'
    from (irb):2
    from /Users/sferik/.rbenv/versions/2.2.0-preview1/bin/irb:11:in `<main>'

I do not get this error on Ruby 2.1.2, 2.0.0, or 1.9.3.

Is safe_yaml safe from CVE-2014-2525?

Does safe_yaml, by default, fall prey to the buffer overflow exploit described (and fixed) below?

Looks like it's not about "smart" parsing, but just about basic string parsing -- yet it also relates to URIs specifically, somehow, and I don't know if safe_yaml makes that distinction?

Announcement: https://www.ruby-lang.org/en/news/2014/03/29/heap-overflow-in-yaml-uri-escape-parsing-cve-2014-2525/
Technical description: http://www.ocert.org/advisories/ocert-2014-003.html

It's fixed in libyaml 1.6.

/cc @drinks

Side-effect on guard (?)

When I add gem 'safe_yaml' to my project and launch guard, guard-rspec launches my tests then report the following error:

Finished in 3.52 seconds 253 examples, 0 failures, 5 pending
ERROR: Error sending notification with : can't convert nil into Hash

This does not not happen when launching with rake spec.

I suspect a guard component is doing something with yaml and notifications; I've not yet been able to track this down but thought I would report it anyway.

Issues with Rails logger?

I've not (yet) shown this to be caused by safe_yaml, but after adding safe_yaml, with Ruby 1.9.3-p327, the Rails log is wonky: It no longer properly shows the calls coming in through Rack etc. It shows some - that frankly do not make sense - and then nothing more (based on the timestamps).

Any ideas?

No support for load_stream

As far as I understand, safe_yaml does not support the load_stream method (or any form of YAML steam loading).

require "safe_yaml"

# File.open "bla.yaml" do |f|
#   YAML.load_stream(f, :safe=>truew)
# end

v = <<EOF
--- !ruby/hash:ClassBuilder
"foo; end; puts %(I'm in yr system!); def bar": "baz"

---
EOF

puts "Load\n\n"
puts(YAML.load(v))

puts "\nLoad stream\n\n"
puts(YAML.load_stream(v))

Gives:

Load

Called 'load' without the :safe option -- defaulting to safe mode.
You can avoid this warning in the future by setting the SafeYAML::OPTIONS[:default_mode] option (to :safe or :unsafe).
{"foo; end; puts %(I'm in yr system!); def bar"=>"baz"}

Load stream

/usr/lib/ruby/2.1.0/psych/class_loader.rb:53:in `path2class': undefined class/module ClassBuilder (ArgumentError)
        from /usr/lib/ruby/2.1.0/psych/class_loader.rb:53:in `resolve'
        from /usr/lib/ruby/2.1.0/psych/class_loader.rb:45:in `find'
        from /usr/lib/ruby/2.1.0/psych/class_loader.rb:27:in `load'
        from /usr/lib/ruby/2.1.0/psych/visitors/to_ruby.rb:360:in `resolve_class'
        from /usr/lib/ruby/2.1.0/psych/visitors/to_ruby.rb:261:in `visit_Psych_Nodes_Mapping'
        from /usr/lib/ruby/2.1.0/psych/visitors/visitor.rb:15:in `visit'
        from /usr/lib/ruby/2.1.0/psych/visitors/visitor.rb:5:in `accept'
        from /usr/lib/ruby/2.1.0/psych/visitors/to_ruby.rb:31:in `accept'
        from /usr/lib/ruby/2.1.0/psych/visitors/to_ruby.rb:276:in `visit_Psych_Nodes_Document'
        from /usr/lib/ruby/2.1.0/psych/visitors/visitor.rb:15:in `visit'
        from /usr/lib/ruby/2.1.0/psych/visitors/visitor.rb:5:in `accept'
        from /usr/lib/ruby/2.1.0/psych/visitors/to_ruby.rb:31:in `accept'
        from /usr/lib/ruby/2.1.0/psych/nodes/node.rb:37:in `to_ruby'
        from /usr/lib/ruby/2.1.0/psych.rb:456:in `block in load_stream'
        from /usr/lib/ruby/2.1.0/psych.rb:456:in `map'
        from /usr/lib/ruby/2.1.0/psych.rb:456:in `load_stream'
        from test.rb:19:in `'

uninitialized constant Psych::Nodes (NameError)

using windows 8 have this uninitialized constant problem with safe_yaml => safe_yaml-1.0.4/lib/safe_yaml/psych_resolver.rb:4:in <class:PsychResolver>' *unless* I run the command with bundle exec before the command. I've triedgem update` and nothing changed.

Any ideas?

Multi line + tab

Hi

jekyll/jekyll#2416 points me to here.

If I use YAML syntax like

name: |
     1 bla bla bla
     1 bla bla bla

The first line in my jekyll output is indented where I include it, but the subsequent ones are not. I'm using the data files feature of jekyll, to include the content of the yaml in my markdown.

Any thoughts?

SafeYAML::Parse::Date discards timezone

> irb
1.9.3-p392 :001 > require 'safe_yaml'
 => true
1.9.3-p392 :002 > t = YAML.safe_load("date: 2012-12-01 10:33:45 +11:00")
 => {"date"=>2012-11-30 23:33:45 UTC}
1.9.3-p392 :003 > t = YAML.load("date: 2012-12-01 10:33:45 +11:00", :safe => false)
 => {"date"=>2012-12-01 10:33:45 +1100}

In lib/safe_yaml/parse/date.rb, L25, the value passed back (an instance of Time) is created using Time.utc, discarding all timezone information encoded in the YAML input. I cannot see any obvious reason or advantage to this; it also appears to be a POLA violation.

Bug in rubinius when handling map nodes

The following bug rubinius/rubinius#2141 affects safe_yaml 0.6.1 and causes problems when handling a map node under rbx 2.0.0.rc1.

One alternative (other than waiting for it to be fixed upstream) would be to change the case statement in SafeYAML::SyckResolver#resolve_node to be something like this:

def resolve_node(node)
  case node.value
  when Hash   then resolve_map(node)
  when Array  then resolve_seq(node)
  when String then resolve_scalar(node)
  else
    raise "Don't know how to resolve a '#{node.kind}' node!"
  end
end

Let me know and I could make a patch for this.

Release version 0.6 gem

I'd like depend on safe_yaml in one of my gems but I can't use version 0.5.2. Now that RubyGems.org is back online, please push a new version of the gem that includes the latest changes. Thanks!

ArgumentError on ruby-1.9.3-p125

Running ruby-1.9.3-p125 under rvm I get this error:

/usr/share/ruby-rvm/gems/ruby-1.9.3-p125/gems/safe_yaml-0.6.3/lib/safe_yaml.rb:45:in `parse': wrong number of arguments(2 for 1) (ArgumentError)

Looks like there's a branch in there checking for ruby version and it's taking the wrong one for that package of 1.9.3?

Support for custom deserializer?

Here's the situation - for our app, safe_yaml works for everything except for a massive number of serialized HashWithIndifferentAccess. With safe_yaml, we have to choose between:

  • Inspecting everywhere the object might be deserialized, and instead calling object.with_indifferent_access. (And potentially descending into the object to transform nested objects too... ick)
  • Auditing anywhere the object is used and verifying that we're string keys instead of symbols (ick)
  • Add support for deserializing HashWithIndifferentAccess within safe_yaml. This is what I've been doing while evaluating safe_yaml, and it works really well for our app. However, it's not a very clean solution:

zwily@036c2cf

Lots of people have been talking about doing whitelists/schemas/whatever to handle situations like this in Psych, but that support still seems like a ways off. It seems like to me a lightweight way to handle a simple use case like this would be to be able to specify in the SafeYAML configuration a mapping of tags -> blocks. So I could do:

SafeYAML::CONFIG[:custom_initializers] = {
    "tag:yaml.org,2002:map:HashWithIndifferentAccess" => Proc.new {
        HashWithIndifferentAccess.new
    }
}

Then when instantiating objects, if the node has one of the supplied tags, it would call the supplied Proc instead of using [] or {} (for example).

I can code up a pull request but I wanted to see if this seemed like a good approach first. If it's not appropriate for the library, then we'll probably continue using my forked version.

Some floating point numbers parsed incorrectly

See the spec for more info.

Scientific notation

YAML.load('123.45e+6') # => 123450000.0
require 'safe_yaml'
YAML.load('123.45e+6') # => "123.45e+6"

Infinity / Negative infinity

YAML.load('.inf') # => Infinity
require 'safe_yaml'
YAML.load('.inf') # => ".inf"

NaN

YAML.load('.NaN') # => NaN
require 'safe_yaml'
YAML.load('.NaN') # => ".NaN"

Base 60

YAML.load('190:20:30.15') # => 685230.15
require 'safe_yaml'
YAML.load('190:20:30.15') # => "190:20:30.15"

With underscores

YAML.load('685_230.15') # => 685230.15
require 'safe_yaml'
YAML.load('685_230.15') # => "685_230.15"

"Unknown YAML Tag '!'" error when serialized strings are prefixed with bang

When serializing a string that is marked with ActiveSupport's html_safe and has trailing whitespace, ruby prefixes the quoted string with a bang. In my understanding this is not necessary, but still valid yaml, so I would expect safe_yaml to load it without raising an unknown tag exception.

Ruby script to reproduce:

require 'active_support/core_ext'
require 'safe_yaml'

SafeYAML::OPTIONS[:whitelisted_tags] = ['!ruby/object:Hash']
SafeYAML::OPTIONS[:default_mode] = :unsafe
SafeYAML::OPTIONS[:raise_on_unknown_tag] = true

s = 'foo '.html_safe

puts s.to_yaml

puts 'Unsafe:'
puts YAML.load(s.to_yaml)

puts 'Safe:'
puts YAML.safe_load(s.to_yaml)

Output:

--- ! 'foo '
Unsafe:
foo 
Safe:
/home/tils/.rbenv/versions/2.0.0-p0/lib/ruby/gems/2.0.0/gems/safe_yaml-0.9.0/lib/safe_yaml.rb:38:in `tag_safety_check!': Unknown YAML tag '!' (RuntimeError)
    from /home/tils/.rbenv/versions/2.0.0-p0/lib/ruby/gems/2.0.0/gems/safe_yaml-0.9.0/lib/safe_yaml/resolver.rb:64:in `get_and_check_node_tag'
    from /home/tils/.rbenv/versions/2.0.0-p0/lib/ruby/gems/2.0.0/gems/safe_yaml-0.9.0/lib/safe_yaml/resolver.rb:59:in `resolve_scalar'
    from /home/tils/.rbenv/versions/2.0.0-p0/lib/ruby/gems/2.0.0/gems/safe_yaml-0.9.0/lib/safe_yaml/resolver.rb:22:in `resolve_node'
    from /home/tils/.rbenv/versions/2.0.0-p0/lib/ruby/gems/2.0.0/gems/safe_yaml-0.9.0/lib/safe_yaml/resolver.rb:55:in `block in resolve_seq'
    from /home/tils/.rbenv/versions/2.0.0-p0/lib/ruby/gems/2.0.0/gems/safe_yaml-0.9.0/lib/safe_yaml/resolver.rb:55:in `each'
    from /home/tils/.rbenv/versions/2.0.0-p0/lib/ruby/gems/2.0.0/gems/safe_yaml-0.9.0/lib/safe_yaml/resolver.rb:55:in `inject'
    from /home/tils/.rbenv/versions/2.0.0-p0/lib/ruby/gems/2.0.0/gems/safe_yaml-0.9.0/lib/safe_yaml/resolver.rb:55:in `resolve_seq'
    from /home/tils/.rbenv/versions/2.0.0-p0/lib/ruby/gems/2.0.0/gems/safe_yaml-0.9.0/lib/safe_yaml/psych_resolver.rb:17:in `resolve_root'
    from /home/tils/.rbenv/versions/2.0.0-p0/lib/ruby/gems/2.0.0/gems/safe_yaml-0.9.0/lib/safe_yaml/resolver.rb:16:in `resolve_node'
    from /home/tils/.rbenv/versions/2.0.0-p0/lib/ruby/gems/2.0.0/gems/safe_yaml-0.9.0/lib/safe_yaml.rb:168:in `safe_load'
    from safe_yaml_repro.rb:16:in `<main>'

It seems that this can be worked around by adding '!' to the list of whitelisted tags. Would that be 1) safe? 2) a correct solution? Or should it be considered a bug in safe_yaml?

Please stop with the security theater and lies.

Not all YAML's at 1.4 are vulnerable and you clearly leave out the fact that several Linux distros will backport security fixes because of the clear rule against feature updates on long term releases. I don't appreciate being told that my libyaml is out of date and insecure when the truth is it isn't: http://changelogs.ubuntu.com/changelogs/pool/main/liby/libyaml/libyaml_0.1.4-2ubuntu0.12.04.3/changelog

If I didn't Linux for a living and know these things I would be freaked out, so now you are just creating security theater and causing problems that don't need to be caused, please either create an algorithm so you can properly detect the vulnerability or end the theatrics.

Unnecessary check for libyaml >= 0.1.6 on Ubuntu / Mint based systems

When attempting to run "octopress" (3.0.0-rc7), which uses "jekyll" (1.5.1), which uses SafeYaml (1.0.2), I get the a warning, and nothing further proceeding. This stops me from actually running the program. All of these packages were installed via "gem".

The warning text is:

  SafeYAML Warning
  ----------------

  You appear to have an outdated version of libyaml (0.1.4) installed on your system.

  Prior to 0.1.6, libyaml is vulnerable to a heap overflow exploit from malicious YAML payloads.

  For more info, see:
  https://www.ruby-lang.org/en/news/2014/03/29/heap-overflow-in-yaml-uri-escape-parsing-cve-2014-2525/

  The easiest thing to do right now is probably to update Psych to the latest version and enable
  the 'bundled-libyaml' option, which will install a vendored libyaml with the vulnerability patched:

  gem install psych -- --enable-bundled-libyaml

Unfortunately, in Ubuntu (and Mint, which uses the Ubuntu packages), CVE-2014-2525 was fixed without changing the version from 0.1.4 (see http://people.canonical.com/~ubuntu-security/cve/2014/CVE-2014-2525.html)

I cannot find any method of disabling the check, even though it is unnecessary on my patched system. I have also tried running the "gem install psych -- --enable-bundled-libyaml" command to no avail.

Is it possible to disable the check, say via an environment variable? Without this, I cannot use any programs that use SafeYAML.

Thanks,

Ian.

"Inheritance" still broken

Please try the following patch and observe the behavior:

diff --git a/spec/safe_yaml_spec.rb b/spec/safe_yaml_spec.rb
index 434efda..e37ddd6 100644
--- a/spec/safe_yaml_spec.rb
+++ b/spec/safe_yaml_spec.rb
@@ -169,10 +169,13 @@ describe YAML do
             foo: foo
             bar: bar
             baz: baz
-          custom:
+          custom: &custom
             <<: *defaults
             bar: custom_bar
             baz: custom_baz
+          breakage: &breakage
+            <<: *custom
+

Slow and very verbose

Hi @dtao , I commited safe_yaml to the gem tolk.
But's now it's loaded on any project loading the tolk gem, and it's very verbose , with no easy possibility to stop it displaying all warnings.

I have no measure but it also seems to be very slow (loading i18n files for example)

Do not patch YAML by default

Please, consider not to patch YAML.load methods by default. I think there can be cases where safe_yaml is installed as dependency for some other gem. This gem will use safe methods and there is no need to patch YAML in whole project.

I've faced the problem where safe_yaml fails to load document with TypeError: can't convert String into Hash, but Psych loads it without errors.

I've fixed it with SafeYAML::OPTIONS[:default_mode] = :unsafe. But I think it's not good. SafeYAML.patch! method would be better option.

WDYT?

Permissions incorrect in 0.9.4

The lib/safe_yaml/transform/to_symbol.rb has its permissions set to 0640 in v0.9.4, this causes issues if the gem is installed as root.

-rw-r----- 1 root wheel 402B Jul 23 15:08 to_symbol.rb

Nils objects are not deserialized

Using ruby 1.9.3p448 & SafeYAML 0.9.5 Nils are not deserialized properly.

1.9.3p448 :001 > SafeYAML::OPTIONS[:default_mode] = :safe
 => :safe 
1.9.3p448 :002 > YAML.load("--- ~")
 => false 

As a workaround we use the whitelist option, but according to the documentation Nils should be deserialized by default.

1.9.3p448 :003 > SafeYAML.whitelist!(NilClass)
 => [NilClass] 
1.9.3p448 :004 > YAML.load("--- ~")
 => nil 

Newbie Problem: unable to require safe_yaml: unitialized constant

Hi,

I'm completely new to ruby and wanted to try the the latest code of "jekyll" (https://github.com/mojombo/jekyll) and run it "myself" (i.e. not installing from gem), but I run into problems with safe_yaml. Maybe it's just a newbie problem I am experiencing?

When I run

ruby jekyll.rb

I get:

/Library/Ruby/Gems/1.8/gems/safe_yaml-0.9.0/lib/safe_yaml.rb:102: uninitialized constant > SafeYAML::Set (NameError)
from /Library/Ruby/Site/1.8/rubygems/core_ext/kernel_require.rb:110:in gem_original_require' from /Library/Ruby/Site/1.8/rubygems/core_ext/kernel_require.rb:110:inrequire'
from jekyll.rb:21

jekyll.rb:21 says

require 'safe_yaml'

That's why I came here. Is there something wrong with safe_yaml or is there something wrong with my system?

Thanks for any help in advance!

ruby -v
ruby 1.8.7 (2012-02-08 patchlevel 358) [universal-darwin12.0]

safe_yaml vs Psych safe_load

On ruby 2.2.2

2.2.2 :001 > require 'yaml'
2.2.2 :002 > YAML.safe_load("foo: !ruby/regexp '/bar/'\n",[Regexp])
 => ("foo"=>/bar/)

if i include safe_yaml:

2.2.2 :001 > require 'yaml'
2.2.2 :002 > require 'safe_yaml'
2.2.2 :003 > YAML.safe_load("foo: !ruby/regexp '/bar/'\n",[Regexp])
 => {"foo"=>"/bar/"} 

Note that safe_yaml does not create a regex but a string.

Support load file for files with `---` starting content

If I have a .txt file like so:


---
title: Blah
key: value

---

Hey, here are some words!

Calling data = YAML.load_file(file) will correctly get me the YAML-as-a-hash in data. However, after requiring this gem, the following error appears:

safe_yaml-0.9.5/lib/safe_yaml/psych_handler.rb:48:in `add_to_current_structure': Don't know how to add to a NilClass! (RuntimeError)

The workaround for now is to do something like

if File.read(file) =~ /\A(---\s*\n(.*?\n?)^---\s*$\n?)(.*)/m
    data = YAML.safe_load($2)
end

Be more descriptive about multiple colons

Jekyll uses SafeYAML to parse all YAML we read in from a user's filesystem. We use it for titles, for dates โ€“ for every piece of metadata about a post or page we can. As such, we rely heavily on SafeYAML's behaviour regarding well-formed and malformed YAML.

I received an issue today by the fantastic @gjtorikian who mentioned that a friend of his had difficulty when a value he wrote contained a colon itself, thus causing what should be a String value to become a Hash. This caused strange behaviour ( jekyll/jekyll#2030 ) and lots of confusion.

Is there a way we can check for colons on non-newline separated lines, such that:

> SafeYAML.load("---\ndescription: we're working with great developers at: large, small, medium, massive, miniscule, and made-out-of-jam companies")
#=> {"description" => "we're working with great developers at: large, small, medium, massive, miniscule, and made-out-of-jam companies"}
> SafeYAML.load("---\ndescription: we're working with great developers\nat: lots of places")
#=> {"description" => "we're working with great developers", "at" => "lots of places"}

The newline before the term that ends with the : signifies that it's a new key.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.