[Summer 2021] Create models to wtforms about pyre-check

[abishekvashok]

This is not in typeshed, without stubs it's not working either, do we really need this as to add stubs @r0rshark

[abishekvashok]

Hey @r0rshark so I tried setting this up with the suggestion and reveal_taint works now, but the problem is the conditional way in which this library creates attributes:
This is an error I am getting while running Pysa:

ƛ  app.py:18:15-18:33 -> Undefined attribute [16]: `wtforms.fields.core.StringField` has no attribute `data`.

The code works in the python interpreter, and I guess this is due to the way in which the attribute is defined:

class StringField(Field):
    """
    This field is the base for most of the more complicated fields, and
    represents an ``<input type="text">``.
    """
    widget = widgets.TextInput()

    def process_formdata(self, valuelist):
        if valuelist:
            self.data = valuelist[0]
        elif self.data is None:
            self.data = ''

    def _value(self):
        return text_type(self.data) if self.data is not None else ''

https://wtforms.readthedocs.io/en/2.3.x/_modules/wtforms/fields/core/#StringField

Field that it inherits from doesn't have data either, so it's getting tough to model this library. I tried DSL as well, just not triggering or registering the data attribute :/

[abishekvashok]

@r0rshark what are your thoughts on defining all attributes as taint sources (ie, all the field classes such as a text field or string field as taint sources itself.) It would be weird to taint entire classes but seems like the only way we could add support. What are your thoughts on this?

[r0rshark]

o I tried setting this up with the suggestion and reveal_taint works now, but the problem is the conditional way in which this library creates attributes:
This is an error I am getting while running Pysa:

@abishekvashok Nice work! Can you please share what pyre query outputs for

pyre query "attributes(wtforms.fields.core.StringField)"
pyre query "defines(wtforms.fields.core.StringField)"
pyre query "attributes(wtforms.fields.core.Field)"
pyre query "defines(wtforms.fields.core.Field)"

Just to test. would taint the entire class work? (https://pyre-check.org/docs/pysa-basics#sources)

[abishekvashok]

the attributes of StringField:

{
  "response": {
    "attributes": [
      {
        "name": "_value",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.StringField._value)[[Named(self, wtforms.fields.core.StringField)], typing.Any], wtforms.fields.core.StringField]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "process_formdata",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.StringField.process_formdata)[[Named(self, wtforms.fields.core.StringField), Named(valuelist, unknown)], typing.Any], wtforms.fields.core.StringField]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "widget",
        "annotation": "wtforms.widgets.core.TextInput",
        "kind": "regular",
        "final": false
      }
    ]
  }
}

The functions in StringField:

```json { "response": [ { "name": "wtforms.fields.core.StringField._value", "parameters": [ { "name": "self", "annotation": null } ], "return_annotation": null }, { "name": "wtforms.fields.core.StringField.process_formdata", "parameters": [ { "name": "self", "annotation": null }, { "name": "valuelist", "annotation": null } ], "return_annotation": null } ] } ```

The attributes of Field:

{
  "response": {
    "attributes": [
      {
        "name": "__call__",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.__call__)[[Named(self, wtforms.fields.core.Field), Keywords(unknown)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "__html__",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.__html__)[[Named(self, wtforms.fields.core.Field)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "__init__",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.__init__)[[Named(self, wtforms.fields.core.Field), Named(label, unknown, default), Named(validators, unknown, default), Named(filters, unknown, default), Named(description, unknown, default), Named(id, unknown, default), Named(default, unknown, default), Named(widget, unknown, default), Named(render_kw, unknown, default), Named(_form, unknown, default), Named(_name, unknown, default), Named(_prefix, unknown, default), Named(_translations, unknown, default), Named(_meta, unknown, default)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "__new__",
        "annotation": "typing.Callable(wtforms.fields.core.Field.__new__)[[Named(cls, typing.Type[wtforms.fields.core.Field]), Variable(unknown), Keywords(unknown)], typing.Any]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "__str__",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.__str__)[[Named(self, wtforms.fields.core.Field)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "__unicode__",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.__unicode__)[[Named(self, wtforms.fields.core.Field)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "_formfield",
        "annotation": "bool",
        "kind": "regular",
        "final": false
      },
      {
        "name": "_run_validation_chain",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field._run_validation_chain)[[Named(self, wtforms.fields.core.Field), Named(form, unknown), Named(validators, unknown)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "_translations",
        "annotation": "wtforms.i18n.DummyTranslations",
        "kind": "regular",
        "final": false
      },
      {
        "name": "check_validators",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.check_validators)[[Named(cls, typing.Type[wtforms.fields.core.Field]), Named(validators, unknown)], typing.Any], typing.Type[wtforms.fields.core.Field]]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "default",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      },
      {
        "name": "description",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      },
      {
        "name": "do_not_call_in_templates",
        "annotation": "bool",
        "kind": "regular",
        "final": false
      },
      {
        "name": "errors",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      },
      {
        "name": "filters",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      },
      {
        "name": "flags",
        "annotation": "wtforms.fields.core.Flags",
        "kind": "regular",
        "final": false
      },
      {
        "name": "gettext",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.gettext)[[Named(self, wtforms.fields.core.Field), Named(string, unknown)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "id",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      },
      {
        "name": "label",
        "annotation": "wtforms.fields.core.Label",
        "kind": "regular",
        "final": false
      },
      {
        "name": "meta",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      },
      {
        "name": "name",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      },
      {
        "name": "ngettext",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.ngettext)[[Named(self, wtforms.fields.core.Field), Named(singular, unknown), Named(plural, unknown), Named(n, unknown)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "populate_obj",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.populate_obj)[[Named(self, wtforms.fields.core.Field), Named(obj, unknown), Named(name, unknown)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "post_validate",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.post_validate)[[Named(self, wtforms.fields.core.Field), Named(form, unknown), Named(validation_stopped, unknown)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "pre_validate",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.pre_validate)[[Named(self, wtforms.fields.core.Field), Named(form, unknown)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "process",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.process)[[Named(self, wtforms.fields.core.Field), Named(formdata, unknown), Named(data, unknown, default)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "process_data",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.process_data)[[Named(self, wtforms.fields.core.Field), Named(value, unknown)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "process_errors",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      },
      {
        "name": "process_formdata",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.process_formdata)[[Named(self, wtforms.fields.core.Field), Named(valuelist, unknown)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "raw_data",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      },
      {
        "name": "render_kw",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      },
      {
        "name": "short_name",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      },
      {
        "name": "type",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      },
      {
        "name": "validate",
        "annotation": "BoundMethod[typing.Callable(wtforms.fields.core.Field.validate)[[Named(self, wtforms.fields.core.Field), Named(form, unknown), Named(extra_validators, unknown, default)], typing.Any], wtforms.fields.core.Field]",
        "kind": "regular",
        "final": false
      },
      {
        "name": "validators",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      },
      {
        "name": "widget",
        "annotation": "unknown",
        "kind": "regular",
        "final": false
      }
    ]
  }
}

The functions of Field:

{
  "response": [
    {
      "name": "wtforms.fields.core.Field.populate_obj",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        },
        {
          "name": "obj",
          "annotation": null
        },
        {
          "name": "name",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.__unicode__",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.check_validators",
      "parameters": [
        {
          "name": "cls",
          "annotation": null
        },
        {
          "name": "validators",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.__str__",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.process",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        },
        {
          "name": "formdata",
          "annotation": null
        },
        {
          "name": "data",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.__call__",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        },
        {
          "name": "**kwargs",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.__new__",
      "parameters": [
        {
          "name": "cls",
          "annotation": null
        },
        {
          "name": "*args",
          "annotation": null
        },
        {
          "name": "**kwargs",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.validate",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        },
        {
          "name": "form",
          "annotation": null
        },
        {
          "name": "extra_validators",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.process_formdata",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        },
        {
          "name": "valuelist",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.post_validate",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        },
        {
          "name": "form",
          "annotation": null
        },
        {
          "name": "validation_stopped",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.pre_validate",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        },
        {
          "name": "form",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.process_data",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        },
        {
          "name": "value",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.__html__",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.__init__",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        },
        {
          "name": "label",
          "annotation": null
        },
        {
          "name": "validators",
          "annotation": null
        },
        {
          "name": "filters",
          "annotation": null
        },
        {
          "name": "description",
          "annotation": null
        },
        {
          "name": "id",
          "annotation": null
        },
        {
          "name": "default",
          "annotation": null
        },
        {
          "name": "widget",
          "annotation": null
        },
        {
          "name": "render_kw",
          "annotation": null
        },
        {
          "name": "_form",
          "annotation": null
        },
        {
          "name": "_name",
          "annotation": null
        },
        {
          "name": "_prefix",
          "annotation": null
        },
        {
          "name": "_translations",
          "annotation": null
        },
        {
          "name": "_meta",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.ngettext",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        },
        {
          "name": "singular",
          "annotation": null
        },
        {
          "name": "plural",
          "annotation": null
        },
        {
          "name": "n",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field.gettext",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        },
        {
          "name": "string",
          "annotation": null
        }
      ],
      "return_annotation": null
    },
    {
      "name": "wtforms.fields.core.Field._run_validation_chain",
      "parameters": [
        {
          "name": "self",
          "annotation": null
        },
        {
          "name": "form",
          "annotation": null
        },
        {
          "name": "validators",
          "annotation": null
        }
      ],
      "return_annotation": null
    }
  ]
}

[abishekvashok]

Just to test. would taint the entire class work?

@r0rshark Yes! (I had to use DSL as forms are created via inheritance)

import flask
from wtforms import Form, TextField, SubmitField

class ContactForm(Form):
    name = TextField("Candidate Name ",[validators.Required("Please enter your name.")])
    submit = SubmitField("Submit")
 
@app.route('/contact')
def contact():
    form = ContactForm()
    return flask.render_template(flask.Markup(form.name.data))

Pysa can detect xss vulnerability in the above code when the all the attributes of ContactForm care marked as taint sources as:

 ModelQuery(
   find = "attributes",
   where = [
     AnyOf(
       parent.extends("wtforms.form.Form", is_transitive=True),
       parent.extends("flaskwtf.form.FlaskForm", is_transitive=True),
     ),
   ],
   model = [
     AttributeModel(
       TaintSource[UserControlled, UserControlled_Parameter],
     )
   ]
 )

If this is acceptable, even though even user.doc would result in a false positive (as mentioned in the doc), I can make a PR solving this and #37 :)

[r0rshark]

Yes this looks like the right way to do it! Checked internally if we can exclude some attributes similarly to what we do for methods arguments https://pyre-check.org/docs/pysa-model-dsl#not-and-anyof-clauses

[abishekvashok]

@r0rshark is that possible? If we can further query the attribute names, of attributes (we are tainting), and exclude some, it will indeed be really cool :)

[r0rshark]

Ah got it it would be an attribute of the tainted attribute! No I do not think this is possible :(

[abishekvashok]

So do you guys want to taint that attribute on its entirety? If so, I can push a pr, else we can close this :)

[r0rshark]

Yes let's taint the entire attribute. My assumption is that in most of the cases this is being used is as a UserControlled input. I imagine the usage of the __ attributes is very rare

[abishekvashok]

Yes let's taint the entire attribute. My assumption is that in most of the cases this is being used is as a UserControlled input. I imagine the usage of the __ attributes is very rare

Yes! Let's get it done :)

[abishekvashok]

Closed via facebook@20115d7