osteele / liquid Goto Github PK

I envision a cache package, with a null cache, a cache that examines file mod time, and a cache that doesn't. Maybe just doc how to do this instead.

Checklist

• I have searched the issue list

Detailed Description

• Add benchmark to the list of other template benchmarks.
• Provide the bechnhmark results in readme until this is re-run by the author of original bench suite.

Add a CI step to build and test on a 32-bit platform. This will detect a regression in #47 or the introduction of similar bugs.

I believe this requires building and testing on a 32-bit Docker container.

Checklist

• I have searched the issue list
• I have tested my example against Shopify Liquid. (This isn't necessary if the actual behavior is a panic, or an error for which IsTemplateError returns false.)

Expected Behavior

I can compile for armv7

Actual Behavior

Throws error: yaccpar:452: constant 4294967295 overflows int

Detailed Description

It appears somewhere there's a constant which is too large for anything but a 64bit environment.

Possible Solution

Would be good to add some build tags to have a different value for other environments.

Checklist

• I have searched the issue list
• I have tested my example against Shopify Liquid. (This isn't necessary if the actual behavior is a panic, or an error for which IsTemplateError returns false.)

Expected Behavior

{{ "a\"b\\c" }}

output

a"b\c

Actual Behavior

Syntax Error

Detailed Description

ref. #5

The Shopify specification does not allow escaping of characters in string literals.
https://shopify.github.io/liquid/basics/types/#string

Liquid does not convert escape sequences into special characters.

But this is inconvenient for me.

Possible Solution

The proposal makes it escapable when ``.
Shopify/liquid#1174

This my branch uses strconv.Unquote to escape only double-quoted.
master...tuchida:escape-string

In this implementation, it seems that only \' and \" can be escaped.
https://github.com/acstech/liquid/blob/5bcb5c9b2d87dbd09e8a08889477167721e09edf/core/parser.go#L135-L136

If you want to support escaping, you can decide on the specification and I will create a Pull Request.

Expected Behavior

{{ 'email' | errors.Get }}

Actual Behavior

syntax error in \"'email' | errors.Get\"

Detailed Description

I have an errors struct with a Get(string) []string method which I would like to use directly in the template. But I can't manage to pass an argument to that method.
Other methods without an argument work (e.g. Key() []string).

Implement {%- cmd -%} and {{- expo -}}. (Note that - need only appear on one side.)

Implementation plan:

• Add trimLeft, trimRight to Chunk.
• Create an io.Writer type trimWriter struct {io.Writer; trimLeft, trimRight bool; pending []byte} that wraps another writer.
• The render wraps the output writer in this before passing it to the node renderers.
• ws.Write: if ws.trimLeft, apply bytes.TrimLeft. If ws.trimRight, apply bytes.TrimRight. If the the trimmed string isn't empty, write ws.pending and reset it to empty. Finally, if any bytes were trimmed from the right, append them to ws.pending.
  • When the renderer is done, it needs to call ws.Flush(). This should write ws.pending.

https://shopify.github.io/liquid/tags/iteration/

<table>
{% tablerow product in collection.products %}
  {{ product.title }}
{% endtablerow %}
</table>

See https://ci.appveyor.com/project/osteele/liquid:

# github.com/osteele/liquid/strftime
could not determine kind of name for C.localtime_r
Command exited with code 2

https://shopify.github.io/liquid/filters/url_encode/

Is it possible to call .AddTag() and .AddFilter() once liquid.newEngine() has been called, to add custom tags/filters at runtime? Or best to fork the repo?

Liquid interprets a.b-c as a['b-c']. We do this for names that include alphanumerics and [_-], in any order. Should this be extended to all non-whitespace?

Checklist

• I have searched the issue list
• I have tested my example against Shopify Liquid. (This isn't necessary if the actual behavior is a panic, or an error for which IsTemplateError returns false.)

Expected Behavior

This proposal originated from #35 but tries to tackle a broader issue, i.e.: Sometimes I might want to have more control over the rendering process, specifically where the renderer writes to.

Example

• I might want to limit the number of bytes written when exposing templating to some untrusted party.
• I might want to set a timeout for the rendering process.
• I might want to render into a file directly.

Actual Behavior

The render package has a Render(node Node, w io.Writer, vars map[string]interface{}, c Config) function which takes an io.Writer, however, this is meant for internal use only. The Template methods have Render(vars Bindings) ([]byte, SourceError) and RenderString(b Bindings) (string, SourceError) which are useful in most circumstances but don't allow for more control as described above.

Detailed Description

Possible Solution

I'd propose a new Template render method FRender that allows to pass a custom io.Writer:

func (t *Template) FRender(w io.Writer, vars Bindings) SourceError

This gives more control over the rendering process when needed, e.g. you might want to set a render timeout to avoid deeple nested loop constructs in a template to cause potential server issues:

type CancelWriter struct {
	context.Context
	bytes.Buffer
}

func (w *CancelWriter) Write(bs []byte) (int, error) {
	if w.Err() != nil {
		return w.Len(), context.Cause(w)
	}
	return w.Buffer.Write(bs)
}

func Render() ([]byte, error) {
	wr := &CancelWriter{}
	if err := liquid.NewEngine().ParseAndFRender(wr, nil /*untrustworthy template*/, nil /*data*/); err != nil {
		return nil, err
	}
	return wr.Bytes(), nil
}

Compared to the ruby implementation, does the go version have similar security/safety guarantees?
What I mean is, liquid was meant for end-user modification and should not allow malicious code to be executed.
Curious as to your comments in this regards.

Great work!

Checklist

• I have searched the issue list
• I have tested my example against Shopify Liquid. (This isn't necessary if the actual behavior is a panic, or an error for which IsTemplateError returns false.)

Expected Behavior

<li class="nav-list-item active"></li>

Actual Behavior

<li class="nav-list-itemactive"></li>

Detailed Description

When using the whitespace trimming flags, whitespaces are trimmed incorrectly, compared to the Ruby liquid implementation:

{%- assign pages_list = "test" -%}
<li class="nav-list-item{% if true %} active{% endif %}"></li>

Checklist

• I have searched the issue list
• I have tested my example against Shopify Liquid. (This isn't necessary if the actual behavior is a panic, or an error for which IsTemplateError returns false.)

Expected Behavior

Respect whitespace flags in for and if blocks.

My friends are:

{%- for friend in people -%}
{{ friend }},
{%- endfor -%}

and I like them!

with

{"people": ["alice", "bob"]}

Should render as

My friends are:alice,bob,and I like them!

https://liquidjs.com/playground.html#TXkgZnJpZW5kcyBhcmU6Cgp7JS0gZm9yIGZyaWVuZCBpbiBwZW9wbGUgLSV9Cnt7IGZyaWVuZCB9fSwKeyUtIGVuZGZvciAtJX0KCmFuZCBJIGxpa2UgdGhlbSE=,eyJwZW9wbGUiOiBbImFsaWNlIiwgImJvYiJdfQ==

Actual Behavior

Whitespace flags are ignored:

My friends are:


alice,

bob,


and I like them!

Detailed Description

It seems the TrimLeft and TrimRight functions aren't invoked at the block level.

Possible Solution

Invoke TrimLeft and TrimRight on the block during rendering.

https://shopify.github.io/liquid/filters/url_decode/

Checklist

• I have searched the issue list
• I have tested my example against Shopify Liquid. (This isn't necessary if the actual behavior is a panic, or an error for which IsTemplateError returns false.)

Expected Behavior

Parse {{ display_price | default: true, allow_false: true }} without any error.

Actual Behavior

It shows Liquid error (line 12): syntax error in "display_price | default: true, allow_false: true" in {{ display_price | default: true, allow_false: true }}

The syntax is from the shopify document:
https://shopify.github.io/liquid/filters/default/

Detailed Description

Possible Solution

Add an option to warn or be silent on undefined filters (currently errors).

Add an option to warn or error on undefined variables (currently silent).

Right now bindings are a map so to use a struct I'd need to do something like the following:

bindings := map[string]interface{}{
    "data": myStructVariable,
}

It would be great if it was possible to pass in a struct directly to keep the liquid code accessing the variables cleaner.

Shopify Liquid supports a for loop else clause. This library does not.

This was noticed by @tanema in issue #42.

Checklist

• I have searched the issue list
• I have tested my example against Shopify Liquid. (This isn't necessary if the actual behavior is a panic, or an error for which IsTemplateError returns false.)

Detailed Description

I want to be able to use liquid on structs from other packages, where I have no control over the struct tags. Specifically I'd like to use the json tag.

Possible Solution

Amend the tagKey constant to be an exported var:

Add benchmarks.

These could be translated into Markdown from the test files in https://github.com/SlinSo/goTemplateBenchmark.

There might also be benchmarks or test files in some of the other Liquid implementations.

Checklist

• I have searched the issue list
• I have tested my example against Shopify Liquid. (This isn't necessary if the actual behavior is a panic, or an error for which IsTemplateError returns false.)

Expected Behavior

bindings := map[string]interface{}{
"app": "test_app",
"描述": "content",
}
template := 应用:{{app}} 描述:{{ 描述 }}
expected := 应用:test_app 描述:content

Actual Behavior

syntax error in "描述" in {{ 描述 }}

Detailed Description

Possible Solution

Summary

The ruby implementation of liquid defines a "FileSystem" interface than can be used to load template files from sources other than the local disk. This allows a developer to (for example) load template files from s3 buckets or a database seamlessly. This feature becomes particularly useful when using include tags.

https://github.com/Shopify/liquid/blob/master/lib/liquid/file_system.rb
https://www.rubydoc.info/gems/liquid/Liquid/BlankFileSystem

I have an unfinished version of this working locally, but wanted to submit this issue for discussion before I got too far down any particular implementation path.

Suggested Implementation

Trying to stay as close as possible to the ruby implementation we define an interface as:

type LiquidFileSystem interface {
    ReadTemplateFile(templatename string) ([]byte, error)
}

The instance of LiquidFileSystem lives in the render.config struct. A basic implementation (and default behavior to provide backwards compatibility) would be a simple wrapper around
ioutil.ReadFile

func (tl *diskFileSystem) ReadTemplateFile(filename string) ([]byte, error) {
    source, err := ioutil.ReadFile(filename)
    return source, err
}

Any custom implementation would be utilized as follows:

type S3FileSystem struct { ...snip... }
func NewS3FileSystem(...) *S3FileSystem { ...snip... }
func (tl *S3FileSystem) ReadTemplateFile(filename string) ([]byte, error) {  ... snip ...}

e := liquid.NewEngine()
fs := NewS3FileSystem(...)
e.RegisterFilesystem(fs)

As yet Unaddressed Issues

1. Template Error Reporting (currently I believe we just report filename as opposed to a full URL)
2. Potentially new classes of errors?
3. Caching. Another github issue addresses caching, but it becomes more important if we are loading templates files from a relatively high latency source.

Next Steps/Suggestions?

If this approach seems generally sane to you and it's a feature you are interested in adding I will finish cleaning up the code and submit a PR. If you have any alternate implementation ideas I'm more than open to incorporating them.

Investigate the interaction of loop reversed, limit, and offset, and whether it matters which order they're specified.

Does Liquid handle e.g. "a\"b\\c" and 'a\'b\\c'? If so, implement them.

Does Liquid handle newline, unicode and other escapes? If so, implement those too.

The lexer could use JSON.Unmarshall on strings that contain \\

Checklist

• [✓] I have searched the issue list
• [✓] I have tested my example against Shopify Liquid. (This isn't necessary if the actual behavior is a panic, or an error for which IsTemplateError returns false.)

Expected Behavior

Ability to parse condition assignment in assign tag.

Actual Behavior

syntax error

2023/02/13 13:33:22 Liquid error (line 1): syntax error in "%assign con_0_Euh43 = user.name == \"Ryan\" and user.email == \"[email protected]\"" in {% assign con_0_Euh43 = user.name == "Ryan" and user.email == "[email protected]" %}

Detailed Description

While liquidjs correctly parses it this go implementation not.
See https://liquidjs.com/playground.html#eyUgYXNzaWduIGNvbl8wX0V1aDQzID0gdXNlci5uYW1lID09ICJSeWFuIiBhbmQgdXNlci5lbWFpbCA9PSAieHhAZ21haWwuY29tIiAlfSAKeyUgYXNzaWduIGNvbl8xX0V1aDQzID0gdXNlci5wcm9qZWN0ID09ICJFYXN5IGVtYWlsIiAlfSAKe3sgY29uXzBfRXVoNDMgfX0Ke3sgY29uXzFfRXVoNDMgfX0KeyUgaWYgY29uXzBfRXVoNDMgYW5kIGNvbl8xX0V1aDQzICV9CiAgPHA+SGVsbG88L3A+CnslIGVuZGlmICV9,ewogICJ1c2VyIjogewogICAgIm5hbWUiOiAiUnlhbiIsCiAgICAiZW1haWwiOiAieHhAZ21haWwuY29tIiwKICAgICJwcm9qZWN0IjogIkVhc3kgZW1haWwiCiAgfQp9
Even though it can be rewritten to use in if statement directly it is not a solution for because we are using https://github.com/zalify/easy-email which is making these assign statements

Possible Solution

I don't know. But I can help if you direct to somewhere.

https://shopify.github.io/liquid/tags/iteration/

{% cycle 'one', 'two', 'three' %}
{% cycle 'one', 'two', 'three' %}
{% cycle 'one', 'two', 'three' %}
{% cycle 'one', 'two', 'three' %}

→

one
two
three
one

All the information is present, but it's not injected into the various error instances.

I played with the library today, and it worked totally fine. Thank you to all the authors for all the work that went into it!

Keep up the good work, please!

Feel free to close this issue after reading.

Currently

	params := map[string]interface{}{
		"message": Message{
			Text: "hello",
		},
	}
	engine := liquid.NewEngine()
	template := "{{ message.Text }}"
	str, err := engine.ParseAndRenderString(template, params)
	log.Printf("Err: %v\n%v", err, str)

doesn't work as expected, but

	params := map[string]interface{}{
		"message": map[string]interface{}{
			"Text": "hello",
		},
	}
	engine := liquid.NewEngine()
	template := "{{ message.Text }}"
	str, err := engine.ParseAndRenderString(template, params)
	log.Printf("Err: %v\n%v", err, str)

does. The text/template and html/template libraries in the go standard library allow you to traverse through any objects (maps, structs, slices and pointers to these) and use them in the templating engine, as in my first example.

A decent workaround for this is the github.com/fatih/structs Map() function, which converts your struct structure into a map based structure, but this has two problems in my use case. 1. My data struct is very large and this is a lot of processing, and 2. it preserves elements of my data struct that are *map[string]interface{} (which need to be like this to work with the rest of my program), which as far as I can tell

	params := map[string]interface{}{
		"message": &map[string]interface{}{
			"Text": "hello",
		},
	}
	engine := liquid.NewEngine()
	template := "{{ message.Text }}"
	str, err := engine.ParseAndRenderString(template, params)
	log.Printf("Err: %v\n%v", err, str)

also doesn't work correctly.

Are you planning on extending to copy this behaviour (from text/template or even https://github.com/acstech/liquid) in your current library, or do you consider it out of scope?

System information

• Version: github.com/osteele/liquid v1.2.4
• Go: go version go1.13.7 darwin/amd64

Problem

I am receiving syntax errors while trying to parse templates that are meant for shopify. I have filters that look like this

{{image | img_url: '580x', scale: 2}}
{{ order.created_at | date: format: 'date' }}
{{ 'customer.order.title' | t: name: order.name }}

I have tried to just define the filters:

	cfg.AddFilter("t", func(value interface{}) interface{} {
		return value
	})
	cfg.AddFilter("date", func(value interface{}) interface{} {
		return value
	})

And I get a generic Syntax error that does not define what the error is.

Steps to reproduce the behavior

Here is a test that will reproduce this:

package lint

import (
	"testing"

	"github.com/osteele/liquid"
	"github.com/stretchr/testify/assert"
)

func TestPathToProject(t *testing.T) {
	engine := liquid.NewEngine()
	template := `{{ 'customer.order.title' | t: name: order.name }}`
	bindings := map[string]interface{}{}
	engine.RegisterFilter("t", func(value interface{}) interface{} {
		return value
	})
	_, err := engine.ParseAndRenderString(template, bindings)
	assert.Nil(t, err)
}

Possible Solution

I have a hunch that you do not support named arguments to filters.

Follow-up question, it does not seem like your for loop supports else and I cannot seem to override the standard for loop with my own

https://shopify.github.io/liquid/filters/sort_natural/

Blocked by golang/go#35781

The CI should test whether the generated files (expressions.go and y.go) match the sources.

The following GitHub Action will do this:

  - name: Test generated code
  - run: |
      make generate
      git diff --exit-code

The missing pieces are:

1. Installing Ragel

This works:

  - name: Install Ragel
    run: |
      sudo apt-get update
      sudo apt-get install 

However, this requires apt-get update and apt-get install on each run.

Is there a way to cache this? Alternatively, should this action use a Docker image that includes Ragel?

2. Installing goyacc

I tried this:

Add a step:

  - name: Install generators
    run: make setup
  - name: Run generators
    run: make generate

(make setup includes go install golang.org/x/tools/cmd/goyacc. make generate runs go generate ./....)

This produces an error:

expressions/parser.go:3: running "goyacc": exec: "goyacc": executable file not found in $PATH
make: *** [Makefile:24: generate] Error 1
Error: Process completed with exit code 2.

Checklist

• I have searched the issue list
• I have tested my example against Shopify Liquid. (This isn't necessary if the actual behavior is a panic, or an error for which IsTemplateError returns false.)

Detailed Description

If a custom block, created with RegisterBlock(), uses render.Context.WrapError it panics due to render.rendererContext.node being nil.

Errorf() and SourceFile() have the same issue.

Test case:

package main

import (
	"errors"
	"github.com/osteele/liquid"
	"github.com/osteele/liquid/render"
)

func main() {
	engine := liquid.NewEngine()
	engine.RegisterBlock("foo", func(c render.Context) (string, error) {
		return "bar", c.WrapError(errors.New("baz"))
	})
	_, _ = engine.ParseAndRenderString(`{% foo %}  {% endfoo %}`, map[string]interface{}{})

}

Possible Solution