Comments (25)
Hi,
Sorry about that.
I can't reproduce that issue. Which version of pfsense are you running on the corrupted firewall ?
from ansible-pfsense.
Would it be possible to get a diff of the config.xml before and after?
from ansible-pfsense.
Hello,
Thanks a lot for your help.
I'm on pfSense 2.4.4p2 (latest stable).
I joined a diff file from before and after applying ansible state. It seems it removed all CDATA, not much else. I wonder if that's the cause of the issue.
Thanks again
config.diff.txt
from ansible-pfsense.
Thanks. It should be ok with the last commit.
from ansible-pfsense.
It works perfectly, thanks a lot.
from ansible-pfsense.
My bad, there's still some issue. The alias is created, but description (for alias and per network/ip) are empty and some error is triggered each time on pfSense:
There were error(s) loading the rules: /tmp/rules.debug:45: syntax error - The line in question reads [45]: scrub on $TRANSIT all fragment reassemble
@ 2019-03-19 13:39:27
from ansible-pfsense.
Some fields in "interface" seems to have the same issue than earlier in the config.xml:
<interfaces>
<wan>
- <enable></enable>
+ <enable />
<if>vmx0</if>
- <descr><![CDATA[TRANSIT]]></descr>
- <spoofmac></spoofmac>
+ <descr>TRANSIT</descr>
+ <spoofmac />
<ipaddr>10.5.0.3</ipaddr>
<subnet>16</subnet>
<ipaddrv6>fd2b:a315:a66f:7f14::3</ipaddrv6>
<subnetv6>64</subnetv6>
</wan>
<lan>
- <descr><![CDATA[HE]]></descr>
+ <descr>HE</descr>
<if>gif0</if>
- <enable></enable>
- <spoofmac></spoofmac>
+ <enable />
+ <spoofmac />
</lan>
</interfaces>
I doubt it'll fix the empty description fields, but I think that's what triggers the error on pfSense.
from ansible-pfsense.
To have a description on your aliases, you have to use the descr parameter in your playbook.
The interface xml looks ok to me. It's just the python xml parser that is shortening elements. I suppose Orion used the 'html' style to prevent that but it was not closing some tags in that mode (like the area), which triggers your initial issue. You can see that around line 1000: https://github.com/python/cpython/blob/2.7/Lib/xml/etree/ElementTree.py.
Is there any information in rules.debug that would help us to identify what is wrong ? Have you created any rule with pfsense_rule ? or is it using an alias created with pfsense_alias ?
from ansible-pfsense.
I used the fields "descr" and "detail", both seems to have no effect:
descr: "Bastion DMZ"
detail: "adm IPv4||adm IPv6"
I'm not refering to the closed brackets, but some CDATA disappears just like on the first diff. Not sure if that's an issue or not.
I'm not using pfsense_rule at all for the moment, only pfsense_alias.
In rules.debug, I have those lines:
scrub on $TRANSIT all fragment reassemble
scrub on $HE all fragment reassemble
It seems to be the cause of the error reported above. Not sure if that really helps you, since I don't know what the value should be.
Please let me know if you need anything else. Thanks.
from ansible-pfsense.
The stripping of CDATA is a known quirk with ElementTree and hasn't been an issue before. I suspect the changing of the closing element style is an issue - pfsense seems to use an odd/custom XML parser/generator. That is why I specified method='html' when generating the config.xml file. I'm not quite sure I follow what the initial issue here was though that led @f-bor to change that to method='xml'.
from ansible-pfsense.
The initial issue was that the configuration was completely broken, making pfSense unusable (no WebUI access at all, reporting error in the first message, and same behavior from ansible).
Changing it to "xml" fixed the issue, the aliases are added, but no description (per alias or ip/network) are provided, and an error (non-critical, does not prevent access to WebUI or ansible) is thrown: There were error(s) loading the rules: /tmp/rules.debug:45: syntax error - The line in question reads [45]: scrub on $TRANSIT all fragment reassemble @ 2019-03-19 13:39:27
from ansible-pfsense.
@Hakujou I can't reproduce the description issue. Can you post your the playbook you're using to create or update the alias ?
from ansible-pfsense.
@opoplawski, I did some tests before pushing the fix. It does not seems to affect the ability of pfsense to read the configuration file. Also, one change in the GUI and pfsense generates again elements with closing style.
from ansible-pfsense.
Sure, here's the playbook:
---
- name: Create aliases
pfsense_alias:
name: "{{ item.name }}"
type: "{{ item.type }}"
address: "{{ item.address }}"
state: present
with_items:
- "{{ pfsense_aliases }}"
With the following variables:
---
ansible_python_interpreter: /usr/local/bin/python2.7
ansible_user: root
pfsense_aliases:
- name: MANAGED_NETWORKS
type: network
address: "10.0.0.0/12 2a01:xxxx:xxxx:dc00::/64 2a0c:xxxx:xxxx::/48"
descr: MANAGED
detail: "HOME IPv4||LAN IPv4||HOME IPv6"
from ansible-pfsense.
The initial issue was that the configuration was completely broken, making pfSense unusable (no WebUI access at all, reporting error in the first message, and same behavior from ansible).
Changing it to "xml" fixed the issue
What I'd like to see is a diff of the initial working config.xml and the broken config.xml generated with method='html'. I don't yet understand the XML output that's making pfsense complain.
from ansible-pfsense.
Diff was provided here: #13 (comment)
from ansible-pfsense.
@opoplawski, I did some tests before pushing the fix. It does not seems to affect the ability of pfsense to read the configuration file. Also, one change in the GUI and pfsense generates again elements with closing style.
Yes, but it makes comparing updates visually much easier. Unless it's just not possible, I want to keep using method='html' to try to keep the formatting as similar as possible.
from ansible-pfsense.
Fatal error: Uncaught Error: Cannot create references to/from string offsets in /etc/inc/xmlparse.inc:71 Stack trace: #0 [internal function]: startElement(Resource id #6, 'UPDATEFIB', Array) #1 /etc/inc/xmlparse.inc(186): xml_parse(Resource id #6, 'rface>\n\t\t\t\t\n\t\t\t\t`
This seems to indicate that it is encountering a line with just tabs after a line ending with 'rface>'. I don't see that in the diff. Is that in the generated config.xml file somewhere?
from ansible-pfsense.
I tried to reproduce the error but I got a different error this time:
Crash report begins. Anonymous machine information:
amd64
11.2-RELEASE-p6
FreeBSD 11.2-RELEASE-p6 #3 518496b29ae(RELENG_2_4_4): Wed Dec 12 07:41:44 EST 2018 [email protected]:/build/ce-crossbuild-244/obj/amd64/ZfGpH5cd/build/ce-crossbuild-244/pfSense/tmp/FreeBSD-src/sys/pfSense
Crash report details:
PHP Errors:
[19-Mar-2019 16:13:52 Europe/Paris] PHP Fatal error: Uncaught Error: Cannot create references to/from string offsets in /etc/inc/xmlparse.inc:71
Stack trace:
#0 [internal function]: startElement(Resource id #19, 'UPDATEFIB', Array)
#1 /etc/inc/xmlparse.inc(186): xml_parse(Resource id #19, 'rvice>\n\t\t<servi...', true)
#2 /etc/inc/xmlparse.inc(147): parse_xml_config_raw('/conf/config.xm...', Array, 'false')
#3 /etc/inc/config.lib.inc(132): parse_xml_config('/conf/config.xm...', Array)
#4 /etc/inc/config.inc(159): parse_config()
#5 /etc/inc/gwlb.inc(23): require_once('/etc/inc/config...')
#6 /etc/inc/functions.inc(33): require_once('/etc/inc/gwlb.i...')
#7 /usr/local/sbin/pfSsh.php(26): require_once('/etc/inc/functi...')
#8 {main}
thrown in /etc/inc/xmlparse.inc on line 71
[19-Mar-2019 16:13:53 Europe/Paris] PHP Fatal error: Uncaught Error: Cannot create references to/from string offsets in /etc/inc/xmlparse.inc:71
Stack trace:
#0 [internal function]: startElement(Resource id #18, 'UPDATEFIB', Array)
#1 /etc/inc/xmlparse.inc(186): xml_parse(Resource id #18, 'rvice>\n\t\t<servi...', true)
#2 /etc/inc/xmlparse.inc(147): parse_xml_config_raw('/conf/config.xm...', Array, 'false')
#3 /etc/inc/config.lib.inc(132): parse_xml_config('/conf/config.xm...', Array)
#4 /etc/inc/config.inc(159): parse_config()
#5 /etc/inc/gwlb.inc(23): require_once('/etc/inc/config...')
#6 /etc/inc/functions.inc(33): require_once('/etc/inc/gwlb.i...')
#7 /etc/inc/notices.inc(24): require_once('/etc/inc/functi...')
#8 /usr/local/bin/notify_monitor.php(23): include_once('/etc/inc/notice...')
#9 {main}
thrown in /etc/inc/xmlparse.inc on line 71
No FreeBSD crash data found.
Following variable used with the above playbook:
---
ansible_python_interpreter: /usr/local/bin/python2.7
ansible_user: root
pfsense_aliases:
- name: SECURE_AD
type: host
address: "10.10.50.1 10.10.50.2 2a0c:b641:xxxxxxx:6db0:987e 2a0c:b641:xxxxxxxxx:3404:48fa"
descr: "Active Directory"
detail: "ad01||ad02||ad01 IPv6||ad02 IPv6"
Here's the diff: config.diff.txt
from ansible-pfsense.
@opoplawski
The issue is here:
- <area>0.0.0.0</area>
+ <area>0.0.0.0
When writing html, the python xml library considers an area tag must be empty (no inner text). So it does not close the tag and generates malformed xml. You can see that around line 1000 in the 2.7 library: https://github.com/python/cpython/blob/2.7/Lib/xml/etree/ElementTree.py
from ansible-pfsense.
@Hakujou, in your playbook, your are not using the detail and descr variables you have defined. That's why you don't have any description. You have to use them like address, name or type.
from ansible-pfsense.
Ah, thanks for being explicit about '<area>' - I didn't catch that.
from ansible-pfsense.
@Hakujou I don't think the rules.debug issue is related to us and I can't reproduce it. Those rules are generated when you uncheck "IP Do-Not-Fragment compatibility" and "Disable Firewall Scrub" in System / Advanced / Firewall & NAT. They are checked by default. Can you try to delete the alert, recheck those two boxes, save, uncheck them again, save, play around with psense_alias to make some changes and tell us if the alert appears again at any time ?
from ansible-pfsense.
My bad for the playbook, I completely forgot to readd them, I deleted them for testing... I feel stupid.
Regarding the other issue, "IP Do-Not-Fragment compatibility" and "Disable Firewall Scrub" are not checked by default (because they are violations of RFC for compatibility, which has its uses but is still bad practice).
Error still reappear after checking and unchecking them again:
There were error(s) loading the rules: /tmp/rules.debug:86: syntax error - The line in question reads [86]: nat on $TRANSIT inet from 10.2.0.0/16 to 10.2.0.0/16 -> 10.5.0.1/32 port 1024:65535
@ 2019-03-19 18:10:30
There were error(s) loading the rules: /tmp/rules.debug:78: syntax error - The line in question reads [78]: scrub on $LAN all fragment reassemble
@ 2019-03-19 18:10:34
It must be related to ansible-pfsense, since I had the same behaviour on five pfSense VMs, which never had any issue (not any error at all) before using it.
I think it somehow breaks the configuration, as any modification of firewall rule (even without making changes, just by re-saving existing parameters) triggers the error again.
from ansible-pfsense.
The issue was caused by aliases having the same name as interfaces. It wan't tested in pfsense_alias. I fixed that. You just have to delete the aliases that have the same name as interfaces and the error will be gone.
from ansible-pfsense.
Related Issues (20)
- Calling the plugin breaks special XML fields which use CDATA and contain ampersands HOT 4
- Managing DHCP and unbound - any plans? HOT 4
- Modules list error in main README.md ? HOT 1
- Feature Request: Provide interface update HOT 1
- Ansible - Error using privilege escalation HOT 1
- pfsense 2.5 Support? HOT 25
- using ca module in pfsense 2.5 HOT 1
- pfsensible.core not compatible with ansible.netcommon 2.0.0 HOT 1
- diff['before'] is equal to diff['after'] HOT 1
- Retire this repo HOT 5
- pfsense_vlan.py error on 2.5.1 HOT 5
- Problem with pfsense_authserver_ldap module - automatically change LDAP config ? HOT 2
- pfsense_ipsec_p2 module overwriting other p2 interfaces in 2.5.2 HOT 1
- pfsense_aggregate: module failuer when try to use non-default gateway with the rule
- cannot add vlan interface HOT 1
- can we have a backup option on this ansible modules HOT 1
- Rule add issue when alias list is empty -> TypeError: 'NoneType' object is not iterable HOT 1
- pfsense_interface module cannot remove a subinterface HOT 1
- ERROR! 'pfsense_rule' is not a valid attribute for a Play HOT 1
- Not a problem - just wondering if there is a way to print a list of users by group? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ansible-pfsense.