badstreff / git2jss Goto Github PK
View Code? Open in Web Editor NEWEasily sync your scripts and EA's from a git repo with the JSS
License: MIT License
Easily sync your scripts and EA's from a git repo with the JSS
License: MIT License
For scripts with the shebang line of #!/bin/zsh
โ performing a download yields the following message:
No interpreter directive found for:
We've been discussing this in #git2jss on Slack, but I thought it was worth opening an issue. When trying to use sync to upload EAs/scripts to a Jamf Pro instance without any scripts for EAs (e.g., a brand new test instance) no EAs or scripts get uploaded, even when using --update_all
Changed Extention Attributes: []
Changed Scripts: []
No Changes in Scripts
No Changes in Extension Attributes
It sounds like a fix has already been identified. Just wanted an issue to attach to it.
In some environments eg a dev environment that is internal only the cert may be self signed..
Would it be possible to add a flag for disabling this verification of the cert?
Traceback (most recent call last):
File "/home/gitlab-runner/.local/lib/python3.5/site-packages/aiohttp/connector.py", line 796, in _wrap_create_connection
return (yield from self._loop.create_connection(*args, **kwargs))
File "uvloop/loop.pyx", line 1769, in create_connection
File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
yield self # This tells Task to wait for completion.
File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup
future.result()
File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/usr/lib/python3.5/asyncio/sslproto.py", line 493, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "/usr/lib/python3.5/asyncio/sslproto.py", line 201, in feed_ssldata
self._sslobj.do_handshake()
File "/usr/lib/python3.5/ssl.py", line 633, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "./sync.py", line 163, in <module>
loop.run_until_complete(main(args))
File "uvloop/loop.pyx", line 1364, in uvloop.loop.Loop.run_until_complete
File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
result = coro.throw(exc)
File "./sync.py", line 147, in main
await upload_scripts(session, args.url, args.username, args.password, semaphore)
File "./sync.py", line 92, in upload_scripts
responses = await asyncio.gather(*tasks)
File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
yield self # This tells Task to wait for completion.
File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup
future.result()
File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
result = coro.throw(exc)
File "./sync.py", line 105, in upload_script
auth=auth) as resp:
File "/home/gitlab-runner/.local/lib/python3.5/site-packages/aiohttp/client.py", line 690, in __aenter__
self._resp = yield from self._coro
File "/home/gitlab-runner/.local/lib/python3.5/site-packages/aiohttp/client.py", line 267, in _request
conn = yield from self._connector.connect(req)
File "/home/gitlab-runner/.local/lib/python3.5/site-packages/aiohttp/connector.py", line 402, in connect
proto = yield from self._create_connection(req)
File "/home/gitlab-runner/.local/lib/python3.5/site-packages/aiohttp/connector.py", line 748, in _create_connection
_, proto = yield from self._create_direct_connection(req)
File "/home/gitlab-runner/.local/lib/python3.5/site-packages/aiohttp/connector.py", line 859, in _create_direct_connection
raise last_exc
File "/home/gitlab-runner/.local/lib/python3.5/site-packages/aiohttp/connector.py", line 831, in _create_direct_connection
req=req, client_error=client_error)
File "/home/gitlab-runner/.local/lib/python3.5/site-packages/aiohttp/connector.py", line 801, in _wrap_create_connection
raise ClientConnectorSSLError(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorSSLError: Cannot connect to host <JSS>:8443 ssl:True [[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)]
Have been looking to get this going for our org for a while now and finally got all the pieces in place only to be stumped by what appears to be a deprecated function.
Wondering if you or anybody else has come across this and has a quick fix?
Short version, I think add sync_path needs to be added to the front of the joins near lines 152, 207, 293, and 335. (the line numbers are from my fork and are a bit off, as I took the long road to get there and changed a bit more thinking it was necessary)
More info on what I saw here: https://macadmins.slack.com/archives/C04QVP86E/p1697777935371559?thread_ts=1697751693.656709&cid=C04QVP86E
More info about my conclusions here: https://macadmins.slack.com/archives/CDLQMGB6E/p1697873801412639
This was with the fork that was just merged.
Before going down the path to fix ourselves, wondering if anyone has done the work to update what seems to just be aiojss.py to support the new tokenized API auth vs old basic auth. Seems like it should be relatively easy, but always prefer avoiding re-inventing the wheel. =)
Thanks!
Hello!
While further testing this in prep for our go live. We've also tested it on v10.
I'm seeing an error through the CI runner hitting the webapp(hosted on the same server), and if i run it locally hitting the same server, or on the server hitting the webapp.
What it seems is aiosync is spawning too many connections in a short period. Which Jamv10 doesn't seem to like. I can confirm this on 10.0.0 and 10.1.1.
I've gone ahead and added to mysql & tomcat threads and still experience the same error.
Traceback (most recent call last):
File "sync.py", line 49, in upload_extension_attribute
resp = await session.put(put_url, auth=auth, data=ET.tostring(template), headers=headers)
File "/home/rderewianko/.local/lib/python3.5/site-packages/aiohttp/helpers.py", line 104, in __await__
ret = yield from self._coro
File "/home/rderewianko/.local/lib/python3.5/site-packages/aiohttp/client.py", line 277, in _request
yield from resp.start(conn, read_until_eof)
File "/home/rderewianko/.local/lib/python3.5/site-packages/aiohttp/client_reqrep.py", line 624, in start
(message, payload) = yield from self._protocol.read()
File "/home/rderewianko/.local/lib/python3.5/site-packages/aiohttp/streams.py", line 554, in read
yield from self._waiter
File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
yield self # This tells Task to wait for completion.
File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup
future.result()
File "/usr/lib/python3.5/asyncio/futures.py", line 266, in result
raise CancelledError
concurrent.futures._base.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "sync.py", line 159, in <module>
loop.run_until_complete(main(args))
File "uvloop/loop.pyx", line 1364, in uvloop.loop.Loop.run_until_complete
File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
result = coro.throw(exc)
File "sync.py", line 145, in main
await upload_extension_attributes(session, args.url, args.username, args.password)
File "sync.py", line 31, in upload_extension_attributes
responses = await asyncio.gather(*tasks)
File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
yield self # This tells Task to wait for completion.
File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup
future.result()
File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
result = coro.throw(exc)
File "sync.py", line 52, in upload_extension_attribute
resp = await session.post(post_url, auth=auth, data=ET.tostring(template), headers=headers)
File "/home/rderewianko/.local/lib/python3.5/site-packages/async_timeout/__init__.py", line 35, in __exit__
self._do_exit(exc_type)
File "/home/rderewianko/.local/lib/python3.5/site-packages/async_timeout/__init__.py", line 80, in _do_exit
raise asyncio.TimeoutError
concurrent.futures._base.TimeoutError
There is a difference in the number of submissions before failure.
CI Runner - 63
Locally - 112
From what I understand, you can not supply --password
in the command to generate a password prompt. However, when the password is entered in this prompt it appears to not be accepted, as I get the following error:
python sync.py --url https://<jss>.com:8443 --username api_git2jss --do_not_verify_ssl
Password:
Traceback (most recent call last):
File "sync.py", line 429, in <module>
loop.run_until_complete(main())
File "uvloop/loop.pyx", line 1417, in uvloop.loop.Loop.run_until_complete
File "sync.py", line 385, in main
session, args.url, args.username, args.password, semaphore)
File "sync.py", line 355, in get_existing_categories
auth = aiohttp.BasicAuth(user, passwd)
File "/Users/ap.orlebeke/.virtualenv/git2jss/lib/python3.7/site-packages/aiohttp/helpers.py", line 127, in __new__
raise ValueError('None is not allowed as password value')
ValueError: None is not allowed as password value
When trying to run the sync.py I'm getting the following error when the --do_not_verify_ssl flag is set. This same flag is working with the download.py.
./sync.py:288: DeprecationWarning: verify_ssl is deprecated, use ssl=False instead async with aiohttp.ClientSession(connector=aiohttp.TCPConnector(verify_ssl=args.do_not_verify_ssl)) as session: Traceback (most recent call last): File "./sync.py", line 330, in <module> loop.run_until_complete(main(args)) File "uvloop/loop.pyx", line 1446, in uvloop.loop.Loop.run_until_complete File "./sync.py", line 289, in main categories = await get_existing_categories(session, args.url, args.username, args.password, semaphore) File "./sync.py", line 281, in get_existing_categories return [c.find('name').text for c in [e for e in ET.fromstring(await resp.text()).findall('category')]] File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xml/etree/ElementTree.py", line 1315, in XML parser.feed(text) xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 0
Would be nice to be able to install this via pip and point it at a directory instead of having the script expect the root dir
Receive the error: File "/usr/local/lib/python3.6/xml/etree/ElementTree.py", line 1314, in XML parser.feed(text) xml.etree.ElementTree.ParseError: reference to invalid character number: line 35, column 20
While trying to download and/or parse our scripts. It seems to be only breaking on &
. I have been doing some research on how to mitigate this. The only thing I could see is to read the XML for each line and handle the &
before the ElementTree gets it.
It looks like there is a practice followed my mac admins to separate their scripts by folders within their repo.
Wanted to start the discussion of possibly refactoring the script and documentation to depend on this structure.
Hello
First of all: Thank you for writing this great tool!
This is a feature request and not an issue as such. Do you have any planes to add support for configuration profiles?
Or would you be willing to accept a PR with this functionality?
Currently each time sync is run the username and password to the JSS need typed in the command line, it would be nice to have the ability to cache them in something like the keychain and verify they are correct so the user doesn't have to keep typing them
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.