GithubHelp home page GithubHelp logo

m2ee-tools's Introduction

m2ee-tools's People

Contributors

achiel avatar djvdorp avatar igor-mendix avatar jtwaleson avatar knorrie avatar markvanments avatar pommi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

m2ee-tools's Issues

Error while starting app:

Hello,
I'm trying to start an on-premise Mendix app, but I get the following error:

m2ee(mxlaba): start INFO: Trying to start the MxRuntime... Traceback (most recent call last): File "/usr/bin/m2ee", line 1071, in <module> cli.cmdloop() File "/usr/lib64/python2.6/cmd.py", line 142, in cmdloop stop = self.onecmd(line) File "/usr/lib64/python2.6/cmd.py", line 219, in onecmd return func(arg) File "/usr/bin/m2ee", line 69, in do_start self._start() File "/usr/bin/m2ee", line 133, in _start if not self.m2ee.start_appcontainer(): File "/usr/share/m2ee-tools/m2ee/core.py", line 86, in start_appcontainer if not self.runner.start(): File "/usr/share/m2ee-tools/m2ee/runner.py", line 155, in start self._pid = proc.pid UnboundLocalError: local variable 'proc' referenced before assignment ERROR: Starting the JVM process did not succeed...

We're using Mx 6.7.1, and m2ee version 0.5.11.3...

Do you have any idea what the problem could be?

Thanks!

"cleanup_runtimes" sometimes delete the runtime currently in use

I've run into a problem with cleanup_runtimes a few times now.
After the first deployment of an app with a new runtime I see this:

m2ee(mendix): about
Using m2ee-tools version 7.2.2
Using Mendix Runtime version 9.12.2.44241
Copyright © 2003-2022 Mendix bv. All rights reserved.
Model version: 0.0.0.6625

m2ee(mendix): cleanup_runtimes
INFO: Cleaning up old runtimes from /opt/app/Mendix/Fieldtool/runtimes...
INFO: Keeping 9.6.11.42323
INFO: Removing 9.12.2.44241...
m2ee(mendix):

The 'about' shows that runtime 9.12.2 is in use.
But when I give the command cleanup-runtime it removes that runtime.

If I download that specific runtime again, restart the application, and enter the cleanup_runtimes command again, then the correct runtimes are removed.

Debian 11 (Bullseye) packages "empty"

busy testing/setting up Debian 11 Mendix instances, and the bullseye distribution seems empty, had to fall back to buster to get packages, but these have other python2 related problems it seem when trying to install m2ee-tools

The following packages have unmet dependencies:
 python-m2ee : Depends: python-yaml but it is not installable
               Depends: python-httplib2 but it is not installable
E: Unable to correct problems, you have held broken packages.

Move PostgreSQL munin graphs in here

At Mendix, we've always had the database statistics separate for munin. Let's move them in here instead, so that they are displayed as part of the Mendix application statistics.

! [new branch] debian/master -> origin/debian/master (unable to update local ref)

PSA: I'm converting the debian branches of this repo to DEP14 format when doing the v7.2.2 bugfix release.

That means that we'll have debian/master and debian/stretch etc.

For whoever has cloned this repo before, this will result in the error shown in the subject, when trying to fetch updates:

error: cannot lock ref 'refs/remotes/origin/debian/master': 'refs/remotes/origin/debian' exists; cannot create 'refs/remotes/origin/debian/master'
 ! [new branch]      debian/master            -> origin/debian/master  (unable to update local ref)
error: cannot lock ref 'refs/remotes/origin/debian/stretch': 'refs/remotes/origin/debian' exists; cannot create 'refs/remotes/origin/debian/stretch'
 ! [new branch]      debian/stretch -> origin/debian/stretch  (unable to update local ref)

The git client will suggest:

error: some local refs could not be updated; try running
 'git remote prune origin' to remove any old, conflicting branches

As long as you don't have a local branch based on the old upstream debian, just follow the instruction once, and you'll be fine.

Handle M2EERuntimeNotFullyRunning in munin code

This is not very useful:

-# munin-run mxruntime_tr10000
Traceback (most recent call last):
  File "/etc/munin/plugins/mxruntime_tr10000", line 69, in <module>
    m2ee.munin.print_values(m2ee_instance, name)
  File "/usr/lib/python2.7/dist-packages/m2ee/munin.py", line 98, in print_values
    stats, java_version = get_stats('values', m2)
  File "/usr/lib/python2.7/dist-packages/m2ee/munin.py", line 141, in get_stats
    stats, java_version = get_stats_from_runtime(m2)
  File "/usr/lib/python2.7/dist-packages/m2ee/munin.py", line 165, in get_stats_from_runtime
    stats.update(m2.client.runtime_statistics(timeout=5))
  File "/usr/lib/python2.7/dist-packages/m2ee/client.py", line 120, in runtime_statistics
    return self.request("runtime_statistics", timeout=timeout)
  File "/usr/lib/python2.7/dist-packages/m2ee/client.py", line 50, in request
    raise M2EERuntimeNotFullyRunning(status, action)
m2ee.client.M2EERuntimeNotFullyRunning: The Mendix Runtime is not fully running, but reporting status 'created'. Unable to execute action runtime_statistics.

Just act like the app isn't running at all, don't explode.

Could not start app

Hi @knorrie

I tried to upgrade m2ee tools to v7.2, I could not start app in the case of not have microflow constants.

image

Thanks,

munin graphing of concurrent users vs named users

Good day,

I have a case were the named users are so much more than the normal concurrent users, that in the munin graphs, the named users are hiding the details of the concurrent user's usage graph, as shown in the attached graph.

mxruntime_sessions_aquacheckweb-day

If I spend the time to fix/change for a pull request, is there any specific way I should split up these data? Is there a use for these together?

I would've liked to have the named users an AREA and then STACK the anonymous connections on top.

m2ee barfing when STDOUT redirected/piped

m2ee about & status both throw TypeError is seems when output is redirected/piped

acw2@agw-acw-mx01-prod ~ % m2ee about
Using m2ee-tools version 7.2.2
Using Mendix Runtime version 8.18.14.33751
Copyright © 2003-2021 Mendix bv. All rights reserved.
Model version: 3.0.0.1432
acw2@agw-acw-mx01-prod ~ % ls
acw2  m2ee-tools-7.2.3  mx-build.sh  rr.awk
acw2@agw-acw-mx01-prod ~ % m2ee status
INFO: The application process is running, the MxRuntime has status: running
INFO: Logged in users: (1) ['Anonymous_24973d9b-35ae-4258-a635-950a920af4a6']
acw2@agw-acw-mx01-prod ~ % m2ee status | grep running
--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3.9/logging/__init__.py", line 1082, in emit
    stream.write(msg + self.terminator)
  File "/usr/lib/python3.9/codecs.py", line 378, in write
    self.stream.write(data)
TypeError: write() argument must be str, not bytes
Call stack:
  File "/usr/local/bin/m2ee", line 1050, in <module>
    main()
  File "/usr/local/bin/m2ee", line 1029, in main
    cli.unchecked_onecmd(' '.join(args.onecmd))
  File "/usr/local/bin/m2ee", line 857, in unchecked_onecmd
    super(CLI, self).onecmd(line)
  File "/usr/lib/python3.9/cmd.py", line 217, in onecmd
    return func(arg)
  File "/usr/local/bin/m2ee", line 283, in do_status
    logger.info("The application process is running, the MxRuntime has status: %s" % status)
Message: 'The application process is running, the MxRuntime has status: running'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3.9/logging/__init__.py", line 1082, in emit
    stream.write(msg + self.terminator)
  File "/usr/lib/python3.9/codecs.py", line 378, in write
    self.stream.write(data)
TypeError: write() argument must be str, not bytes
Call stack:
  File "/usr/local/bin/m2ee", line 1050, in <module>
    main()
  File "/usr/local/bin/m2ee", line 1029, in main
    cli.unchecked_onecmd(' '.join(args.onecmd))
  File "/usr/local/bin/m2ee", line 857, in unchecked_onecmd
    super(CLI, self).onecmd(line)
  File "/usr/lib/python3.9/cmd.py", line 217, in onecmd
    return func(arg)
  File "/usr/local/bin/m2ee", line 294, in do_status
    total_users = self._who(max_show_users)
  File "/usr/local/bin/m2ee", line 815, in _who
    logger.info("Logged in users: (%s) %s" %
Message: "Logged in users: (1) ['Anonymous_24973d9b-35ae-4258-a635-950a920af4a6']"
Arguments: ()
acw2@agw-acw-mx01-prod ~ %
 % m2ee about | cat
Traceback (most recent call last):
  File "/usr/local/bin/m2ee", line 1050, in <module>
    main()
  File "/usr/local/bin/m2ee", line 1029, in main
    cli.unchecked_onecmd(' '.join(args.onecmd))
  File "/usr/local/bin/m2ee", line 857, in unchecked_onecmd
    super(CLI, self).onecmd(line)
  File "/usr/lib/python3.9/cmd.py", line 217, in onecmd
    return func(arg)
  File "/usr/local/bin/m2ee", line 361, in do_about
    print('Using m2ee-tools version %s' % m2ee.__version__)
  File "/usr/lib/python3.9/codecs.py", line 378, in write
    self.stream.write(data)
TypeError: write() argument must be str, not bytes

Multi line variable support in m2ee.yaml?

Good day,

How/which multi-line constant string values are allowed?
Don't see any example/

Have the need to insert a JSON set of values, that would be inserted with jinga2 (from Ansible) template, but the looping makes it a challenge to put in a single JSON line.

The alternative is to have a complex yaml variable, but the developer seems to be insistent that is not possible, only JSON & XML from the from the m2ee.yaml file.

Any other advice directions?

Exception in thread "main" java.nio.NoSuchFileException .... When starting m2ee

INFO: ApINFO: Application Name: My Project Name
INFO: The application process is not running.
m2ee(myapp): start
INFO: Trying to start the MxRuntime...
OpenJDK 64-Bit Server VM warning: Ignoring option MaxPermSize; support was removed in 8.0
Exception in thread "main" java.nio.file.NoSuchFileException: /path/to/project/directory/data/tmp/platform-public___12589025704097317480
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389)
at java.base/java.nio.file.Files.createDirectory(Files.java:690)
at java.base/java.nio.file.TempFileHelper.create(TempFileHelper.java:135)
at java.base/java.nio.file.TempFileHelper.createTempDirectory(TempFileHelper.java:172)
at java.base/java.nio.file.Files.createTempDirectory(Files.java:1007)
at com.mendix.container.deployment.internal.Interpreters$ClassLoaders$$anon$1.(Interpreters.scala:87)
at com.mendix.container.deployment.internal.Interpreters$ClassLoaders$.createClassloader(Interpreters.scala:83)
at com.mendix.container.deployment.internal.Interpreters$ClassLoaders$.$anonfun$newDependency$1(Interpreters.scala:145)
at cats.data.StateFunctions.$anonfun$modify$4(IndexedStateT.scala:363)
at cats.data.StateFunctions.$anonfun$apply$2(IndexedStateT.scala:348)
at scala.Function1.$anonfun$andThen$1(Function1.scala:85)
at scala.Function1.$anonfun$andThen$1(Function1.scala:85)
at cats.data.AndThen.loop$1(AndThen.scala:107)
at cats.data.AndThen.runLoop(AndThen.scala:116)
at cats.data.AndThen.apply(AndThen.scala:68)
at cats.data.IndexedStateT.$anonfun$run$1(IndexedStateT.scala:64)
at cats.Eval$.loop$1(Eval.scala:358)
at cats.Eval$.cats$Eval$$evaluate(Eval.scala:363)
at cats.Eval$FlatMap.value(Eval.scala:284)
at com.mendix.container.deployment.Runners$.runClassloadersHierarchyBuilder(Runners.scala:111)
at com.mendix.container.componentregistry.internal.ClassloaderComponentReader$.buildHierarchy(ClassloaderComponentReader.scala:215)
at com.mendix.container.componentregistry.internal.ClassloaderComponentReader$.read(ClassloaderComponentReader.scala:22)
at com.mendix.container.componentregistry.ComponentRegistry$.run(ComponentRegistry.scala:14)
at com.mendix.container.boot.Container$.doStart(Container.scala:114)
at com.mendix.container.boot.Container$.doStart(Container.scala:97)
at com.mendix.container.boot.Container$.$anonfun$startAndWaitForever$2(Container.scala:74)
at scala.util.Either.flatMap(Either.scala:352)
at com.mendix.container.boot.Container$.$anonfun$startAndWaitForever$1(Container.scala:73)
at com.mendix.container.boot.Container$.time(Container.scala:156)
at com.mendix.container.boot.Container$.startAndWaitForever(Container.scala:73)
at com.mendix.container.boot.Main$.$anonfun$new$2(Main.scala:15)
at scala.util.Either.flatMap(Either.scala:352)
at com.mendix.container.boot.Main$.delayedEndpoint$com$mendix$container$boot$Main$1(Main.scala:14)
at com.mendix.container.boot.Main$delayedInit$body.apply(Main.scala:11)
at scala.Function0.apply$mcV$sp(Function0.scala:39)
at scala.Function0.apply$mcV$sp$(Function0.scala:39)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
at scala.App.$anonfun$main$1(App.scala:76)
at scala.App.$anonfun$main$1$adapted(App.scala:76)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
at scala.collection.AbstractIterable.foreach(Iterable.scala:926)
at scala.App.main(App.scala:76)
at scala.App.main$(App.scala:74)
at com.mendix.container.boot.Main$.main(Main.scala:11)
at com.mendix.container.boot.Main.main(Main.scala)
ERROR: JVM process terminated without reason., errno: 0xa, subprocess output available

plication Name: My Project Name
INFO: The application process is not running.
m2ee(myapp): start
INFO: Trying to start the MxRuntime...
OpenJDK 64-Bit Server VM warning: Ignoring option MaxPermSize; support was removed in 8.0
Exception in thread "main" java.nio.file.NoSuchFileException: /path/to/project/directory/data/tmp/platform-public___12589025704097317480
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389)
at java.base/java.nio.file.Files.createDirectory(Files.java:690)
at java.base/java.nio.file.TempFileHelper.create(TempFileHelper.java:135)
at java.base/java.nio.file.TempFileHelper.createTempDirectory(TempFileHelper.java:172)
at java.base/java.nio.file.Files.createTempDirectory(Files.java:1007)
at com.mendix.container.deployment.internal.Interpreters$ClassLoaders$$anon$1.(Interpreters.scala:87)
at com.mendix.container.deployment.internal.Interpreters$ClassLoaders$.createClassloader(Interpreters.scala:83)
at com.mendix.container.deployment.internal.Interpreters$ClassLoaders$.$anonfun$newDependency$1(Interpreters.scala:145)
at cats.data.StateFunctions.$anonfun$modify$4(IndexedStateT.scala:363)
at cats.data.StateFunctions.$anonfun$apply$2(IndexedStateT.scala:348)
at scala.Function1.$anonfun$andThen$1(Function1.scala:85)
at scala.Function1.$anonfun$andThen$1(Function1.scala:85)
at cats.data.AndThen.loop$1(AndThen.scala:107)
at cats.data.AndThen.runLoop(AndThen.scala:116)
at cats.data.AndThen.apply(AndThen.scala:68)
at cats.data.IndexedStateT.$anonfun$run$1(IndexedStateT.scala:64)
at cats.Eval$.loop$1(Eval.scala:358)
at cats.Eval$.cats$Eval$$evaluate(Eval.scala:363)
at cats.Eval$FlatMap.value(Eval.scala:284)
at com.mendix.container.deployment.Runners$.runClassloadersHierarchyBuilder(Runners.scala:111)
at com.mendix.container.componentregistry.internal.ClassloaderComponentReader$.buildHierarchy(ClassloaderComponentReader.scala:215)
at com.mendix.container.componentregistry.internal.ClassloaderComponentReader$.read(ClassloaderComponentReader.scala:22)
at com.mendix.container.componentregistry.ComponentRegistry$.run(ComponentRegistry.scala:14)
at com.mendix.container.boot.Container$.doStart(Container.scala:114)
at com.mendix.container.boot.Container$.doStart(Container.scala:97)
at com.mendix.container.boot.Container$.$anonfun$startAndWaitForever$2(Container.scala:74)
at scala.util.Either.flatMap(Either.scala:352)
at com.mendix.container.boot.Container$.$anonfun$startAndWaitForever$1(Container.scala:73)
at com.mendix.container.boot.Container$.time(Container.scala:156)
at com.mendix.container.boot.Container$.startAndWaitForever(Container.scala:73)
at com.mendix.container.boot.Main$.$anonfun$new$2(Main.scala:15)
at scala.util.Either.flatMap(Either.scala:352)
at com.mendix.container.boot.Main$.delayedEndpoint$com$mendix$container$boot$Main$1(Main.scala:14)
at com.mendix.container.boot.Main$delayedInit$body.apply(Main.scala:11)
at scala.Function0.apply$mcV$sp(Function0.scala:39)
at scala.Function0.apply$mcV$sp$(Function0.scala:39)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
at scala.App.$anonfun$main$1(App.scala:76)
at scala.App.$anonfun$main$1$adapted(App.scala:76)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
at scala.collection.AbstractIterable.foreach(Iterable.scala:926)
at scala.App.main(App.scala:76)
at scala.App.main$(App.scala:74)
at com.mendix.container.boot.Main$.main(Main.scala:11)
at com.mendix.container.boot.Main.main(Main.scala)
ERROR: JVM process terminated without reason., errno: 0xa, subprocess output available

Please add GPG keys to SKS keyservers

Good day, could you please add the GnuPG keys to the SKS/etc. GnuPG keyservers instead of having to fetch it from the mendix website?

wget -q -O - https://packages.mendix.com/mendix-debian-archive-key.asc | apt-key add -

Clean up configuration handling code

This is about src/m2ee/config.py, which didn't get it fair share of spring cleaning yet in the last years. The problem is not that there are many bugs. The problem is that the code is a mess and hard to work with and change, which is not nice for the developer, and makes it easier to introduce new bugs.

When removing Mendix 2.5 compatibility most of the hacks already got removed, now it the time to clean up the rest.

So:

  • Have a proper in-memory model of settings, not just a yolo style dictionary.
  • Add input validation so invalid settings are detected early without causing unexpected program behavior.
  • Make a clear distinction between just reading the config file and assembling derived config. Currently a lot of things are already done immediately, like setup_classpath (which should be done during start instead).
  • Move config handling that happens outside config.py into it (e.g. conversion of format of params in send_runtime_config in core.py)
  • Transform the get_blah functions into properties. Put the old functions into a corner as deprecated functions (warn when using them, remove in v8).

Fix hanging client actions when runtime does not respond.

Also see #12

Moving over from there:

"Currently if someone will damage their running app so badly that it keeps accepting connections and requests but never answer them, you're also quite out of luck with the CLI. Starting m2ee will just hang on the first runtime_status call, and you won't get your prompt to try stop the app.

I'd propose to change the timeout=None in client.request(action, params, timeout) and set it to a sane low default that should be sufficient to do almost all admin actions, which should return instantly. The other ones which can take longer, like start and stop already define their custom timeouts."

PostgreSQL dumpdb/restoredb/emptydb and stored procedures/functions

Goodday,

I'm in a situation (Mendix 6) where there have been a need to optimize the DB queries to prevent tight loops inside the modeller, and that is/was done using stored procedures.

Now a dumpDB stores those functions/procedures "correctly"
The EmptyDB clears everything except these stored procedure/functions (which isn't quite Mendix's domain I'll concede)
the restoreDB then fails 'cause these functions/procedures pre-exists, and gives an error (and my Ansible scripts fails).

I'm in need of the restoredb to include --clean --if-exists when it executes the pg_restore command.

Could we add these (will create a pull request if acceptable)?
Perhaps a separate option/argument like --force (or part of --yolo ?) to enforce these clean if exists?

ERROR - Core: An exception occurred while running the after-startup-action.

Hello,

I am attempting to setup a Mendix instance for testing purposes. I've managed to set it up on EC2 place the database on on RDS instance. I got a dummy application to start/stop with m2ee-tools and I am able to visit it with a public URL.

Once I moved onto uploading the database and inputing in constants for the actual test application, I started running into issues. I don't recall specifically what the prompt mentioned, but it asked to update the database. There were a few options I tried before saying "Yes" to the update. From there I ran into an error starting the application (after said update)..

Attached the full log file, I'll insert a snippet of where the error starts below.

2018-02-05 17:29:01.887 ERROR - Core: An exception occurred while running the after-startup-action.
2018-02-05 17:29:01.887 ERROR - Core: com.mendix.core.CoreException: com.mendix.modules.microflowengine.MicroflowException: com.mendix.core.CoreRuntimeException: com.mendix.systemwideinterfaces.MendixRuntimeException: java.lang.NullPointerException at RestServicesCustomized.ACT_AfterStartup_InitializeREST (JavaAction : 'StartPublishesServicesJava') at Core.AfterStartup (SubMicroflow : 'Start REST')

Full Advanced stacktrace in attachment:

feb-5-logfile-m2ee.txt

Java: 1.8.0_151

Exception occurred in action '{"xpath":"//System.User[System.Session_User]"

We just upgrade to mx7 and when starting the m2ee tools we always get the below error. It looks as if it is the retrieve is somehow messed up.

`2017-07-27 14:26:52.974 ERROR - M2EE: com.mendix.m2ee.api.AdminException: An unhandled exception occurred!
at com.mendix.m2ee.appcontainer.server.handler.HttpAdminHandler.handle(HttpAdminHandler.java:142)

Caused by: com.mendix.core.CoreRuntimeException: com.mendix.systemwideinterfaces.MendixRuntimeException: com.mendix.core.CoreRuntimeException: Exception occurred in action '{"xpath":"//System.User[System.Session_User]","amount":-1,"depth
":0,"offset":-1,"sort":{},"type":"RetrieveXPathAction"}', all database changes executed by this action were rolled back
at com.mendix.basis.actionmanagement.ActionManager.executeSync(ActionManager.java:178)

Caused by: com.mendix.systemwideinterfaces.MendixRuntimeException: com.mendix.core.CoreRuntimeException: Exception occurred in action '{"xpath":"//System.User[System.Session_User]","amount":-1,"depth":0,"offset":-1,"sort":{},"type":"Retr
ieveXPathAction"}', all database changes executed by this action were rolled back
at com.mendix.util.classloading.Runner.doRunUsingClassLoaderOf(Runner.java:37)

Caused by: com.mendix.core.CoreRuntimeException: Exception occurred in action '{"xpath":"//System.User[System.Session_User]","amount":-1,"depth":0,"offset":-1,"sort":{},"type":"RetrieveXPathAction"}', all database changes executed by thi
s action were rolled back
at com.mendix.basis.actionmanagement.CoreActionHandlerImpl.processErrorState(CoreActionHandlerImpl.scala:152)

Caused by: com.mendix.basis.connectionbus.ConnectionBusException: An exception has occurred for the following request(s):
InternalXPathTextGetRequest (depth = 0): //System.User[System.Session_User]
at com.mendix.connectionbus.RequestAnalyzer.doRequest(RequestAnalyzer.java:49)

Caused by: com.mendix.basis.connectionbus.ConnectionBusException: Exception occurred while retrieving data. (SQL State: 42883, Error Code: 0)
at com.mendix.connectionbus.connections.jdbc.JdbcDataStore.getCorrectException(JdbcDataStore.java:1010)

Caused by: org.postgresql.util.PSQLException: ERROR: operator does not exist: bigint = boolean
Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.
Position: 611
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2412)
`

Allow setting log levels before starting

In Mendix, log categories ("nodes" in the internal pub sub handling) are created when something is logged to them the first time. Since Mendix 6, it's possible to set log levels for nodes that do not exist yet, which makes it possible to set log levels to a value at startup of the application before actually starting to process log messages (the start_logging admin api call).

So, allow configuring log levels in the config file, and have them set up during start.

Previously params for the set_log_level admin call would be {"subscriber": subscriber, "node": node, "level": level}.

Three changes have been made to the API:

  • Instead of "node" and "level", another param "nodes" can be provided, in which case "node" and "level" will be ignored. "nodes" has to be a list of dictionaries containing "name" and "level" as fields.
  • The "subscriber" param can be omitted (or explicitely set to the value "*"), in which case the log level will be set for all active subscribers.
  • A "force" param is added, which changes the behavior if the specified node name is not found: instead of erroring, it will create it.

Since the original set_log_level call is overloaded with new behavior, the caller has to determine if this new functionality is available based on Mendix Version, it's not possible to dynamically find out except for a trial and error method.

What's the idea for m2ee-tools?

  • The existing change log level command stays the same
  • In the config file, we want to be able to provide a mapping of node and level which will be set early.

Related proposal: #6 I'm not going to merge this, since it also starts overloading configuration file structures and internal functions with different sorts of behavior, resulting in a forest of if else.

Implementation proposal:

  • Per logsubscriber section in the config, allow setting "loglevel" which simply contains a mapping of node -> level for all levels that we initially want to set.
  • Use the client set_log_level call to set create these loglevels with the nodes + force param, keep the code path separate from the CLI path.
  • Don't change anything in the CLI code paths.

Publish m2ee python module on pypi

Jouke mentioned it would be nice to be able to pip install m2ee.

Only the m2ee library is in scope here, not all documentation or the cli program.

Non-zero exit status on error

In deployment automation, I want my scripts to halt when an error occurs. The simplest way to do this is to use set -e in bash, but that depends on the command returning a non-zero exit status on errors. m2ee does not currently do this.

apt-key deprecated

According to the documentation the signing key has to be added as follows:

root@mx-precision-5570:/home/reinout# wget -q -O - https://packages.mendix.com/mendix-debian-archive-key.asc | apt-key add -
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).

As you can see it complains that apt-key method is deprecated. Expectation is that the documentation follows the officially supported method.

How do I redirect to HTTPS?

Hello,

Using m2ee, I am not sure how to force https. I have current traffic automatically connecting to http. Is there a way to set it up that any requests to http redirects to https?

I currently have a developer attempting to test REST from one server to another and his application is connecting to http port even though his configuration is pointed at https.

Installing m2ee-tools from source. Unable to look up mendix runtime files because product version is yet unknown. Try unpacking a deployment archive first.

Hi,
We are using CentOS servers, so I have followed m2ee -tools installation from source found here:
https://github.com/mendix/m2ee-tools/blob/develop/doc/non-root-install.md
But when I issue any command I’m getitng:
INFO: Unable to look up mendix runtime files because product version is yet unknown. Try unpacking a deployment archive first.

I have tried to run it with m2ee.yaml from examles-same issue.
m2ee unpack same.

Any hint how to resolve that?

Python 3 compatibility

Let's attempt to make the code python 2 and 3 compatible at the same time. There will probably be headaches, but we can't simply force users to jump to python 3, and other use cases are begging for it.

Specify runtime path instead of give possible locations

For debugging/runtime CI purposes, it would be good to be able to say: use runtime path /srv/app/runtimes/x.y.z instead of "look for the runtime in /srv/app/runtimes/".

Usecase: the runtime wants to run CI on cloud foundry / docker and also ship the runtime dev build along with the MDA.

jvm metrics for mendix 8/java 11 are not retrieved correctly.

In munin.py:get_stats_from_runtime the Java JVM memory values are retrieved from the runtime statistics. The response on Mendix 8 (with Java 11) is this:

{
  "feedback": {
    "sessions": {
      "user_sessions": {
      },
      "named_users": 20,
      "anonymous_sessions": 0,
      "named_user_sessions": 2
    },
    "cache": {
      "total_count": 0
    },
    "memory": {
      "used_nonheap": 254877936,
      "init_nonheap": 7667712,
      "max_nonheap": 780140544,
      "committed_nonheap": 272007168,
      "init_heap": 1073741824,
      "memorypools": [
        {
          "is_heap": false,
          "usage": 1425280,
          "name": "CodeHeap 'non-nmethods'",
          "index": 0
        },
        {
          "is_heap": false,
          "usage": 140633704,
          "name": "Metaspace",
          "index": 1
        },
        {
          "is_heap": true,
          "usage": 458138760,
          "name": "Tenured Gen",
          "index": 2
        },
        {
          "is_heap": false,
          "usage": 52676096,
          "name": "CodeHeap 'profiled nmethods'",
          "index": 3
        },
        {
          "is_heap": true,
          "usage": 270960112,
          "name": "Eden Space",
          "index": 4
        },
        {
          "is_heap": true,
          "usage": 8968920,
          "name": "Survivor Space",
          "index": 5
        },
        {
          "is_heap": false,
          "usage": 18281096,
          "name": "Compressed Class Space",
          "index": 6
        },
        {
          "is_heap": false,
          "usage": 41861760,
          "name": "CodeHeap 'non-profiled nmethods'",
          "index": 7
        }
      ],
      "committed_heap": 1037959168,
      "max_heap": 1037959168,
      "used_heap": 738067792
    },
    "languages": [
      "en_GB",
      "nl_NL"
    ],
    "entities": 604,
    "requests": [
      {
        "last_request_timestamp": 1581619663617,
        "name": "",
        "value": 1186
      },
      {
        "last_request_timestamp": 1581547423367,
        "name": "n/",
        "value": 0
      },
      {
        "last_request_timestamp": 1581547423093,
        "name": "ws-doc/",
        "value": 0
      },
      {
        "last_request_timestamp": 1581547423166,
        "name": "eba/",
        "value": 0
      },
      {
        "last_request_timestamp": 1581547423155,
        "name": "api-doc/",
        "value": 0
      },
      {
        "last_request_timestamp": 1581547423258,
        "name": "rest-doc/",
        "value": 0
      },
      {
        "last_request_timestamp": 1581547423168,
        "name": "odata-doc/",
        "value": 0
      },
      {
        "last_request_timestamp": 1581615172790,
        "name": "rest/",
        "value": 1332
      },
      {
        "last_request_timestamp": 1581547423084,
        "name": "ws/",
        "value": 0
      },
      {
        "last_request_timestamp": 1581614405199,
        "name": "file",
        "value": 150
      },
      {
        "last_request_timestamp": 1581547421263,
        "name": "link/",
        "value": 0
      },
      {
        "last_request_timestamp": 1581547423385,
        "name": "debugger/",
        "value": 0
      },
      {
        "last_request_timestamp": 1581620021125,
        "name": "xas/",
        "value": 85165
      },
      {
        "last_request_timestamp": 1581616818238,
        "name": "p/",
        "value": 27
      }
    ],
    "connectionbus": {
      "select": 8147638,
      "insert": 123969,
      "update": 98424,
      "delete": 132182,
      "transaction": 49484
    }
  },
  "result": 0
}

The methods make a decision based on the existence of memory.memorypools which exists and then retrieves values from the response. But, comparing the variable names with the assignment we see a mismatch.

M2EE Runtime response
stats['memory']['code'] = memorypools[0]['usage'] "CodeHeap 'non-nmethods'"
stats['memory']['permanent'] = memorypools[2]['usage'] "Tenured Gen"
stats['memory']['eden'] = memorypools[3]['usage'] "CodeHeap 'profiled nmethods'"
stats['memory']['survivor'] = memorypools[4]['usage'] "Eden Space"
stats['memory']['tenured'] = memorypools[5]['usage'] "Survivor Space"

This results in internal ticket DEP-2484.

Improve handling of constants (defaults, overview, warnings)

When a constant is not explicitely configured, we should use the default value which is set in the project. By doing so, we avoid the situation of not being able to start because a constant value is missing.

However, the user might quickly want to see for which constant the default value is being used, so add some sort of check_constants command to give a nice overview (wich values), grouped in 1. explicitely configured constants 2. constants for which the default value is used 3. constants which are configured, but are not present (any more?) in the current application.

Also, stop spamming the user with this info directly at startup of the program (warn_constants).

"Modern" metrics collections (ala Netdata)

Good day,
Years ago (as in 2014 when I first got to administer a Mendix stack), Munin was the thing, and though it's not bad per se, I've been busy deploying https://Netdata.cloud as the next big thing and one of the "funs" for me is the Mendix stacks I manage, so I started to consider to import the stats into Netdata as discussed here https://learn.netdata.cloud/docs/agent/collectors/python.d.plugin but then I started to think and my questions:

  • Is there a "simple" method to generically extract the stats directly from the Mendix admin socket? (ie. without the Python?) This might be the "beterer" method, as I can then point my GoLang colleague in the direction of the Admin port's stats documentation (which I haven't yet found) and life would be great again especially as we could make this a Prometheus type endpoint as an encore (Actually if the Mendix itself was a Prometheus endpoint, that would've been more awesome ;) )

Failing that "simple"/documented/etc. API to the stats, I see the need for a plugin, but that plugin would then be either:
(a) a major hack to parse Munin output of m2ee (okay, that might be a general help for more munin people, but that feels just like a yet another trailer at the back that can fail type solution) or
(b) I integrate it as part of the m2ee and provide a PR for you to integrate in m2ee-tools

  • Which route would Mendix etc. advise me to follow?

In the case of (b), any advice specifics to follow?

m2ee 7.2 won't start when custom configuration option '-c' is used and default ~/.m2ee/m2ee.yaml is missing

Typical m2ee -c m2ee.yaml output when you don't also have a ~/.m2ee/m2ee.yaml file:

WARNING: Error reading configuration file ['[...non-default path...]/m2ee.yaml']: coercing to Unicode: need string or buffer, list found, ignoring...
CRITICAL: No configuration present. Please put a m2ee.yaml configuration file at the default location ~/.m2ee/m2ee.yaml or specify an alternate configuration file using the -c option.

I verified 7.1 does work.

'isProduction = false' at information_license

isProduciton

Hi,

so, basically after i installed and activate license on my app
i was wondering that if the information about isProduction means anything?

because at the Runtime mode it s valued Production but at field isProduction its false

Cannot run 2 mendix apps on linux

I have install m2ee-Tools on debian and centos7 (non-root install) but when I add second application, it will run into java error.

How can I run 2 mendix apps on a *nix server

Connection to the database is failing

Hi, I'm trying to create a Mendix project and run it with minikube. I was able to successfully run the example mendix starter here https://github.com/pommi/mendix-docker with docker but when I try to run and deploy it locally, "m2ee start" hangs and the application never starts. I checked app.log and this is what I'm seeing:

018-03-28 16:53:41.640 INFO - Logging: Logging to file: /root/data/log/app.log, max size: 10240KiB, max rotation: 10
2018-03-28 16:53:41.667 INFO - Core: Mendix Runtime 5.13.1 (build 3976). Copyright © 2003-2015 Mendix bv. All rights reserved.
2018-03-28 16:53:42.371 WARNING - ConnectionBus: Database connections: 0 active, 0 idle.
2018-03-28 16:53:42.371 ERROR - ConnectionBus: Opening JDBC connection to db:5432 failed with SQLState: 08001 Error code: 0 Message: The connection attempt failed. Retrying...(1/4)
2018-03-28 16:53:42.473 WARNING - ConnectionBus: Database connections: 0 active, 0 idle.
2018-03-28 16:53:42.473 ERROR - ConnectionBus: Opening JDBC connection to db:5432 failed with SQLState: 08001 Error code: 0 Message: The connection attempt failed. Retrying...(2/4)
2018-03-28 16:53:42.674 WARNING - ConnectionBus: Database connections: 0 active, 0 idle.
2018-03-28 16:53:42.674 ERROR - ConnectionBus: Opening JDBC connection to db:5432 failed with SQLState: 08001 Error code: 0 Message: The connection attempt failed. Retrying...(3/4)
2018-03-28 16:53:43.075 WARNING - ConnectionBus: Database connections: 0 active, 0 idle.
2018-03-28 16:53:43.075 ERROR - ConnectionBus: Opening JDBC connection to db:5432 failed with SQLState: 08001 Error code: 0 Message: The connection attempt failed., giving up...(4/4)

My yaml's and Dockerfile are based off of this project https://github.com/mendix/azure-docker-mendix/tree/master/Docker-Azure

m2ee (adjust) loglevel does not work if the lognodename has a space in it

Today I was trying to set the loglevel for the following lognodename: REST Consume
However, the Python cli input validation/parsing for the m2ee cli doesn't seem to like this, no matter what I do, it doesn't accept it. I tried adding it as displayed, with ' and with " but to no avail.

m2ee(tr10000): loglevel
Current loglevels:

syslogsubscriber RestConsume INFO

To adjust loglevels, use: loglevel <subscribername> <lognodename> <level>
Available levels: NONE, CRITICAL, ERROR, WARNING, INFO, DEBUG, TRACE
m2ee(tr10000): loglevel syslogsubscriber REST Consume TRACE
To adjust loglevels, use: loglevel <subscribername> <lognodename> <level>
Available levels: NONE, CRITICAL, ERROR, WARNING, INFO, DEBUG, TRACE

m2ee(tr10000): loglevel syslogsubscriber 'REST Consume' TRACE
To adjust loglevels, use: loglevel <subscribername> <lognodename> <level>
Available levels: NONE, CRITICAL, ERROR, WARNING, INFO, DEBUG, TRACE

m2ee(tr10000): loglevel syslogsubscriber "REST Consume" TRACE
To adjust loglevels, use: loglevel <subscribername> <lognodename> <level>
Available levels: NONE, CRITICAL, ERROR, WARNING, INFO, DEBUG, TRACE

Ideally, your lognodename should probably not contain spaces, but I can not imagine that I am the only one that experiences this. The reason why I suspect this is the cli input validation/parsing is that I think it would result in perfectly fine JSON from what I can tell, however you never get to that point:

$ m2ee -vvvv
m2ee(tr10000): loglevel syslogsubscriber RestConsume TRACE
TRACE: Executing command: loglevel syslogsubscriber REST Consume TRACE
TRACE: M2EE request body: {"action": "set_log_level", "params": {"subscriber": "syslogsubscriber", "node": "REST Consume", "level": "TRACE"}}
TRACE: M2EE response: {"feedback":{},"result":0}
INFO: Loglevel for RestConsume set to TRACE

m2ee start - timeout setting??

Goodday,

so, busy with Ansible deploys, and I have errors with database connectivity, which causes the m2ee start to hang indefinitely.

The errors that scrolls through the log file:

2018-03-26 15:52:33.022 WARNING - ConnectionBus: Database connections: 0 active, 0 idle.
2018-03-26 15:52:33.022 ERROR - ConnectionBus: Opening JDBC connection to Some(192.168.200.22:5432) failed with SQLState: 08001 Error code: 0 Message: Connection to 192.168.200.22:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections., giving up...(4/4)

Yes, I also have a 2nd JDBC connection (where the Mendix App in this case fetches data to populate the "main"/modeller database).

What I'm in need of, is a sorta time-out for the start to fail with some errors if the Mendix doesn't start up in time.
This might be to consider in the light of things like model (database structure) changes that does take a long(er) time to complete...

Add possibility to use units (ms,s,min,h,d) for time length values

Some configuration options take a number representing time length, e.g. ClientQueryTimeout or LogMinDurationQuery. Some of them want a value in seconds, some in milliseconds, etc. This is confusing for the user.

Allow using more intuitive values, like 30ms or 10s instead.

Take PostgreSQL as an example to copy from:
ms = milliseconds
s = seconds
min = minutes
h = hours
d = days

Capture JVM startup fail stdout/stderr when starting detached

When starting an application in the background, this can fail during the phase before we have a working admin port. It should not happen, but sometimes it does, thanks to a bug or I've it also seen happen when a disk is full.

When running m2ee interactively, the output is dumped on stdout/stderr and visible because the file descriptors are still inherited, and the intermediate process didn't detach and call close_stdio yet. However, when starting from an unattended program (import m2ee; etc...) we also want to be able to intercept this output.

So:

  • intercept stdout / stderr from the JVM project during early startup
  • if stdout is a tty, just also copy it there (unbuffered).
  • if stderr is a tty, just also copy it there (unbuffered).
  • where to store the output is TBD, maybe extend M2EEException with some details attribute or feedback-like nested structure

download_runtime SSLHandshakeError

Hello,

Attempting to run m2ee download_runtime I get the following error:

ERROR: Checking download url https://download.mendix.com/runtimes/mendix-7.23.8.58888.tar.gz failed: SSLHandshakeError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)

What's bizarre is I can easily do a

wget https://download.mendix.com/runtimes/mendix-7.23.8.58888.tar.gz

and the package will download into my ~/ directory.

Is there a work around for this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.