tcalmant / ipopo Goto Github PK

A report shell command could be used to generate a full report of the current state of the framework, containing:

• OS description (using os.name and the platform module)
• Python description (version, path to the interpreter)
• Current state of Pelix (list installed bundles, with state, version and path)
• Current state of iPOPO (list of instances, with factory name, instance name and source bundle)
• Memory consumption (if possible)

The report might be generated for different levels of information (e.g. small, framework, memory, full, ...).
The generated report might written to the standard output or to a file. It could also be sent through a HTTP POST request or via an email (configuration to be specified...)

The MQTT service provided by pelix.services.mqtt is not really usable as is.
Its API must be redefined in order to support real use cases or it will be removed.

It should also use the MQTT client class instead of having a parallel implementation.

Prepare a directory associating a framework UID with its host address.

That way, a discovery protocol will be able to send multiple host addresses to access the framework, and the "Framework Directory" will select the first working one or the last one used.

This shall allow to use the MQTT discovery with protocols other than MQTT-RPC.

Add pwd and cd methods to change of working directory

Support for Python 2.6 forces to use ugly tricks to ensure compatibility with this version.
It also causes a lot of trouble to correct bugs, as each correction must then be rewritten to work with Python 2.6.

Same as @requires for single (non-aggregate) dependencies, but ensures that the injected service will always be the "best" one, according to service ranking and registration orders.

The injected service will always be the one with the highest ranking (greatest service ranking integer).

When updating the "service.ranklng" property of a service, the service registry doesn't re-sort the service references.
As a result, the reference returned by get_service_reference() is not the expected one.

Pelix should override the import method when installing a bundle, in order to avoid duplicated imports and to have a better control on the Python path.

The code could be inspired of the "exocet" project (https://launchpad.net/exocet/).

Allow a component to inherit some of the services provided by its parent class (a component or a decorated class).
The services included or excluded from the inheritance should be listed in the @ComponentFactory or an @InheritProvides or an @InheritHandler decorator.

When instantiating an HTTP service component using the instantiate shell command, the port property must be converted to an integer, else Python 3 fails with the error below.
Also, the invalidate method must be protected to avoid using a None reference.

$ instantiate pelix.http.service.basic.factory httpd pelix.http.port=9000
Component 'httpd' : error calling callback method for event VALIDATE
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\pelix\ipopo\instance.py", line 522, in safe_callback
    return self.__callback(event, *args, **kwargs)
  File "C:\Python34\lib\site-packages\pelix\ipopo\instance.py", line 472, in __callback
    result = comp_callback(self.instance, *args, **kwargs)
  File "C:\Python34\lib\site-packages\pelix\http\basic.py", line 796, in validate
    if self._port is None or self._port < 0:
TypeError: unorderable types: str() < int()
Component 'httpd' : error calling callback method for event INVALIDATE
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\pelix\ipopo\instance.py", line 522, in safe_callback
    return self.__callback(event, *args, **kwargs)
  File "C:\Python34\lib\site-packages\pelix\ipopo\instance.py", line 472, in __callback
    result = comp_callback(self.instance, *args, **kwargs)
  File "C:\Python34\lib\site-packages\pelix\http\basic.py", line 867, in invalidate
    self._server.shutdown()
AttributeError: 'NoneType' object has no attribute 'shutdown'
Component 'httpd' instantiated.

See https://ipopo.coderxpress.net/wiki/doku.php?id=contrib:shell_v2

The ThreadPool class has been copied and enhanced in the jsonrpclib-pelix project.
Its code and tests should be backported in iPOPO.

Add a decorator replacing either @ComponentFactory or @Instantiate, to ensure that one and only one component from this factory is instantiated at a time.

When an exported service is unregistered or when its endpoint name is updated, the dispatcher should look for services which export was refused due to an endpoint name conflict.
For example, if a service S1 and a service S2 require the endpoint name E1, S2 must be exported with this name as soon as S1 is unregistered or requires another endpoint name.

The options of .coveragerc have changed with coverage 4.0.
The exclude option seems to have been renamed:

py27 installed: coverage==4.0,iPOPO==0.6.3,jsonrpclib-pelix==0.2.6,nose==1.3.7,wheel==0.24.0
[...]
coverage.misc.CoverageException: Unrecognized option '[report] exclude=' in config file .coveragerc

This causes Travis-CI builds to fail.

Support a service property to define synonyms in order to be compatible with cohorte/cohorte-remote-services#4.

Keep in memory the full stack trace, as a string, of the exception raised during component validation.
It should be accessible using the instance or a new shell command.

If two bundles instantiate components of factories with the same name, no error is raised and only the first component factory will be used.

It seems that the https://github.com/tcalmant/demo-ipopo-nao project has problems with the MQTT configuration process, based on ConfigurationAdmin.

The demo fails to start because the MQTT component is not configured.
The configuration admin service prints the following line:
PID error: mqtt-connector -- <internals.mqtt.MqttConnector object at ...>

If a Pelix framework is started using start-stop-daemon with the pelix.shell.console bundle, the latter stops the framework immediately as it detects an EOF in the standard input.

The bundle should at least check if it is run as a daemon or log a warning before stopping the framework.

Current implementation only took into account part of the Remote Services specification (Chapter 100 of OSGi Compendium Specifications).

A new iteration should be made to comply with both Remote Services and Remote Services Admin specifications (Chapter 122).

Instead of letting the multicast discovery service send a request to the remote dispatcher servlet, the local dispatcher access service (provided by the servlet) should do the communication itself.

Implement a basic support for XML-RPC (using xmlrpclib) and JSON-RPC (using jsonrpclib or a fork) for exported services, i.e. with the service.exported property.

The implementations might be provided as servlets for the HTTP service.

Implement a Remote Service Discovery provider, which could work as a centralized discovery protocol.

It would also be interesting to try to implement a JSON-RPC-like transport through MQTT.

In most cases, it is acceptable to replace an injected service with a new one without restarting the client component.
For example, C depends on a service provided by A and B, and A is injected in C.
When A is invalidated, B could be immediately injected in C, without invalidating C.

Shell sample:

$ install pelix_boot                                                                           
Bundle ID: 12                                                                                  
$ uninstall 12                                                                                 
Uninstalling bundle 12...                                                                      
ERROR:pelix.shell.core:Error calling default.uninstall: need more than 1 value to unpack       
Traceback (most recent call last):                                                             
  File "c:\users\thomas\documents\git\ipopo\pelix\shell\core.py", line 604, in execute         
    result = method(session, *args, **kwargs)                                                  
  File "c:\users\thomas\documents\git\ipopo\pelix\shell\core.py", line 1381, in uninstall      
    bundle.uninstall()                                                                         
  File "c:\users\thomas\documents\git\ipopo\pelix\framework.py", line 423, in uninstall        
    self.__framework.uninstall_bundle(self)                                                    
  File "c:\users\thomas\documents\git\ipopo\pelix\framework.py", line 1098, in uninstall_bundle
    parent, basename = name.rsplit('.', 1)                                                     
ValueError: need more than 1 value to unpack                                                   

sys._current_frames() is not available in Jython, therefore threads and thread shell commands fail.

$ threads
ERROR:pelix.shell.core:Error calling default.threads: '<reflected field public org.python.core.PyObject o' object has no attribute '_current_frames'
Traceback (most recent call last):
  File "C:\jython2.7b3\Lib\site-packages\pelix\shell\core.py", line 703, in execute
    result = method(io_handler, *args, **kwargs)
  File "C:\jython2.7b3\Lib\site-packages\pelix\shell\core.py", line 1152, in threads_list
    frames = sys._current_frames()
AttributeError: '<reflected field public org.python.core.PyObject o' object has no attribute '_current_frames'

When a handler event occurs, e.g. when a controller changes state, all the handlers are called back, whereas only those of "service_provider" kind should.

Proposal: the StoredInstance.__safe_handlers_callback() method should have the target "kind" as first parameter to select those specific handlers.
If this kind is None, then all handlers must be notified.

It seems possible to implement an HTTP service using Tornado.
This should increase the performances of Pelix HTTP (Tornado is faster than the HTTP server from the standard library).
It should also bring new features like websockets support.

An experimental implementation could be written before the release of 0.6.3.

Increase the number and the quality of unit tests.
Mainly on Pelix service handling and on dependency injection.

Update the documentation and tutorials at https://ipopo.coderxpress.net/

• Update docs & tutorials with the new APIs (@BundleActivator, iPOPO waiting list, ...)
• Tutorials for new services (MQTT, ConfigAdmin, ...)
• Rewrite the Eclipse IoT day page: link to code, Youtube videos, ...
• Review pages style

Currently, Pelix Remote Services supports the pelix.remote.export.reject property to reject the export of a specific interface when using remote services, even if it has been indicated in the service.exported.interfaces property.

The main use case is when a component provides multiple services but disallows the export of only some of them, like local-only services (shell commands, ...).
This is useful when a component is instantiated manually, with the services.exported.interfaces property set to * (all).

Two other properties should be defined to ease the description of those constraints:

• pelix.remote.export.none: no export is allowed for this component, even if the instantiator says otherwise.
• pelix.remote.export.only: this is the opposite of reject. Only interfaces listed in this property can be exported.

pelix.remote.export.none is the top-priority property: when set all others are ignored.
If both export.onlyand export.reject properties are set, the exported interfaces those in export.only minus those of export.reject.
if the result is an empty set, then the export is aborted.

The LDAP Filter module (pelix/ldapfilter.py) is the slowest part of the code of iPOPO.
As it is often used, mainly for service filtering, it should be way faster.

According to line profiling, the evaluation of an LDAP filter takes more than 70% of the call time when using the get_service_references() method.

The @PostRegistration decorated method would be called right after a service has been registered by the @Provides handler.
This would allow to configure the component in @Validate, to register the service, then to start threads/handlers which require that the services to be provided.

It seems that the "try-except ImportError" trick to have a code working with both Python 2 and Python 3 doesn't work well with IronPython.

It seems the except block should intercept both ImportError and AttributeError exceptions, as shows this trace, running "run_remote.py" :

Traceback (most recent call last):
  File "C:\Program Files (x86)\IronPython 2.7\Lib\importlib\__init__.py", line 37, in import_module
  File "run_remote.py", line 316, in <module>
  File "run_remote.py", line 203, in main
  File "C:\Program Files (x86)\IronPython 2.7\lib\site-packages\pelix\framework.py", line 1528, in create_framework
  File "C:\Program Files (x86)\IronPython 2.7\lib\site-packages\pelix\framework.py", line 1337, in install_bundle
  File "C:\Program Files (x86)\IronPython 2.7\lib\site-packages\pelix\framework.py", line 728, in install_bundle
  File "C:\Program Files (x86)\IronPython 2.7\lib\site-packages\pelix\remote\dispatcher.py", line 60, in <module>
AttributeError: 'LightException' object has no attribute 'client'

Let the instantiator of the HTTP server be able to configure the request_queue_size server class member.

It should act like the @Temporal handler of iPOJO.

It has the same signature as @requires, but it behaves differently: when calling a dependency that has disappeared, the called will be paused until a new dependency is injected or a timeout exception will be raised.

• Update the tutorials, according to the new API and capacities
• Add a page about Pelix and iPOPO internals (for developers)

If an exception is raised during a call to the methods decorated with @Validate or @Invalidate, the state seen by iPOPO is kept as is, i.e. "Validating" or "Invalidating".

These components should be considered in either an "invalid" or "error" state, to reflect the problem.

The current README file is not informative nor attractive enough.

See matiassingers/awesome-readme for inspiration

pelix.shell.console is the script executed by most of Pelix/iPOPO users to quickly test their bundles/
Therefore it should accept some arguments to ease those tests.

Here are some ideas:

• --version: prints the version of Pelix and its installation folder, then quits returning 0;
• --init <script>: runs the given script file (in fact, calls run <file>) before accepting shell input;
• --run <script>: runs the given script file then exits;

When calling BundleContext.getService(ref), where ref is a ServiceReference to a service factory, then the framework should return the result of the get_service() method of this factory.
(See the OSGi specification for more information on this concept : http://www.osgi.org/javadoc/r4v43/core/org/osgi/framework/ServiceFactory.html).

In Pelix, this could be implemented by using a pelix.service.factory service property, and by modifying the API :

• get_service_reference(clazz, ldap_filter, prefer_factories=False):

  if prefer_factories is True, services factories are sorted separately and placed at the beginning of the result list.

• get_service(reference, factoryParameters=None):

  if the given reference points to a service factory, then the given parameters will be forwarded to its get_service() method; else the second argument will be ignored.

In pelix.http.__init__.py , if no meme-type is given to the function send_content, the content-type of the response will always be sent with the value text/html.

def send_content(self, http_code, content, mime_type="text/html", http_message=None, content_length=-1):
    ...
    self.set_response(http_code, http_message)
    if mime_type:
        self.set_header("content-type", mime_type)
    ...

So if it was set prior to the call of that function, the value will be overriden.

When two servlets services are to be bound on a same path but on two different HTTP services, the bound_to() method is not sufficient to have a clean behaviour: an exception will be raised when the second servlet path will be tested.

The solution could be:
1/ let the servlet register itself in the bound_to() method, if it accepts the server description
2/ let the servlet implement a new method (e.g. accept_server()), called before the servlet path check.

When trying to start the Remote Shell on Windows, the following error occurs:

ERROR:InstanceManager-rshell:Component 'rshell' : error calling callback method for event VALIDATE
Traceback (most recent call last):
  File "C:\Users\Thomas Calmant\Programmation\git\ipopo\pelix\ipopo\instance.py", line 507, in __safe_callback
    return self.__callback(event, *args, **kwargs)
  File "C:\Users\Thomas Calmant\Programmation\git\ipopo\pelix\ipopo\instance.py", line 455, in __callback
    result = comp_callback(self.instance, *args, **kwargs)
  File "C:\Users\Thomas Calmant\Programmation\git\ipopo\pelix\shell\remote.py", line 359, in validate
    _create_server(self, self._address, self._port)
  File "C:\Users\Thomas Calmant\Programmation\git\ipopo\pelix\shell\remote.py", line 249, in _create_server
    server = ThreadingTCPServerFamily((server_address, port), request_handler)
  File "C:\Users\Thomas Calmant\Programmation\git\ipopo\pelix\shell\remote.py", line 228, in __init__
    self.socket.setsockopt(socket.IPPROTO_IPV6, opt_ipv6_only, 0)
AttributeError: 'module' object has no attribute 'IPPROTO_IPV6'

When using a filter in @Requires, it would be interesting to have use a property variable to have an instance-dependent filter.

Something like:

@ComponentFactory()
@Property('_prop_field', 'my.property')
@Requires('_dependency', 'dep_spec', ldap_filter="(svc.value=${my.property})")
...

iPOPO should support a new kind of handlers, which could be added to live instance managers at runtime.
There activation couid be requested in two ways:

• the component indicates that it accepts to be handled by an optional handler: if it's here, the handler is bound to the component, else the component is started as normal.
• the handler indicates the components it wants to be bound to.

The idea is to enable optional non-functional properties handling using iPOPO handlers.
For example, the logger sample handler could be enabled for all components as soon as its bundle is started, but it must not prevent the instantiation of components if it is absent.

As they can be enabled after the instantiation of the component, and event after its validation, those handlers should not be able to manipulate the component.

Feature release of iPOPO

• New Remote Services implementation, closer to the OSGi specification and compatible with ECF (with mDNS and Jabsorb-RPC)
• New iPOPO "Waiting List" service, instantiating components as soon as their factories appear
• New iPOPO handler @RequiresMap
• First implementation of an MQTT client service
• New @BundleActivator decorator for Pelix bundles
• Bug fixes in both Pelix and iPOPO

The @Requires injects lists in instances when working with aggregated dependencies.
When a new dependency is found, it is appended to the list, which therefore not sorted according to service ranking, but keeps item in place.

We should check if using collections.deque would be more efficient, as it is optimized for right-append operations.