roflincopter / dmp

Distributed Music Player

DMP

Distributed Music Player

The network protocol

Each message passed over the TCP control connection has the following format

A network packet
1 byte	4 bytes	1 byte	4 bytes	size bytes
encryption sentinel	possibly encrypted message::Type	encryption sentinel	size of message uint32_t	possibly encrypted message

• The encryption sentinels are either 0 or 1, for unencrypted and encrypted respectively.
• The Type is a uint32_t which can be casted to a message::Type
• The size of the message is a uint32_t and is the size of the message in bytes.
• The message is a custom Archive format, which serializes structs with boost::fusion.

For the time being all these components are sent separately resulting in 5 network packets. This is somewhat inefficient, it could be reduced quite easily to 1 packet when sending unencrypted and 2 packets when sending encrypted, due to way messages are currently handled. But it's not a priority until this becomes a performance bottle neck.

The serialization protocol

The Serialization archive format is straight forward. For each member of a struct serialize it members from top to bottom.

If the variable is

• a char, uint8_t int8_t cast it to a wider type and write it to avoid non printable characters.
• any other primitive, write the primitive.
• a string, write the length of the string followed by a space and the string itself.
• a container, write the size_t number of elements in the container followed by a space and serialize the elements.
• a pair or tuple, serialize all elements in order, space separated.

separate each type you have serialized with a space.

The encryption

DMP uses libsodium for encryption and password hashing for more information regarding the encryption used in the protocol specified above, please refer to libsodium

Building on Mac OSX

First we install the dependencies (for the client)

brew install icu4c taglib boost qt5 gstreamer libsodium gst-plugins-good
brew install gst-plugins-ugly --build-from-source

Then we can run cmake as usual. BUT: linking qt5 does not work out of the box (there is no macosx-clang binary in /usr/local/?). But one can set the following cmake options:

Qt5Core_DIR      /usr/local/Cellar/qt5/5.4.1/lib/cmake/Qt5Core
Qt5Gui_DIR       /usr/local/Cellar/qt5/5.4.1/lib/cmake/Qt5Gui
Qt5Widgets_DIR   /usr/local/Cellar/qt5/5.4.1/lib/cmake/Qt5Widgets

dmp's Issues

{n,}curses gui

For anybody up to the challange, make a TUI for those people who would want to run this in a terminal. This also gives us an idea if the models and communication between client and UI is generic enough or if there are some hidden assumptions on the capabilities of the UI.

DmpCallbacks and DmpConnection are becoming too tightly coupled.

Today I ran into a bug that was indeterministically crashing my client. with a boost serialization exception "invalid signature". This most of time indicates some sort of corruption of the boost::asio::tcp::socket, first and foremost a "double" call to async_receive. But this time it happened because the DmpCallbacks was not past as a reference.

The connection should not have these weird limitations but I cannot think of a better way to do this. The DmpCallbacks class in conjunction with MessageSwitch has saved me considerable dev time because they force you to implement a lot of necessary boilerplate. Without these the program would crash or become quiescent.

So on the one hand I want to keep it just they way it is, but on the other hand, it introduces a hidden requirement on the Connection class.

Moving the searchbar model to the client broke the "syntax error coloring"

Normalize filter input

I think it's better to normalize the input of 'contains' and such a bit, to work around issues where you'd expect something would match but it doesn't. Exact matching could perhaps be enforced with artist contains exact("2Pac").

Pausing and unpausing a song does not work

I have no clue as what is going wrong with the gstreamer pipelines but they are not resuming correctly when unpausing the sender. I suspect the TcpClientSink just gives up on us, but that's just a suspicion at the moment.

Use boost::serialization version facility for dmp_library cache files.

Everytime a member of either the dmp_library::Library or dmp_library::LibraryEntry changes the program will crash and you have to manually delete the cache file. This can be fixed by using the boost::serialisation versioning facility.

Remove using namespaces in libdmp-library

I thought having using namespaces would make the code look more elegant but it has kinda backfired as I cannot use the dmp_library namespace without using the boost and std namespace. I should have used type aliases, much cleaner and faster.

Client hangs on joining client thread after trying to create a radio (failed serverside with #13)

Operation not permitted

Current version can trigger an "Operation not permitted" in the server when queuing a song. This is most likely a race condition, but I really wanted to commit this first before starting to fix it.

Cross compile XDispatch with mxe for x86_64-w64-mingw32

The Devs dropped support for the mingw32 64 bits builds because of issues they had with atomics. This might be resolved now so we should check if we can make it work.

Find out how Qt handles it icons.

I have tried to find an add and delete button for my add_radio and delete_radio buttons. But to no avail. There is the possibility to add your own custom resources and use them for icons and such, but i was hoping to use the Qt standard ones for a standardized look and feel on all systems.

So either this isn't possible and a resource bundle has to be created and integrated or the code should be adapted to use Qts standard icons.

Clean up inclusions

There are a couple of files that include too many headers, as they were rewritten a couple of times and were not cleaned up accordingly. It not that important but it makes the code a bit cleaner especially in boost::fusion and boost::asio heavy classes.

Investigate if get_nth can return the original type.

I have the feeling I can improve upon the get_nth function in the static dispatch. making returning variants and applying visitors. This way I can couple vieuw to certain data types for instance the length/duration field in the dmp-library. This should not be a flat string with seconds but should be converted by Qt into a duration like representation. This can only be done if I have some original knowledge of the type. Boost fusion still knows the type but i throw it away when getting the nth member for convenience.

The playlist type in libdmpcommon:message.hpp should become It's own class

There are 2 places where this type is magically introduced in the code. This should be fixed by creating a class. That contains the same type but more preferably a struct and a vector of structs to also remove the magic std::get(tuple) code that is used in the dmp_radio.{hpp,cpp}.

Move the models to the dmp-client.

The models are now owned by the UI classes, this is not desirable.

Rename DmpClientUiDelegate to DmpClientDelegate

The class has nothing to do with the UI (besides being useful there).

Show all music when nothing is searched for

The search now acts as a filter. Without a filter, dmp should show everything, not nothing.

Link the gstreamer plugins static in the windows build

Linking the gstreamer plugins statically in the windows build will reduce the dependencies.

Remove Dmp-prefix, use namespaces?

There are several classes prefixed with Dmp. While useful, wouldn't it be nicer if they were put into a namespace? Allows you to have a using namespace.

Replace all std::thread with boost::thread.

Std::thread is not supported by the current MXE builds as they are simply not implemented yet. Boost threads are available though. Places where threading code cannot (easily) be replaced with Xdispatch code should be replaced with boost::thread.

blocks on #4

nullptr when dmp-client thinks server is still playing, but isn't

Reproduce

• Setup client
• Create a radio
• Queue a song on that radio
• Tunein on that radio
• Tuneout
• Wait for the song to finish
• Tunein, client will think is still playing
• Press play/pause twice.

Stacktrace

ASAN:SIGSEGV
=================================================================
==29063==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000abb4e7 bp 0x7fffe9848810 sp 0x7fffe98486e0 T0)
    #0 0xabb4e6 in DmpClientGui::PlayPauseToggled(bool) /tmp/../qt-client/dmp_client_gui.cpp:206:3
    #1 0xb9c066 in DmpClientGui::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) /tmp/qt-client/moc_dmp_client_gui.cpp:96:17
    #2 0x7f4054be240c in QMetaObject::activate(QObject*, int, int, void**) (/usr/lib/libQt5Core.so.5+0x2b840c)
    #3 0x7f4055863711 in QAction::triggered(bool) (/usr/lib/libQt5Widgets.so.5+0x137711)
    #4 0x7f4055865d97 in QAction::activate(QAction::ActionEvent) (/usr/lib/libQt5Widgets.so.5+0x139d97)
    #5 0x7f405596646c (/usr/lib/libQt5Widgets.so.5+0x23a46c)
    #6 0x7f4055966683 in QAbstractButton::mouseReleaseEvent(QMouseEvent*) (/usr/lib/libQt5Widgets.so.5+0x23a683)
    #7 0x7f4055a2e8c9 in QToolButton::mouseReleaseEvent(QMouseEvent*) (/usr/lib/libQt5Widgets.so.5+0x3028c9)
    #8 0x7f40558a8fa7 in QWidget::event(QEvent*) (/usr/lib/libQt5Widgets.so.5+0x17cfa7)
    #9 0x7f4055a2e9a8 in QToolButton::event(QEvent*) (/usr/lib/libQt5Widgets.so.5+0x3029a8)
    #10 0x7f405586af4b in QApplicationPrivate::notify_helper(QObject*, QEvent*) (/usr/lib/libQt5Widgets.so.5+0x13ef4b)
    #11 0x7f4055870be5 in QApplication::notify(QObject*, QEvent*) (/usr/lib/libQt5Widgets.so.5+0x144be5)
    #12 0x7f4054bb370a in QCoreApplication::notifyInternal(QObject*, QEvent*) (/usr/lib/libQt5Core.so.5+0x28970a)
    #13 0x7f405586f0d2 in QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool) (/usr/lib/libQt5Widgets.so.5+0x1430d2)
    #14 0x7f40558c785c (/usr/lib/libQt5Widgets.so.5+0x19b85c)
    #15 0x7f40558c9c72 (/usr/lib/libQt5Widgets.so.5+0x19dc72)
    #16 0x7f405586af4b in QApplicationPrivate::notify_helper(QObject*, QEvent*) (/usr/lib/libQt5Widgets.so.5+0x13ef4b)
    #17 0x7f405587036d in QApplication::notify(QObject*, QEvent*) (/usr/lib/libQt5Widgets.so.5+0x14436d)
    #18 0x7f4054bb370a in QCoreApplication::notifyInternal(QObject*, QEvent*) (/usr/lib/libQt5Core.so.5+0x28970a)
    #19 0x7f40550dc635 in QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) (/usr/lib/libQt5Gui.so.5+0xe4635)
    #20 0x7f40550dde74 in QGuiApplicationPrivate::processWindowSystemEvent(QWindowSystemInterfacePrivate::WindowSystemEvent*) (/usr/lib/libQt5Gui.so.5+0xe5e74)
    #21 0x7f40550c358e in QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) (/usr/lib/libQt5Gui.so.5+0xcb58e)
    #22 0x7f404a1757ff (/usr/lib/qt/plugins/platforms/libqxcb.so+0x557ff)
    #23 0x7f4052ec4a0c in g_main_context_dispatch (/usr/lib/libglib-2.0.so.0+0x49a0c)
    #24 0x7f4052ec4cf7 (/usr/lib/libglib-2.0.so.0+0x49cf7)
    #25 0x7f4052ec4dab in g_main_context_iteration (/usr/lib/libglib-2.0.so.0+0x49dab)
    #26 0x7f4054c0b136 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) (/usr/lib/libQt5Core.so.5+0x2e1136)
    #27 0x7f4054bb1131 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (/usr/lib/libQt5Core.so.5+0x287131)
    #28 0x7f4054bb8aeb in QCoreApplication::exec() (/usr/lib/libQt5Core.so.5+0x28eaeb)
    #29 0x9cd8e8 in main /tmp/../qt-client/main.cpp:78:12
    #30 0x7f40517a603f in __libc_start_main (/usr/lib/libc.so.6+0x2003f)
    #31 0x9cbf5c in _start (/home/nick/code/dmp/build-clang-Release/qt-client/qt-client+0x9cbf5c)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /tmp/../qt-client/dmp_client_gui.cpp:206 DmpClientGui::PlayPauseToggled(bool)
==29063==ABORTING

Moving the searchbar model to the client broke the "syntax error coloring"

The most likely cause is that the data access is done wrong now.

Make the parser more manageable.

Although the parser is in working order, it is a 160 line beast and should be split up in multiple parts if possible. The high recursive nature of the query language might not allow it but maybe it is worth looking in to.

Make a progress bar for library scanning.

Now that I switched to accurate scanning of the music folder, it would be nice to include a progress bar. It is becoming "unresponsive" with less than 100 songs already.

Create and use custom exception class in libdmp-library

In the file libdmp-library.cpp there are a couple of std::runtime_errors that should become their own exception. This way it is much nicer to distinguish between std and dmp errors.

Review all threading code and use Xdispatch where possible

Some threading issues require sequential execution. These can be enforced by for instance condition variables. Unfortunately there are currently some issues with condition variables in libc++ so to avoid these issues we are going to use xdispatch. But first we need to identify the pieces of code we want to rewrite using xdispatch and how we should do that.

blocks on #1