Kaffeine hangs on startup if ffdecsawrapper is running about ffdecsawrapper

[bas-t]

If you configured ffdecsawrapper the right way, the join parameter in /etc/default/ffdecsawrapper, and you point your program (kaffeine, mythtv, whatever you use) to the right loopback interface, it should work, I think.
I've never used kaffeine though.
If it starts without ffdecsawrapper running, my best guess is that your configuration of ffdecsawrapper and/or kaffeine is to blame.

[AndreyPavlenko]

I've a single physical adapter. I run ffdescawrapper with the following command:

/usr/bin/ffdecsawrapper --join 0:1 --cam-budget --sid-allpid --sid-filt 20 --buffer 16M --cam-dir /etc/ffdecsawrapper

It seems running ok. A new adapter appears in /dev/dvb, but if I run kaffeine it hangs and I need to kill it. If ffdescawrapper is not running kaffeine works just fine.

[bas-t]

Did you ever have a working situation with old sasc-ng and kaffeine?

[AndreyPavlenko]

No, I never tried kaffeine with sasc-ng since it requires kernel patching. From the ffdescawrapper description I understood that it does not require to patch and recompile the kernel. My system is Ubuntu and the kernel is 3.11.1-031101-generic.

[bas-t]

It does need the patch for the kernel, but the kernel should be patched while running ./configure
Did that happen?
If not, do 'make update' and './configure' again and tell me if your kernel got patched properly

[AndreyPavlenko]

Sure I did ./configure. I'm trying to create a deb package consisting of the sources and install script. The install script (postinst) does the following:

USER='ffdecsawrapper'
uname="$(uname -r)"

if sudo -n -u $USER ./configure
then
    rm -f /usr/bin/ffdecsawrapper
    rm -rf /lib/modules/$uname/updates/ffdecsawrapper
    mkdir -p /lib/modules/$uname/updates/ffdecsawrapper
    install ffdecsawrapper /usr/bin
    install dvbloopback.ko /lib/modules/$uname/updates/ffdecsawrapper
    depmod -a
fi

Is it enough?

[bas-t]

I don't see the user ffdecsawrapper being created, and I'm not that familiar with creating packages

[AndreyPavlenko]

This is a part of the script. The user is also created:

USER='ffdecsawrapper'
GROUP='ffdecsawrapper'

create_user() {
addgroup --quiet --system $GROUP || true
adduser --system --home /etc/ffdecsawrapper --shell /bin/false
--gecos "FFdescaWrapper user" --no-create-home
--disabled-login --disabled-password
--ingroup $GROUP $USER > /dev/null 2>&1 || true
adduser $USER video > /dev/null 2>&1 || true
}

I can share the package so you can try it if you have an Ubuntu/Debian system.

[bas-t]

Yes, please, do share.

I'll test coming week on a test server.

In the meantime, I'd like to invite you to have a look at another issue, it may or may not be related to this one.

#8

[AndreyPavlenko]

Here is it - https://docs.google.com/file/d/0B21_ooef7x8Ad2ZHRV9mQUFQZ0k/edit?usp=sharing&pli=1
You can find the installation script (postinst) if you extract the package:
ar xv ffdecsawrapper_1.1.2-1004005285b-ppa1raring_all.deb
tar xzf control.tar.gz

Concerning #8 - my g++ version is 4.7.3.

[bas-t]

Are you willing to install g++-4.8 package?

Short way to reproduce the error:

./configure --ffdecsa_mode=PARALLEL_64_MMX --cxx=g++-4.8

and run make

[AndreyPavlenko]

Tried to build with 4.8 - succeeded.

[bas-t]

I'm so glad, it's not a g++ issue but just a Debian one.

Thanks.

[AndreyPavlenko]

I've installed g++ from this ppa - ppa:ubuntu-toolchain-r/test.

[bas-t]

g++-4.8 is available in regular Ubuntu repos as of saucy, so I'm going to install a test server with saucy next week and see what happens. I don't want to use ppa's, go mainstream. Most other users will, I think.

[AndreyPavlenko]

I know, but I'm on Raring since Saucy is not yet released, so I installed from ppa.

[bas-t]

Saucy not released? That means the shit has not yet hit the fan in Ubuntu. If it ever will, that is.
Ok , I'll reinvestigate when Saucy is released.

Some thoughts on packaging FFdecsawrapper:

If you want to hit the front page of digg (I did once, it's fun), you should consider splitting FFdecsawrapper into 3 packages.

One regular package for the main binary, one DKMS package for the kernel patch and one DKMS package for the dvbloopback module. Also make either one of them depend on the other two. That would be a cool way to go now, wouldn't it?

As it seems, some effort in that direction has been taken allready on AUR (the Archlinux equivalent of ppa), so you may get some pointers from there.

[AndreyPavlenko]

I thought about it, but a few points are not clear to me:

Which --ffdecsa_mode= option to chose when building the first (binary) package? Does this option impact on the binary?
The configure script does all the stuff - binary, patch, module. How can I split it into 3 independent build procedures?

I would appreciate very much if you help me with these questions.

[bas-t]

I'll sort it out, just give me a week or two. I'm a little short of time right now.
I've allready created a new branch for this purpose: packaging.
Nothing new in there yet, of course.
I'll give you notice when I'm done.

[AndreyPavlenko]

Finally I've got it working with kaffeine. Seems there are some problems with the configure script - it does not call the kernelpatch-ubuntu script in my environment. Besides, the kernelpatch-ubuntu script deletes the /usr/share directory...

I've created a dkms package - https://docs.google.com/file/d/0B21_ooef7x8ANVFDeE96dlRtWVE/edit?usp=sharing&pli=1. The dkms make file is located at /usr/src/ffdecsawrapper-1.1.2/.

[bas-t]

If it does not call the kernelpatch script, it means that you have a dvb-core.ko module in /lib/modules/uname -r/updates dir. That is expected. I don't want to patch the kernel if it allready has been done.

The kernelpatch script does NOT delete the /usr/share directory

[bas-t]

Please explain how you did get it working with kaffeine and close this issue.

[AndreyPavlenko]

Sorry, not /usr/share but /usr/src directory. I think here is it:

SHORT_KERNEL=$( echo uname -r | grep -o '[[:digit:]]+.[[:digit:]]+'.0 )
...
TEMPDIR=/usr/src/$SHORT_KERNEL
...
sudo rm -rf $TEMPDIR
sudo mkdir -p $TEMPDIR

My kernel version is 3.11.4 and $(echo '3.11.4' | grep -o '[[:digit:]]+.[[:digit:]]+'.0) returns empty string.

After proper installation of the patched module kaffeine works fine. These are my dkms scripts:

https://docs.google.com/file/d/0B21_ooef7x8AMGk5SmVXaHYwS1E/edit?usp=sharing
https://docs.google.com/file/d/0B21_ooef7x8AUHE1UlpINWw1NVU/edit?usp=sharing

[bas-t]

There is no line in any file like

$(echo '3.11.4' | grep -o '[[:digit:]]+.[[:digit:]]+'.0)

Tested FFdecsawrapper yesterday on Ubuntu Precise, runs as expected

[AndreyPavlenko]

There is line SHORT_KERNEL=$( echo uname -r | grep -o '[[:digit:]]+.[[:digit:]]+'.0 ) at kernelpatch-ubuntu line 39 - https://github.com/bas-t/ffdecsawrapper/blob/stable/kernelpatch-ubuntu#L39. I've just substituted uname -r with my kernel version.

[bas-t]

The original line does not return an empty string.

[bas-t]

https://docs.google.com/file/d/0B21_ooef7x8AMGk5SmVXaHYwS1E/edit?usp=sharing contains a typo.

line 13: make celan should be make clean

[AndreyPavlenko]

The original line does not return an empty string.

It does in case uname -r returns 3.11.4. In the above exampe I've replaced "uname -r" with 3.11.4: $(echo '3.11.4' | grep -o '[[:digit:]]+.[[:digit:]]+'.0). The expression "'[[:digit:]]+.[[:digit:]]+'.0" does not match 3.11.4.

line 13: make celan should be make clean

Thanks! Fixed.
I've uploaded the package to my ppa - https://launchpad.net/~aap/+archive/cam/. My packaging script is here - https://github.com/AndreyPavlenko/cam-builder/tree/master/ffdecsawrapper.

[bas-t]

"The expression "'[[:digit:]]+.[[:digit:]]+'.0" does not match 3.11.4."
Strange, I never experienced this. The other day I installed Saucy, no problem.
But if you give me a better alternative, I will use that of course

[AndreyPavlenko]

You din not have this issue in Saucy because the current version of the Saucy kernel is 3.11.0-12.19 which does match the expression: echo '3.11.0-12.19' | grep -o '[[:digit:]]+.[[:digit:]]+'.0. However, the version 3.11.1-12.19 will not match.

As an alternative, I think, this may work - echo $(uname -r) | awk -F '-' '{print $1}'
And for the version comparison you may use dpkg --compare-versions.

[bas-t]

Or I could do grep -o '[[:digit:]]+.[[:digit:]]+' , like in the debian script, and then install $SHORT_KERNEL.0
The 0 is there because th Ubuntu source, unlike the Debian source, adds a .0, so linux-source-3.11.0
while in Debian linux-source-3.11

[AndreyPavlenko]

Yes, but it will not work for kernels built with make-kpkg. Currently I use vanilla kernel built with make-kpkg and the sources are located at /usr/src/linux-source-3.11.4+.tar.bz2.

[bas-t]

The script is ment to be used ONLY with clean, unmodified Debian/Ubuntu kernels.
If you compile your own kernel, just use the proper dvb-mutex patch

[bas-t]

However, should you want to use the kernelpatch script with a vanilla kernel, just alter line 311 and maybe line 312 in configure, and change the kernelpatch script to download $SHORT_KERNEL.tar.xz from
https://www.kernel.org/pub/linux/kernel/v3.x/

[AndreyPavlenko]

Downloading kernel sources is a good solution, I think. Especially if the kernel is installed from http://kernel.ubuntu.com/~kernel-ppa/mainline/ which does not provide the source packages.

[AndreyPavlenko]

I've implemented sources downloading - https://github.com/AndreyPavlenko/cam-builder/blob/master/ffdecsawrapper/debian/dkms/Makefile

Tested it with 3 different kernels without installing the kernel-sources package:
Ubuntu kernel from ppa
Kernel installed from http://kernel.ubuntu.com/~kernel-ppa/mainline/
Kernel built with make-kpkg

[bas-t]

Hi!
I've seen the thread on the Russian forum, of course I can't read it.
But Google sort of "translated" it for me in Dutch. That's no good, google translation sucks...

I have a question: why are you packaging? Shouldn't you first get things up and running the proper way and then go from there?

I for one will never test your package as it is, if only because of the initscript.

Oh, and don't EVER patch a kernel that is compiled by yourself after compilation!!!

There is a good reason why I don't support that.

Generally speaking: if you compile your own kernel, you have to do the patching prior to compiling, not afterwards

[AndreyPavlenko]

Hi,

I prefer packaging because a proper package can always be properly uninstalled/cleaned up. Besides, there are other people who use this package and do not want to compile anything. The package has been tested by different people on different Ubuntu systems and it works.

Usually, when I compile a kernel, I build packages with image/headers/sources and then install these packages on different computers. For this case the package works just fine.

[bas-t]

I still disagree with you, there are only two files generated from FFdecsawrapper setup. Well, three if you count the kernel patch.

But more important: you don't run configure at all. That means that all users of your package get stuck with a predefined optimization. That's not good, to say the least. Same goes for transferring to another computer: ok when it has the exact same hardware and software configuration. If not, and that is most likely the case, you should not transferr it at all.

Possible solution:
Make your package run (parts of) the configure script, at least the part where the optimization gets done and the part that chooses the right compiler version.

You mention that some people don't want to compile anything. That may be so, but by installing any DKMS package they are... You get my drift? It's not so much that they don't want that anything gets compiled, they just want another person to write a script that takes care of everything. And that's exactely what both you and me have done. With that in mind, the only difference between installing a package and running my script is "apt-get install " or "./configure". Mine is shorter.

That gets me back to my first question: why packaging? My scripts take care of all compilation matters, just to ensure that people don't have to understand anything of configuring and compiling. So I think that we both have the same goal, but you present people with a fixed configuration and I don't.

And some more: what about out of kernel dvb drivers? You don't address that at all, unlike Archlinux guys

Archlinux, as you may know, has several vesions of FFdecsawrapper available, because there are many people using out of kernel, mostly v4l (based) dvb drivers. They all have one thing in common: they run the configure script for FFdecsawrapper to ensure proper optimization for any given system and only include a (DKMS?) kernelpatch and archlinux specific system configuration files. Now that's the good way to go!

[AndreyPavlenko]

In my initial version I just called the configure script from the package postinstall script but it didn't work at all because:

There are many apt-get install in the configure script that do not work because we are already installing and dpkg is locked.
The configure script is interactive with stdin reads and echo messages - this does not work in packages. Assume that you are installing the package from a gui, not console.
An finally, the configure script deleted all my /usr/src directory.

What I did - I just implemented almost the same logic as in your scripts but in the way suitable for packaging.

Yes, it does not recompile the binary when installing, but I thought you proposed the same above:

---

you should consider splitting FFdecsawrapper into 3 packages.

One regular package for the main binary, one DKMS package for the kernel patch and one DKMS package for the dvbloopback module.

[bas-t]

You can get around the apt-get issues by making your package depend on the packages installed by the many apt-get instances. pre depend, that is. Not just depend.

[bas-t]

More thoughts:
is dpkg locked too whiie running a DKMS script? If not, you hit the jackpot.

How does the deleting of your /usr/src dir happen? It never happened here, so please point me to the possible bug, I will fix that ASAP

About the interactive thing: there are many packages that support this, like mysql. It asks for a password. No matter if you do it from a shell or in a gui. So that's a non issue in my opinion.

[bas-t]

OK, one last thought for today:
Your package should indeed recompile the binary whenever a kernel update occurs, thus defining the fastest ffdecsa optimization parameters available

My routine:
cd /path/to/git checkout of/FFdecsawrapper
make update
./configure

[AndreyPavlenko]

You can get around the apt-get issues by making your package depend on the packages installed by the many apt-get instances.

Yes, most of the dependencies can be added to the package depends, but not this one:

  apt-get purge linux-source* -y && apt-get build-dep linux --no-install-recommends -y
  apt-get install linux-source-$SHORT_KERNEL -y || die "Error installing linux-source-$SHORT_KERNEL"

(btw, why do you remove sources for all installed kernels but install only for the running one? Dkms re-configures packages for all kernels and may fail if you've removed the sources)

is dpkg locked too whiie running a DKMS script? If not, you hit the jackpot.

I'm not sure since dkms is triggered by dpkg... Need to check it.

How does the deleting of your /usr/src dir happen? It never happened here, so please point me to the possible bug, I will fix that ASAP

You seems have fixed it. I mentioned about it above. It happened because SHORT_KERNEL=$( echo uname -r | grep -o '...' ) initialized to empty string because the grep expression did not match my kernel version.

About the interactive thing: there are many packages that support this, like mysql. It asks for a password. No matter if you do it from a shell or in a gui. So that's a non issue in my opinion.

Please look into the mysql package internals:

apt-get download mysql-server-5.5
ar xf mysql-server-5.5_5.5.32-0ubuntu7_amd64.deb control.tar.gz
tar xzf control.tar.gz

You will find the templates file here. This file contains all (localized) messages required for the user interaction. The package scripts interact with user via debconf but not via echo and read. In this way user interaction should work in gui as well as in console. In case of echo/read the package may hang, waiting for user input, while updating with an update manager.

Your package should indeed recompile the binary whenever a kernel update occurs, thus defining the fastest ffdecsa optimization parameters available

I thought about that and the initial version of the package did recompile on update, but it was not easy to extract the optimization tests from the configure script, so I decided to just compile the binary with predefined parameters. This is not the best solution but it works. If you extract the optimization test into a separate script or a function I would appreciate very much.

[bas-t]

Hey, I'm not a packager, I don't know that much about it.

BTW the optimization script is allready a seperate one: FFdecsa_optimizer

[AndreyPavlenko]

Thanks. I'll implement binary compilation when I have time.

[bas-t]

One more thought: is it possible to do 'cat /proc/version' in a packaging script?
If it is, you should be able to install the right gcc/g++ version, I think.
Or if you make installing the binary some kind of DKMS thing too, that should give you a lot of extra options. Again. I think...
How does nvidia driver package do it? Maybe worth to look in to.

[AndreyPavlenko]

I think it's possible. Why do you need to install gcc the kernel was built with?
Another question - are the performance optimizations noticeable to a user? I've 2 packages that use FFdecsa - vdr-plugin-dvbapi and vdr-plugin-sc. Both are built with predefined flags and they ... just work, with minimal cpu consumption. How does a user benefit from the binary recompilation with an optimization?

[bas-t]

Benefit?
Yes user will notice, by optimizing + compiling with the right gcc version one can gain up to 35% speed without being an expert
That means the difference between beeing able to decode like 6 or 9 HD shows at the same time. OK in most cases the gain will not be so spectacular, but at least 10 to 20 percent is a realistic scenario.

Of course, you will only notice the difference when you have installed plenty adapters in your system.
If only one or two are present, well, discard optimization.

Ok, I currently run a backend with a Mobile AMD Sempron(tm) Processor 3000+.
That is a very low power cpu.
It contains 5 dvb-c tuners
Without optimization, I can decode 3 or 4 max HD shows at the same time.
With proper optimization, choosing the right compiler etc. I am up to 8.
Now, that of course takes an expert to accomplish that. Nevertheless, it can be done

[AndreyPavlenko]

I thought it's not possible to use more than 4 adapters with ffdecsawrapper.

[bas-t]

But it is. You have to recompile your kernel with CONFIG_DVB_MAX_ADAPTERS=twice your adapters

[bas-t]

Default is 8, that's why you cannot use more then 4 adapters without recompiling your kernel.