yorvex / xobsbeam

OBS plugin to transmit video and audio feeds between OBS instances, raw, or with lossless or lossy compression. NDI alternative.

Beam

⚠️ Seeking help with translations, please go here if you want to help, thanks!

Beam (technical project name "xObsBeam") is an OBS plugin to transmit video and audio feeds between OBS instances, raw, or with lossless or lossy compression. An alternative to NDI and Teleport for A/V transmission. Raw transmission and some compression options have alpha channel and HDR support.

Prerequisites

• OBS 29.1.X+ 64 bit
• Windows
  • tested on Windows 10 and 11
• Linux
  • occasionally tested, but not regularly
  • binary build created on Ubuntu 20.04 WSL environment, therefore linked against glibc 2.31

Overview

This plugin transmits a video and audio feed from one OBS instance to another. The video feed can be compressed lossy or lossless, or even transmitted raw with no added compression CPU usage at all but at the expense of extreme bandwidth needs.

The most common scenario would be transmitting audio and video feeds from OBS on a gaming PC to OBS on a streaming PC, another would be to run two OBS instances on the same PC, one for recording with a higher quality setting, and then transmit the video feed to a second OBS instance for streaming with lower quality, which adds some panels, alerts, animations and other effects only for the stream which shouldn't be visible in the recording (other solutions to achieve that currently don't seem to be really stable).

Comparison

Both NDI and Teleport can only transmit their video feeds using a lossy compression (albeit almost visually lossless to be fair). The JPEG compression that Teleport uses is also available in Beam, but with the additional option to only compress a certain percentage of frames, giving a fine grained control over the CPU usage vs. bandwidth usage tradeoff. Using this makes it possible to achieve both lower bandwidth and CPU usage than NDI, but this may depend on the content and other factors.

Beam also offers lossless compression options, including some that support alpha channels and HDR, which is not available in either NDI or Teleport. Also only Beam offers a frame buffer on the receiver that is able to compensate for network jitter and packet loss, which can be a big advantage in some scenarios, especially when trying to use this on unstable networks like Wifi (note that using Wifi for this kind of transmission is still not recommended and should be avoided if possible). Optionally a fixed delay between sender and receiver can be configured, which can be useful if outside audio or video sources need to be synced with the transmitted feed.

Beam basically aims to offer more options, flexibility and stability, however, in the end no solution is better or worse in general, it's just different tradeoffs regarding CPU/GPU usage, bandwidth needs, quality, ease of use and compatibility. What works best in a given scenario depends (among other things) on the specific use case and available resources. It's also very hard to predict how any solution will perform for any given setup, so it's best to test it out and play with the settings.

Compression options

You want to pick raw transmission (simply don't select any compression) if you stay within the same computer (Named Pipe connection setting), because available bandwidth is not an issue in this case and you can save on CPU usage. It might also be feasible if you only transmit a feed with low resolution and/or low FPS or if you're on a 5G or 10G network.

If you're on a standard 1G network you will likely need to use compression to keep bandwidth needs in check. Try to stay lossless with QOY, and if that doesn't work you can always pick JPEG, which is still mostly visually lossless.

See here for more details on the available compression options and raw bandwidth usage.

Usage

Install the same version of the plugin (different versions are never guaranteed to be compatible to each other) into all OBS instances that you want to transmit video/audio feeds between.

Installation

Before installing make sure that OBS is not running.

For regular OBS installations see the operating system specific instructions below. For portable mode simply extract the .7z file into the root directory of the portable folder structure.

🟦 Windows

For automatic installation just run the provided installer, then restart OBS.

For manual installation extract the downloaded .7z file (= copy the contained obs-plugins and data folders) into the OBS Studio installation directory. The default location for this is

C:\Program Files\obs-studio

This needs admin permissions.

🐧 Linux

The folder structure in the downloaded .7z file is prepared so that you can extract the file (= copy the contained files) into your user home and on many systems this will just work already, e.g. on Ubuntu with an installation through apt. If you installed through flatpak, copy the obs-studio folder from the .7z file to: ~/.var/app/com.obsproject.Studio/config/

However, depending on the distribution and OBS installation method (manual, distro repo, snap, flatpak...) the location of this folder can vary, so if it doesn't work from the user home you might have to look around a bit.

Example locations for the plugin .so (and .so.dbg) file are:

• ~/.config/obs-studio/plugins/ (The structure the .7z is prepared for)
• ~/.var/app/com.obsproject.Studio/config/obs-studio/plugins/ (OBS flatpak installation, if this works it should also cover locales)
• /usr/lib/obs-plugins/
• /usr/lib/x86_64-linux-gnu/obs-plugins/
• /usr/share/obs/obs-plugins/
• ~/.local/share/flatpak/app/com.obsproject.Studio/x86_64/stable/active/files/lib/obs-plugins/
• /var/lib/flatpak/app/com.obsproject.Studio/x86_64/stable/active/files/lib/obs-plugins/

Unfortunately the expected location of the locale, which can be found in the data folder, can vary also.

If you get missing locale errors from the plugin you can try to copy the "locale" folder found inside the data folder to:

• /usr/share/obs/obs-plugins/<plugin name>/locale
• ~/.local/share/flatpak/app/com.obsproject.Studio/x86_64/stable/active/files/share/obs/obs-plugins/<plugin name>/locale
• /var/lib/flatpak/app/com.obsproject.Studio/x86_64/stable/active/files/share/obs/obs-plugins/<plugin name>/locale

If in doubt, please check where other "en-US.ini" files are located on your system.

The steps to update an older version of the plugin to a newer version are the same, except that during file extraction you need to confirm overwriting existing files in addition.

Sender configuration

One OBS instance will be the sender, on this instance go to the OBS main menu and select Tools -> Beam Sender Output.

A dialog will appear where you can configure the sender identifier and how the sender will accept receiver connections. Named pipe connection is the recommended connection type for local (within the same machine) connections, as it has the least overhead and therefore should come with the smallest resource impact. If you need to connect receiver and sender from different machines you need to use a TCP socket connection. Compression configuration depends on the setup and use case, read above sections for more information.

For the "Compress from OBS render thread" option the rule of thumb would be to leave it enabled as long as you're not dropping frames in any of the OBS instances, if you do, then disable that option (will be more likely to be necessary the higher the compression level you pick).

Check the "Enable Beam Sender Output" box if you want your output to be active now. Press OK to save the settings.

Note that as soon as the output is active your resource usage from OBS will go up, as OBS is now providing data to this plugin (regardless of whether the plugin is doing something with it at this point). Also certain video and audio settings (e.g. video resolution, FPS...) are locked in the OBS settings as long as an output is active. If you want to change those settings, you first need to disable the Beam output again.

Another option for a sender would be a filter. For this right click an async source and choose Filters from the context menu:

Now in the "Audio/Video Filters" section click the + button and choose one of the Beam filters:

After creating the filter the sender options are the same as for the output.

Note that Beam filters are not available for sync sources, which include Game Capture or scenes, they don't have the "Audio/Video Filters" section. There is currently no plans to add this, you can read more about the reasons here.

Receiver configuration

At least one OBS instance will be the receiver (multiple receivers can connect to a sender), on a receiver instance add a new source of type Beam Receiver.

Double click the source to edit its properties.

Make sure to select the same connection type that you previously selected for the sender, then you should be able to find the sender on the list of available Beam feeds (an output named "BeamSender" in this case). Press OK to save the settings.

Now you can show the new Beam source (click the eye icon next to it) and it should connect to the sender and start to receive data. Play with the other options as you wish, a render delay limit can make sure that your source is reconnected (resetting delay to a lower level again) if it ever becomes too high, a frame buffer can be used to compensate for network hiccups or other lags or to get a fixed delay between sender and receiver, by automatically increasing or decreasing the buffer to counter any changes in delay within OBS.

Troubleshooting

See the Wiki for troubleshooting help.

FAQ

• Q: Why is the plugin file so big compared to other plugins for the little bit it does, will this cause issues?

  • A: Unlike other plugins it's not written directly in C++ but in C# using .NET 7 and NativeAOT (for more details read on in the section for developers). This produces some overhead in the actual plugin file, however, the code that matters for functionality of this plugin should be just as efficient and fast as code directly written in C++ so there's no reason to worry about performance on your system.

• Q: Will there be a version for MacOS?

  • A: NativeAOT doesn't support cross-compiling and I don't have a Mac, so I currently can't compile it, let alone test it. You can try to compile it yourself, but note that MacOS is currently only supported by the next preview version of .NET 8, although people do already successfully create builds with it.

• Q: Will there be a 32 bit version of this plugin?

  • A: No. Feel free to try and compile it for x86 targets yourself, last time I checked it wasn't fully supported in NativeAOT.

• Q: Does this work with all color formats and color spaces including HDR?

  • A: It should work fine when not using compression for every available OBS color format/space setting including HDR, since OBS data is just transferred 1:1. This is also true when using LZ4 or Density compression. QOI and QOIR compression, however, support alpha but are designed only for RGB(A), and QOY is designed for YUV formats like the NV12 OBS default or I420, it doesn't support alpha either. The current JPEG implementation in Beam also doesn't support HDR formats or alpha channels, even if the JPEG spec in general would offer this.

For developers

C#

OBS Classic still had a CLR Host Plugin, but with OBS Studio writing plugins in C# wasn't possible anymore. This has changed with the release of .NET 7 and NativeAOT, and this plugin is fully written in C#.

Used libraries/technologies

This plugin uses the following libraries/technologies:

• NetObsBindings as a base for building OBS plugins in C#
• System.IO.Pipelines PipeWriter/PipeReader for high-efficiency data transfer between sender and receiver
• K4os.Compression.LZ4 for LZ4 compression
• QOI - ported 1:1 to C#, also some comments were added, see here for the Beam implementation
• a modified version of QOY - see source code comments for the differences to the original
• QOIR - optionally loaded from an external dynamic library, more details on the Beam implementation of it here
• Density - optionally loaded from an external dynamic library, more details on the Beam implementation of it here
• libjpeg-turbo - optionally loaded from an external dynamic library, more details on the Beam implementation of it here

Compression is simply applied to each frame separately, QOI, QOIR, QOY and JPEG are image codecs and not video codecs anyway and LZ4 and Density not even tailored for images.

Building

Refer to the building instructions for my example plugin, they will also apply here.

Credits

Many thanks to kostya9 for laying the groundwork of C# OBS Studio plugin creation, without him this plugin (and hopefully many more C# plugins following in the future) wouldn't exist. Read about his ventures into this area in his blog posts here and here.

Also thanks to fzwoch for his wonderful Teleport plugin and for many interesting discussions on Discord that helped me along the way of creating this and other plugins.

xobsbeam's Issues

[BUG]

Describe the bug
Installation cancels with error message.

To Reproduce
Steps to reproduce the behavior:

1. Start installation. <- Even tried to "Start as Administrator"
2. Define installation path (in my case other than default).
3. Get error message.
4. Tried on the other PC also, error stays the same.

Expected behavior
Installation finishes without errors.

Screenshots
If applicable, add screenshots to help explain your problem.

Screenshot message translated:
"Error overwriting the file: ..."

Logs and details about your system

• Windows 11 Pro 23H2 <- Using local administrator account
• OBS 30.0.2 <- In portable mode, not installed in default folder.
• Beam 1.0.0 <- Installer

PNG support with fast FPNGE encoder

FPNGE is only an encoder. Decoding can be done using ImageSharp and in case that doesn't perform well enough later be replaced by a different decoder.

Unfortunately FPNG (without E) doesn't support decoding FPNGE encoded files, so the next best fast decoder would be wuffs, however, implementing this would take a lot of effort. Maybe this adapter code could be compiled into the library instead.

Density support

Density is a promising candidate for a generic (i.e. not image specific) compression. These have the advantage that regardless of what color format is configured in OBS no conversions are necessary and all formats are supported that OBS supports, including HDR. Currently the only option xObsBeam has for this is LZ4, which is already very good for the "low compression but also low CPU usage" scenario, but Density is supposed to be even better.

[BUG] Can't select JPEG compression on OBS (Fedora 38 host)

Describe the bug
Cannot enable JPEC Compression entry in Bream Output Settings.

To Reproduce
Steps to reproduce the behavior:

1. Open Beam Output Settings
2. Check JPEG Compression

Expected behavior
JPEG Option enabled and sliders enabled

Screenshots

Screencast.from.2023-10-01.20-22-31.mp4

Logs and details about your system
https://obsproject.com/logs/GL8OnCXK6vRE4ETQ

Additional context

• Using the RPMFusion build of obs, not flatpak.

Logs show dropped frames (1sec 60 frames) almost every 10 seconds

So not sure how this is happening. I have a beefy laptop, AC powered. It is connected to my game PC through a Thunderbolt to 10Gbit connection. The game PC has a USB 3.2 gen 2 to 5Gbit connection. It's only configured for OBS, so nothing else is on that network.

With compression or without compression it seemingly stutters every couple of seconds, sometimes 2, sometimes 10.

Filter that can transmit A/V feed from a single source

Currently only an output exists to transmit the combined A/V feed from OBS. Just as for NDI and Teleport a filter should exist that can also send the feed of a single source only.

JPEG support (lossy and lossless) based on the libjpeg-turbo library

The libjpeg-turbo library should be the fastest available library for this. The main focus for Beam so far has been binary lossless, to close a gap (both NDI and Teleport don't offer these options) and this is also the motivation to implement JPEG (lossless), however, lossy should also be offered as part of this implementation, since it's not much extra work while already at it.

• Support both stable v2 and beta v3 versions of libjpeg-turbo and auto-detect which one is available
• Lossless compression (v3 version only)
• Lossy compression (v2 and v3)
• Support for BGRA color format
• Support for color formats other than BGRA
• Linux support

QOIR support

QOIR is a derivate of QOI that has both improved compression speed and ratio compared to QOI. That pretty much sounds like a "can only win" situation, however, the one nice thing about QOI is that it's so simple and Beam has a pure C# implementation for it that will always automatically be platform independent, without the need for any external library.

For QOIR the code is much more complex, not really making me want to rewrite it as a full C# implementation, so it will be yet another native library that has to be separately managed and installed and so on. Right now I think the managed QOI implementation will stay in Beam in addition as an always available fallback option even when QOIR support was added.

Improve settings dialog (Settings presets or advanced vs. simple mode)

Beam settings are quite complicated, they should either be divided into a simple vs. advanced mode, or simple settings presets should be offered and selecting them auto-configures the detailed advanced settings.

Peer discovery (UDP broadcast based sender discovery)

Right now for TCP socket mode users have to manually enter an IP address or hostname of the sender. This should be simplified by using peer discovery based on UDP broadcasts.

There should be an additional setting to always use peer discovery (for every new connection) to identify the sender that is enabled by default. This mode is the most simple and should work in most scenarios, it is useful when 1. peer discovery always works reliably in the given network environment 2. the sender can't be found by hostname and changes its IP address from time to time, e.g. because it's auto-assigned through DHCP.

When it is disabled peer discovery can still be used as a helper to fill in the sender data (IP/host and port), but then Beam will only use this data for future connections and not automatically repeat the discovery process. This mode is useful when peer discovery cannot be done reliably or not at all, e.g. because involved network gear doesn't support UDP broadcasts, but then the sender needs to keep the same IP address or hostname all the time.

Add Indonesia translation

adding translation indonesia
id-ID.ini

Improve frame ordering and receiver frame buffering

In the 0.6.0 release when not syncing to the OBS main render thread there could still be situations where frames are processed by receivers in the wrong order. Some progress has been already made in this area, but it's not part of a release yet. The FrameBuffer.cs source code has extensive comments explaining the situation in more detail and the current state of the implementation.

• Reorder frames if received in wrong order
• Compensate frames received delayed due to network lags
• Support sender and receiver having different frame rates
• Support fractional frame rate settings

The January 3rd version of Beam can no longer be adapted to OBS 30.2.0-Beta-1.

Previously, OBS versions 30.1.2 and even lower were running normally. Today (2024.06.08), after updating to OBS 30.2.0-Beta1, OBS can only be launched in safe mode, and Beam cannot properly adapt to the new UI interface of OBS. It has been confirmed that only the Beam plugin is installed, and it is 100% certain that the inability to open the software is due to Beam.

Crash 2024-06-08 18-13-15.txt

WebP lossless support

Only putting this here for documentation that I already had this implemented and while indeed the compression ratio was better than with e.g. QOI (170 mbps peek with WebP on a test feed where QOI peeks at 270 mbps) the CPU usage was so insanely high that I can't see the scenario where this would ever be useful. It's also visible in this benchmark.

Code formatting cleanup

The Beam code should be run through code analyzer checks (OmniSharp/.NET Analyzer). An .editorConfig should be created for this project and then all code should be run through it and have any warnings that come up handled. See here or here for options.

At the time of writing the standard code check already comes up with almost 200 points, so this should only be done when there is currently no other branch being worked on, otherwise merging might become a nightmare.

• Analyzers
• Code formatting rules

Odd resolutions like 1279x719 cause screen distortions when using YUV color formats

Happens because alignment is not properly handled by the GetPlaneSizes() function. Check the OBS video_frame_init function as a reference.

[BUG] Memory leak on insufficient bandwidth

It was reported here that a memory leak seems to occur when Beam tries to keep on sending more data than bandwidth is available. This needs further investigation.

Don't use packet send queue when sending uncompressed through local named pipe

The System.IO.Pipelines library that Beam uses does already come with its own buffer, but that's a pure binary buffer that doesn't allow reordering frames or any monitoring to notice when the buffer fills up (e.g. because the network bandwidth is insufficient), which is why a frame send queue was introduced. There was also the internal reason that the Pipelines FlushAsync() calls may not overlap with access to the buffer.

For this one relatively stable situation where the feed is only transmitted locally (no potential network bottlenecks), synced to the OBS render thread (no ordering issues) and uncompressed (no potential CPU bottlenecks) Beam should actually send the data directly without the frame send buffer and just take the relatively low risk of buffer overflows (they will still be detected indirectly by the timestamp age check).

It will introduce extra code only for this case, because some specific Pipelines FlushAsync() synchronization needs to be done, basically this should get an entirely separate send function outside of the normal send loop. But having a way to just directly send raw data with the lowest possible CPU usage was my motivation to invent Beam in the first place (and only then came the idea to add lossless compression options and so on), so I am willing to put some extra effort into really making this one specific case as efficient as possible.

This is an experiment that might also fail but it's certainly worth a try.

Frame buffer option: Fixed delay

When synchronizing events or feeds (e.g. a separate audio feed) outside of OBS this can only be done when the delay between the sender and receiver is known and stays stable throughout a session.

With existing solutions like NDI or Teleport this is not possible, often the delay would start off at 50 ms but reach 200 ms 2 hours later.

This usually happens

1. slowly and constantly because of clock drift between the systems (so that frame timestamps differ)
2. as a sudden jump after a short lag in sender or receiver OBS occurred

That this happens cannot be avoided, but a frame buffer could be used to compensate for this. On a 30 FPS feed a video frame is shown for 33.33 ms. If the frame buffer is set to 500 ms, it would hold 15 frames. With the current implementation the frame buffer will always keep exactly those 15 frames.

With the new "Fixed delay" option if the delay of the feed increases by 33.33 ms or more the frame buffer would reconfigure itself to now hold only 14 frames, effectively handing a frame out to OBS for rendering 33.33 ms earlier, therefore countering the added delay and restoring the original resulting delay.

The same in the other direction, the buffer would be increased to hold 16 frames then.

All of this is an experimental idea, it needs to be tested how the switch can be done so that the frame jump caused by this has a minimal negative effect.

When decreasing the buffer an additional frame is sent to the OBS output, however, by the timestamp of that frame OBS would know it's not due yet and just queue it internally, since it is coming from an async source, and it wouldn't make any visual difference. So in addition the offset of the timestamps of all frames need to be adjusted too. This needs also be applied to audio frames, otherwise it would cause A/V desync.

On the contrary, when increasing the buffer one frame will simply not be sent to the OBS output when it would be due. This is what is already happening when a frame is skipped because of lag, the frame buffer should already have mechanisms to deal with that.

#12 should be finished already as a base for this.

Make network bind interface selectable for the receiver

As Beam makes extensive use of network bandwidth it's not unlikely that users want to have this traffic on a dedicated network interface in systems that have two. Some mainboards even come by default with 2 LAN ports, and it's quite cheap to add a PCIe network card additionally to an onboard LAN port.

For the sender the bind network interface can already be selected, but for the receiver this currently isn't possible.

Frame buffer: prevent OBS audio buffering, force timestamps within thresholds

Extensive testing could show no more OBS audio buffering issues with the current release, therefore closing this for now. Might reopen with a specific scenario if one is found later.

[BUG] OBS crash on startup with sender enabled

Beam crashes host OBS on startup if the sender is enabled.
I used to have this issue only when the receiver is already running, but now I receive it regardless.

22:02:23.935: [xObsBeam] Peer Discovery server: Starting for Output "Beam Sender Output" on ****:****...
22:02:23.935: [xObsBeam] Listening on ****:****.
22:02:23.935: [xObsBeam] Failed to start TCP listener for Beam Sender Output on ****:****, try configuring a different port or use a different interface.
22:02:23.939: [xObsBeam] InvalidOperationException in BeamSender.Start: Not listening. You must call the Start() method before calling this method.
22:02:23.939:    at System.Net.Sockets.TcpListener.AcceptSocketAsync(CancellationToken) + 0x12f
22:02:23.939:    at xObsBeam.BeamSender.<Start>d__38.MoveNext() + 0x77b

The configured port is not used by anything else according to TCPView.
My config: xObsBeam.json
Full log: 2024-04-19 22-07-13.txt

[BUG] Can't overwrite plugin file installer error

Describe the bug
As mentioned in #31, I also get the error message from the installer that it couldn't overwrite the plugin file.

To Reproduce

1. Download latest release installer
2. Execute with administrator privileges
3. Keep "Install OBS-Plugin" checked in the installer
4. Proceed with the installer as usual

Expected behavior
The installer should finish as usual and OBS-Beam plugin should be available in my OBS instance.

Screenshots
Same error message as in #31

Logs and details about your system

No installer logs available.

• Windows 11 Pro 23H2
• OBS 30.1.2
• Beam 1.0.0
• Video Settings N/A (1920x1080, 50fps)
• 10G, 2.5G and 1G Networks attached

Additional context
The installation is contained in C:\Program Files\obs-studio so I am not running in portable mode.

P216/P416 HDR color format support for lossless transmission

This issue is just here to document that currently xObsBeam in theory already supports transmitting these formats for raw transmission and lossless compression, however, they just produce a blank (black) feed on the receiving source, since sources outputting these formats is not implemented in OBS right now. As of OBS release 29.1.X this can be seen e.g. here.

As soon as this is implemented in OBS in a future release it should automatically start to work with Beam.

Lossy formats are an entirely different story, the JPEG library used by Beam does support 12 and 16 bit color formats, but only RGB and not YUV a(whereas OBS supports only YUV formats for this) and without SIMD optimizations, so it can be expected to be really CPU intensive to the point of not being usable in most scenarios. QOIR lossy doesn't support more than 8 bit RGBA anyway.