migueldeicaza / xtermsharp Goto Github PK

XTerm emulator as a .NET library

License: MIT License

C# 99.86% Makefile 0.03% C 0.11%

xtermsharp's Introduction

XtermSharp

XtermSharp is a VT100/Xterm terminal emulator for .NET, the engine is intended to be agnostic from potential front-ends and backends.

Consumers use a higher-level implementation that provides integration with their UI toolkit of choice. This module contains both implementation a Cocoa/Mac and a Terminal.Gui implementations that can serve as a reference to other ports.

The terminal itself does not deal with connecting the data to to a process or a remote server. Data is sent to the terminal by passing a byte array with data to the "Feed" method.

Convenience classes exist to spawn a subprocess and connecting the terminal to a local process, and allow some customization of the environment variables to pass to the child.

The SampleTester program is an internal test program that exercises some of the Terminal API.

Long term, I want to have an embeddable console for MonoDevelop/VisualStudioMac that is a proper terminal, so I can debug things like gui.cs without launching an external window as well as having a full terminal for my Gui.cs library.

XtermSharp.Mac

This is a library built on top of XtermSharp that provides a MacOS NSView that can be used in your applications to embed a terminal emulator. This maps the Mac input to send data to the underlying terminal.

Like XtermSharp, this does not wire up the terminal to a backend or a process, this is something that your code still needs to do.

An example terminal on how to connect a Unix shell process is available in the MacTerminal project.

Features

The engine is pretty good at this point and lets you use Emacs, vi, top, mc, blessed and gui.cs both with the keyboard and the mouse. Some features are only available on the Mac version, as the text-version with curses imposes some limitations.

The unicode handling is pretty decent, but does not have support for Grapheme clusters yet, that is coming up later.

Scrollback and reflow are currently missing, those are tied together, and I think it can be done better than the original xterm.js design supported.

Roadmap

There are many issues filed that track some of the capabilities that are missing, the list is by no means complete. What I would like to do is package this in a reusable form for all available platforms (Windows, Mac, Linux, iOS, Android) so others can use this to embed terminal emulators where necessary.

There is currently a native piece of code needed to fork/exec safely on MacOS in the presence of MacOS recent "Runtime Hardening".

Some of the Gui.cs code needs to be moved to its XtermSharp.GuiCs library, and the SubprocessTerminalView needs to introduced for MacOS.

History

I wanted to have a console emulator for .NET since the days of Silverlight, but never quite got around it - but having an engine that could have different front-ends and backends is something that I have desired for a long time. I did some bits and pieces some years ago which are not required nowadays.

This code was originally a port of xterm.js, but has been adjusted for proper processing Unicode using NStack and to make the front-end pluggable in a .NET way. While xterm.js was a useful starting point, the code has deviated and there is limited value in continuing the port, this code can now live on its own.

xtermsharp's People

Contributors

Stargazers

Watchers

xtermsharp's Issues

Add a way to specify the desired TERM and Mouse Support

Events in XtermSharp

XtermSharp started with a TerminalDelegate pattern to notify of events, and overtime got a handful of .NET-style events later on added. This was because the TerminalDelegate was useful for Terminal, but the higher-level constructs did not use this.

I want to make XtermSharp mirror the design of SwiftTerm, where the notification pattern is always the delegate, even if this means re-publishing events to other delegates.

So for example, the TerminalView for AppKit implements TerminalDelegate and in addition, reports events via its own TerminalViewDelegate and so on.

Buffer recycling glitch

I am still trying to understand the root issue here, but if you set the scroll back to something low, like 5, and cause the buffer to start recycling rows there are 2 glitches that happen. The first is a blank line inserted in the display buffer and the second is a ghosting of the first line of data.

Debugging this in UpdateDisplay, the buffer has 53 lines, the first contains the ghosted data, the last contains a blank line (which maps to what we see as line[0] is recycled).

With a breakpoint in UpdateDisplay that stops when the content we are updating contains the string "Xterm" and a breakpoint in Terminal.Scroll(bool) on line 301 (buffer.Lines.Recycle ().CopyFrom (newLine)), we see that the str contains the text before we start to recycle lines. What appears to be happening is that the terminal view's buffer is being loaded with text from the terminal buffer, the terminal buffer is full. CircularList.Push bumps startIndex immediately after adding an element when length == array.Length.

It is possible that CircularList.startIndex is being incremented too soon, since no additional items has been added yet. Actually, this might be the case, looking at xterm.js, length is incremented after the check to adjust startIndex.

Needs support for OptionAsMetaKey

......otherwise I can't use Emacs :)

Test Coverage

TerminalController.StartShell should be StartProcess

That reflects better what this does.

some remarks regarding the parser

xterm.js' parser was written with JS string (UTF-16 codepoints) in mind. Plz note that it will not work correctly with raw UTF8 bytes (for any byte with the 8th bit set). For this to work you would have to rewrite Parse in a way to correctly decode multibyte chars on the fly. Another and prolly easier approach is to simply decode the whole byte chunk beforehand to UTF16 or UTF32, this way the parser can stay mostly untouched. In xterm.js we went with UTF32 after some longish testing, mainly for performance reasons (with a small memory sacrifice compared to UTF16).

Furthermore the older version of the parser has some loose ends, that are already fixed upstream:

some rare control codes are not handled correctly (not a big deal, since those are not part of typical terminal data, see the updated transition table)
support for the colon subparam notation, which is needed for true color sequences (see xtermjs/xterm.js#2241) with explicit ZDM setting. ZDM is a tricky topic, its deprecated since mid 80s and not part of the latest ECMA spec, but since all DEC VTs are operating in ZDM we kept it for now. Note that this might change with newer sequences though.
function identifier based handler registration to better reflect the ECMA defined function space determined by prefix, intermediate and final bytes (see xtermjs/xterm.js#2346)
support for REP (repeat previous character, ECMA-48, see xtermjs/xterm.js#2199)
some bug fixes regarding abort conditions during OSC/DCS sequences. The old parser would always enter the handlers, which is not wanted for SUB/CAN (again not a big deal since hardly seen)

Last but not least the parser has seen some code/perf improvements, but those are mostly JS engine related and less likely show a big impact in C#. On a sidenote - the somewhat complicated print handling with aggregating chunk slices shows a major perf improvement in JS (5 to 10 times compared to single char handling), which is not true for the original C parser. In C jumping for every single char into print is much faster, prolly due to heavy inlining/unrolling (JS JITs are still lacking in this regard). Not sure how C# would do here. In general the perf of the parser should not be an issue, in JS it handles typical sequences with 80-150 MB/s (the C variant only being twice as fast).

Kill TerminalOptions

It really should just be properties set on the terminal.

Add support for CSI 't' sequence

This has a handful of operations, implemented now in SwiftTerm

TerminalView.SelectedRange implementation

This should return the range of bytes, currently it returns nothing

This is expected by NSTextInputClient

TrueColor implementation

Implement scrollback

Screenshots

Needed: Winforms front end

Parse() should be based on bytes, not Rune

Now that I figured out how this is working, that needs to be updated.

Move Mac specific pty code from XtermSharp to XtermSharp.Mac

XtermSharp has mac specific methods in it HeavyFork, SetWinSize, and AvailableBytes.

The latter two I'm not sure about - the ioctl codes are prefixed with MAC_.

Audit Parse() for allowing utf-8 stuff.

GUI.cs backend

This describes the tasks to complete an Xterm control for https://github.com/migueldeicaza/gui.cs:

Upgrade to Terminal.Gui 1.0
Mouse support
missing colors/attributes
explore the effect of utf8 envy variable on output support (why are drawing characters not rendering at work)
terminal size vs visible region: auto adjust or support scrolling
inout handler - need an “escape” hit key

Implement gui.cs support

Cursor keys don't work in Mac terminal

Cursor keys don't work in Mac terminal but do in GuiCsHost.

Move PTY/Fork support into the core library, so that clients can easily use.

Pending backports from SwiftTerm

These are changes for improved correctness, few people need them, but they make the test suite work better:

Mac App:

229b22e643ef7e4965786d79f03fb0bc34f238c8
b69995142d7ac4acece0d60139e6b917f3b51ff5

Implement Buffer Resize

Some keys auto-repeat, some don't

E.g. i & n repeat, j & k don't

SGR attribute 9 and 39 (crossed-out and font-crossed out)

Implement Selection

Cocoa Implementation

Things pending:

Cursor drawing (block, underline, blink)
Complete color rendering
Default colors, customizable colors
Scrollback capability
Blink support in text
Allow fonts to be configured, and add fallback for Bold/Italic to use properties in NSAttr

Screen scraping API

Now that I made HeadlessTerminal public, as I need it for fuzzing, perhaps I should surface more information about the contents of the screen and make a handful of internal classes public.

APIs to surface:

Saving:

save (to: URL, format: SaveFormat = .txt)

Accessing content"

Writing to the screen, some basic operations, could surface more:

Record input/output:

setOutputRecorder (...)
setInputRecorder (...)

24bit fg/bg color is not implemented and I haz questions...

https://github.com/migueldeicaza/XtermSharp/blob/master/XtermSharp/Terminal.cs#L683

		public int MatchColor (int r1, int g1, int b1)
		{
			throw new NotImplementedException ();
		}

The code in InputHandler.cs looks right:

				} else if (p == 38) {
					// fg color 256
					if (pars [i + 1] == 2) {
						i += 2;
						fg = terminal.MatchColor (
							pars [i] & 0xff,
							pars [i + 1] & 0xff,
							pars [i + 2] & 0xff);
						if (fg == -1)
							fg = 0x1ff;
						i += 2;
					} else if (pars [i + 1] == 5) {
						i += 2;
						p = pars [i] & 0xff;
						fg = p;
					}
				} else if (p == 48) {

But without MatchColor implemented, 24bit color is not implemented. The spec is:

ESC[ 38;2;⟨r⟩;⟨g⟩;⟨b⟩ m Select RGB foreground color
ESC[ 48;2;⟨r⟩;⟨g⟩;⟨b⟩ m Select RGB background color

I tried a naïve try:

	public int MatchColor (int r1, int g1, int b1)
	{
            return (int)System.Drawing.Color.FromArgb(r1,g1, b1).ToArgb();
	}

I obviously don't know what I'm doing because this always returns 255 even though in the debugger I see my values for r1, g1, and b1 being passed in.

What is the value of the the int fg and int bg supposed to hold?

I built this too to help:

        public class XtermCharAttribute {
            public int Attribute { get; set; }

            public static XtermCharAttribute FromAttribute(int attribute) {
                return new XtermCharAttribute(attribute);
            }

            public XtermCharAttribute(int attribute) {
                Attribute = attribute;
            }

            public bool Bold => ((FLAGS)(Attribute >> 18)).HasFlag(FLAGS.BOLD);
            public bool Italic => ((FLAGS)(Attribute >> 18)).HasFlag(FLAGS.ITALIC);
            public bool Underline => ((FLAGS)(Attribute >> 18)).HasFlag(FLAGS.UNDERLINE);

            public System.Drawing.Color FgColor => System.Drawing.Color.FromArgb((Attribute >> 9) & 0x1ff);
            public System.Drawing.Color BgColor => System.Drawing.Color.FromArgb(Attribute & 0x1ff);
        }

What am I doing wrong/missing?

Rework buffers to keep track of data per-line, rather than on-grid to allow reflow.

Cocoa: in mc, in the text viewer there are subtle lines between rows.

Enter key doesn't always do the right thing

Enter key works for submitting a terminal command, but does not work for selecting a file in Vim or Emacs in their respective file browsers.

It appears to always be trying to insert a line in a read only buffer.

TODO - fold Input and terminal into one.

Lots of crashes when running `cat /dev/random`

There are a few places in the code that can crash when bad input is received from cat /dev/random

Ship libpty in the nuget package for Mac.

List of terminal emulator apps built with xtermsharp?

I've been trying to find more examples of code that uses XtermSharp.

Besides the samples included here, what else is out there?

Mouse selection: double click should select word, triple click should select line.

Currently it does not do this.

Needed: Windows backend

This would spawn a child cmd process or launch bash on WSL

CharData should probably contain strings representing graphemes.

Currently this holds Runes, which is limiting, needs to hold graphemes.

TerminalController

The recently introduced TerminalController was originally introduced to make the run-a-child-process scenario simpler to reuse without duplicating the code in the sample app.

Then the code was extended to handle scrolling. And now we have both TerminalView and TerminalController. I suspect this division could be better.

The scroll handling should be moved into TerminalView and the TerminalControl should probably be renamed ProcessTerminalView or something like that - reflecting the fact that this is a terminal that hosts a local process vs say, be a front-end to a remote network connection

.net Core

Any plan for a future .net core version?

Lucida Sans Typewriter isn't installed by default on MacOS

Maybe use Menlo or Monaco or Andale Mono instead as the default.

This is how it looks without the font installed - not sure which font is being used here.

Mouse support

Mouse support to enable reporting of mouse events, sending mouse events, and wiring that up in the UI frameworks.

Needed: WinUI frontend

Crashes with "vi", due to an invalid sequence being received

InputHandler: Implement RestrictCursor.

Alt/Option + ";" crashes Rune.ColumnWidth

XtermSharp as a Console Terminal?

Can I use XtermSharp to render a VT stream to the console? Kind of doing a Putty but inside the windows console. I looked at the source a little bit and it lacks a Console driver, which I could find in the Gui.cs repo, but I am not completely sure how it works... Does it just copy the whole XtermSharp buffer frame to the Console on each input?

What approach would you recommend?

Needed: WPF frontend

Terminal.SizeChanged is never raised.

This event is currently not being raised.