While implementing Sixel, it is important to test with images that contain transparency.
Transparency can be achieved by drawing pixels of different colors but not drawing some pixels in any of the Sixel colors, leaving the background color as it.
I believe this is the only way to properly draw non-rectangular Sixels, and would be especially nice with the background acrylic transparency in the new Windows Terminal.

Testing using WSL with Ubuntu for example, in mlterm such images are properly rendered as having a transparency mask and the background color is kept, while in xterm -ti vt340, untouched pixels are drawn black, even though the background is white, which seems to imply they render sixels on a memory bitmap initialized as black without transparency mask or alpha before blitting them into the terminal window.

OOh. Sixel is very cool stuff.

I've decided that I need that. NEED.

I'll happily review a PR :)

Caught the Build 2019 interview today that mentioned this request. I still maintain that Xorg on sixel is just wrong. So very very wrong.

The ffmpeg-sixel "Steve Ballmer Sells CS50" demo never gets tired tho. Gotta say, it is a little disappointing the video lacks sound (sound really makes the video). Consoles already have sound, naturally. They totally beep. Precedent set. What we really need is a new CSI sequence for the opus clips interleaved with the frames, amirite?

Related: #120

Need.

LOL I was watching the stream and I just thought to myself "here's my boss assigning me work live in front of a studio audience".

Please make this a priority for v1.0!

3d animations can be v1.5 😛

OMG

Upvoting this request, Sixels would be such an amazing thing to have in the Terminal.

This weekend I finished implementing sixel read support for my MIT-licensed Java-based TUI library, and it was surprisingly straightforward. The code to convert a string of sixel data to a bitmap image is here, and the client code for the Sixel class is here.

I have done very little for performance on the decoder. But when using the Swing backend, performance is still OK, as seen here. (The snake image looks bad only because byzanz used a poor palette creating the demo gif.) I was a bit taken aback how quickly it came together. It's very fair to say that the "decode sixel into bitmap" part is the easy bit, the hard bit is the "stick image data into a text cell, and when that is present blit the image to screen rather than the character".

Just want to mention it to other folks interested in terminal support for sixel, and hoping it could help you out.

I'll upvote if someone else writes a Jupyter notebook client ;)

We already have an example of Sixel support in mintty which is written in C (vice java). Only thing needed is a refactor to C++ (at least for initial support). Still always good to see how it's been implemented in other projects.

At the moment, I don't care 😋 We know that sixel is something we need to work on, we've got some steps towards getting it to work done already. I'm fairly confident that @j4james is continuing to experiment with it. When we need to wrest control of this thread for our own feature tracking, I'll come back through and mark it all as off topic. Till then, go for it.

1. rasterize terminal text characters to image/texture.
2. overlay any graphics (sixel or otherwise) on image/texture created in 1.
3. blit image to screen (the compositor, really)

once 1 is working, which requires 3 to display, injecting 2 should be of relatively little concern. for game developers, I bet all three of these could be done in a week. they live and breathe rasterization and blitting.

keep your eyes open for a performance-delivering update to Terminal in the coming year. I bet you that sixel support will not be far behind. I have no inside information, and this is all educated guesswork.

my understanding is that 1 and 3 are being worked on, now, for performance reasons. number 2 won't be far behind.

Haha. Github comment of the day! Source code or it didn't happen. :-P

Source code or it didn't happen. :-P

Here you go:
https://gist.githubusercontent.com/j4james/9c2e67686306e2c37aa07e71fe1d2504/raw/7e0a1d7c6a0206e801241b4f23324c5a6a2d0997/owl.txt

But note that it requires a terminal that can emulate a 10x20 cell size for the owl to be positioned and sized correctly. I've also simplified the original code a little, and cut the image palette down to 15 colors, so it should theoretically work on a real VT340 now.

Success! Tested on a real VT340 and it worked perfectly.

For evidence, here is the output from the VT340 after I told it to send a MediaCopy to the host (essentially a screenshot) of the VT340 screen in sixel format:

https://gist.github.com/hackerb9/fb5eb56391e51de23af6dd5cedb12464/raw/ac9032df3afd62ea6e5f6f0b6a5621923c1a1630/vt340mediacopy.six

And here is that MediaCopy file converted to a PNG:

Note the occasional glitches where a byte has gotten its eighth bit set high is an artifact of my mediacopy.sh script. The owl looks perfect on the VT340's screen.

@naikrovek cough cough Scroll all the way to the bottom for some notes on mixing images and text in a single cell.

Has anyone requested SGR-Pixels (mouse mode 1016) yet? I see it mentioned here as not supported. Decode sixel, add SGR-Mouse, and you've got a viable new gaming medium. Sure would be nice to have some roguelikes that could put real images in when they needed them.

Or if we need to be all business-y to justify it, how about a nice little MSPaint app that works over ssh?

@PhMajerus

While implementing Sixel, it is important to test with images that contain transparency.

Transparent sixels are available here from @hackerb9 and here from me, and can be generated by @hpjansson 's chafa and the git head version of this.

I wasn't considering 1016, but I do have a POC of the DEC locator mode. There's no point in doing either of them until we have sixel, though, because the pixel coordinates will need to be tightly coupled to the sixel resolution.

Moving a discussion comment from #13024

In theory, we could work on DRCS and Sixel support without having to figure out how they transit ConPTY up front, which will let us parallelize that work

Could we now just flush the frame with TriggerFlush(false) to end any current buffered conpty content, and then pass that string through? I think we're getting pretty close here to getting the strings to the Terminal, albeit not rendered yet

Could we now just flush the frame with TriggerFlush(false) to end any current buffered conpty content, and then pass that string through?

I don't think so, no. I suppose if all you want to do is "cat" an image from the command line with something like img2sixel, that might suffice, but anything complicated will probably break. We either need the full passthrough mode working, or for the conpty renderer to be capable of regenerating the sixel on the fly so it can repaint areas of the screen that have been invalidated. Of the two, passthrough mode seems more feasible.

sixel support is rapidly becoming as mandatory as ANSI escape sequence support. Please do not allow Terminal to become as obsolete as the old Windows console host by not implementing sixel.

As far as I know, terminal emulators are the only platform capable of running cross platform, statically linked, mouse driven apps that can even be run remotely over ssh.

The main feature missing from this platform is a universally implemented graphics API. I've spent a bit of time researching this, and I believe implementing at least one of sixel, kitty, or iTerm2 would be well worth the effort.

IS there something similar to sixel, but for audio? I think it would makes sense over SSH connections...

IS there something similar to sixel, but for audio? I think it would makes sense over SSH connections...

Similar, insofar as that representing anything even moderately complex will require extreme downsampling? Absolutely there is: DECPS.

@piranna : Off-topic here, but apart from what @DHowett mentioned, you probably have more success to stream audio via a pulseaudio TCP connection which you can tunnel via SSH. 💡

@rgpublic I know it's offtopic, sorry for that, but hoped you could guide me. Good to know something exists, thank you to both of you :-)