Transcript: Yjs Deep Dive - Part 1 - TagTeamTalk #004

This is a transcript. For the full video of this talk, see A Deep Dive into Yjs part 1 - Tag1 Team Talk #004.

Hello. Good evening. Good morning. Running wherever you are in the world, and welcome to yet another Tag1 Team Talk. It's a real pleasure to be here today. My name is Preston So. I'm a contributing editor to Tag1 Consulting and also the moderator of today's session. I want to welcome all of our viewers and our listeners, whether you're joining us by video or audio today. It's a real pleasure to welcome you to our session today. Today is gonna be a really interesting session for me personally. I have long been interested in learning more about Yjs, a collaborative shared editing system. And joining us today to talk about it is our three guests who are from all around the world. I'm going by the way, from Mumbai, India. And first and foremost, I want to introduce our special guest Kevin Jahns who's a real-time collaboration systems lead at Tag1 and the creator of library, the framework that is all the rage they we're gonna be talking about today, Yjs, based in Berlin.
Hello. Thanks for having me.
Also joining us today all the way from Switzerland is Fabian Franz. Senior technical architect and performance lead at Tag1 Consulting. Hey, Fabian.
Hey. Glad to be here again.
And of course, last but not least, I do want to welcome Michael Meyers, our managing director of Tag1 Consulting, joining us out of New York City. Hey, Mike.
Hey everybody. Thanks for being here. Thanks for having me.
Alrighty. So for those of you who haven't joined us before, we have several other interesting Tag1 Team Talks that you can find on Tag1Consulting.com. And one of these talks actually is very relevant to today. If you want to learn a little bit more about the foundation of real-time shared editing and some of the things we're gonna be talking about today, I highly encourage you to check out the very first inaugural Tag1 Team Talk available on the website about collaborative editing and it ties directly into this.

By the way, just a quick note for those of you who are joining us by audio, our listeners, also anyone who's joining us using assistive technologies, we are gonna be doing some visual whiteboarding later today during this webinar. And so we do apologize for not having an audible component of that available. However, we're gonna do our best to talk through things and describe everything that's going on in the video as effectively as possible. And we are gonna make sure that you're able to follow along. So please be sure to check out the video. If you're normally a listener to get the most out of the visual aid that we'll be presenting. And without further ado, I want to go ahead and pass the mic over to Mike, no pun intended, was going to talk a little bit about Tag1 and why we're here today.
Awesome. Thanks Preston. Tag1 is a consulting company. We have over 50 people around the world. We specialize in building mission-critical applications, highly available, highly scalable. If you have difficult problems to solve, we're the organization that we partner with. We're here today to talk about real-time collaboration. We think that this is an important part of the future of applications, not just web applications, but mobile and desktop applications. And we can have a fall webinar on that.

But being able to collaborate and work together with your peers on the team is changing the way that we do business. We spend a lot of your time and applications like Google Docs, collaborating with your team. And there are a lot of single page applications, things like project management kanban boards, where Stephen, a few seconds in delay, provide some friction in your workflow. And so being able to see changes in real time, really facilitates working together. And so Tag1 with Kevin and Fabian are building more applications that do that. Right now we're working on the exciting internet for a top Fortune 50 company, which has collaborative shared editing, collaborative shared drawing, whiteboarding and features that enable their workforce to do exactly that. And so, we're interested in doing more of these applications and are excited to begin to the internals today with Kevin.
Absolutely. And for me this is a very interesting topic. Myself, I have been long interested in just a very unique difficulties here are the main challenges that come about with regards to collaborative editing. But first I want to turn to Kevin a little bit and get a little bit of patience from you, just from a personal side of things. What is likely got to interested in this whole idea of shared editing in real time collaboration? I understand that you did a lot of studies in Aheim. Can you tell me more about sort of, what's your background in real-time collaboration? What got you interested in it?
Yeah. So I think it all started back in 2013. I was working. I had a student worker position at the institute for databases and information systems. And I was working on the ROLE SDK which is basically a clone of Google Wave. I don't know if you remember what Google Wave was. But I think it was a pretty novel approach to how collaboration could work. You have these spaces that you could design yourself. And these spaces were calibrated for by default. For example, you could add a widget, an e-mail widget for example to this space and with your emails. And the site, you could have note taking widget, which was also calibrated. So while you read your emails, you can make notes on this widget. For example, if you have a group account, other people could join, also read those emails and make notes too. This may be a bad example. But everything was calibrated by default in Google Wave. And we tried to reproduce that. We had a similar product because The Wave was discontinued, unfortunately. We redevelop that. They had some pretty novel approaches there. For example, everything that you did there, this widget, they communicated directly with each other. So just give an example, in Google Wave, these widgets communicated with the server the server would culminate everything, like sync those changes back to the other widgets and do all that stuff.

But in the ROLE SDK, everything was peer to peer by default. So there was no notion of the server. How this would work, a widget could send a message, a message on a specific topic and other widgets could listen to that topic. And how that message was propagated was actually, it's a bit complicated. It was part XMPP protocol. It was multiuser chat room and it was part cross browser, cross window communication by a special browser communication layer. This was easily adapted features in WebRTC. So a coworker of mine did a really cool demo, Ish Van Curran. He implemented. He put a Game Boy game, Mario, and one of these widgets have the controls to the other widget. And he could control Mario from a different computer using the control widgets because this, it was a peer to peer communication. So, back then everything that I wanted to do was implement a collaborative text area. I had very limited computer science knowledge.

I didn't know about all of these algorithms like CRDTs an operational transformation. Back in the days, actually CRDT was not that. There was not a lot of research about that. So we just try to plug-in an operational transformation approach and found out somehow it doesn't work. Why doesn't it work? This got me interested in this topic. Why doesn't operational transformation work in a peer-to-peer manner? So is about to start my bachelor thesis and I was still hooked on this problem, and I proposed to write a framework for peer to peer collaboration on text, and because the main application here in the ROLE SDK was state sharing, I also wanted to share any arbitrary kind of state between those widgets in peer-to-peer manner. So this is how I got started.
It's a very sort of amazing background. And one of the things that you just mentioned, that was very interesting I think, just in light of our previous conversations was the notion of being peer-to-peer, decentralized, right. Distributed systems. And I want to kind of delve into that a little bit, and talk a bit about kind of the state of shared editing back in the day. One of the things I know about operational transformation, for example is that the biggest issue with it is that there is no a unique identifier that is focused on every single node in what you're editing.

And as a result, you can have certain complex that actually arise because of that. And what I understand about the CRDT is that because there are unique identifiers, no matter where you insert, let's say, a piece of text or perform a deletion, it's always gonna be tied to that location. Sorry. Tied to that single character as opposed to the location, as what happen in OT. And so, can you talk a little bit about sort of, why is it that CRDT is so much more interesting and how does it actually benefit things like these advantages you mentioned around peer-to-peer, off-line editing for example. And especially around things like real-time synchronization, which I think is a very big issue for a lot of people watching this today.
Yeah. So the thing is an operational transformation, it can work peer to peer. There have been research papers about that. But the advantage, in my opinion about CRDTs in general is that all the operations that you define are idempotent and connotative by default. So this means no matter how often you apply the same operation, you will get the same result. You can apply the same operation twice, the second time it won't do anything, for example. And connotative means it doesn't matter in which order you apply those operations.
I think it might be a good idea if you could use a whiteboard to give a little example. When I did my own research, I saw for example, a counterexample. So is pretty good for something like that. Maybe you have something.
Sure. I will share my screen. You wanted to get into the whereabouts of the algorithm. And part of the interview actually was about sharing how this algorithm works. Let's just have 10 minutes and talk about the algorithm. Maybe we can even compare operational transformation and CRDT just to give you a quick idea of what the advantages are. I don't want to say that any of them is better, but I know that CRDT's work better for me because they are in concurrency, they are easier to understand. At least to me. All right.
Kinda interesting, even though it's a little bit even more deeper topic. But those research papers that directly compare those two approaches and kinda provide general transformation framework where instead of working on the baud rate, raw data you are transforming into another model. And then there's more similarities to bolster those technologies. I found that idea, this approach pretty interesting.
Yeah. Definitely.
Before you begin, Kevin, I just want to point out that the whiteboarding application that you're using is actually an implementation of Yjs for collaborative whiteboarding and drawing.
Yeah. That's why wanted to use this whiteboarding application. So, during this demo you will see the other users the cursors and other users can do stuff to the whiteboard too. I don't know if we're going to do this this moment, but I highly recommend. Maybe I can show this to you. It's room.sh. It's a really cool application.

All right. So, let's just again. I want to give a quick example what operational transformation is and how that works. So let's say we have two users and these users want to do something. What you doing operational transformation is you represent the operations that happen as a timeline. So let's say user one. This is user one. This is user two. He wants to insert character A at position zero. This is going to be our notion for this operation. And he sent that operation to the other user. Now the second user also does an operation. And so at character B at position zero. And now you immediately see this is conflicting. There are two possible outcomes for this. Either we end up with the document AB or the document BA. In operational transformation, the transmission is really easy because we just compare by user ID. So, the left operation here, it was created by user one. So it gets precedence over the operation that was created by user two. So we will end up. So we start with the document A, and we get operation B, inserted at position zero, because two is higher than one. We insert B to the right of A. And at the right side, we have the document B. And we see that when the operation A, insert a position zero comes in, the client ID one is smaller. So we should insert A to the left of B. And all the peers end up with the same document. And I think this model of showing concurrency is really helpful if you think about how two users work concurrently. You can model pretty complex cases like this. For example, what happens when based on this operation, something happens here and then maybe a third client also insert something here. So these are just some examples.
I think it would be very good to show now, at the timeline part of it because when now you insert client, that's user 83 is inverting at C, but also at position zero. But it counts timewise after these other operations have completed already, then something else happens.
Yeah. We could go over all these examples. The thing is there are some examples where the simple approach won't work. Though I need to mention that operation transformation can work on the peer who came in at like, even with three clients. But you need to figure out the right algorithm for that. So it's just important to know not to get lost here. I just wanted to give you a quick overview of how that could work. I'm not sure if you should go into the details why are there are some scenarios where this is complex.
I know. Just what I meant is, that if you now have this with the three and they're inserting a C at position zero, but it happens at that time point where you're doing it, the again end at the CAB like it should be, because that operation comes after those other operations of completed. So there's a time component to it. Not just the conflict kind of problem.
That's actually right. You're right about that. So for example, these two operations now come in before this user creates, insert C a position zero. So this means, this operation here and this operation, they got distributed to user three. And now user three insert something at position zero. Now he propagates a change to the other clients, and they know that this operation insert C at position zero is not concurrent anymore to the other operation. So they don't need to transform it. The peers only need to figure out which operations are concurrent to each other and then do some transformations. This is a general idea. Transformation by position. Maybe I can. Oh, hello. Maybe I can give a quick example of how that works in Yjs.
Yeah. CRDTs. Will be really interesting to see this compared.
Yeah.
I agree. I agree, absolutely.
So. In CRDTs, the concurrency model is a bit different, and I think that this model of having these three users in the timeline, is really hard to understand as soon as you get to three or four or five users, can have these complex scenarios. Like for me, it's really hard to understand those timelines.
The principal to understand the timeline, the other part that's really hard for me. But operational transformation is. Then also, you now have to look at the. Now there's not just three users, but four. Like 12 users et cetera. All those operations are coming in and it just gets really complicated really fast.
Right. Actually, my first attempt of implementing this was based on operational transformation. I tried to figure out all those cases, and I tried to reproduce scenarios. I had implemented framework to show me cases where my operation transformation approach didn't work. It would give me those examples, with sometimes seven users sending 15 operations, and I needed to figure out, okay, where is the real problem here and how do I really narrowed the problem down to a specific thing? And then you find out, okay, here's the problem. This is why this approach of operational transformation doesn't work from peer to peer.
Investor has a problem with peer to peer, that many people are familiar with gets leery now because we could have a petition, and then clients are collaborating outside of each other and then client, Swede comes in says, hey, I'm also here. I've done some work. Please take me. And all the others are like, hey, we already collaborated on this document for five hours. Where have you been?
Yeah.
Kevin, have a quick question. I know that Google Docs for example is based on OT, and it caps out at a certain number of users. is there any way related to what you're talking about? No, not at all. So in Google Docs and most other implementations of operational transformation, you always have central server that does the transformation. So this is a really sweet thing, because as soon as you have a central server, you have a central point where you can do the decisions of how document should look like. And the problem becomes much, much easier if you have centrality. So, because now you don't have to think about the case that user wants, sends an operation to user three, because this always happens over a server.
Yeah. And the server indexes are very, very important. It can determine a global order of all operations, which is the main thing you need. Because then you have a finite state machine and not this mix and match of everything, every user doing anything.
Right.
That's a really great illustration, I think, of the primary benefit of CRDT, right. Is that no longer dealing with something that's relative, you're dealing with an absolute sense of what's changing. This would bring it down to the level of some of our viewers who might not understand. What that means is that you no longer rely on previous things going on. You can get a sense of what is going on based on that share history as you both mentioned
Right.
Maybe we go over and make the whiteboard for the CRDT now.
Yeah, let's do that. Okay. I have a special notion of how I like to represent the algorithm in Yjs. And so, this is not really this general for CRDTs. This is just specific for Yjs. And I came up with the algorithm with a visual representation of the concurrency model. And it somewhat goes like this.

So we start with a document AB, then we want to insert something. For example, the orange user comes in and wants to insert character C between character A and character B. And I'll always give each character a unique identifier. This identifier is known as a Lamport timestamp. And it doesn't really have anything to do with actual time. It's not like Unix time stamp. It's just a position in the system. So something that you usually do. Sorry for that. Is you define the Lamport timestamp as the user ID or client ID. So in this example, the client ID will be zero and a clock. An ever-increasing clock. And this is going to be one. So user zero. Oh. Now we always started zero. We should always started zero. That's my opinion. Computers time should always start with zero. So, character C is inserted between A and B. This is the semantic of this operation. Now we can insert another character, for example D, between C and B. This is how it would look like, user zero, and clock increased by one. So now we have the document A, C, D and then B. You can see the link lists, and make an arrow always here. Will represent the concurrency model and the operations that happened. And outcomes to show you how concurrency would look like in this model, would look somehow like this. And let's say the blue user comes. And he wants to insert E between A and B. You see that this operation insert E between A and B, that is concurrent to C and D. And in order to resolve those conflicts, we do the same thing as an operation of transformation. We just compare by user ID or client ID. This client, the blue client will have client ID one. Clock is going to be zero. So since one is bigger than zero, we end up with the document. Whoops. A, C, D. This is here. C and D. And then E, because he has a bigger client ID, and then B. So this is the same approach, the same idea as an operational transformation. That it has different representation of how things are inserted. You can see that instead of using absolute positions, we always use IDs, because each character is identified by a unique ID. And the really cool thing about this is if something happens not between character A and B. Like somewhere else in the document, there is no concurrency at all, because they don't need to be transformations. This doesn't really affect these operations.
Just to give some examples for people. Not many of us knows how they can visualize that. It's similar to how if in the internet, instead of saying, hey, give me that server with the IP address, whatever from Google or from Tag1, you would say hey, give me that server that's three hops away, where we have to go through bridges through the internet. Everyone would totally say, hey, that's difficult. Where should I now? How that is, et cetera? The other part where you can think about is, if you're familiar with object three into programming. Inside all of those are kind of like rich objects in your ecosystem of things.

And for now, just think about them as understanding machine conducts, like processors interacting with each other then saying, hey, I want to be here and here. And now we can see every object, kinda lives on itself and they can all see and communicate with each other, kind of through those messages. And with that, we can give every object in this distribute system kind of this unique address, like the number times number, like Kevin said. And with that you can really address it. If you have a unique identifier here for every character. And that's also important because regardless of where the character moves within the last 24 hours, like you're trying to soft line for 24 hours, and the B has moved to the very, very, very, very end. The client would still insert between A and B, the new F client that really wants to do this operation, because A and B can still be determined of where it should be. Determine also with this order here, which before was defined by the global server, can be determined by the Times itself because there is a logical order to it. And then makes this so powerful in that, so instead of position based like this is here, this is here in the position now changes from position two, to position 20,500 and 781. You could just say, hey, it's still B. Well, it still D. It still has the identifier they both kinda want. And that doesn't change. And that's so great about it. But it also brings some challenges. And yeah. For example like, large text. Every character needs.
Yeah, every character basically needs a unique ID. And this is one of the challenges. Like, one of the problems actually in CRDT is. It is a good concern. Is this actually usable for text editing? Think about a book that has like, hundred thousands of characters in each character is an object. And do that and JavaScript, this seems really imperformant, right. So these are some of the challenges that you you to consider when designing an CRDT. It needs to work with hundreds of thousands of IDs, and everything is an object. And there are ways to optimize that. I basically have two work really closely with how V8 or how the JavaScript engines work in order to optimize correctly, so that objects get represented really efficiently in memory instead of. Like for example, instead of using objects for everything, you can find these objects in a way that JavaScript will represent similar to a struct with C. So this is way more efficient than defining everything as a hash map, a hash table. Because hash tables will be overkill for this.
Yeah, I think you both have really well shown the fact that what we're really doing here is adding a lot of complexity. But it's also gonna make things a lot more efficient. As you mentioned, Kevin, this notion that every character is an object and that means every single one of those characters has to have a unique identifier. That leads to a lot of issues around memory and garbage collection. And one of the things that I'm curious about, actually myself now, just to bring this back to really, some of the questions that our audience is gonna have. One of the things that I know is a big problem for me. And I know this is a problem that all of us share. Is that trying to let's say, perform a large copy paste into a document. In Google Docs for example, it can take up to 10, 15 seconds for the synchronization, that sync process to finally complete. Or when you come back online and you get that message on Google Docs saying, oh, there's a lot of changes in this document. Can you talk through a little bit about how you can deal with those kinds of issues using this kind of, this approach with CRDT?
Oh yeah, definitely. So there's a really nifty trick that I do. And I create a lot of tabs. Okay. So instead, I told you that each character is uniquely identified by some kind of code vector like this. And most of the time, users type in a row. For example, they type A and then they type B. This is going to be like this, and then they type C. So, two of course. So they write words. They write characters in a row. And especially when you insert a lot of content, it will always look like this. The clock will increase always by one, with every structure or every operations that you do. So something that you can do is, instead defined those operations like this. This is find zero. Sorry, I need to redo that. Instead of defining this operation like this, you can also define it like this. And it has a length of three. So it basically is A, B, C. It is attached to clock zero. But it has three operations. And this is what I do basically in JavaScript on text objects. Concatenate those operations so you don't need an object for every single character. You can actually have a bunch of characters together, defined as a singe operation. But this makes addressing things with more complex. For example, when another user wants to insert something between B and C, for example, you need to split up this operation into two operations and then insert it in between. So addressing is going to be more difficult, but the memory representation and also the footprint of the database what you store your document updates is going to be much lower. This is also the way.
So basically compressing into a range. You're just saying, hey, this is kinda just increasing the order so it can just compress into a range. And if someone even wants to change something within this logical bit text, just then I'll split it up. The thing that interests me personally, are you also doing like compression after the fact or only when inserting? Like, if someone was inserting an X and then way later someone else was removing that X again. So now that phrase could be combined again, are you doing that or--
No, I'm probably not. I can never ever combined again because in CRDTs you cannot delete characters.
No?
You can never. You can never actually remove a character from the documentary. You can always only mark it as deleted. But again, here's something really, really cool that we do. Just an example. Let's say we mark this X as deleted. Let's just say underline is deleted. Actually better, make a D here. So this means this X is marked as deleted. You can remove the content of that operation, which really makes sense when you have concatenated operations. So sometimes you have operations that have a lot of content. When you market is deleted, you can delete all the content.

And this really pays off in production, especially when you have a large insertion, and then you delete this large insertion. And an example of ProseMirror, or like any structured text editor. So what we do here is, we have a node, something like a paragraph node. And in this paragraph node, you have several text nodes. For example, A, B, C. When you delete the paragraph, you mark it as deleted. Oh, what did I do here? You mark it as deleted. You can actually mark all these structs also as deleted. And you also don't care about the order anymore. So, you can actually remove these structures or these operations completely from memory. It doesn't matter, because these were children of paragraphs that were deleted. I hope this makes sense to you. I actually don't know.
It does to me, certainly. Absolutely.
Okay. So, this address some ways of how you can optimize CRDT implementation, remove content, handle memory efficiently. Especially consider how the JavaScript engine works, and almost all. Actually, all JavaScript engines like Mozilla and V8 that Chrome users and Safari, and also Internet Explorer is going to use V8. Always have a special optimization techniques that you can use to optimize how an object is represented in memory. If you always define an object, for example, as. So roughly, this is how and operation will look like. It has an ID. It has associations to the left. Left and right character. And what does it have too? It knows if it is deleted. Yeah, that's basically. And it has the content, right. So this is how an object is represented. And if you always define an object like that, if you never add things to that object, the compiler can actually figure out that this is always the range that matters. So we can store it as a struct in memory. So instead of using a hash table, that what you would normally do like an object, basically hash tables, you can define the JavaScript compiler will optimize this as a struct, which will use very little memory. And yeah. In the browser or in the JaaS.
Okay, that makes sense. Because say the browser does not. So we're talking about leg with optimization here, of how we can optimize JavaScript itself. Because if I have an object that can grow, et cetera, I need a mesh table with my address, so I can find it, can garbage collected, et cetera. But if I have a fixed length data structure, then I can just allocate 10,000 of that and can use them, and can de-allocate them again. So yeah, that's a really neat way of optimizing the end terms here. Let we quickly go again, so that I've understood this correctly just were also explaining again for the readers. So, when we were talking about those ranges, we were basically saying, even though we are representing this is arrange, internally we still now, that B for example, still has the adverse of zero point comma one. And now you are saying that. Now I've started a paragraph, and now I've deleted the content of the paragraph. And now all the information that's in the paragraph doesn't matter anymore. How does it deal, if for example, someone. In operation comes in very, very late that wants to insert the Y between A and B, but ABC has already been deleted. How does it deal with that?
So, that's a really good question. So what happens when these marker is deleted. It doesn't matter, actually. If the mark is deleted, they are still there. Maybe their content also was deleted, but the operation itself, it is still there. And this is why concurrency still works, because you can still address a deleted character. Right. You just market is deleted. You just say it is deleted. But, it is actually still there in the internal format. To the user, you wouldn't so that character, but internally, you still need it to handle concurrency.
Okay. But you said you had some optimizations for that, so that for example, I'm pasting the Wikipedia. Probably not the hole, but some very large pages. Like the largest page in Wikipedia. And now I'm not using a node, but I'm just deleting all the content again or Mike is deleting all the content again. I paste Wikipedia again, and Mike is deleting it again. How do you deal with that case if you don't end up with 1000 copies of the Wikipedia?
Right. So, how do I handle that? They're are basically two modes in JavaScript. You can have garbage collector enabled. The garbage collector actively removes stuff. When you copy, paste Wikipedia article, you can delete content. Let me give you an example. Assuming your post is really, really, really long amount of text. So it really goes on, and goes on. And this is addressed by user zero, clock zero. So it is a really huge operation, and it has like, let's say 100,000 characters. Right. When you mark this as deleted, it gets transformed to. You can delete the content, right. So it will look somehow like this. Nothing. But you know it is deleted. It is a deleted operation. And still, client zero, clock zero, it has a length of 100,000. And it is marked as deleted, and it doesn't have any content. So this is really cool. Now you can insert a lot of content you deleted, and the only randomness and. The only thing that is still there is a single operation that is basically just three integers and a Boolean. Let's say, there was once a really, really long insertion here. But now it isn't anymore. We know it is really, really long. But we don't know what it is anymore.
B and C which would be between one and two? So now, it was inserted in this whole chain that was deleted basically. And so it could still position it relative to the other text that is around this large pasted text.
Right. So, assuming something was inserted before deletion, between B and C, right. I have a green user here, and he inserts X between B and C. Now another user wants to insert X also, but now the operation is deleted. What will happen now is, is operation gets split up into two operations. The first one is zero, zero. How much is this? One, two? So, this is a deleted operation of a length of two characters and now we have X. Now we have a deleted operation, zero, three. Actually, zero, one. So it should be two, right. I can't count. Counting with zero is pretty hard sometimes. Okay, so it is two. And it has 100,000 minus two. Just say minus two. So, it gets split up into two operations, basically. So. And this is the whole trick. Now we have a bit more operations, but the splitting up, it is really worth it if you do a lot of huge insertions and then delete all this content. Basically what authors of, what book authors do or like any. A lot of people, even me, they write a huge paragraph and then delete the use paragraph does it wasn't good enough or they move the paragraph to a different location. And what happens now is, this paragraph was deleted and then inserted in a different position. And this is really beneficial or Yjs because now the amount of text is represented way more efficiently. Especially in long term, this really pays off with huge document storage improvements. Yeah.
Very cool.
Another advantage how you can make this efficient is with efficient coding techniques. So when you send all these operations to the other peers, they really need to be represented efficiently. When you have 100,000 characters, they are not merged or concatenated. They're just operations. If you send that as JSON objects, you would really blow up your memory buffer, right, because JSON is a pretty inefficient format if you have a lot of objects. So, something else that really paid off is binary encoding for that, because now we can. You can encode each integer way more efficiently. Integers in binary are represented as four bits. In Yjs you can often represent an integer as three bits because of variable length encoding. This is what Protobuf, for example, also does. For example, the integer one to 127 is represented as a single bite. Other numbers, more space than that. But, it can be the really efficiently. If you do that and JSON, it depends only on the link because they are conformed to a string. So a number, simple number can be represented in 128 bytes. Not bytes fights, in like five bytes. So, this is also one of the advantages of binary encoding. So I need to drink something.
Of course, you've been going on. You both have been going on for a long time. And you know, we are coming up on time. So I do want to bring things back to. You know, we've heard a lot about the technical underpinnings. Not only CRDT and how it differs from traditional OT, but also some of the ways in which you've introduced these amazing efficiencies into Yjs. I'm curious though, just to jump right back into how the audience can help out maybe contribute some of their energy towards the project. First of all, what's sort of your. I wanna ask two questions to you, Kevin. What are some of the things that you're working on next? Like, what are some of the things that you wanna add to Yjs? And you know, given that Yjs is open source, given that we're all such open source fanatics here, how can we get involved in helping you build out Yjs in the future?
I am gonna first answer your second question. I would be really happy to, if more people get involved. It should be really easy but now to do that. So, discussion board. They can post your projects, your additions to Yjs. Yjs itself is a pretty small library. So, the Yjs project is only 10 kilobytes of JavaScript code. It's a very small module, and it only handles the concurrency part and how to encode document updates. And all the rest, for example, the communication over WebRTC, over web sockets, the communication over for example the IPFS network, that is another underlying communication protocol or XMPP. Like, there's so many additions to Yjs. All these additions are separate modules. If you're working actively with one of those projects. For example, if you have a lot of experience with that or IPFS, I would be so happy to work with you on implementing a communication module for Yjs. There are also other things. For example, editor support for example for CodeMirror, for Ace, for ProseMirror, for Karel Editor. There are already a lot of existing editors that is supported by Yjs as separate modules. And there are still a lot of editors missing. For example, I would love to have support for Draft.js and Slade. It would be so interesting to have people working on additions for Yjs. Because it is already a very modular project, it would be great to see more people getting involved writing more additions, because this is what will drive this community. There's a discussion board that I set up just last week, that we can use to collaborate, share knowledge. And I would be really happy to hear more about your experience with Yjs, because all these things that we talked about in this interview and the last interview, they're already existing. They can already be used. But there's very little experience in using that. So if we have like, this big. If we can share knowledge on how to implement an actual product with that, it would be really great. Especially when you work on peer-to-peer technology, I would be so happy to work on more. To see Yjs on more peer-to-peer technology.
The other thing, what I'm working on right now is, actually getting Yjs version 13 released. It's been a really, really long ride on Yjs version 13. Version 12 was basically so good, and it had so many additions that it is really hard to replace version 12. But now it is time to actually do that. And there still some work to do, according all the additions that were existing for Yjs version 12 to 13. The next thing that would be really interested in working on is peer-to-peer technology. Adding Yjs support, like writing a batch module Yjs. For those who don't know, it is the peer to peer network where their users share documents with each other in a peer-to-peer manner. It's basically like Bittorrent. It has the same underlying technology to look for other peers that are interested in the same topic. But, you can make additions to a document. And there's a recent addition to that network where you can have multiple users making additions to the same document. In order to make that happen, you need some algorithm to figure out how concurrency works in that case. And Yjs would be perfect for that. So this is what I'm going to be working on next. Yeah, definitely do that. That's even a peer-to-peer YouTube clone like thing. It's kind of crazy because the data stored like, nowhere. No central server at all. You always have to wait a little bit till your video loads, but then is coming from all over the world, because everybody has a little chunk of what your, of what you want to watch right now. So these peer-to-peer technologies are really, really fascinating and interesting. Also speaking for the Drupal community here, as a tech monster, being one of the Drupal experts obviously. I'd really love to see if someone would be trying SDK forward which all of our one million, Drupal websites. My Drupal websites as 2% maintainer. So the one million sites could just use and order templates sites. All the new ones could also. So that would be really cool if there was just like a Yjs backend, someone's fondling it and then all of those that came before. I've seen how great it worked with ProseMirror. If they all could be calibrated that would be fantastic. And also just as a journalist, saying I've seen several of the source project in my life. Develop several of my own. This is kind of stage where you're kind of like, they don't fully, that adoption. And if now, many people would be using Yjs, would be developing apps with it, and that will be totally great in getting the feedback. It's not just code contributes. It's really just using this, giving feedback of what works, what doesn't? What's great, what isn't? And I think this is really what pushes open source projects forward, what kind of the first step large adoption. And the contributions come from themselves in my experience.
Absolutely. Yeah, I think we can all. Four of us on this webinar agree that Yjs for me is really interesting because it is part of this revolution towards peer-to-peer technologies. And we're seeing a lot more of this kind of decentralization occur. It's a very important trend, and will have a lot of impact in the ways that we collaborate in the future. We are out of time. But I did want to mention a couple of things. As Kevin mentioned, we're looking for people to help contribute and offer their insights into how they're using Yjs, how you're implementing Yjs, what you're doing and also some insights into other integrations. You know, Fabian mentioned Drupal. There's also a lot of interest in other respects editors. I wanna just call it a couple of things about Yjs, very quickly here. If you're interested in contributing to Yjs, or if you're interested in looking into Yjs, please go over and visit Y-JS.org also it's available on Github at Github.com/Yjs/Yjs. Is that correct, Kevin?
[Kevin] That's correct.
You grabbed that namespace. That's a great namespace to have. Those three letter ones on Github are tough to have.
That's right. I recently got that. Also, did you mention the the old URL for the project? I think also have the Yjs.dev URL now. So, this is what were going to use from now forward. All the sweet little two letter domains.
Yes, because adding that hyphen is so much work, right?
I did want to save you time.
Absolutely. So Yjs.dev. And I think what's interesting, is I just went to Yjs.dev and you actually get this really awesome live demo, right on the side right there, that you can take a look at. Hey, I want to say thank you so much to Fabian, and to Kevin, it's been a real pleasure chatting with you today. All the links that we've been talking about during this session are gonna be available in this description, wherever this page is. Please don't forget to check out our in order webinar about this very topic on share, where you can also hear more from Kevin and from Fabian as well. And if you're interested in learning more about a particular topic, you want to hear more about some of those things we were talking about today, you want to learn more about CRDT, you want to learn more about some of these interesting ideas, go ahead and shoot an email over to tagteamtalks@tag1consulting.com. Once again, I want to say thank you to Kevin and to Fabian and to Mike today. We're gonna post these talks by the way at Tag1.com/tagteamtalks. And without further ado, thank you so much and until next time.