This is a transcript. For the full video, see Introduction to DrupalSpoons, a new developer workflow for Drupal contributors - Tag1 TeamTalk #017.
Preston So: [00:00:00] Hello and welcome to Tag1 TeamTalks, our series on emerging web technologies at Tag1 Consulting. Today's episode is about Drupal Spoons, a configuration of groups and projects.
Built on GitLab, that offers issues, merge requests, and continuous integration for contributed projects in the Drupal ecosystem. I'm Preston So, and I'll be the host of today's episode. I'm joined today by two of my dear friends and colleagues. Moshe Weitzman. Located in Boston, he's a subject matter expert, senior architect and project lead at Tag1.
We’re also joined today by Michael Meyers, who's located in the Berkshires, Massachusetts, Managing director at Tag1.
I’m your host, editor-in-chief Preston So here in New York City, it's currently raining cats and dogs. Go ahead and jump right into this really interesting conversation.
I don't know much about this topic, so I'm really excited for today's episode. first off, Moshe, at a high level. What are Drupal Spoons? Should we explain what a Spoon is first? Maybe to?
Moshe Weitzman: [00:00:58] For sure. People are very curious about the name. In this ecosystem, I've built this platform.
There are Spoons and there are Forks, Forks is easier to explain than Spoon. So, in Git you have a public repo, people are sharing. And then, people are encouraged to make Forks of that repo and that's where they make changes. Forks, to use merge requests,~~ ~~order to move their changes into the main repo.
So that's what a Fork is. And, so Spoons, are the other half of Drupal Spoons. And, you know, we'll definitely talk a lot more about why we need Spoons and Forks and that kind of thing.
Preston So: [00:01:46] It's a very evocative name, Moshe. And, you know, to be honest, I didn't really think about Forks when I heard the name Drupal Spoons.
So I really appreciate that explanation. so, so I guess, you know, for folks who are a little less familiar with how, how modules and projects are built in the Drupal ecosystem, you know you, you, you kind of alluded to very briefly. Some of the kind of, deal with today.
What does the development process look like for a typical contributor? Right now who's managing a module or a theme. And how does that kind of change with the motion over to GitLab? And of course, how things have been working on drupal.org.
Moshe Weitzman: [00:02:26] Sure. So, the way the development process works today on drupal.org is that we have an issue queue for each project, devel and C tools and Pathauto and so forth.
And in that issue queue, people are encouraged to post the patch files, and that's the mechanism that they use in order to propose changes to those projects. Those patch files. are picked up by a bot called Drupal CI and Drupal CI has a recipe where it assembles, the project like Pathauto applies, the proposed change, the patch file brings in the right version of Drupal core and the right database and so forth and runs the suite of unit tests on it.
So that's how development works today, and it certainly works. Drupal has achieved lots of success with it. and, in a sense, with Drupal Spoons, we're doing the same kind of thing. We're just doing it a lot more elegantly in my opinion. And so, yeah, I want to get into like, what are the pain points?
For the current, drupal.org development process. so I have a list here. People will bear with me. so when you're authoring, issues and replies, on drupal.org, it's extremely hard to upload images. in order to supplement the point you're trying to make. There is an image button in the toolbar.
If you click that, you would think that you could just like drag an image in - you can't. The process which isn't well described is that you have to attach an image file to your, to your comment and then reference that image file when you click that image button. so there's a two step process there.
It's only two steps, but it's enough to discourage most people from using issues. using images. Sorry. you might think that like using images is not a big deal, because well, we never had that. , but it is a big deal. Okay. So when you're talking about bug reports, when you're talking about support requests and feature proposals, the conversation goes way faster and clearer.
When you have images supporting that conversation. so the drupal.org flow is actively inhibiting our conversation by not making it easy to add images. In GitLab, you can copy from one place and paste into the text area and you have an image uploaded, you know, no fuss. you can drag it from somewhere on your desktop and move it into the text area and you have an image.
in fact, you know, Drupal 8, works that way. But drupal.org doesn't have Drupal 8. It has Drupal 7, and so we don't have nice images and it really hurts us. Another problem with, or another pain point with drupal.org is the nitpick culture. when people propose patches, inevitably, people follow through and, they might comment on the material points about your patch, but they will definitely comment on the variable names they disagree with or the white space they disagree with, or, various other coding standards issues.
And, you know, it's fine that those things should get fixed before they get into the code. However, it's not fine in my experience, to just say what those problems are without also fixing them. And, but the tools didn't make that easy. So the culture on drupal.org is generally that it's okay to just give a nitpick review and walk away.
The way this is solved on GitLab, there are a couple of different ways. When you're giving a review in a merge request, you can actually submit a suggestion, which is this line should be this way. and you write the new line for that way. New new version of that line. And the author of the merge request can accept your change, your proposal.
And so neither the, the reviewer or the merge request author had to get their command line tools out to do this work. it was all done with a browser, was all done easily. So I, as a patch author, would love to get suggestions and would not love to get nitpick discussion. And there's this other tool called the Web IDE in GitLab, which is amazing, really cool engineering that someone did and product engineering.
On any merger quest you can say open in Web IDE. And what happens then is that, you get this. IDE in the browser that has syntax highlighting. It has the ability to open. It opens automatically all the files that were changed in that merge request. And you can go ahead and make the changes that you have spotted for that merge request.
So it's like a much enhanced version of suggestions where you can make a substantially better version of the patch again, without resorting to the command line, pulling down the branch, opening up the editor and so forth. you have a quick way of making changes. So I think that our velocity will improve once we adopt the Web IDE, you know, the Web IDE does other stuff that are amazing.
Like as you're making changes, you can make commits and those commits automatically run the CI pipeline and you can see the CI pipeline, pipeline status as you're developing in the Web ID. So you can, iteratively make changes so that your tests pass. so it really is quite smart engineering there.
And I think will really benefit from that. The GitLab issue queue supports bulk editing, you know, there's this problem on drupal.org where like, when we go to a new version, it's a real pain to enter into every issue, change its version and submit it. but that is the process. When we have to do hundreds of those, like when there's new versions of Drupal core, the process is that I've seen is that X jam opens up an infrastructure issue and asked Neil to run a script on the server so that we can move hundreds of issues and change their version.
That's a service that is not open to all projects. and it's, it just doesn't have to be that cumbersome. this is something that is out of the box, with GitLab. More pinpoints.
Preston So: [00:08:37] Absolutely. Yeah. One of the things I think you've done really well here Moshe, is to illustrate kind of the, you know, the fact that, you know, not only are we grappling with the fact that, you know, drupal.org is, is sort of this older, you know, it's kind of, it's kind of this well-established approach, but.
Obviously, there are certain areas of friction. You identify the images problem, for example, batch editing, and that's another issue as well. and you know, as we know very well, the Drupal Association has been working really hard to migrate all the Drupal's code over to GitLab. If you're interested in hearing more about that, we have a couple of episodes with the Drupal Association and engineering team as part of Tag1 TeamTalks on that subject.
but I'm kind of curious, you know, I know that Moshe, you don't have any, any special privileged access to GitLab, which is a shame. Cause I was going to ask you that help me, you know, hack into some of those really cool features you described. But, you know, you said that it uses, you know, zero custom code except for the issue migration.
What exactly do you see as kind of the next steps in terms of kind of, you know, do you have plans to integrate at all with the work that the developer that the Drupal Association is doing? you know, is there a kind of, you know, synergy between what you're working on and what they're working on, what's the kind of next steps in terms of where this is headed?
Moshe Weitzman: [00:09:49] Yeah. so I have been in close contact with the DA, and, and they. they're watching Drupal Spoons really closely. it's my impression that they are rooting for Drupal Spoons. They would love for the community to embrace it. if you look at it from their shoes, they have a lot of work in front of them and not much resources to do it in.
and that was only made worse by the cancellation of DrupalCon North America recently. and so, wouldn't it be great if they could just move everyone off the project modules, onto a vanilla GitLab? I think they would love that, they would remove maintenance of the project modules and thus be able to upgrade to Drupal 9 on drupal.org.
In a much simpler fashion and they could potentially remove maintenance of Drupal CI. Which is quite a big, time, time and resource investment they have too, and like all those dollars and hours can go into, you know, the next highest priority. So I think that's why the DA is really interested in this project.
At the same time, the DA basically follows what the community says and wants. Right? Above all else. And so, you know, I've been saying that triple spoons is a model and a movement. It's a model. And that I'm showing the DA and showing the community that we can use vanilla GitLab and it'll work fine.
But, but that's not enough if it was just a proof of concept, I would have announced it on Twitter and been done with it. But I feel like we need modules to move over. We need people to move over. We need people to want it and to say they want it. And to tweet about it. In order to overcome the status quo, I think that like there's no villain in this case, like people are generally supportive of this move.
The only villain is the status quo. It's, you know, people have their pet features that work differently from drupal.org to GitLab, and they might prefer the way they work on get on drupal.org. But I think, you know, I have more pain points that I can illustrate here. I think when you look at the sum of the pain points and the sum of the improvements to GitLab offers, we just might have to bid a, you know, fond farewell to a few things that were preferable on drupal.org.
Preston So: [00:12:14] Absolutely. Absolutely. And I think that, you know, this is part of a lot of, you know, Obviously there's been discourse since the very beginning of Drupal's move to Git, for example, about, you know, leveraging some of these source control providers and their full feature sets. and it's been a very fraught kind of discussion obviously, cause there's a lot of different passions and moving parts and you know, really, really challenging issues involved.
So, you know, I think, I think, one of the things I can say is that, you know, we definitely, you know, I sort of appreciate Moshe of your, really kind of concentrated thinking on how can we actually, you know, challenge that status quo and move into this kind of much more kind of developer friendly direction for this environment, which I love.
But you know, so I think we glossed over a little bit of, kind of the, the core features of Drupal Spoons. I think that, you know, your description was incredible, but I want to take just a little bit further into some of these kind of three core pillars that you identified. so, you know, you mentioned that Drupal Spoons offers obviously issue support.
you know, you know, I imagine, you know, this is very similar to kind of the ways in which GitHub or GitLab normally does issue queues. one of the things that I know that get lucky GitLab provides is a much richer kind of customizability when it comes to things like issue management and things of that nature.
Could you speak to a little bit kind of, you know, how does kind of Drupal Spoons improve upon the issue management or issue process within the GitLab context nowadays?
Moshe Weitzman: [00:13:40] Yeah, for sure. yeah. So the features from GitLab that we, you know, we're primarily using our issues, merge requests and continuous integration.
so, you know, GitLab offers relatively simple issues. There are better in some ways and worse in some ways than drupal.org issues. And, You know, the sort of management of them is the same. You have a set of maintainers, who accept code changes and that sort of thing. One place where I think that Drupal spoons really innovated here is the local development process for contrib modules.
is a lot better defined than it is on drupal.org. when you clone a Can trip project from dribble spoons and run composer install, you have a fully functioning website and installation of Drupal there. You don't then have to like symlink it into an external dribble core. For example, in order to have tests run, it all is cohesive.
Right there, that's beneficial. for a couple of reasons. One is that. in your IDE you have triple core, you can do completion and so forth, with the, with the right version of Drupal core, furthermore, any other dependencies are right there. next up, what was I going to say? I had a list. Well, the, the test running. Okay. So you can just run composer unit and your, your tests run similar thing. Yes. But, but apart from that on, on triple CGI, you kind of have this problem where your tests don't run exactly the same as they do on triple CGI. And so you have people who are following the Drupal CIA docs.
How do I get triple CCI locally so that I can. Replicate the failure that I'm seeing on dribble out of our testing, but that's not a thing with Drupal spoons. Like it's exactly the same commands that you run locally and that you run, at, at, at GitLab so, I think that's another real benefit for developers is that you have a clear way to develop.
And run tests.
Preston So: [00:15:42] I think, you know, especially as, as, as tests become more and more critical to, every developer's workflow, obviously, you know, I think the adoption of test driven development has been relatively slow, but, but picking up, picking up some steam, this'll be very, very good to see a lot of adoption in the Drupal community of, of these kinds of mechanisms.
And I'm very excited to see, you know, you mentioned for example, merger requests. Of course are unique to GitLab really, really great feature. you know, a kind of unique, feature suite, but you also mentioned a web IDD, which I thought was one of the things that, you know, has been really challenging.
That's really hard to kind of directly, edit code within the environment that you're reviewing code in. so I think that that's a really good, kind of. a component as well. So it was, I guess, you know, to move away a little bit from the kind of engineering side, because obviously everyone can read your blog posts.
it's it's, you know, an incredible description of exactly what Drupal Spoons does. I want to get a little bit of a kind of human side here, the personal side here. like how did Drupal Spoons come about, you know, what were some of the problems that you saw and that were affecting your own kind of personal workflows and why did you end up creating it?
Moshe Weitzman: [00:16:44] Yeah. one of the projects I maintain is called devel. And if people will want to check out Drupal spoons, that might be a good project to check out on there. we've got a problem with testing on drupal.org, where it wasn't showing all of the deprecations, that we expected it to show.
And so we were running, the, the test suite on Travis CI, which is pretty cool system, but had a little bit of frustration with that. And rather than improve our Travis tests, I wanted to give GitLab's CI try for running our tests. And in order to run the tests, I had to kind of reorganize the code base.
and once I had the code reorganized and the test running, I was like, well, this is really cool. This is the way I want to work. Wouldn't know why not bring issues and merge requests in and just do the whole shebang. So it kind of spiraled in that way. I didn't intend to, to build a whole CI system for devel and I certainly didn't intend to build a platform for other projects to join on.
But like it made every step of the way made sense and it made it cooler and we were under lockdown and it was a Corona project. and it was really a lot of fun, you know, while the world was stopped. So, I'm happy with how it has turned out. I think it's totally usable by projects that want to use it. and you'll recognize benefits on day one from doing so
Preston So: [00:18:16] I think illustrates it very well. Also be kind of scratch your own itch. You know, tendency in the Drupal community, and also in open source at large. you know, obviously this sprung up from, you know, a kind of task that you needed to do for maintenance of the devel module.
You know, it wasn't necessarily something that was, you know, this grandiose idea. And, you know, I love that the pandemic helped, you find the time to really build this out. you know, I wish I could have been that productive and prolific, but maybe next pandemic. We'll see. Hey, so one question I had for you Moshe though, was, you know, we talked about how this is going to completely reinvent and, you know, revitalize in some ways the kind of contributed developer process and the developer workflows that people use to create contributed projects.
Is this going to be for the Core too? Is there any kind of potential or any sort of way that this could possibly be something that could be really beneficial to, you know, core committers or core maintainers? like Fabian who's a guest on our core confidential show all the time.
Moshe Weitzman: [00:19:15] Yeah. so, you know, my personal opinion is that core could definitely Could definitely use this process. you know, I think that. Politically core is a bigger deal to many people. so I haven't tried to really advocate for core using Drupal Spoons. so my goal really is to like get contribute projects, to use it. first on Drupal spoons, and then maybe the DA will offer.
For contrib, of vanilla GitLab experience. I think that would be an awesome next step. and then I think it's up to the community and the core maintainers to decide, do we want to stick with Project Module or do we want to make GitLab work? And, I am like thrilled to be part of that conversation and do prototyping and talk through the pain points.
You know, ultimately we'll, we'll see where that goes. I think that Core might be sufficiently different, in its needs that may be, it's going to be its own group on GitLab that is separate from Drupal Spoons. Like it might not be the exact same configuration, but I think all of the same ideas and workflows apply.
Preston So: [00:20:37] I think that's a very worthy goal. It sounds like a very lofty vision. You know, I think we all know how challenging it is to get some of these things, you know, going for not just contrib, but also for core. So, out of curiosity, you know, I know that this originally started out as a. kind of a milestone for the devel module.
And I, you know, as I understand it, it's, live and working right now for devel module. could you speak a little bit to kind of like, how can somebody leverage Drupal Spoons today?
and you know, for example, me, you know, I have very, you know, I have a couple of projects that I've got that I'd love to introduce Drupal Spoons to.
What's kind of the first step for someone who's totally new to the Drupal Spoons approach and how can I get started?
Moshe Weitzman: [00:21:22] Yeah. yeah, there's like a, you know, 25 modules or so that are enjoying Drupal Spoons right now. the way you, join is that, first you join as a user. where there's like one, issue you have to create in a queue called the webmasters issue queue.
And the only thing you have to do is, post an issue that says join Drupal Spoons. At that point, a bot will pick up your new issue requests. I'm using the, the GitLab API, and it will add you to the Drupal Spoons group and the Drupal Forks group. the difference between the two being, you have, only read permissions in Drupal Spoons, but you have full right permissions, interval forks, and you can make any changes you want and get to any project.
So we always collaborate in Drupal Forks. But, Drupal Spoons is where like the golden accepted code is. So you fill out that issue to join and then you fill out a second issue that says migrate and then the module name to Drupal spoons. That's it. then someone will come in, run a CI pipeline.
That's going to migrate your project over and then migrate all your project over means that the code comes over, the issues come over. And, you get a project created for a spoon and you get your project created as a fork. and, you get to somewhat, you know, the bot replies or the person who replies in the post and says your project has been migrated.
The cool thing or one cool feature about that is that a Drupal Spoons automatically mirrors all your code changes. to drupal.org. So any changes that you're making on spoons are automatically come back to drupal.org, and so when people download packages, through composer or download tarballs, they're getting the latest code and you didn't have to do any work there.
Preston So: [00:23:14] Well, that's incredible. And, and, you know, I think that, you know, first of all, it's, it's, it's an incredibly, kind of, you know, straightforward process. And, that makes sense. I'm looking forward to migrating over to Drupal Spoons. If I, if I find the time, we'll see. but, you know, it's great to hear that there's 25 modules.
You know, are there any kind of prominent names on that list that we would know about? I'm sure there are, and I'm sure there are a lot of enthusiastic, Drupal Spoons users who would be very, or are Drupal students, folks who also were interested in Drupal Spoons would be interested to hear kind of who, who is currently using.
Moshe Weitzman: [00:23:42] I think that the one that most people will know about is Migrate. yeah. Lucas Hedding has done a great job of moving, the migrate suite over. so most of my grade is now on Drupal Spoons. Otherwise, Jurgen Haas move to a bunch of his modules, the Drupal Remote Dashboard. And, you know, there, there are other like interesting, but less known modules that have moved over too.
Yeah. So, if you look at the Drupal Spoons, group, you can see the listing of who's in there.
Preston So: [00:24:15] That's amazing. now, you know, one thing I wanted to ask is, you know, obviously of course it, you know, there's, there's a lot of discussions that we had coming soon about, you know, whether this, you know, what this means for Core, what this means for theDrupal Association, what this means for our GitLab migration.
but from the standpoint of, you know, open source governance and the community had actually begun to foster, which is really incredible here, this ecosystem, you know, is every aspect of Drupal Spoons, open source being open source. you know, how would you like people to get involved?
Is there a place where, you know, maybe there's some good initial steps that contributors could work on, and, you know, how can people get involved in, in helping you out in supporting this amazing ecosystem?
Moshe Weitzman: [00:24:56] For sure. so like the simplest thing is to tweet about it and blog about it and, make your support known, more involved would be to come to the Webmasters issue queue, (gitlab.com/) drupalspoons/webmasters. And look at the open issues there, and participate in them. you can also look at the code that's there.
Yes. Drupal Spoons is all open source. You can browse the code, you can open up a merge request and make it better. so I would welcome those kinds of participation for sure. And, and yeah, to bring your projects over, would be a great way to show your support.
Preston So: [00:25:34] Wonderful. so, you know, one thing I did want to, kind of, kind of ask you is, you know, in terms of the ways in which, this kind of approach is changing, you know, I recognize for example, that the ways in which developer processes work, one of the things I know is coming soon, for example, is, you know, I'm not sure exactly what it entails, but.
GitHub code spaces, for example, how does this approach with Drupal Spoons fit into the larger kind of, you know, debate happening right now around workspaces issue workspaces and, and GitHub's coming code spaces?
Moshe Weitzman: [00:26:07] so I mean, I think the bottom line is it gets Drupal into the modern world of, you know, Web based and Git based development.
So, so that's how it fits. Like, you know, we have millions and millions of dollars going into developer, experience, right? As you mentioned, GitHub and Microsoft have poured tremendous resources into that. And on the other side, GitLab has poured tremendous resources into a platform. So. It's not, it's not feasible for the Drupal community to keep up with that kind of investment.
We don't have the UX team, the product leadership, to build at that velocity. and you can see because project module rarely get touched by people outside of the Association. So, so I, you know, it's about time for us to like jump on that train. And benefit from it. GitLab is like an amazing product and amazing company.
If people haven't been paying attention, they have a velocity that's outstanding. they have like, a super transparent company. You can read all of their policies. You can read their backlog directly, as GitLab issues. you see who all the product managers are for each section, you can just mention them in an issue and they will reply like, they really have been impressive in how they run their company and how they execute their product development.
So, you know, one thing that I'm really happy that the DA did was pick GitLab as a platform. I think that's really gonna, it's going to serve us well.
Preston So: [00:27:40] I couldn't agree more. I think that, you know, GitLab is, is, is a resolute step into the future, for Drupal. And, it's going to be exciting to see how, how we begin to leverage some of those exciting features and especially how we begin to really, I, you know, get the value that you described out of, out of Drupal Spoons.
I'm very, very excited. So we have a segment here, now for something completely different. We have a little segment here on Tag1 TeamTalks called the Aside Tag. And what we do is we just spend about one minute per panelist and we share something going on in our personal lives. Something that is, you know, something that we've been, that we've learned about recently or that we've been playing with or.
Something that we want our audience to check out. obviously we have a lot of listeners and a lot of viewers from around the world. So one of the first, you know, obviously the last couple of weeks have been really, you know, tougher for, black communities around the country and around the world.
One thing that I was really happy to do was to share the conference that I organized. Decoupled Days, we released a statement last Monday, or earlier this week, actually, in support of, Black Lives Matter and also against a lot of these horrible things that are tearing down. You know, all of the communities that we hold dear, namely, you know, issues like anti-blackness, white supremacy, police violence, all of these are, are.
Horrible, kind of, you know, sins in America. So one thing that I had done is I, you know, we had a really great meeting where we discussed the statement and released it. And one of the things that I'm excited to report is that we're also going to be issuing a donation to Black Girls Code.
So very, very excited to do that with the team. And, you know, this labor of love and this nonprofit conference is something that. you know, has really shown up in terms of the kind of emphatic and strong statement that we wanted to issue in light of these events. So that's, what's been going on in my world.
Meyers or Moshe what's been going on in your world?
Moshe Weitzman: [00:29:30] I could talk a little bit about my garden.
I've been growing my garden here with my girlfriend, Julie, and it's coming along great. We're doing lettuce for the first time. and like the lettuce is sprouting, like mad. So, lots of salads happening around here.
I got to figure out maybe additional ways to use lettuce, beyond the salad. I might try lettuce on the grill. I'm not sure. but, yeah, the tomatoes will come in soon and the herbs are doing great. And, just want to give a shout out to all the other Drupal gardeners out there.
Michael Meyers: [00:30:02] When I, when I lived in Denver Moshe there was a restaurant that had a grilled Caesar salad.
Really?
Grilled Lettuce is amazing. I've never tried it, but, I, I'll make that my Aside Tag as well, building on yours.
I've started a window garden, something I've always wanted to do, even though we have outdoor space now, I'm still the mindset of living in a city, but, I've got, avocados, sprouting and, you know, growing scallions in a cup and all the, the Covid things that people are doing.
And it's awesome. You know, now we have scallions every time we need to
Moshe Weitzman: [00:30:36] awesome.
Preston So: [00:30:38] Wonderful. Well, we did have a shorter episode today. You know, I think that this was a wonderful introduction to Drupal Spoons, I think we can dig much more into some of the nitty gritty and details, within Drupal Spoons.
I'd love to dig into, especially some of the CI capabilities that Moshe mentioned earlier. so to finish things off today, all the links that we mentioned today, whether it's related to Drupal Spoons or GitLab migration or the episodes that we mentioned. Are going to be posted online with this episode of Tag1 TeamTalks.
And if you liked this episode, if you enjoyed what we had to say today, and if you're interested in the topics that we had to cover for today, please remember to upvote, subscribe, share this with your friends and family showed us with your colleagues and your coworkers. And as always, you can check out our past episodes at tag1.com/tagteamtalks , as always. We'd love your feedback and insight and any, you know, any sort of ideas or proposals you have for topics you want to hear about. If you want to bring the Moshe back, talk more about Drupal Spoons, if you want to bring, you know, someone else on board to learn about something that they're working on, please write to us at tagteamtalks@tag1consulting.com.
I want to thank my dear friends and colleagues today Moshe and Meyers who joined me on this episode. And we'll see you next time.