This is a transcript. For the full video, see The Drupal Association is the Drupal DNA giving Life to our Code & Community - Tag1 TeamTalk #014.
Preston So: [00:00:00] Hello, and welcome to a very special edition of our Tag1 TeamTalk series. A webinar and podcast series about emerging web technologies. Today we've got a really special episode for you. We're kicking off a new mini series with the engineering team at the Drupal Association, one of my personal favorite nonprofits in the entire world.
Drupal, of course, is one of the largest and most active open source projects in the world, and the Drupal Association, also known as the DA. Is responsible for enabling it to thrive by constantly working on tooling and projects to improve the ecosystem. And this mini series, we're going to give you a little bit of an overview and a survey, a little bit of insight into the technology and the tooling behind Drupal and what it takes to foster open source communities and enable open source development at scale.
Joined by our guests, the DA Engineering Team, the people responsible for making it thrive. So if you are interested in open source software development at scale or enterprise web development, we hope you get some insights from this very exciting episode today. By the way, in case you didn't know, my name is Preston So.
I'm the host and moderator of Tag1 TeamTalks. I'm senior director of product strategy at Oracle, editor in chief at Tag1 Consulting, and the author of Decoupled Drupal in Practice. I'm joined today by some very good friends of mine. First, Tim Lehnen, in Portland, is the Chief Technology Officer of the Drupal Association.
We're also joined today by Narayan Newton, who is the CTO of Tag1, based out of Portland. And we're also joined today by Michael Meyers, the Managing Director of Tag1 out in the Berkshires. I'm joining you here from New York City and let's go ahead and jump right in. So, you know, I think a lot of us in this audience and on the Tag1 TeamTalks, kind of fan base, know that the Drupal Association, the DA is the nonprofit behind Drupal.
It's that community force behind Drupal. But, you know, I'm not really sure. A lot of people understand very well. What's your, what you and your team do from week to week. Can you give us a little bit of insight into what exactly the DA does and why it's such an important fixture of the Drupal landscape?
Tim Lehnen: [00:01:56] Sure. Yeah, I'd be really happy to. I think it, as an overall kind of overarching concept, the Drupal AC sociation focuses on three areas. We focus on supporting the Drupal community on putting on DrupalCon as the event to gather us together and on drupal.org, which is really where we're going to focus our conversation today, and all of the engineering infrastructure that supports the Drupal project.
it's kind of interesting because there's this misconception, this fallacy in the Drupal community. or at least among some of the Drupal community, that there's a firewall between the Drupal Association and the Drupal project. This idea that the Drupal project remains this very volunteer oriented contribution culture, and the DA is there to, you know, put up some tools, maintain the website and run the event.
And that's not really true. I think people don't realize how much the Drupal Association is actually responsible for key parts of Drupal as a software product. So, the Drupal association manages and maintains the infrastructure that hosts the translations, the localization of Drupal, all of the updates, that every Drupal site calls home for, security PSA, composer, metadata, all of these kinds of cloud services that are part of the Drupal software product come from the Drupal Association.
And, we on the engineering team in particular, and make those happen. So what we really like to say is, you know, the Drupal Association builds the tools that empower the community to build Drupal.
Preston So: [00:03:21] And I think one of the things that's really interesting is that especially in recent years, we've seen a lot of really amazing projects emerge from the Drupal Association. One really good example of this, a journey that I was following very closely, from the comfort of my living room was the transition over to GitLab.
and I know that there are a lot of other things going on as well. I know that there's work that's been done on Composer. There's some amazing things happening around the Contribution Credit System. What are some other kinds of tangible examples of the tools that you refer to as really empowering the community and helping them to build this amazing open source software?
Tim Lehnen: [00:03:56] I mean, I think probably the most obvious tool for anybody who's contributed to Drupal or anyone who you know, maintains an integration, a module that works with your is Drupal CI, the testing infrastructure for the project. It's a custom built CI pipeline, based on some standard tools that allows the project to test in a huge variety of environment combinations.
What versions of PHP, what mySQL and verify that the software is going to work, that it's compatible with the latest version of Drupal, that there aren't regressions and things like that. and it was a huge undertaking for the Association because when we first looked at building this new test infrastructure, you know, the obvious question was, you know, why don't you build something on Travis, and just run tests there.
And we quickly found out that like running a Drupal Core test would go for about an hour and a half and then completely timeout and just max out the kinds of resources that we had there. So we had a sort of slightly bespoke solution just to handle the scale of Drupal testing and Drupal test coverage.
I think maybe Narayan can talk a little bit about the components that kind of make up DrupalCI, cause it's a really cool project.
Narayan Newton: [00:05:10] It is very interesting. slightly bespoke can describe so many elements of the Drupal infrastructure. it is just very difficult to run the entire Drupal test suite.
And so it is interesting what was built because some of it was built before the standardized tool and that would have made it easier now. So it's quite cool to look at a Git repository of what are basically Docker files that would, would now be like helm templates, but were done in standard Unix tools instead to template them out to build the required environments for all these different test targets.
The actual orchestration of testing, instead of using something like Kubernetes, which would be the choice now, It is using Jenkins and the EC2 Fleet plugin. So Jenkins is basically managing the orchestration of VMs, and spinning them up and using them as test targets and then actually running the tests on them in a clean room, clean room environment.
Because obviously that is required for testing.
It is a very interesting quirk of the Drupal infrastructure as a whole, that a lot of aspects of infrastructure were put in place right before standardized tooling came out to resolve that issue. So as Tim has noted, we end up with these like little bespoke systems.
And Drupal CI is a, is a large bespoke system, although one of the reasons it is a little bespoke is what Tim said. It just takes so long to run a test that a lot of the actual services you could buy into would never work, and then a lot of the standardized tooling wouldn't quite work either at the time it was built.
An example is on the back-end when Jenkins collects these logs from the test runners. The logs are so chatty, but important that we end up doing things like those logs are stored on a ZFS volume with compression. Because we tested a few different file systems and a few different, asynchronous compression, and that was the one that actually performed well. So DrupalCI, it works well, but there are a lot of layers to it.
Michael Meyers: [00:07:21] Just to give people a sense of the scale that we're talking about. I think I read in a, in a Dries blog post that the CI system did something like 7 to 10 years of concurrent testing in a year?
Tim Lehnen: [00:07:35] Yup. Yeah, it's, it's something like that.
It's really pretty close and it's just, you know, because it dynamically scales to the, to the demand, you know, we might go from running just one or two test spots and. You know, some time of the day without a lot of contribution activity to having as many as, I think we've had as many as 70 concurrent machines running.
And these are all enormous, like 32 core boxes, running the core test suite. And each one of those runs takes maybe an hour and 10 minutes. So yeah, the scale is really. overwhelming and it's allowed us to do other things too. I mean, one of the biggest contributions of DrupalCI hasn't even just been to Drupal.
We found progressions in new versions of PHP that have then been fixed upstream, before those, versions became final. So yeah.
Preston So: [00:08:20] So out of curiosity. Oh, so out of curiosity, Tim, you know, who, who's responsible for kind of financing all of these, you know, the, the, the 32 core boxes, you mentioned this amazing test infrastructure you've got, and, there's some amazing thinking that's gone into here.
I mean, I'm just mind boggled by, the kind of things you all had to do, who's, who's kind of supporting all of these efforts. And, over seeing all of this work.
Tim Lehnen: [00:08:45] I mean, it's a, it's a really great question. It's the community as a whole financing the Drupal Association, so that testing can cost us as much as $6,000 a month, with roughly the current load, to run all of those machines.
those ephemeral machines that run individual test suites. so it can be quite expensive and we offset those costs through partnerships. we get sponsorships, you know, when you're buying a ticket to DrupalCon, for example , or to our new virtual DrupalCon event, those ticket proceeds go into funding, the engineering team and the infrastructure we need.
Whether it's, you know, the AWS fleet that runs these tests or any other. Of these kind of initiatives that we've talked about. So it's, it's the support of individual donors, of Drupal Association members and organizations like Tag1 and others that kind of funds this and makes it possible to offer this as a service to the Drupal community.
Preston So: [00:09:38] And I think it's a, it's a, it's an amazing service. you know, I have seen so much value come out of Drupal CI, just in terms of the ability for the community to feel like they're, what they're contributing is stable, that they're able to release and tag new releases. It's a really great thing for the Drupal community.
And, you know, I think we're all very grateful for that. So Narayan and Tim, I wanted to ask a little bit about some other projects that the Drupal Association has been working on. Now. I understand that one of the recent projects that the DA had to do in addition to, you know, this, this, creation of the Drupal CI infrastructure was also, this.
Something that a lot of people don't really think of as being a very important thing, but it really is, translating some of the metadata in drupal.org and I'm sure there's a lot of really interesting things we can take into there into a standardized, Composer format. So I'm kind of curious, like what did that involve?
You know, and I'm curious to hear from both of you on this, you know, how did this impact the adoption of Composer into Drupal? How did this impact. the way that Drupal 8 was developed, semantic versioning. let's see. The full, skinny.
Tim Lehnen: [00:10:42] Yeah, there's, I mean, there's a lot to talk about here because this is another one of those kind of fundamental technology shifts in the PHP space, right?
The use of Composer and Packagist to do dependency management in PHP is kind of a tectonic shift for PHP projects in general. But, you know, as we said before about Drupal being, you know, slightly ahead of these oncoming changes, we always find ourselves in these situations where we've built a solution kind of before the ecosystem settles on something. So Drupal has had its own metadata, of course, for, you know, the, the life of the project. any contributed module, for example, has its info YAML file that talks about, you know, the, the attributes of that project and projects on Drupal.org, or it can also have sub projects within them and things like that.
And so when with the transition from Drupal 7 to Drupal 8, when people began saying, Hey, we'd like to build our sites using Composer dependency management and Drupal's PHP, why can't we do that? we began to understand that we would need to create a sort of transition layer. To allow, to allow the Drupal kind of file structure and metadata structure to be translated into something that Composer understands as their sort of standard structure.
So that was a huge undertaking. and it's kind of an ongoing project because it was first used , Composer support was first used actually way back in Drupal 7 , in the contributed space.
There was a Composer manager module and then in Drupal 8, over the course of the life of Drupal 8, it went from, you know, being one of the equally supported ways to install Drupal. You know, you could use Composer or you could download the tarball to being the preferred way to do it. and, and things have been changing throughout Drupal 8's lifecycle as recently as version 8.8 there have been changes to the Composer scaffolding files that ship with Drupal. So as of 8.8 and beyond, for example, whether or not you start with the tar file or you start with Composer, you get a properly scaffolded Composer compatible install.
So all of this required building this translation layer that takes the Drupal metadata and turns it into, composer compatible data.
And that means, You know, earlier versions of Drupal didn't have semantic versioning for contributed projects, or even for Drupal itself, Drupal 7 was not a semantically versioned project. So we have to take a Drupal version, you know, if you had 7x-1.0 we have to turn that into SemVer (semantic version) into a, you know, a 1.0.0 or something like that, and then say, serve that from the Drupal 7 end point. And that's how we do platform compatibility. So we built this layer where there's a, this Composer facade, which says, here's the 7x end point. Here's the 8x and beyond end point, which will now be continuous through Drupal 9 and beyond that takes these older versions of modules and preset for projects and translates all that data. And there's some cool things we do from a technical point of view to make that happen. We, that generally when we generate the metadata, process it for all these contributed projects, we then write it to flat files and then put those flat files into our CDM layer to cache them with a routine that purges them on update whenever new information is written.
And that means that this dependency data. when you're doing a complex composer calculation, which can be pretty heavy, doesn't even have to get all the way back to drupal.org. Your local, a local, CDN server to your region probably has that data cached. So we don't even see the request, traffic for that.
So that's very cool. but it's a complex problem and it'll probably be, even though Drupal 9, will be encouraging the use of semantic versioning in the contributed space and drupal.org now supports semantic versioning for all contributed project projects. It probably won't be the required way of doing things until Drupal 10 which means this is a service we're going to have to maintain for a while.
Preston So: [00:14:50] That's very interesting. And I think, you know, semantic versioning obviously has, has really only recently kind of spread into, I would say, the, the wider world of version management in software. and I, and I wanted to call out something you said earlier, Tim, when you said that, you know, Drupal is a little bit early to the game sometimes in terms of creating technologies that soon have standardized approaches or soon have new specs. One thing I'm glad about is that we didn't do that with jQuery. I'll just say that. So, now it's getting into another interesting area here and Narayan, I want to get you in on this too.
So, you know, there's, there's a, there's this really interesting kind of historical, undercurrent of throuple where, you know, we started out with, CVS, version control. We went over into Git and, and, and now we've actually moved drupal.org itself in terms of all of the code and all of the, kind of, things you can do around code management over to GitLab.
So, I'm kind of curious, you know, we've, we've heard a little bit about the transition to GitLab, but I don't think we've heard much about it from the people behind it. So, what is this migration over to GitLab? Why was it so important? What were some of the biggest challenges and how can people use this today?
Narayan Newton: [00:16:02] Yeah, the GitLab migration was a big project. I was on the infrastructure side of that. I was also on infrastructure side of the original CVS to Git migration. So in comparing them, I'd say the GitLab migration was a little bit nicer than the original CVS to Git but, GitLab has done pretty well for us so far.
From my perspective, a lot of the impact of GitLab has been interestingly , yes , less than using it and more supporting it because we went from our, as you know, kind of bespoke Git set up that had our own Git daemon that we wrote and see Git as the web front end. And all of that mixing with composer because composer fetches things off of Git web front end sometimes. and all of that combining together made performance, especially when the large updates was pushed, rather difficult to maintain. So going to GitLab and having a full suite that is not built or supported by us and we can just use and has, much more mature performance characteristics, especially during high load; has been really nice.
And being able to keep it up to date from security perspective when they push things out and how reactive they are, security issues has been great as well.
The actual migration was, I think one of the smoothest I've been part of. I've been through a lot of the DA migrations and pre DA migrations and there's kind of a wide span in how much support migration got, and therefore how the project management went, how the resourcing went, and GitLab was an interesting one just because it was very professionally done and the testing was. Pretty in depth and we rolled out without major issues, really. One of the interesting bits about rolling out and GitLab is it wasn't just installing it and migrating everything into it and then turning it on.
GitLab has to interface with drupal.org and drupal.org has features that are present on GitLab, but we don't want to use the GitLab version of those features. So the integration points between Drupal.org and GitLab and being able to either turn off or block those features in GitLab was a large portion of my task during the migration is making sure we had filters that would make it so that you can only change options on projects in one direction because we didn't obviously want to sync inter directionally like for everything.
But as far as these migrations go, it generally went well, Tim.
Tim Lehnen: [00:18:45] Yeah, I mean, my observation was also that this was one of our smoothest projects, since I've been a part of the DA. So over the last six years, in terms of a lack of downtime, a lack of disruption to, you know, changing the entire Git code management infrastructure out from under our very active community without causing us to miss, say, a minor version release.
Like there was no, we didn't have to delay any of the new minor versions of Drupal. We didn't miss any release windows. We didn't have any significant periods of downtime, like it was a really impressive migration and it's given us a lot of advantages in terms of usability and project acceleration for the end users.
Right. So the parts of the GitLab interface that we do expose, like the code viewing and commenting tools, and a number of the like diff tools and things like that are significantly improved over the, you know, bespoke versions that we used before. And, while we got delayed by a number of other projects and then, doing some big fundraising campaigns, we're still really on track to having the merge requests enabled very soon.
Those are working on a development site in kind of an Alpha level and that's going to be another just massive leap forward. Not only will we have proper merge requests for collaboration on drupal.org but we'll have these single threaded fork merge conversations. So, you know, unlike other platforms, we'll maintain the Drupal way of having that single conversation about a solution, while still using those tools.
So it's just going to be a really cool, really cool feature.
Narayan Newton: [00:20:23] Plus, I think it's the only migration I can remember where we didn't have a community member just run for the Hills afterwards.
Tim Lehnen: [00:20:28] Yeah, that 's very very true. There's it's just so tricky cause in a lot of these things, especially back in the day before the Association had really, I want to say kind of taken ownership of some of this infrastructure.
It was this combination of like. These three volunteers have been working on something nights and weekends and they feel like it's their baby and it's their thing. And you know, they haven't had the resources to do what, what might be as much as they would like to, even if they have the talents, because they're doing it as a volunteer gig.
And over the course of five or six years, we've been able to take over those things, professionalize them, and then have more stable and more frequent deployment of new features for the Drupal ecosystem. So.
Preston So: [00:21:18] Well, I think I definitely share in, the, sort of, you know, happiness and contentment of the Drupal community. When I say that GitLab is amazing and, I think it's going to lead to a lot of amazing innovation and it wasn't possible at all without, you know, both of you being involved in the project and all of your support.
So, you know, I think that, it's amazing also that we're keeping the same exact way of doing discourse in drupal.org. I know that was a big concern of the community, was, you know, how do we avoid these scattered issues everywhere? These constellations of, you know, these constellations of conversations that just never go anywhere and never can refer back.
So I'm very, very glad to hear you're looking forward to the merge requests, especially, to see how that goes.
So, you know, Tim, you mentioned that, there's a lot of work that goes into the Drupal Association's infrastructure for drupal.org and you also mentioned something that I think is really interesting and really important from the open source perspective, which is that when it comes to volunteerism and open source innovation, sometimes you just can't rely on volunteers to do the mission critical infrastructure. So I'm kind of curious, you know, there's, there's another example of modernization that I know that Drupal Association recently undertook, or still currently in the process of doing, Narayan or Tim, what's this whole infrastructure modernization phrase for in drupal.org and, what are some of these words that I see on the page here in my notes that I don't understand.
Tim Lehnen: [00:22:42] Let me pull back for a moment and talk a little bit about that, that concept, right, of volunteerism when it comes to infrastructure and services. Because I think there's actually a really interesting point that the open source world could understand better and should talk about more so, Volunteer contribution is one of the most powerful forces in the world for creating software.
It's the basis of open source, right? It's the reason that these things work, and it's the reason that open source software is better. It's just a superior way of building the tools that we use. But a while you can easily in nights and weekends, do patch reviews and make merge requests and do code reviews and contribute to software itself.
Software and services require two very different kinds of commitment and architecture. And that's why when it comes to infrastructural tools, testing, Git Lab, drupal.org itself, just keeping the site, the site up, all of those things, those are services that need to be provided. And sure, they're built on open source software, contributed by individuals, by volunteers, as well as by paid staff.
but the day to day process of maintaining those services really requires full time staff. It's one of the main reasons why we need an engineering team and why you've seen this difference in service uptime and quality over the time that we've built a real team rather than relying on purely a volunteer space.
So I just wanted to, to set that context cause there's amazing things that volunteers can do to create something, but the ongoing kind of maintenance and care and feeding often requires people who can be dedicated to that in a fulltime way. and so to get into what some of this modernization project is, I'll definitely hand it over to Narayan because it's, it's, it's really cool work.
Narayan Newton: [00:24:29] Yeah, the drupal.org infrastructure is old. It has been around for quite a while, and it's existed in the same data center for quite a while, and it's gone through a lot of transitions. When I first started working on it was the configuration management tree for it was literally CFEngine version two, which is a configuration management system that probably a good percentage of people have never heard of at this point.
The backups at one point are literal, were literal tape drives. It has gone from Gentoo to CentOS to Debian. It has really gone through a number of changes. And speaking of volunteerism, I was a volunteer lead sys admin for eight years, something like that, and now have a more formal arrangement with the DA.
But a lot of our work recently has been taking this. System that has gone through a lot of transitions and has a lot of legacy components because of those transitions and modernizing it too, be more what we would walk in and build today if we were starting from scratch, which is sometimes difficult, especially when all the lights need to stay on while you're doing it.
So one of the big things we've been working on recently is going from a centralized log server that had minimal integration with everything around it, and you could basically search it with. Standard Unix tools. it had had Splunk and other things on it at some point, but at the end of the day, it was basically just an aggregation point for logs and going from that to having all the logs in Datadog with all of our monitoring so that you can cross reference between logs and the actual load graphs of the servers or the error rates from our CDN, which is also in there.
And basically having a more modern log platform that you could integrate all of this into one central location and have a good view for the health of the system and more, more easily track down problems.
We did Datadog for that. You can do something similar with Elk and Prometheus, but the idea is just modernizing how we're tracking our statistics and logging so that we can more easily track down issues.
Also a big focus has been updating our configuration management tree. We did originally a decade ago, started with CFEngine. We went to Puppet pretty quickly. and the public tree has gone through several iterations and we're working on the latest to more easily support, quickly updating the base OS.
My current goal is I really want all of our systems to be able to quickly. Transition from, Debian and old stable to stable when it's tagged. just because we have a tendency to lag behind a little bit and it's good to just. Keep that going as quickly as possible. we're getting to, after I think about a year and a half of work, something like that.
We're getting to the point where a lot of the legacy points in infrastructure are either gone or reworked it to be less legacy. And we get to now focus on some true modernization projects, which is going to be. Like switching from NFS for the files Mount to a recent version of Gluster FS. So we can more easily scale that out, into the introduction of some containerization to not just Drupal CI, but to actual production workloads, using the, some of the old systems that are still at our data center.
That's at the Oregon, Oregon State's Open Source Lab. To build a Kubernetes cluster that we can run some of our batch of jobs on and more easily scale out those batch jobs so that we can track them and monitor them and have them cleaned up afterwards. I'm just, in essence, trying to bring the Drupal.org Infrastructure that, at one point was like leading edge for what people were doing for Drupal deployments bring that, from what had become a more legacy infrastructure back to something that if not is bleeding edge, it is at least in that area and can be something that can be looked at as a, an example of one way you can do things and also is something of interest because it is a, we are a community project and like having people more interested in infrastructure and more, able to help where they can is always useful.
Tim Lehnen: [00:29:01] I would add to that, one of the most important elements of this is, you know, Dries likes to tell the story about the first, the. Not even the DA didn't exist at the time, but drupal.org first kind of financial crisis when the servers melted down because of too much traffic and the shared host kicked him off and he had to fundraise for the very first server and it wound up getting a donated, you know, Sun 2U rack and then that even that original Sun machine, I think wound up in that Oregon State Open Source Lab at one point. I'm not sure if it's still around. but then we moved on to other bare metal servers. and things like that.
And the more recent configurations have, you know, made this a portable infrastructure. Right. So the other advantage we have is we can now use this mix of like VM based or containerized portions of the infrastructure, whether they're within the Oregon State Open Source Lab, which still provides kind of deep discounts to open source projects on bandwidth and things. They're still a really valuable partner, but also we can combine that with cloud deployments of portions of the infrastructure as we might need them, or if we ever had to, we could wholesale lift and shift.
So we're in a much safer position for, you know, significant disaster recovery and things like that as well. It's, it's a lot of, it's a lot of really key backend work that's just gonna make sure that Drupal.org will still be around for the next decade.
Narayan Newton: [00:30:29] My first project on the Drupal.org infrastructure was, splitting the web component off of that, shared Sun server onto the new web nodes that he purchased on the second, second generation of servers.
Tim Lehnen: [00:30:41] Yeah.
Preston So: [00:30:44] Well, you know, I definitely couldn't agree more that, especially in this day and age when, you know, there's a lot of people thinking about disaster recovery, and hedging against risk, risk management, being able to lift and shift, as you said, forklift, the entirety of the drupal.org structure, potentially in a situation where.
You know, things are happening. That's a really, really amazing development in Drupal's infrastructure, Drupal.org's infrastructure. So I'm very, very happy about that because it means that even if the apocalypse happens, potentially we'll be able to put it all onto a spaceship and go to Planet B,
Tim Lehnen: [00:31:15] Another apocalypse? You mean?
Preston So: [00:31:19] About another pandemic. Yeah.
Tim Lehnen: [00:31:22] Yeah.
Preston So: [00:31:24] This time, hopefully it's a pandemic of joy instead of the pandemic of, of, a virus. But anyways.
So, you know, we've, we've talked a little bit about the infrastructure and I think, you know, I joined in with the rest of the community and saying how much we appreciate the efforts of the Drupal Association.
Obviously, supported by Tag1 and a host of other folks in the community to really raise up the stature and the caliber of what Drupal is capable of. And it's a testament to both of you. I'm so curious now, I know we've talked about some of the history, recent history of drupal.org recent history of Drupal infra.
What are you working on now? I know that you mentioned that you're working on GitLab still, you're working on some of the last milestones, I believe in Drupal CI. What are, what are some of the other projects you're working on?
Tim Lehnen: [00:32:07] Yeah, so there's, there's a number of cool things going on and a lot of it centers around supporting Drupal 9's release and then supporting the next generation of what Drupal will need throughout Drupal 9's life cycle.
Right? So, we're actually adapting a lot of what was built for the Drupal CI system for, to support the Drupal 9 readiness push, right?
A lot of the deprecation testing, there's a, there's a weekly batch, That takes place using the Drupal CI infrastructure that feeds Gabor's (Hotjsy) , dashboard of what projects still have deprecated code they need to change.
So we've got that system built out in order to support that effort. Very recently, we've worked with the Drupal Rector team, and that, that project is being maintained and financed by Palantir and it has some partners, working on it from a number of other organizations, Acquia and others and so now Drupal CI actually automatically generates Drupal Rector patches, which are these patches that will fix.
Fix changes that you need for Drupal 9 compatibility for you automatically. So it generates these fixes, and that's already been used to create, to make a significant portion of the contributed module ecosystem, Drupal 9, ready. So that's super exciting. there's tons of other things coming up coming around though.
So. You know, we want to further emphasize and educate the community on how to use semantic version properly. We're working on spinning up with the first Virtual DrupalCon. So we're working on infrastructure needs and the platform for doing that. You know, we've talked on this previous podcast in an earlier episode about the automatic updates initiative.
That's an initiative that's still in progress. The first phase of work was really great, but there's a second phase coming and, and still something we want to work on. So. There's all sorts of things. I won't run through all of them, but the main goal here is because of the work that's been done over the last several years and because of the help that we've received from Tag1 and some others, supporting this.
We've moved from. We've moved from getting out of the, the whole of legacy projects and legacy code and into building true community multipliers, things that empowered the community to do more, to spend less time working around old processes and older tools and more time actually pushing Drupal forward.
So, that's what a lot of these are really centered around.
Preston So: [00:34:28] And I know that, there's some amazing, community multipliers that come out from, contributing to some of this work or, that are involved in what you're managing as well. You know, I know that there's some initiatives going on around contribution credit, GitLab merge requests, as we mentioned earlier, an identity provider for drupal.org, you know, I'm kind of a, you know, I know that there's a lot of topics here, so, you know, is there one particular, let's say community multiplier from this list that is particularly interesting to both of you?
Tim Lehnen: [00:34:58] Well, if I were to pick one, and Narayan's may be different, I'm actually really excited about the contribution credit system that we've built, for the Drupal project on drupal.org. I actually think that the notion of measuring the open source contribution economy.
and really studying that in, pretty much an academic way, is, first of all, fairly unique. I'm, I'm aware of very few other projects that have attempted something on this scale. The only one that I'm really aware of is the Chaos Initiative, which is a Linux Foundation sub project that is studying contributions within open source.
And I think it's, it's just remarkably interesting because. And you know, how is measuring contribution actually a multiplier for contribution? It's a multiplier in a number of ways. It allows us to create incentive and direction, like historically, in a large open source project, your incentive for contributing is basically scratch your own itch, find a thing that you need, find the way to contribute to that.
and then the project goes, it grows in the direction of everybody's individual needs, but when you have an ability to tune weights and measures about, what people are contributing to, how much credit they receive, how that credit goes back to the organization that's sponsoring their time. You can actually begin to create some natural incentives and direction for where the project goes.
So we're in the early phases of that. There's a committee that's helping us govern that algorithm, but I just think the whole idea is fascinating from an academic point of view, and potentially might be a turning point in understanding how we govern and incentivize open source contribution. So that would be my answer.
Preston So: [00:36:34] Absolutely. And I think, and I think we've all read, you know, that that piece from Ashe Dryden, right? And all those years ago about the problem of unpaid labor and open source, I'm very interested to hear about how this community contribution credit system, really impacts the way that we get rid of the notion of, you know, meritocracy and get rid of the notion of code contributions are the only contributions in Drupal. I applaud you for, for being at the forefront of that.
Now, Narayan, what about you? I mean, you know, a lot of these community multipliers are supported by Tag1 are part of this, you know, kind of Drupal Association meilleur that we're talking about.
What's, you know, is it the same? Do you agree that, you know, this contribution credit system is most enticing to you? Are there other aspects of kind of ongoing work you're doing that are more interesting to you? Maybe one single project that you're working on?
Narayan Newton: [00:37:21] I feel like always having an interesting perspective on this sort of thing, just because I've been in the Drupal community for a long time, but I'm not really a developer.
I'm a system administrator.
So I would say my two would be upgrading Drupal.org and its substate sites to Drupal 9, because a lot of my tasks right now are kind of not directly related to that, they're about getting the underpinnings ready for that, making sure we can run, we're not gonna be able to do that all at once.
So making sure we can run various versions of both Drupal PHP and the stack underneath. So there's that. And then one thing I'm definitely very interested in is supporting a virtual DrupalCon and selecting the platform for that and making sure that Drupal.org As a whole is ready for that. I would say that's a mixture of interest and fear, but I'm very focused on it.
Tim Lehnen: [00:38:10] Yeah.
Preston So: [00:38:11] And you've identified two really interesting, things on the two opposite ends of the spectrum when it comes to the diversity of open source contributions, right? You're talking about very deep into the infrastructure about PHP and, and, and, and enabling that stack underneath. And then you're also talking about event organizing and how you're going to make the admin infrastructure work.
I think it's, it's, it's remarkable that we all have such an amazing interest, not only in the technology, but also in the community. I think that's what makes Drupal special. So we are out of time to talk about some of these challenges about infrastructure. We're going to talk about that more in future special additions of Tag1 TeamTalks.
But I want to finish here by asking both of you. How can the community help besides, you know, obviously DrupalCares and community donations and community contributions. Are there actual, let's say things that community members can do and organizations like Tag1 can do to join Tag1 in supporting these efforts.
Tim Lehnen: [00:39:05] Yeah. Well I'm gonna, I'm not gonna slide right over DrupalCares cause I think it's a really important campaign. So I will still spend one moment on that. If any of the listeners are not, are not familiar with what that is, please do check that out on drupal.org the link under the main hero image at the top of D. Dot. O will tell you all about it. We're really trying to support. the whole organization and ecosystem through some financial crisis caused by the Covid -19 situation. So that's really important. But from a, from another point of view, other organizations or individuals really anyone could help support us.
with this upcoming Drupal 9 upgrade drupal.org, uses a ton of contributed modules just like any other Drupal site. So anybody out there who wants to take a look at what drupal.org uses and help get it ported and ready for Drupal 9 would be making a great contribution and I think we can put a link in the show notes to drupal.org make file that has that list of, of all of those modules that we use.
The other thing I would ask, and then I'll, if Narayan wants to add to it, he can, the other thing I would ask is, please get the word out to your clients, if you work in Drupal and you have end users who aren't really part of our community, help them join our community, help them understand what the Drupal Association does, help them get involved.
Because. You know, the success of the project is going to be based on us growing that pie of engaged community members. And that end user audience is in many ways the hardest for us to reach. We can reach all of you, all of the contributors, all of the people who build Drupal very well, but you're, the ones who can reach, the, the actual end users themselves and help us reach them.
So please, try and do that.
Preston So: [00:40:43] Anything to add there Narayan?
Narayan Newton: [00:40:46] No, that all sounds good to me.
Preston So: Alrighty. Well, you know, I think that there's a lot of really interesting things in DA's working on. Obviously I definitely want to take a look at that make file and see if there's any modules that I can actually work on. Probably not given that it's been a while since I've written a Drupal 8 module, but I can still give it a shot.
And now for something a little new, we have a little segment on the. Tag 1 TeamTalks show called the Aside Tag where we share a little bit about what's going on in our real lives outside of these little boxes that we're currently speaking in. So I'll go ahead and share. We're going to each take one minute to share something that we've been working on or something cool.
Something that's really interesting about what we're doing right now. So for me, you know, this week has been a really interesting struggle to figure out how to get. Groceries delivered to my apartment. And you know, it's one of those things where, because of this whole pandemic, the concept of grocery delivery has skyrocketed.
And now everyone is really into the idea of having groceries delivered. There's a conglomerate very close to where I live that offers grocery delivery, but the minimum is $250 for every order. And I'm thinking, you know, I live alone. I live, I lived in my own apartment and I don't know how I could possibly make it through $250 of non-perishable.
Perishable food, within any reasonable timeframe. So I'm currently trying to figure it out. Do I risk it and go out or do I stay, you know, it's kind of that classic song. Do I stay or do I go. Anyways? What's on your mindset? What's going on with you?
Tim Lehnen: [00:42:11] Oh gosh. Well, with all the extra indoor time, I've been watching a lot of quarantine YouTube. and there's a few shout outs that I would give to things that I've found just amusing or reassuring or relaxing or just a way to take my mind off things. Alton Brown, who I'm sure most people out there know from the food network, like just a fantastic presenter and chef, has done just this short little series on his personal channel with his wife.
They call it quarantine kitchen, kitchen, spelled with the Q U like quarantine. It's hilarious. It's some of the funniest media that I have ever seen. So I'd highly recommend that to anyone out there who just wants to see the humor in this whole situation. and the way we're living now. I've also been watching a lot of the Adam Savage Tested YouTube channel, where he's doing a bunch of, solo one day builds and all this other kind of content.
It's just, Yeah, there's something soothing about watching someone work with their hands, focus on just some little projects that can be completed in a single sitting. and there's some insightful discussion about self-reflection in times like this, when it seems like the world is changing. So those, those are my, my wins for this little period.
Preston So: [00:43:19] I love that. I love that. That's wonderful, Narayan how about you?
Narayan Newton: [00:43:24] I feel like I'm coming at this from a bit of a different perspective, but I've been working on a load testing tool.
I, my friend Jeremy and I have been working on taking Locust, which is a load testing tool and Python that we use heavily and rewriting it in Rust or re-implementing it. It's not really a rewrite. It's a completely different tool based on the ideas of Locust, and Rust to take advantage of multiprocessors because we want to be able to more cheaply run a test trust to saturate large infrastructure.
I'm not sure this really fits the goal of something different from technology.
Michael Meyers: [00:44:06] It could definitely be about technology. And, Jeremy, for folks who don't know, is the founder and CEO of Tag1, and he is very much in love with Rust. He wants to talk about it all the time.
Preston So: [00:44:17] Really interesting. Oh, interesting. You know, I definitely need to talk to you both about this offline because I've been getting into Rust, myself.
I know one of the people who co authored the book on Rust. Very fascinating language, very fascinating things you can do. so, but sorry, we interrupted you, Narayan, sorry. You keep going. So
Narayan Newton: [00:44:35] That's it. I don't do much of that or any of the programming, but I do a lot of the help with design and testing and designing the load test to prove whether it's better or not, which is always really interesting.
Like it's, currently what we're working through is that it's much, much better at saturating a very fast response, but in a more real world test, we're seeing segments of time where it stops using the CPU and there's like stagnation that's clearly hitting a lock of some kind. So we're going to need to profile that.
So things like that, that can be very interesting. It's a little side project.
Preston So: [00:45:09] Very cool. Very cool. And how about you Meyers's going on in your world.
Michael Meyers: [00:45:13] Man, I'm going to take us back to the food theme cause I think like many of us, you know, being at home have more time to cook and certainly, you know, our challenge to get fresh groceries.
We're also trying to support local businesses. And I never thought I'd say this, but, I went to the farm down the road. There's a farm around the corner and they had short ribs. Actually, I have to shop online now and they drop it off at your doorstep. but I've been in love with the short ribs for, you know, 20 years, trying to find ways to cook them.
And I, I recently started playing around with sous vide I know it's been around for ages and I'm late to the game, but. It's perfect for stuff like short ribs. And, I did a 48 hour sous vide that came out with the absolute most amazing short ribs I've ever done in my life. I was blown away. so definitely recommend folks, if you haven't already checked out sous vide.
I'm, I did a Pork Loin last night, doing Ribeye tonight. Like, I just can't stop doing it now. It's, it's, it's wonderful.
Narayan Newton: [00:46:13] Do you sous vide and then finish on the grill or smoke or.
Michael Meyers: [00:46:17] Yeah, I, it's, we've, charcoal grills. I haven't finished on the grill. I've done a lot of like cast iron pan searing, but yeah, that's, that's critical.
Otherwise, yeah, that's a real big part of it. I'm still learning that too.
Narayan Newton: [00:46:31] I have two grills in my backyard right now. So this is near and dear to my heart.
Preston So: [00:46:36] Well, obviously I need to learn how to grill from, from the, from, from the people who are, who are experts here. you know, not just infrastructure experts clearly, but also how to get a nice char on a, on a, on a hunk of meat.
That's great.
Narayan Newton: [00:46:48] Do involve things being inflamed a lot.
Preston So: [00:46:53] Dumpster fires. Right. And before we do a wrap up here, we are at the other more time, but I know that Tim has a message for all of our listeners and viewers today.
Tim Lehnen: [00:47:03] Yeah. So, I wanted to say a series of thanks to everybody who's participated out there in the recent DrupalCares campaign. but I wanted to say a particular thank you to the Tag1 Team.
We've been partners with Tag1 for many years now. As you know, Narayan has mentioned he's been involved in the drupal.org infrastructure since before there was a DA. So like there's a lot of, this goes way, way back. Our history is really strong and Tag1 in particular has really helped us and supported us in getting through this difficult time.
So I just wanted to say a huge thank you for the work that you've done on the infrastructure, the support you've provided organizationally to the Drupal Association, and all the work you do to support the Drupal project. I really appreciate that and it makes me proud to join these kinds of conversations, these podcasts, and, and talk about how we've done such great work together.
So thank you.
Preston So: [00:47:55] Absolutely. And I want to echo, you know, the gratitude and appreciation that all of us in the, in the Drupal community have for the Drupal Association, all of its supporters, including Tag1, and all of the initiatives that are going on right now. I feel like every time I turned my head around, there's a new development, a new advancement, a new innovation in Drupal that is going to change every single developer's workflow in the community. And it's an amazing thing that we have such great folks working on these problems. So without further ado, thank you so much for joining us. All the links that we mentioned today are going to be posted online with this show episode.
And if you like this Tag1 TeamTalks episode. Please remember to upvote, subscribe and like it. Share it with your grandparents, your friends and family, socially distanced, of course, and as always, please check out our past talks at tag1.com/tagteamtalks. If you have any ideas or suggestions or feedback or proposed topics that you want to hear us talk about, you want to bring Tim back, talk more about the stuff that he's working on.
You want to bring Narayan back? Talk about. All the stuff he's doing around load testing, or if you want to bring back Michael Meyers, talk about some of his latest culinary escapades, let us know. Write to us. We value your feedback, tagteamtalks@tag1consulting.com I want to thank our dear friends here today and Narayan, Tim, and Michael Meyers for joining us today.
This was Tag1 TeamTalks, Special Edition about Drupal Association, and thank you so much for joining us. Until next time.