MySQL

Unraveling the Extract, Transform, Load (ETL) Data Migration Process: A Deep Dive on Load

janez — Wed, 24 Jan 2024 13:37:01 +0000

In this episode of Tag1 Team Talks, our team of Drupal experts delve into the essential "Load" phase of the ETL (Extract, Transform, Load) process in Drupal migrations.

janez Wed, 01/24/2024 - 05:37

The golden Goose egg, a compile-time adventure

lynette@tag1consulting.com — Wed, 06 Jan 2021 17:17:44 +0000

Goose, the highly scalable load testing tool created by Tag1 Consulting, has undergone a number of improvements since its creation. Asynchronous support created a significant boost in performance, but Jeremy felt there was still room for improvement.

lynette@tag1co… Wed, 01/06/2021 - 09:17

Stop Disabling SELinux!

Jeff — Thu, 22 Dec 2011 15:48:37 +0000

I see a lot of people coming by #centos and similar channels asking for help when they’re experiencing a problem with their Linux system. It amazes me how many people describe their problem, and then say something along the lines of, “and I disabled SELinux...”. Most of the time SELinux has nothing to do with the problem, and if SELinux is the cause of the problem, why would you throw out the extra security by disabling it completely rather than configuring it to work with your application?

Jeff Thu, 12/22/2011 - 07:48

Performance Tuning Your Database From Drupal

Jeremy — Thu, 14 Jan 2010 08:28:56 +0000

I began porting the dba module to Drupal 7 as an opportunity to learn more about the new database abstraction layer. In this upcoming Drupal release, our database layer is now built upon the PHP Data Objects (PDO) data-access abstraction layer, introducing new syntax and symantecs into writing queries with Drupal. A powerful new query builder makes it possible to easily write queries that run on any database type supported by Drupal, currently including MySQL, PostgreSQL and SQLite. The dba module was always intended to be a useful tool for any database administrator, and toward that end I have recently merged in mysqlreport functionality which utilizes MySQL's server statistics to offer an overview of server health and performance. As an enhancement, I also merged in the knowledge shared in this earlier Tag1 Consulting article , so now problem areas detected in the mysqlreport output are automatically highlighted along with some useful help text offering tips on better tuning your database. Screenshots The following screenshot shows a sample of the output generated by the enhanced MySQL reporting built into the Drupal 7 version of the dba module: When holding your mouse pointer over any of the text that's highlighted in red...

Jeremy Thu, 01/14/2010 - 00:28

Tag1 Now Hiring Interns

nnewton — Thu, 22 Oct 2009 04:52:02 +0000

At the beginning of 2009, I was hired by Tag1 Consulting as Jeremy Andrews' full time partner. A decidedly questionable decision on his part, but a great change for me! I used to work at the Open Source Lab at Oregon State University and I am currently the sysadmin for drupal.org. Working at the OSL and drupal.org spoiled me, I'll be completely honest about that. I got used to working with interesting new technologies and consistently pushing the limits of my knowledge. I have nothing against applying well known best practices and I certainly do a lot of that, but its not why I do the job. What I love is learning new and cool technology. The OSL and Drupal gave me the ability to do that and after graduation I was looking for a place to continue doing it. Tag1 is a perfect fit. On any average day we go from assisting smaller sites with their growth, mostly using tried and true solutions, to working with large enterprise-class installations which require customized and interesting solutions. Success has lead to a problem for us...we need clones. Now, we have researched current cloning technology and its frankly a little disappointing. Not...

nnewton Wed, 10/21/2009 - 21:52

Cacti Memcache Multi-Port Templates

nnewton — Sat, 25 Apr 2009 04:12:16 +0000

We at Tag1 have used the Cacti Memcache templates many times. ( http://dealnews.com/developers/cacti/memcached.html ) They have become a staple for us and many others. However, we and one of our clients have had issues making them work for monitoring multiple Memcache instances (running on different ports) on the same host. It is possible to make this work with the vanilla templates, but it is a bit difficult. Because of this, we have created a re-roll of the latest templates that allows you to add multiple templates to a host and set the port on each of them. This work was kindly sponsored by http://www.looforyoo.com/ . Download Cacti Memcache Templates - Multiport Instructions: These graphs work exactly the same as the vanilla ones, but there is a port field to fill out when you create the graph. You can add multiple graphs to the same host by going through the create step multiple times. (i.e. selecting the graph in the create graph drop down box and pressing create)

nnewton Fri, 04/24/2009 - 21:12

Cool Little Utilities

nnewton — Wed, 25 Feb 2009 08:27:10 +0000

I've had the opportunity in the last few days to use quite a few cool little utilities that made my life considerably easier. I've known about most of them for awhile, but discovered some new tools or new uses for old tools and wanted to share two of them. AutoSSH SSH tunneling is a way of life. I can't count how many times I've had to throw one into place because "there was no other option." I've even seen people use them to protect MySQL replication, not for security interestingly enough but because SSH doesn't "trust the network" the way MySQL does. However, using an SSH tunnel brings the risk of its premature death due to network errors or timeout. Enter autossh stage left. There are many ways to script an ssh wrapper to prevent tunnel death, but autossh does this all for you. It basically creates a tunnel loopback, where ssh is forwarding ports to the remote server and then back to the local server so that it can send a packet to one of the ports and have it loop all the way to the remote and back to local. If it doesn't get the packet back...ssh has...

nnewton Wed, 02/25/2009 - 00:27

Tag1 MySQL Monitoring

nnewton — Sat, 14 Feb 2009 06:23:07 +0000

I recently joined Tag1 Consulting full time. My main project since I joined was building up our monitoring infrastructure. My personal goal being world domination, but first the best monitoring system I can build. Baby steps...baby steps. For quite awhile now, I've used the monitoring system we built at the Open Source Lab to detect and diagnose problems with drupal.org, apache.org, kerneltrap.org and many more Free Software projects hosted at the lab. These experiences convinced me of the absolute necessity of a robust monitoring system. It began simply at the OSL, a very stripped down Cacti instance to monitor load, memory usage and network usage. Combine this with Nagios monitoring ping latency and not much more. We went from this to monitoring disk space, load, users, MySQL availability and much more with Nagios. We also added some simple Cacti graphs for monitoring MySQL performance. At Tag1, I've been allowed to take this even farther. First, I tracked down some newer MySQL Cacti Templates developed by Xaprb. These templates are extremely cool and allow you to graph: InnoDB Buffer Pool Activity, IO Activity, Log Activity, Processlist, MyISAM Index Usage, Connections, Network Traffic, Sorts, Temp Tables and much more. You can see...

nnewton Fri, 02/13/2009 - 22:23

MySQL Replication

nnewton — Thu, 31 Jul 2008 08:08:37 +0000

Until recently, I was a student employee at the Oregon State University Open Source Lab. My career there ended, like many, with that painful process known as graduation. I got invaluable experience at the lab, not the least of which being the knowledge gained as their main (only) database administrator. One of my great pleasures in that position, was learning how to configure MySQL replication and manage clusters of replicating database servers. Even the simple case of a single master and a single slave has its edge cases. There are endless tips and tricks to make long-term maintenance easier and to allow you to make full use of this technology. Running replicating database servers for Drupal in particular has several tricks to it: Replication Settings - Timewarp A major issue with replication in MySQL is that it has a tendency to fall behind. There are various tricks you can employ to assist with this and some big ones for Drupal specifically. First, if you are going to use the search module, you will need to stop the replication of the temp tables it uses. These will be replicated to the slave as they are technically volatile queries, however, they are...

nnewton Thu, 07/31/2008 - 01:08

Chapter 1 Rough Draft Complete

Jeremy — Mon, 28 Jul 2008 03:27:07 +0000

I have completed a rough draft of the first chapter of " Drupal Performance and Scalability ". The first chapter of this online book is divided into four sections, the first of which focuses on the importance of fully defining your performance and scalability goals, helping you to identify what you need to accomplish and how to set concrete and attainable goals. The second section discusses monitoring and measuring your ongoing progress, helping you decide what you need to monitor, and how to monitor it. The third section stresses the importance of making regular backups, discussing what needs to be backed up, and offering example scripts for backing up your entire website, including the database. Finally, the fourth section takes an in depth look at using revision control tools to manage your website, providing useful recipes showing how Git can track changes to your website, helping you update to new releases and push those updates into production. It is important to realize that this is a rough draft, and as such it may contain spelling or grammatical errors, it may be missing key points, and the writing style may not be very polished. However, the book has to start somewhere...

Jeremy Sun, 07/27/2008 - 20:27