WordPress updates raise CMS security questions

“By failing to prepare, you are preparing to fail.” –Benjamin Franklin

WordPress vulnerabilities and exploits have filled the press lately, with 3 urgent security releases in the past month alone. The popular content management service (CMS) powers nearly one quarter of the web, making WordPress a ripe target for exploitation and a big concern for website owners and visitors to those sites.

A recent study from W3Techs shares that 47% of WordPress users only back up their sites “every few months,” with 25% saying they’re not trained in using WordPress at all. If exploited, however, nearly 25% say they would pay “almost anything” to get the lost data back. Add that to another 20% who would pay “several thousand dollars” to recover, and we’ve got almost half of WordPress users who recognize the price of inaction.

Why are these sites so vulnerable? In failing to prepare. Nearly half of the respondents reported having no IT or Website Manager. They are using WordPress because the CMS makes editing and adding site content easy for an everyday user. Sales and marketing teams are often left minding the shop with little to no technical training on the backend of the website.

WordPress itself is very responsive when it comes to releasing updates to patch holes as they come up, but they have to be implemented.

Where do you stand?

If you’re using the fully hosted WordPress.com solution, then the updates are part of the package. If you’re using the self-hosted WordPress.org solution, then it’s up to you to be prepared. (Not sure? More here.)

On self-hosted websites, the key is in how rapidly the security updates and patches are adopted. Security updates and patches are boring maintenance items, often not included in the overall website plan.

What can you lose?

Walking through the door to Customer Service, you see they’re really hopping already with email, chats, phones… Wow! Then, you catch the conversation in the room, and you realize there’s something wrong. Really wrong.

The site is not doing what it’s supposed to be doing. Content has changed. Customers are being redirected to odd places. They report suspicious pop-ups and installation requests. Some even, trusting you, have allowed these things to run.

Your website is doing a lot…of all the wrong things. You’ve been hacked.

Websites drive revenue, provide information, collect donations, and communicate on your behalf. When the site stops working, your mission and reputation are in jeopardy. If you collect information on your users, breach of privacy may open you up to additional cost and liability.

What can you do?

Designers and developers build content, behavior, graphics, features, shopping carts, etc. It’s tested and then deployed to a hosting provider. After a final check, the keys are turned over to the company.  If it’s up to you, what do you do?

First, WordPress is not alone.  Open-source (WordPress, Drupal, Joomla) and proprietary CMS platforms are all susceptible to exploit.

As a hosting provider for many self-hosted websites built on a variety of CMS platforms, what do we see? How do successful sites not only launch, but also remain secure and successful?

Know your CMS.

Communication is the key. When you’re building your website, get everyone together, and keep them talking on a regular basis. Ask a lot of questions along the way.

  • What CMS forms the foundation of the site? What permissions can be set for users? Many users posting content to the site may not need permission to change core elements about the site’s base architecture.
  • What plug-ins or add-ons contribute to the functionality? While recent exploit targeted the core WordPress CMS, these integrated programs interact with the sites and can also be targets for getting past site security.
  • Does your CMS have an auto-update function? WordPress offers a few options that can be set to help keep you current. If not, does the CMS offer alerts or an update blog site so that you can stay current on any issues?
  • What must be managed manually? Major version changes of the core CMS often require compatibility testing and are often not automatic as a result. Set up a testing and release schedule for these big changes.
  • Staying on the current version means that you’ll also stay up to date with the most recent security patches. Seen in every technology platform, end of life for a version means no one will be trying to keep it patched and stable.

Know your role.

Remember that survey. Nearly half of site owners would spend thousands, if not “almost anything” to recover lost data. That’s a lot of Benjamins.

It is like a good insurance policy, and a better use of resources, to plan for maintenance instead of praying and paying for miracles after you experience a loss.

IT resources are needed to manage and maintain the live website. You don’t have to be that expert yourself, but don’t forget to budget for routine maintenance and updates when you’re allocating IT resources.

If you don’t have in-house expertise, consider a management agreement with your developer. Like oil changes for car, it’s part of the price of ownership.

Know your hosting provider.

Not all hosting providers are created equal. Your hosting provider should be an active partner: at a minimum, keeping the Windows- or Linux-based infrastructure stable, secure, and updated in its own right.

As a provider ourselves, we’re a bit biased here at DataYard. We think that a hosting provider should be so much more!

Customer service and support expertise are vital. At DataYard, we love being included in a website’s overall design and development from Day One. We’re experts on our hosting platform, the options available, and maximizing the architecture’s performance for your site.

We talk to you about the site architecture itself, and making sure it’s backed up on a regular schedule. Regular backups mean no mad scramble to see if anything can be recovered.

“When we work with the developers as a site is under construction, we look for bottlenecks to performance,” shares Ryan Chewning, DataYard Systems Administrator. “Most security plugins in WordPress, for example, are incompatible with one another even when they’re fine separately. Some of the security plug-ins help drop malicious connections rapidly, keeping system resources readily available.”

For those building a new website to replace an existing site with active customers, we have extra considerations. Ryan explains, “The user experience during any change is important, from incorporating plug-ins that seamlessly bridge site versions to minimizing any downtime needed to complete the transition.”

The relationship with your hosting provider should not stop when the site goes live. Active management is a valuable element to keeping your website healthy. A DataYard managed account is monitored for performance. With success and more traffic, needs change over time. Ryan concludes, “We watch for performance degradation and make proactive recommendations to keep the site growing along with you.”

What next?

Since your website ties to your bottom line, the bottom line is that you don’t want to trust it to just anyone. If you’re not sure what you have, now’s the time to ask. If you need some help looking at where you are and where you want to be, remember to ask your trusted partners here at DataYard.

Monthly Technical Operations Maintenance Window – May 16, 2015

Monthly Technical Operations Maintenance Window – May 16, 2015

The DataYard Technical Operations team will begin the monthly maintenance window for May at 6:00AM EDT, Saturday, May 16, 2015 with an anticipated completion time of 12:00PM EDT on Saturday May 16, 2015.

During this month’s maintenance window, DataYard System Administrators will be installing security patches on all managed and shared servers.  Service interruptions are expected to be brief, if they occur at all.

If you have any questions regarding this maintenance please contact our technical support team at 1.800.982.4539 or support@datayardworks.com. Remember to follow us on Twitter (@datayard) for the latest service and maintenance status updates as well as other happenings at DataYard.

In the Spotlight: Cope’s Jumps into NASCAR

As a new NASCAR sponsor, Cope’s Distributing races to new heights. 

Spring rains, budding flowers, green grass, and the sound of racing! Greenville’s Cope’s Distributing hops on board with NASCAR, as a sponsor  on Tommy Regan’s #45 — “The Gun Truck.”

With both a retail store and large Internet business that takes firearms distribution nationwide, a NASCAR sponsorship brings in a new audience, and an increased demand on the technical resources of their Internet portal.

It all started when DataYard Account Manager Alek Mezera called Roger Cope, President of Cope’s Distributing, to check in.

“When Roger told me of Cope’s plans to sponsor Tommy’s truck, their video series, and Facebook ticket contest, I knew we needed to take a look at the hosting setup to ensure uptime and availability during this high-visibility campaign,” says Mezera.

“We started by segmenting the server roles onto two VIPs for web and database services, and built the new configuration to quickly accept new additional web nodes if necessary.”

This level of communication is part and parcel of the relationship DataYard has built with Cope’s, in listening and customizing to their specific needs. The attention doesn’t surprise Roger. “We make a phone call and they’re right there,” he says. “I recommend DataYard to literally everybody that’s looking.”

How is The Gun Truck doing? The team kicked off their season on March 28 in Martinsville, placing 32nd, and are headed for the Kansas Speedway on May 8! You can follow the team on their website, via Facebook, and on Twitter (@theguntruck45).

Thanks to Cope’s Distributing for letting us share their story! We want to tell YOUR story too!  Let us know what you’re doing, so we can shine the Spotlight on you!

LEARN MORE about Cope’s Distributing and DataYard.

 

The End is Near… for Windows Server 2003

3 months. No reprieve.

Microsoft executives and media spokespeople are out in force, emphasizing that Windows Server 2003 is headed out to pasture, with end of life (EOL) slated for July 14, 2015 (Yes, THIS July 14). With EOL, Microsoft ceases providing updates, service patches, and support for between an estimated 1 and 12 million servers worldwide.

So, if you’re still facing this challenge, you are clearly not alone. This one’s not just a simple update, as infrastructure that supported Windows Server 2003 (WS2003) can’t handle what’s now needed: Windows Server 2012 R2 (WS2012 R2).

“If it ain’t broke, don’t fix it.”

Aside from appalling grammarians everywhere, this feeling is easy to understand. Most users don’t have visibility to the full brokenness of older operating systems. After all:

  • The applications that the old systems have been running for over a decade (!) now still run.
  • The systems have been that solid for that long, so support or updates shouldn’t be needed because you know they work and are used to their quirks.
  • Besides, upgrading can break things and will surely make them different.
  • With expensive hardware required, this is a messy and costly proposition.

Assuming that most of us haven’t been using the same toothbrush for a decade, and that we don’t expect the same showroom performance out of a 10 year old car, it’s time to face this one head on.

Why change then?

Servers don’t exist in a vacuum. They provide essential services that need to stay up and running. Failing to stay current opens businesses up to serious vulnerabilities:

  • Productivity. Crashes, system downtime, and wasted personnel resources all drain energy from business tasks as you waste time fixing IT issues and patching workarounds. This costs in time and in money. It costs nearly double to fix systems that are over 4 years old.
  • Security. Starting July 15, no new updates or patches means no fixes to newly found security threats. No change means risking the availability of services, the confidentiality of data, and the privacy of customers’ data.
  • Compliance. Most industries have legal compliance requirements in meeting current technical standards, and customers themselves have compliance and regulatory demands included in contractual service level agreements (SLAs). Being out of compliance breaks the SLAs, resulting in fines or loss of business. Accepting credit card payments? Without updates, Visa and MasterCard are on the record that they’ll have to refuse business due to lack of PCI (Payment Cards Industry) compliance.
  • Support. Most hosting companies and technology professionals will be unable to continue to support WS2003 systems. No updates means that they won’t be able to meet their own SLA requirements.
  • Competition. Competitors using new technology are at an advantage. What is out of date? If your operating system is over 5 years old, 61% of your customers say you’re outdated! Moving on to the newest platform will let you more quickly add new systems, set up new users, allow for remote operation, faster response times, etc., bringing a better service to internal users and to customers. The newer platform (WS2012 R2) offers a streamlined and efficient approach for remote accessibility, more options for storage and automation, and greater security.

What now?

IT professionals know the problem and don’t really need convincing. Time being short, it’s time now to push through organizational inertia to get moving.

Since it’s not advisable to cross your fingers and hope for the best with WS2003, migration is the answer with two choices of destination:

  • New assets. Invest in new replacement hardware and software for your infrastructure.
  • Take it to the cloud. Take this time where migration is required anyway and move your infrastructure to a hosted solution. Divest yourself from taking on the cost of owning the assets or the headaches of managing them.

Why are we here?

DataYard’s Infrastructure as a Service (IaaS) provides a cloud solution that’s still close to home and managed by those you’ve come to trust. We understand that the ultimate connection for the technology we maintain are the people relying upon it. We give you people you can rely on to get the job done. Take it to the cloud without sacrificing control.

Our mission at DataYard is simple. We want our customers to focus on their businesses while we serve as trusted technology partners, keeping essential technology infrastructure stable, reliable, secure and redundant. Mission critical applications and data need to be available and error-free so that, like electricity, you forget about it and just run your business.

If you’re facing the migration from Windows Server 2003, you want experts on your side. 90 days is not too late to start. A review of migration stories across the web shows many successful and fairly complex migrations were actually achieved in 60-90 days. In fact, most migrations took two to three times as long to go through the approval process (“Do we have to?”) than through the actual planning and execution of the migration.

DataYard has scalable cloud options that get you out of the hardware business, while keeping your operations secure and local with people you know and trust. No solution is canned and out of the box. We’ll sit down with you to see where you are and where you need to be. Then we find you the best path to get there.

Get current now, and then stay current as technology continues to evolve. Staying with our hosted solutions will keep you current moving forward. As updates and newer platforms are released, we’ll be with you on that path together.

Each DataYard solution is tailored to fit your needs exactly, including making sure you have the level of management control over your own system that you need. Give us a call and let us help you make the best decisions you can for your business.

Monthly Technical Operations Maintenance Window – April 18, 2015

The DataYard Technical Operations team will begin the monthly maintenance window for April at 6:00AM EDT, Saturday, April 18, 2015 with an anticipated completion time of 12:00PM EDT on Saturday April 18, 2015.

During this month’s maintenance window, DataYard System Administrators will be installing security patches on all managed and shared servers.  Service interruptions are expected to be brief, if they occur at all.

If you have any questions regarding this maintenance please contact our technical support team at 1.800.982.4539 or support@datayardworks.com. Remember to follow us on Twitter (@datayard) for the latest service and maintenance status updates as well as other happenings at DataYard.

Monthly Technical Operations Maintenance Window – March 14, 2015

The DataYard Technical Operations team will begin the monthly maintenance window for March at 6:00AM EDT, Saturday, March 14, 2015 with an anticipated completion time of 12:00PM EDT on Saturday March 14, 2015.

During this month’s maintenance window, DataYard System Administrators will be installing security patches on managed and shared servers. Network Administrators will be performing routine Hosted Firewall security updates. Service interruptions are expected to be brief, if they occur at all.

If you have any questions regarding this maintenance please contact our technical support team at 1.800.982.4539 or support@datayardworks.com. Remember to follow us on Twitter (@datayard) for the latest service and maintenance status updates as well as other happenings at DataYard.

Utilizing Cgroups to Balance Resources

Cgroups-image1

How often have you endured complaints from users about server or service performance because of a small subset of users who are taxing the system, utilizing the majority of the available resources? How do you manage resource sharing on a multi-purpose server that may have a database server and web server running together? We have experienced these situations many times, but rather than write exotic process management scripts, maintain heavily modified init scripts or complicated user environments; we have switched to using cgroups.

What are Cgroups?

Cgroups or container groups started in September 2006 as a patch for the Linux kernel initially developed by a few Google engineers. Process containers, as they were initially called, allowed administrators to restrict or prioritize processes and their access to server subsystems. In the 2.6.24 kernel release process containers was merged into the mainline kernel and were renamed to cgroups.

Cgroups provide hooks into various kernel subsystems which allow an administrator to limit the resource utilization of a single process or group of processes on one or more subsystems on the server. Cgroups are currently able to limit CPU, memory, network, and disk I/O. They are also able to perform resource accounting and have the ability to pause and restart processes. As cgroups have grown and developed they have become part of LXC, an OS level virtual server implementation similar to Linux-Vserver or OpenVZ.

Take the following example as an example of how cgroups can be used to balance resource utilization on a server. Company XYZ, Inc has a multi-purpose application server with three departments utilizing it, Accounting, HR, and Sales. This server is also providing XYZ, Inc’s intranet site. Administrators frequently receive complaints from internal users about the performance of the corporate intranet site when HR is running their payroll report. dditionally, Accounting and HR are unable to run their reports at the same time due to the degradation of performance. By using cgroups on this application server it is possible to improve the end users experience for all departments.

Since cgroups work in a hierarchical fashion and allows for multiple hierarchal groups to co-exist we can setup multiple, nested cgroups to limit users or processes from consuming all the system resources. This will protect system resources for other processes ensuring they will always have access to system resources. For XYZ, Inc’s application server we can create two “top level” cgroups, one we would call ‘departments’ and the other ‘daemons’. Inside the root level of each hierarchy we can set the maximum amount of CPU time the whole group will have available to it. We can give our ‘departments’ group 80% of the CPU and memory and ‘daemons’ group 20% of the CPU and memory. The application accounting uses is CPU and memory intensive and we want to prevent those applications from negatively impacting the intranet site and other department’s applications. Within our ‘departments’ hierarchy we can create three additional cgroups, one for each department, giving each a percentage of the 80% of the CPU and memory that has been allocated to the ‘departments’ cgroup. This now limits the memory and cpu that can be consumed by each department and allows a more equitable sharing of the resources. We could continue on adding additional sub-cgroups in ‘departments’ limiting or guaranteeing the amount of block I/O or network throughput for certain departments, applications or users.

Cgroups-image2

Utilizing Cgroups

While the explanation provided is fairly straight forward, when it comes to configuring cgroups, it can be rather daunting as the process isn’t well documented. Cgroups utilizes a pseudo file system for configuration and reporting very similar to the proc or sys files systems. There are two ways to setup cgroups. The first is to write scripts that will mount the pseudo file system which is composed of the subsystems that will be used, create the hierarchy that will be used and and finally set the values for the cgroups. The second way uses the scripts and daemons that come in Debian’s cgroup-bin package or Red Hat’s libcgroup package. With the second method only two configuration files need to be modified.

In Debian based distributions those files are /etc/cgconfig.conf and /etc/cgrules.conf. The basic tools to configure and manage cgroups comes in packages for Debian and Redhat, cgroup-bin and libcgroup which provide the cgconf init script for setting up cgroups and the cgred daemon which places processes in cgroups based on user defined rules.

cgconf

The first init script is cgconf. This script will setup cgroups based on the /etc/cgconfig.conf configuration file. In this file you are able to set the subsystems that you wish to be able to control and then set the values for the various subsystems. Currently cgroups supports the following subsystems: blkio (block device I/O), cpu, cpuacct (CPU accounting), cpuset (CPU core affinity and NUMA access), devices (allows and denies access to block and character devices), memory, freezer (suspend and resume of processes), net_cls (allows for tagging and classification of packets), net_prio (allows for setting the priority of the outbound traffic on a specific interface), ns (limits access to a specific namespace), perf_event (allows for monitoring of processes via the perf tool).

An example from the Red Hat documentation of a cgconf configuration file.

mount {   

    cpu     = /cgroup/cpu_and_mem;

    cpuacct = /cgroup/cpu_and_mem;

    memory  = /cgroup/cpu_and_mem;

}

group finance {

        cpu {

                cpu.shares=”250″;

        }

        cpuacct {

                cpuacct.usage=”0″;

        }

        memory {

                memory.limit_in_bytes=”2G”;

                memory.memsw.limit_in_bytes=”3G”;

        }

}

group sales {

        cpu {

                cpu.shares=”250″;

        }

        cpuacct {

                cpuacct.usage=”0″;

        }

        memory {

                memory.limit_in_bytes=”4G”;

                memory.memsw.limit_in_bytes=”6G”;

        }

}

group engineering {

        cpu {

                cpu.shares=”500″;

        }

        cpuacct {

                cpuacct.usage=”0″;

        }

        memory {

                memory.limit_in_bytes=”8G”;

                memory.memsw.limit_in_bytes=”16G”;

        }

}

When editing the configuration file for cgconf you will need to restart the /etc/init.d/cgconf init script. Restarting only unmounts and remounts the cgroups file system, but this will clear all current cgroups, their configuration directives and will also free all processes from cgroup restrictions. I have found that it is best for persistent daemons such as Apache or MySQL to be restarted after cgconf is restarted to ensure all processes are placed under cgroups control again.

cgrules

The second init script will start the cgred daemon which is the cgroup rules engine daemon. This daemon moves processes into the correct cgroup as the process is created. The daemon takes its configuration from /etc/cgrules.conf. You can configure processes to go into specific cgroups based on the user and/or group the process is running under and/or the process name.

Most documentation that I found while implementing this in some of our production environments was basic; either referencing or coming straight from the Linux kernel documentation of cgroupsone of the most helpful documents was from Red Hat. We are almost entirely a Debian based shop so there are a few difference in how Debian configures cgroups. One of the main differences is the location of the root cgroup. In Redhat based distributions this file system is mounted at /cgroups/. Debian based distributions mount this in /sys/fs/cgroups/. While you can change where this mounts, I would recommend keeping this at the default location as it makes it much easier with the default init scripts to work with the cgroups.

While deploying cgroups I encountered a known issue in Debian 7 “Wheezy” where the init scripts are broken. This was with libcgroup 0.38-1 which shipped with Wheezy. There was an attempt in Wheezy to fix a bug in the init scripts, which could cause processes to crash when loading a new configuration; however, the bug was not correctly fixed when Wheezy shipped. I had to use the init script for cgconf that shipped with Debian 6 “Squeeze” to get cgconf to behave as expected.

We have cgroups implemented on our shared Linux Fusion Web Hosting platform. Being able to restrict the amount of memory and cpu that processes like Apache or PHP can consume has helped to limit the impact of DoS attacks against our customer sites. Beyond this, we are implementing cgroups in other areas of our Linux infrastructure to provide better access to servers during times of high utilization. We have found cgroups as an effective way to manage and enforce resource sharing in our multi-user Linux environments.