User:OrenBochman/Labs 2

From Meta, a Wikimedia project coordination wiki
Jump to navigation Jump to search
Presa de decissions.png This is an assignment for Simone's adoption program. You are welcome to edit this page if you notice any errors or have any additional information to add, but as a courtesy, please notify OrenBochman if you make any major changes to avoid any possible confusion between him and his adoptee(s). Thanks!

Puppet and Labs[edit]

So you have learned about labs and how to create an instance. Behind labs interface there is a configuration management system called puppet. Puppet is important in another more crucial sense - it is also used to manage MediaWiki's production cluster. A necessary step to bringing your work to production will require writing puppet manifests to manage it. This study unit covers basic Puppet usage as well as a number of best practices.

What is puppet?[edit]

Puppet allows users to:

  • Setup and manage multiple virtual machines with predetermined configurations.
  • It is a DSL built on top of Ruby and uses a declarative model.
  • It allows to work using a recipe based approach.
  • It provides an abstraction layer for configuration.

Getting Started[edit]

To proceed with this tutorial you will require sysadmin access to the testing lab - which will enable you to define variables within your instance. .

  • Create a new instance and configure it as follows:
    This last one configures the instance to install a Puppetmaster. Puppetmaster is the server used for running puppet - which means that your system will not be able to work locally without having to access an external puppet server.

now wait for a bit for the lab to go online, it takes time since it requires puppet to compile the definitions, create the vm and boot it. Now:

  • login to bastion
"ssh bastion"
  • access the lab
ssh puppet-demo4 (accept the first lack of certificate message)
sudo puppetd -tv

TODO: insert screenshot of puppetd -tv output

The last command invokes a puppet run by the puppet demons to run and updates the local configuration

  • add a puppet master to local instance.
  • adds the operations repository.
  • and configures the instance to talk to to itself rather than the puppet master.


without a puppet master self - you need to work with a centerelized puppet master configuration as used in production by WM operations . To make changes you need to push changes into gerrit, get them code reviewed and use them from there.

by using the what is

Directory Layout[edit]

\puppet
|\manifests
|+site.pp
|\files
|\private
|\modules

Resources Definitions[edit]

Puppet simplifies configuration management by abstracting resources. In puppet a resource is defined by a type, title and a list of attributes. Further more resources can be bundled together into collections. Some of the built-in types include:

  • package
  • service
  • file
  • cron

an example is

     file {
           "/etc/apache2/sites-enabled/000-default/":
                 ensure => absent
                 notify => Service["apache2"];
     }

this defines a file resource of type file at "/etc/apache2/sites-enabled/000-default/" which should be removed if available.

Creating a Class Definition in Puppet[edit]

Puppet is declarative and highly reusable. This allows working in a recipe like approach.

nodes (only used in productions)

variables can be used to parametrize the installation.

TODO: screenshot she used icinga which is a opensource monitoring application which is a fork of ngios

#import ...

class apache::demo {
...
}


Packages[edit]

Crystal Clear action bookmark.png Best Practice Tip :Breaking your puppet manifests into small modules makes them easier to read and maintain


the advantage of packages is that they can be used to combine different types together.

class name::monitor::packages {

     #this will install the latest version of the apache web server

      package { [ "apache2"]:
              ensure => latest;
      }

}

Services[edit]

Crystal Clear action bookmark.png Best Practice Tip :Try to use similar paths for similar objects

services:

   service{
         "apache2":
              ensure => running,
              require => Package["apache2"];
              subscibe => [File[$....],
                           File[$....],
                           File[$....],
                          ]
   }

this spells out explicitly that the service should not be started until apache has been installed.

subscribe here means that the service must be restarted if the configuration files are changed.

$ before a file names indicates that these are variables.

Files[edit]

Crystal Clear action bookmark.png Best Practice Tip :use fully qualified variable names — these include the full name of the package the element belongs to.

     file {
           "/etc/apache2/sites-enabled/000-default":
                 ensure => absent,
                 notify => Service["apache2"];
     }

which removes the file and then request apache server to restart or refresh it's execution context.

Dependencies[edit]

Crystal Clear action bookmark.png Best Practice Tip :Without dependencies the order of exception is undefined in the puppet environment - which will break your configuration if possible

To organize the order of operation on resources four metaparameters are used. These are:

  • before
  • require
  • notify
  • subscribe

they all accept a resource refrence or an array of resources.

Before and require are very similar. before is used to declate the earlier resource, while require is used to declare the later resource.

notify and subscribe are analogous to before and require, but also declare refresh relationships that they define

Templates variables and Conditionals[edit]

Crystal Clear action bookmark.png Best Practice Tip :Put in the service start/stop before creating/deleting the file - since the serviced won't be started/stopped once the file has been created

It is possible to use templates to parametrize a puppet manifest. For this however we will need to define variables, set the variables and use them in conditional constructs.


     file {
              "/etc/apache2/sites-enabled/000-default":
                    ensure => absent
                    notify => Service["apache2"];
              #this file name is deined in the template apache demo
              "/etc/apache2/sites-enabled/apachedemo":
                    owner => root,
                    group => root,
                    mode => 0444,
                    content => template('apache/sites/apachedemo.wmflabs.org');
                    notify => Service["apache2"];
     }

to set it we go to the Labs UI and

TODO: add screen shot of using

The variables can then be set using:

  • an external file (in production) or
  • using the labs UI in labs.

Using Modules[edit]

  MODULEPATH/
    MODULE_NAME/
      files/
      templates/
      manifests/
      ...
      README

Debugging, Tracing, Logging & Troubleshooting[edit]

Crystal Clear action bookmark.png Learning Tip :You can often borrow a lot from similar resources, so try to have a look at other people's manifests









Trouble Shooting & Error Logs[edit]

Finding and fixing errors

TODO: Add error log location

Style[edit]

  1. quote resource names
  2. quote parameter values which are not reserved words in Puppet.
  3. include curly braces {} around variable names when referring to them in strings.
  4. end lines that declare parameters with a comma.
  5. when declaring a resource with a single parameter, use a single line declaration without a trailing comma.
  6. multiple parameters of a single resource are placed each on their own lines.
  7. arrows should be lined up within each resource.
  8. more at http://projects.puppetlabs.com/projects/puppet/wiki/Style_Guide

Further Reading[edit]

Discussion[edit]

Any questions or would you like to take the test?