My little backup solution

Hi All,

I get a few people that I know asking about backup solutions for home, so I thought I would put document what I use and works well for me.

What I needed was a solution that would allow for the following:
1. backup multiple iterations
2. allow for centralized backup for multiple computers
3. simple to use (I am a simple person)

What I ended up with was the following:
1. D-Link DNS-323 2-Bay Network Storage Enclosure
2. WD 500GB Drive (bought a while ago, so increase as needed)
3. Tivoli Continuous Data Protection

Now I know you all are going, "of course you went with a Tivoli product", but I downloaded the trial and was quite pleased with how it worked and did the tasks I needed it to do.

Here are some screen shots

This is the Start page for TCDP. I have meshed together the flyouts for the My Files, Local Storage and Remote Storage to show some more info.


The General Settings (Accessed from My Files flyout > Settings) allow for some configuration options such as where to backup locally, how many iterations and how much space to use.


In the Files to Protect settings, you can set the files/directories to include and exclude.


Email protection will allow for some various email programs for backup.


Remote Storage allows for TCDP to backup files to a remote storage device. In my case it is the DLINK NAS as listed above.


Advanced Settings allow further configuration to tune TCDP


As files are modified, TCDP will back them up locally and remotely (if available). Below is the Report feature that shows the image files I used for this blog being backed up. The popup in the screen occurs when you click on the Date/Time for the row that you want to see to provide more detail.


I really have not noticed any performance impact, but you also have to look at what is being backed up. I made sure to exclude any of the VMWare extensions and a few others that I know I do not want backed up. This is just like setting exclusions for Antivirus software :)

The restore is also quite simple and will allow for selection either from the local or remote source. When restoring, you have the option of restoring to the original location or to a new location. I would put screen shots for the restore steps, but this one is getting long enough already.

The backup and restore does work, and works well, I tested it out recently after someone had their drive crash with all their pictures. I wanted to make sure this worked!

There is a "demo" on IBM's site at http://www-01.ibm.com/software/tivoli/library/demos/cdp.html?S_CMP=rnav, but it is more of a high level of what the product does rather than a look at using the actual product.

Now the only issue I have with it is that there is not currently a linux client, which may or may not be a problem for you. It is not a big deal for me as anything linux I have does not really require a backup solution.

Script to retrieve Tivoli patches

For the longest time, I have been visiting the IBM FTP site looking for patches that have recently become available. This is especially true when I am waiting for one to come out that I am expecting. If anyone else has done this, then you know how much fun that is. I am on the various emails that let you know when patches are released, but these seem to have no rhyme or reason on when they are run or what they report on. The best source seems to be the FTP site.

Recently I decided to do something I have been meaning to do for the longest time and that is write a simple script that would connect to the ftp site and find any recently released patches. In order to use this script there are a couple modifications that will need to be done. I have kept all the variables in the Initialize subroutine in order to make these modifications easier.

Variables:
SMTP Server - This will need to be modified to supply any smtp server.
    $gMailDomain = "";

Sender address - Whatever the address you want the "From" field to use
    $gSenderAddr = "\@";

Recipient - Address to send the listing to
    $gToAddr = "\@";

    Note: to send to multiple addresses, just comma delimit:
    $gToAddr = "name1\@domain.com, name2\@domain";

Also there is a variable to specify the number of days to look back for patch updates. Just set the global variable $gDayDiff to the number of days you want to look back for updates.

When the script is executed, it will create a file called updates.html (name can be modified by changing the global variable $gHtmlFile). This file is then used as input into the email.

This script does have a requirement for the Mail::Sender module. If you do not want to receive the emails, then comment out the following lines
    use Mail::Sender
    SendMail();
    The entire SendMail subroutine

The file can be downloaded from: http://www.gulfsoft.com/downloads/blog_downloads/patches_list.zip

I have tested the execution on Windows XP, Vista and Red Hat 4. Please let me know of any errors you might encounter.

BTW, leave comments and let us know you are out there.

Thanks

Martin Carnegie

Changing OMNIBus password in TIP/ITNM

The Object Server password is stored in encrypted form in these three files on the TIP server:

/opt/IBM/tivoli/tip/properties/objectserver.properties
/opt/IBM/tivoli/tip/profiles/TIPProfile/config/cells/TIPCell/wim/config/wimconfig.xml
/opt/IBM/tivoli/tip/profiles/TIPProfile/etc/webtop/datasources/ncwDataSourceDefinitions.xml


The last file states the encryption used, which tells you which ncw_*_crypt command to run (this stanza is directly above the ObjectServer hostname):

< !-- ! Login information to access the datasource specified in ! .
! - encrypted : If true, the password is encrypted (using nco_g_crypt)
! - algorithm (optional) : [AES|FIPS]
! If set to AES and encrypted attribute is set to true,
! the password is encrypted (using ncw_aes_crypt).
! If set to FIPS and encrypted attribute is set to true,
! the password is encrypted (using ncw_fips_crypt).
!-->

<ncwdatasourcecredentials username="root" password="xxxxxxxxxxxxxxxxxxxxxxx==" encrypted="true" algorithm="FIPS">

</ncwdatasourcecredentials>


To change the password, in this case you would run ncw_fips_crypt and place the encrypted passwd in the three files listed above.

Parallelizing processes with xargs

I came across some useful information today on using the Unix xargs command to speed up processing:

http://www.spinellis.gr/blog/20090304/

The speed increase with the addition of a single flag is pretty amazing.

Integrating Impact with ITM via Web Services

At Pulse 2009 IV Blankenship gave a short session on Integration Impact with ITM via Web Services. I recorded the video and have posted it to youtube.com.

Part 1 - http://www.youtube.com/watch?v=re8CjQ0Irx4

Part 2 - http://www.youtube.com/watch?v=tToFieRylfQ

Contact tony.delgross@gulfsoft.com or iv.blankenship@gulfsoft.com for more information.

ITM 6.2.1: What is new in Agent Deployment

If you have been using the "tacmd createnode" to deploy agents in the past, you are in for a pleasant surprise in ITM 6.2.1!  IBM added few welcome changes to the above command.  These new features not only makes the deployment easier, they enable you keep track of agent deployment like you'd do in sophisticated products like ITCM.   This article discusses the new features in detail.

Background Execution

If you invoke the tacmd createnode with the usual parameters (remote system name, user and password), it will NOT hang until the agent deployed as it was before. Instead, it will give you a transaction ID and return back control. Kinda similar to "winstsp" command in ITCM. Are you already seeing some ITCM like behavior? Wait there is more! To get the status of the deployment, use the new tacmd subcommand, "tacmd getdeploystatus".  It lists all the distributions you have done so far and their status. A sample output is shown below. 

Transaction ID : 12292212812729130001281138021

Command        : INSTALL

Status         : SUCCESS

Retries        : 0

TEMS Name      : VSITM621WIN

Target Hostname: 192.168.162.169

Platform       : WINNT

Product        : NT

Version        : 062100000

Error Message  : KDY2065I: The node creation operation was a success.

Batch Deployment

ITM 6.2.1 "tacmd createnode" supports batch deployment of agents. For example, you can create a deployment group named AppServers, add all appservers to that group and send deployment to them using single tacmd createnode command.  There are new additions to subcommands viz. tacmd creategroup, tacmd addgroupmember and tacmd deletegroupmember to manage groups and its members. 

Workspaces

To keep track of all your agent deployments, ITM 6.2.1 adds new workspaces to TEPS at the Enterprise level.  The workspace provides a summary of all your deployments including number of agents Succeeded, Failed, Progress, etc, similar to MDist2 Console. There is also another workspace called "Deployment Status Summary By Transaction" providing status of deployments by Transaction ID.  A sample Workspace view is shown below.

 

Remote Execution

One other subtle difference documented is that you can execute "tacmd createnode" from anywhere and you don't have to be logged on to TEMS server.   

These feature additions are steps in the right direction for managing agent deployment and I hope you will find the features helpful.  

Adding disk space to a Linux VM in VMWare - Take 2

A little while back Frank posted a blog about increasing the size of a logical volume in Linux (http://blog.gulfsoft.com/2008/11/adding-disk-space-to-linux-vm-in-vmware.html). I tried this out and it worked great.

I also had to do something similar in Windows, which turns out to be much simpler by using the VMWare Converter. Unfortunately this is not supported for Linux. The one thing that got me thinking though is that Windows can increase from a single drive and not have to use a separate drive. So why could I not do this in Linux?

The reason I needed to do this was for the new version of TPM (7.1). I originally had my VM set up with 20GB allocated, but the new version now requires approximately 50Gb to install, so the steps below are what I used to increase the LVM for my TPM install.

Here is my setup:
VM Ware: VMWare Server 2.0.0
Host OS: Windows 2003 SE SP2 x64
Guest OS: Red Hat ES4 U5

Increase VM Disk Size
Use the VMWare tool vmware-vdiskmanager.exe to increase the size

C:\Program Files (x86)\VMware\VMware Server>vmware-vdiskmanager.exe -x 50GB "f:\
VM_Images\VM TPM7.1 RH\rhes4srv01\Red Hat Enterprise Linux 4.vmdk"
Disk expansion completed successfully.


Use fdisk to create new partition
Even though the previous step reports that more disk was added, it still is not recognized by the OS.

Current file system:
df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00 19448516 1964128 16496436 11% /
/dev/sda1 101086 13232 82635 14% /boot
none 257208 0 257208 0% /dev/shm

1. Enter the FDISK utility
fdisk /dev/sda

2. Print the existing partition table
Command (m for help): p

Disk /dev/sda: 53.6 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 2610 20860402+ 8e Linux LVM

3. Create new partition. This will be a Primary partition and is the 3rd partition. For the cylimder values, press enter to accept the defaults

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 3
First cylinder (2611-6527, default 2611):
Using default value 2611
Last cylinder or +size or +sizeM or +sizeK (2611-6527, default 6527):
Using default value 6527

4. Set the Partition type to Linux LVM (hex 8e)
Command (m for help): t
Partition number (1-4): 3
Hex code (type L to list codes): 8e
Changed system type of partition 3 to 8e (Linux LVM)

5. Print the Partition table again to see the new partition
Command (m for help): p

Disk /dev/sda: 53.6 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 2610 20860402+ 8e Linux LVM
/dev/sda3 2611 6527 31463302+ 8e Linux LVM

6. Write the new partition information
Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

7. Reboot

Create the Logical Volume
Now that the partition is created, the physical and logical volume needs to be created

1. Create the Physical Volume - use the new partition information from the df command. In this case the new partition is /dev/sda3
pvcreate /dev/sda3
Physical volume "/dev/sda3" successfully created

2. Add the new Physical Volume to the Logical Volume
vgextend VolGroup00 /dev/sda3
Volume group "VolGroup00" successfully extended

3. Extend the Logical Volume.

vgdisplay
--- Volume group ---
VG Name VolGroup00
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 2
Act PV 2
VG Size 49.88 GB
PE Size 32.00 MB
Total PE 1596
Alloc PE / Size 635 / 19.84 GB
Free PE / Size 961 / 30.03 GB
VG UUID bzOq45-o5yO-ruYY-Ffx1-DxCx-2e2j-ardXtu

lvextend -L +30.03G /dev/VolGroup00/LogVol00
Rounding up size to full physical extent 30.03 GB
Extending logical volume LogVol00 to 48.88 GB
Logical volume LogVol00 successfully resized

Note: The lvextend can either use the value of the Free PE from the vgdisplay command or the command lvextend -l +100%FREE /dev/VolGroup00/LogVol00. I found out about the %FREE after, so I did not test this.

4. Extend the filesystem
The resize2fs does not work for this situation. The command ext2online will allow for the disk to be resized while disk is still mounted.

ext2online /dev/VolGroup00/LogVol00

5. View the new file system
df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00 50444996 1972848 45911716 5% /
/dev/sda1 101086 13232 82635 14% /boot
none 1825828 0 1825828 0% /dev/shm

And there is a system with a nice large drive. Hope this helps you.