Search This Blog

Wednesday 6 April 2016

How to Increase the size of a Linux LVM by expanding the virtual machine disk

Identifying the partition type

As this method focuses on working with LVM, we will first confirm that our partition type is actually Linux LVM by running the below command.

fdisk -l

fdisk

As you can see in the above image /dev/sda5 is listed as "Linux LVM" and it has the ID of 8e. The 8e hex code shows that it is a Linux LVM, while 83 shows a Linux native partition. Now that we have confirmed we are working with an LVM we can continue. For increasing the size of a Linux native partition (hex code 83) see this article.

Below is the disk information showing that our initial setup only has the one 20gb disk currently, which is under the logical volume named /dev/mapper/Mega-root – this is what we will be expanding with the new disk.
disk free

Note that /dev/mapper/Mega-root is the volume made up from /dev/sda5 currently – this is what we will be expanding.

Increasing the virtual hard disk

First off we increase the allocated disk space on the virtual machine itself. This is done by right clicking the virtual machine in vSphere, selecting edit settings, and then selecting the hard disk. In the below image I have changed the previously set hard disk of 20gb to 30gb while the virtual machine is up and running. Once complete click OK, this is all that needs to be done in VMware for this process.

vSphere settings

If you are not able to modify the size of the disk, the provisioned size setting is greyed out. This can happen if the virtual machine has a snapshot in place, these will need to be removed prior to making the changes to the disk. Alternatively you may need to shut down the virtual machine if it does not allow you to add or increase disks on the fly, if this is the case make the change then power it back on.

Detect the new disk space

Once the physical disk has been increased at the hardware level, we need to get into the operating system and create a new partition that makes use of this space to proceed.

Before we can do this we need to check that the new unallocated disk space is detected by the server, you can use "fdisk -l" to list the primary disk. You will most likely see that the disk space is still showing as the same original size, at this point you can either reboot the server and it will detect the changes on boot or you can rescan your devices to avoid rebooting by running the below command. Note you may need to change host0 depending on your setup.

echo "- - -" > /sys/class/scsi_host/host0/scan

Below is an image after performing this and confirming that the new space is displaying.

fdisk

Partition the new disk space

As outlined in my previous images the disk in my example that I am working with is /dev/sda, so we use fdisk to create a new primary partition to make use of the new expanded disk space. Note that we do not have 4 primary partitions already in place, making this method possible.

fdisk /dev/sda

We are now using fdisk to create a new partition, the inputs I have entered in are shown below in bold. Note that you can press 'm' to get a full listing of the fdisk commands.

'n' was selected for adding a new partition.

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to           switch off the mode (command 'c') and change display units to           sectors (command 'u').    Command (m for help): n

'p' is then selected as we are making a primary partition.

Command action     l   logical (5 or over)     p   primary partition (1-4)  p

As I already have /dev/sda1 and /dev/sda2 as shown in previous images, I have gone with using '3' for this new partition which will be created as /dev/sda3

Partition number (1-4): 3

We just press enter twice above as by default the first and last cylinders of the unallocated space should be correct. After this the partition is then ready.

First cylinder (2611-3916, default 2611): "enter"  Using default value 2611  Last cylinder, +cylinders or +size{K,M,G} (2611-3916, default 3916): "enter"  Using default value 3916

't' is selected to change to a partition's system ID, in this case we change to '3' which is the one we just created.

Command (m for help): t  Partition number (1-5): 3

The hex code '8e' was entered as this is the code for a Linux LVM which is what we want this partition to be, as we will be joining it with the original /dev/sda5 Linux LVM.

Hex code (type L to list codes): 8e  Changed system type of partition 3 to 8e (Linux LVM)

'w' is used to write the table to disk and exit, basically all the changes that have been done will be saved and then you will be exited from fdisk.

Command (m for help): w  The partition table has been altered!    Calling ioctl() to re-read partition table.    WARNING: Re-reading the partition table failed with error 16: Device or resource busy.  The kernel still uses the old table. The new table will be used at  the next reboot or after you run partprobe(8) or kpartx(8)  Syncing disks.

You will see a warning which basically means in order to use the new table with the changes a system reboot is required. If you can not see the new partition using "fdisk -l" you may be able to run "partprobe -s" to rescan the partitions. In my test I did not require either of those things at this stage (I do a reboot later on), straight after pressing 'w' in fdisk I was able to see the new /dev/sda3 partition of my 10gb of space as displayed in the below image.

For CentOS/RHEL run a "partx -a /dev/sda3" to avoid rebooting later on.

fdisk

That's all for partitioning, we now have a new partition which is making use of the previously unallocated disk space from the increase in VMware.

Increasing the logical volume

We use the pvcreate command which creates a physical volume for later use by the logical volume manager (LVM). In this case the physical volume will be our new /dev/sda3 partition.

root@Mega:~# pvcreate /dev/sda3    Device /dev/sda3 not found (or ignored by filtering).

In order to get around this you can either reboot, or use partprobe/partx as previously mentioned to avoid a reboot, as in this instance the disk does not appear to be there correctly despite showing in "fdisk -l". After a reboot or partprobe/partx use the same command which will succeed.

root@Mega:~# pvcreate /dev/sda3    Physical volume "/dev/sda3" successfully created

Next we need to confirm the name of the current volume group using the vgdisplay command. The name will vary depending on your setup, for me it is the name of my test server. vgdisplay provides lots of information on the volume group, I have only shown the name and the current size of it for this example.

root@Mega:~# vgdisplay    --- Volume group ---    VG Name               Mega  ...  VG Size               19.76 GiB

Now we extend the 'Mega' volume group by adding in the physical volume of /dev/sda3 which we created using the pvcreate command earlier.

root@Mega:~# vgextend Mega /dev/sda3    Volume group "Mega" successfully extended

Using the pvscan command we scan all disks for physical volumes, this should confirm the original /dev/sda5 partition and the newly created physical volume /dev/sda3

root@Mega:~# pvscan    PV /dev/sda5   VG Mega   lvm2 [19.76 GiB / 0    free]    PV /dev/sda3   VG Mega   lvm2 [10.00 GiB / 10.00 GiB free]    Total: 2 [29.75 GiB] / in use: 2 [29.75 GiB] / in no VG: 0 [0   ]

Next we need to increase the logical volume (rather than the physical volume) which basically means we will be taking our original logical volume and extending it over our new partition/physical volume of /dev/sda3.

Firstly confirm the path of the logical volume using lvdisplay. This path name will vary depending on your setup.

root@Mega:~# lvdisplay    --- Logical volume ---    LV Path                /dev/Mega/root

The logical volume is then extended using the lvextend command.

root@Mega:~# lvextend /dev/Mega/root /dev/sda3    Extending logical volume root to 28.90 GiB    Logical volume root successfully resized

There is then one final step which is to resize the file system so that it can take advantage of this additional space, this is done using the resize2fs command for ext based file systems. Note that this may take some time to complete, it took about 30 seconds for my additional space.

root@Mega:~# resize2fs /dev/Mega/root  resize2fs 1.41.12 (17-May-2010)  Filesystem at /dev/Mega/root is mounted on /; on-line resizing required  old desc_blocks = 2, new_desc_blocks = 2  Performing an on-line resize of /dev/Mega/root to 7576576 (4k) blocks.  The filesystem on /dev/Mega/root is now 7576576 blocks long.

Alternatively if you're running the XFS file system (default as of RedHat/CentOS 7) you can grow the file system with "xfs_growfs /dev/Mega/root".

That's it, now with the 'df' command we can see that the total available disk space has been increased.

disk free after expansion

Summary

With this method we have increased the virtual disk drive through VMware, created a new partition out of this newly unallocated space within the guest OS, turned it into a physical volume, extended the volume group, and then finally extended the original logical volume over the newer physical volume resulting in overall disk space being increased successfully.

Tuesday 5 April 2016

DB2 Backup and Restore

Introduction

Backup and recovery methods are designed to keep our information safe. In Command Line Interface (CLI) or Graphical User Interface (GUI) using backup and recovery utilities you can take backup or restore the data of databases in DB2 UDB.

Logging

Log files consist of error logs, which are used to recover from application errors. The logs keep the record of changes in the database. There are two types of logging as described below:

Circular logging

It is a method where the old transaction logs are overwritten when there is a need to allocate a new transaction log file, thus erasing the sequences of log files and reusing them. You are permitted to take only full back-up in offline mode. i.e., the database must be offline to take the full backup.

Archive logging

This mode supports for Online Backup and database recovery using log files called roll forward recovery. The mode of backup can be changed from circular to archive by setting logretain or userexit to ON. For archive logging, backup setting database require a directory that is writable for DB2 process.

Backup

Using Backup command you can take copy of entire database. This backup copy includes database system files, data files, log files, control information and so on.

You can take backup while working offline as well as online.

Offline backup

Syntax: [To list the active applications/databases]

db2 list application  

Output:

Auth Id  Application    Appl.      Application Id                                                  DB       # of              Name           Handle                Name    Agents    -------- -------------- ---------- ---------------------  ----------------------------------------- -------- -----    DB2INST1 db2bp          39           *LOCAL.db2inst1.140722043938                                     ONE      1  

Syntax: [To force application using app. Handled id]

db2 "force application (39)"   

Output:

DB20000I  The FORCE APPLICATION command completed   successfully.      DB21024I  This command is asynchronous and may not   be effective immediately. 

Syntax: [To terminate Database Connection]

db2 terminate  

Syntax: [To deactivate Database]

db2 deactivate database one   

Syntax: [To take the backup file]

db2 backup database <db_name> to <location>   

Example:

db2 backup database one to /home/db2inst1/ 

Output:

Backup successful. The timestamp for this backup image is :   20140722105345  

Online backup

To start, you need to change the mode from Circular logging to Archive Logging.

Syntax: [To check if the database is using circular or archive logging]

db2 get db cfg for one | grep LOGARCH   

Output:

First log archive method (LOGARCHMETH1) = OFF     Archive compression for logarchmeth1  (LOGARCHCOMPR1) = OFF    Options for logarchmeth1              (LOGARCHOPT1) =      Second log archive method             (LOGARCHMETH2) = OFF     Archive compression for logarchmeth2  (LOGARCHCOMPR2) = OFF     Options for logarchmeth2              (LOGARCHOPT2) =   

In the above output, the highlighted values are [logarchmeth1 and logarchmeth2] in off mode, which implies that the current database in "CIRCULLAR LOGGING" mode. If you need to work with 'ARCHIVE LOGGING' mode, you need to change or add path in the variables logarchmeth1 and logarchmeth2 present in the configuration file.

Updating logarchmeth1 with required archive directory

Syntax: [To make directories]

mkdir backup   mkdir backup/ArchiveDest    

Syntax: [To provide user permissions for folder]

chown db2inst1:db2iadm1 backup/ArchiveDest 

Syntax: [To update configuration LOGARCHMETH1]

db2 update database configuration for one using LOGARCHMETH1   'DISK:/home/db2inst1/backup/ArchiveDest'

You can take offline backup for safety, activate the database and connect to it.

Syntax: [To take online backup]

db2 backup database one online to   /home/db2inst1/onlinebackup/ compress include logs   

Output:

db2 backup database one online to   /home/db2inst1/onlinebackup/ compress include logs    

Verify Backup file using following command:

Syntax:

db2ckbkp <location/backup file>   

Example:

db2ckbkp   /home/db2inst1/ONE.0.db2inst1.DBPART000.20140722112743.001 

Listing the history of backup files

Syntax:

db2 list history backup all for one    

Output:

                    List History File for one       Number of matching file entries = 4      Op Obj Timestamp+Sequence Type Dev Earliest Log Current Log    Backup ID     -- --- ------------------ ---- --- ------------ ------------    --------------    B  D  20140722105345001   F    D  S0000000.LOG S0000000.LOG      ------------------------------------------------------------    ----------------         Contains 4 tablespace(s):    00001 SYSCATSPACE        00002 USERSPACE1      00003 SYSTOOLSPACE       00004 TS1     ------------------------------------------------------------     ----------------     Comment: DB2 BACKUP ONE OFFLINE         Start Time: 20140722105345          End Time: 20140722105347            Status: A   ------------------------------------------------------------    ----------------    EID: 3 Location: /home/db2inst1         Op Obj Timestamp+Sequence Type Dev Earliest Log Current Log     Backup ID   -- --- ------------------ ---- --- ------------ ------------    --------------      B  D  20140722112239000   N       S0000000.LOG S0000000.LOG      ------------------------------------------------------------    -------------------------------------------------------------    -------------------------------       Comment: DB2 BACKUP ONE ONLINE        Start Time: 20140722112239         End Time: 20140722112240              Status: A    ------------------------------------------------------------    ----------------      EID: 4 Location:   SQLCA Information       sqlcaid : SQLCA     sqlcabc: 136   sqlcode: -2413   sqlerrml: 0       sqlerrmc:      sqlerrp : sqlubIni     sqlerrd : (1) 0                (2) 0                (3) 0                 (4) 0                (5) 0                (6) 0    		      sqlwarn : (1)      (2)      (3)      (4)        (5)       (6)                  (7)      (8)      (9)      (10)       (11)     sqlstate:       Op Obj Timestamp+Sequence Type Dev Earliest Log Current Log     Backup ID    -- --- ------------------ ---- --- ------------ ------------     --------------      B  D  20140722112743001   F    D  S0000000.LOG S0000000.LOG         ------------------------------------------------------------    ----------------    Contains 4 tablespace(s):       00001 SYSCATSPACE       00002 USERSPACE1       00003 SYSTOOLSPACE       00004 TS1    -------------------------------------------------------------     ----------------     Comment: DB2 BACKUP ONE OFFLINE        Start Time: 20140722112743         End Time: 20140722112743             Status: A    -------------------------------------------------------------     ----------------    EID: 5 Location: /home/db2inst1       Op Obj Timestamp+Sequence Type Dev Earliest Log Current Log    Backup ID       -------------------------------------------------------------     ----------------      R  D  20140722114519001   F                                  20140722112743      ------------------------------------------------------------    ----------------     Contains 4 tablespace(s):        00001 SYSCATSPACE        00002 USERSPACE1        00003 SYSTOOLSPACE        00004 TS1   ------------------------------------------------------------    ----------------    Comment: RESTORE ONE WITH RF       Start Time: 20140722114519         End Time: 20140722115015         Status: A    	    ------------------------------------------------------------    ----------------      EID: 6 Location:  

Restoring the database from backup

To restore the database from backup file, you need to follow the given syntax:

Syntax:

db2 restore database <db_name> from <location>   taken at <timestamp>    

Example:

db2 restore database one from /home/db2inst1/ taken at   20140722112743  

Output:

SQL2523W  Warning!  Restoring to an existing database that is   different from       the database on the backup image, but have matching names.   The target database       will be overwritten by the backup version.  The Roll-forward   recovery logs    associated with the target database will be deleted.      Do you want to continue ? (y/n) y      DB20000I  The RESTORE DATABASE command completed successfully.   

Roll forward all the logs located in the log directory, including latest changes just before the disk drive failure.

Syntax:

db2 rollforward db <db_name> to end of logs and stop   

Example:

db2 rollforward db one to end of logs and stop  

Output:

                                 Rollforward Status     Input database alias                   = one     Number of members have returned status = 1     Member ID                              = 0     Rollforward status                     = not pending     Next log file to be read               =     Log files processed                    = S0000000.LOG -    S0000001.LOG     Last committed transaction            = 2014-07-22-    06.00.33.000000 UTC    DB20000I  The ROLLFORWARD command completed successfully.

How to Change Tomcat Admin & Manager Default Password

​Creating Tomcat User:-

To create user account edit conf/tomcat-users.xml file in editor and copy below configuration inside <tomcat-users> </tomcat-users> tags.


<tomcat-users>      <!-- user manager can access only manager section -->    <role rolename="manager-gui" />    <user username="manager" password="_SECRET_PASSWORD_" roles="manager-gui" />        <!-- user admin can access manager and admin section both -->    <role rolename="admin-gui" />    <user username="admin" password="_SECRET_PASSWORD_" roles="manager-gui,admin-gui" />      </tomcat-users>

As per above configuration user manager only can access manager web interface but admin can access both admin + manager web interface.

After making above change Stop and Start your Tomcat server.


Access Roles in Tomcat:-

Tomcat 7 and onward releases has following roles defined for accessing Tomcat Admin and Manager interfaces. Use the following roles while creating users for tomcat with specific access levels.

Roles for Admin (Host Manager) Access:

admin-gui – This allows access to the HTML GUI
admin-script – This allows access to the text interface

Roles for Manager (Manager App) Access:

manager-gui – This allows HTML GUI and the status pages
manager-script – This allows text interface and the status pages
manager-jmx – This allows JMX proxy and the status pages
manager-status – This allows access to the status pages only