How-to

Articles On Single HOW-TO Tutorial

Manual Restore Bacula Without Database

OK, another problem i have. I though my data was gone for good although i do remember my Bacula was doing all the backup! And i finally found a way to get those 1TB files back! Well, as much as you don't know anything about Bacula, you do know where those files are stored right? These files are called 'Volume'. And we will be using these volumes to restore our backup! We will be using bacula volume utility tools to assist us in extracting these precious data!

What's in the Bacula Volume?

Before you can do anything at all, the first thing you need to do is to scan your volume to see whether your stuff is located in the volume!

bls -j -V volume-0177 devicenamehere

and the above will show you something like the one below,


Begin Job Session Record: File:blk=0:8814 SessId=161 SessTime=1480534092 JobId=481
   Job=job.name.com.2017-01-20_01.00.00_33 Date=25-Jan-2017 21:26:12 Level=I Type=B
End Job Session Record: File:blk=0:8814 SessId=161 SessTime=1480534092 JobId=481
   Date=25-Jan-2017 22:53:20 Level=I Type=B Files=2 Bytes=942 Errors=0 Status=T

And what's important on the above are SessId and SessTime. So that we can create a Bootstrap file! Create a file call bootstrap.bsr as show below,

Volume = volume-0177
VolSessionId = 161
VolSessionTime = 1480534092

Now, with this information, we will be able to extract the information out of Bacula Volume!

Extracting Bacula Volume?

In order to extract from Bacula volume, there are a few ways to do it. You can either use your bootstrap file as created above and fire the below command

bextract -p -b ./bootstrap.bsr devicename /home

or you can specific which volume you want to extract without using a bootstrap file as show below,

bextract -p -V volume-0177 devicename/home

and file will starts extracting to /home directory where volume-0177 is the file name and devicename is the actual device name you found on /etc/bacula/bacula-sd file that you wish to restore.

The following shows you some options you can add to your command,

Usage: bextract [-d debug_level] <device-name> <directory-to-store-files>
       -b <file>       specify a bootstrap file
       -dnn            set debug level to nn
       -e <file>       exclude list
       -i <file>       include list
       -p              proceed inspite of I/O errors
       -V              specify Volume names (separated by |)
       -?              print this message
  • -p is useful if your backup is like 1TB and it throws off an i/o error after 50 hours of extracting. -p basically prevent that.
  • -i takes in a file path to include only these files or folder to your restoration plan
  • -e takes in a file path to exclude these files or folder out of your restoration plan
  • -V specific a volume as shown on my example
  • -b takes in a file path which is a bootstrap file to tell bextract what you want to do

Now, go save your own ass from getting whoop! Peace out!

Schedule Rsync Backup From Windows to Linux Server

Windows, WHY ARE YOU ALWAYS SO DIFFICULT! Gosh. Damn you are. This time. i wanted to do schedule a backup from my windows server 2012r2 to my linux backup drive. Its as simple as that (while i though it was at least). Google doesn't help with so many rubbish online. Hence, here is a guide that will help us out (me included)

Environment

Enterprise server (Windows 2012 R2)

This is a windows environment server 2012 R2 where our data is

Backup server (Debian Linux)

This is my backup server where i would like to rsync over.

 

Installation

On Windows server 2012 r2

  • Download cwRsync
  • Unzup cwRsync and copy to "C:\cwRsync".
  • Add "C:\cwRsync\bin" to PATH.
  • Create the directory "C:\cwRsync\home" and "C:\cwRsync\home\USER" (USER should be the name of the user who will run the Rsync in my case its "admin").
  • Create public/private keys with the following command:
  • ssh-keygen -t rsa
    • Paths with "/home/USER/" correspond to the directories that we created in "C:\cwRsync\".
    • Leave the password blank.

On Linux

  • Install openssh-server and rsync.
  • Provide data to a partition (eg.: /backup/).
  • Place the public key in /home/USER/.ssh/ and rename the file to authorized_keys. (assuming its root)

On Windows

  • Test the connection without a password with the following command:
ssh USER@BackupServerIP
  • Test Rsync:
rsync -v -rlt -z --delete "/myfiles/" "USER@BackupServerIP:/backups/"
  • where cygdrive is the directory on C:\cygdrive so the above  C:\cygdrive\myfiles
  • To Test Other port
rsync  -e "ssh -p 14000" -arv "--exclude=.svn/" /myfiles USER@BackupServerIP:/backups/
  • Create a bat file with the rsync command and place it in C:\cwRsync\bin.
  • Schedule execution every day at 0:30 (half past midnight).

Helpful Resources

  • http://stackoverflow.com/questions/34147565/rsync-uid-gid-impossible-to-set-cases-cause-future-hard-link-failure-how-to
  • http://www.smellems.com/tiki-read_article.php?articleId=14

PID:4 using Port 80 In Windows Server 2012 R2

i will cut the chase, if you are suspecting something is using Port 80 and is trying to find out what is it, here are some suggestion. Try stopping the following services

  • IIS
  • World Wide Web Publishing service
  • IIS Admin Service
  • SQL Server Reporting services
  • Web Deployment Agent Service

And if the  NT Kernel was still listening on port 80, you just hit the jackpot with me. Its BranchCode. Try removing it under "Remove Roles" in "Server Management" as show below,

Once you remove that, restart your server and port 80 should be all yours. Verify using the following command,


netstat -nao | find ":80"

and it should show you this.

Good LUCK!

Error response from daemon: Container xxxxxx is already active

If you face this issue where it said "Error response from daemon: Container xxxxxx is already active", and your Container is unable to start, the only way it seems to work for me, is to do the following

sudo service docker restart

and the so call 'active' container will starts itself, using other command fails to start this baby up.

Backlog limit exceeded error and freeze in CentOS 6

"Backlog limit exceeded error", basically what happen is that your OS audit folder is getting flooded with audit events and is unable to write to /var/log/audit directory as the write are too damn fast. It cause the whole system to freeze and you won't be able to login either. Here are a few solutions,

Disable Audit Log

The easiest way is to disable audit log altogether. This will prevent the problem altogether but leaving you with empty audit log to figure out what is going on in your system

/etc/init.d/auditd stop
chkconfig auditd off

Usually we want the log hence this might not work for everyone.

Increase the buffer size

By increasing the buffer size, we can prevent the system from crashing 

 auditctl -b 8192

of you can head over to /etc/audit/audit.rules and change the value permanently

# This file contains the auditctl rules that are loaded
# whenever the audit daemon is started via the initscripts.
# The rules are simply the parameters that would be passed
# to auditctl.

# First rule - delete all
-D

# Increase the buffers to survive stress events.
# Make this bigger for busy systems
-b 8192

# Feel free to add below this line. See auditctl man page

or change the priority at /etc/audit/auditd.conf

#
# This file controls the configuration of the audit daemon
#

log_file = /var/log/audit/audit.log
log_format = RAW
log_group = root
priority_boost = 6

But then, there are times when it grows so big, you want to look into the source of the problem. Hence, might not work for everyone too.

Debugging it

Now, the last solution i have is to look for the issue.

aureport --start today --event --summary -i

which will show us what happen today that crashed my server

-bash-4.1# aureport --start today --event --summary -i

Event Summary Report
======================
total  type
======================
9474  AVC
1196  USER_ACCT
1196  CRED_ACQ
1195  USER_START
1194  LOGIN
1194  CRED_DISP
1194  USER_END
83  NETFILTER_CFG
5  CRYPTO_KEY_USER
3  USER_AUTH
2  CRYPTO_SESSION
1  USER_LOGIN
1  DAEMON_START
1  CONFIG_CHANGE

Now notice this AVC. It has a total of 9474! Gosh, what the heck is that.

-bash-4.1# aureport --start today

Summary Report
======================
Range of time in logs: 11/07/2016 00:00:01.323 - 11/07/2016 10:49:26.950
Selected time for report: 11/07/2016 00:00:00 - 11/07/2016 10:49:26.950
Number of changes in configuration: 86
Number of changes to accounts, groups, or roles: 0
Number of logins: 1
Number of failed logins: 0
Number of authentications: 3
Number of failed authentications: 0
Number of users: 5
Number of terminals: 7
Number of host names: 2
Number of executables: 27
Number of commands: 26
Number of files: 1705
Number of AVC's: 9746
Number of MAC events: 0
Number of failed syscalls: 1263
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 7
Number of integrity events: 0
Number of virt events: 0
Number of keys: 0
Number of process IDs: 4475
Number of events: 17107

Google a bit gives me this

For SELinux there are two main types of audit event:

AVC Audit Events - These are generated by the AVC subsystem as a result of access denials, or where specific events have requested an audit message (i.e. where an auditallow rule has been used in the policy).

SELinux-aware Application Events - These are generated by the SELinux kernel services and SELinux-aware applications for events such as system errors, initialisation, policy load, changing boolean states, setting of enforcing / permissive mode, relabeling etc.

Ah! My SELinux is enable! More Info on AVC (https://selinuxproject.org/page/NB_AL) Looking into /etc/selinux/config


# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=permissive
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

Now disabling it won't flood my audit log! Since i don't need SELINUX on my server.