Setup MongoDB 3.2 Geographical Replica Set in Ubuntu 15.10

Interestingly, i needed to setup a Replica Set on Ubuntu 15.10 for MongoDB 3.2 which is the latest Ubuntu and MongoDB version. This also serve as a tutorial for anyone who is interested in setting up a MongoDB Replica Set as well.

Import the public key used by the package management system.

Login to your server as root, we will need to import the public key use by the package manager for mongodb, just fire the following command,

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10

And we are good here.

Create a Source list file for MongoDB and Installation

Next, we need to add the source list for MongoDB. However, since MongoDB did not support 15.10 at this time, we will use debian ones

echo "deb http://repo.mongodb.org/apt/debian wheezy/mongodb-org/3.0 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list

Now, we will need to update the server and install mongodb

sudo apt-get update
sudo apt-get install -y mongodb-org

And after everything finished running, you should have your mongodb running.

sudo service mongod start

if no error is given, meaning your MongoDB has successfully installed.

Setup Replica Set

Now, assuming you did the above on 3 machines, you will need to setup each replica with the following steps,

head over to /etc/mongod.conf and replace your config with the one show below,

# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

storage:
    dbPath: "/data/mongo"
    directoryPerDB: false
    journal:
        enabled: true
    engine: "wiredTiger"
    wiredTiger:
        engineConfig:
            cacheSizeGB: 1
        collectionConfig:
            blockCompressor: snappy
systemLog:
    destination: file
    path: "/var/log/mongodb.log"
    logAppend: true
    logRotate: reopen
    timeStampFormat: iso8601-utc
net:
  port: 27017
  bindIp: 0.0.0.0

replication:
   oplogSizeMB: 500
   replSetName: dstTest

Next, create the folder for MongoDB data,

mkdir -p /data/mongo
chown -R mongodb:mongodb /data

Once you have done that, restart MongoDB and make sure there is no error.

sudo service mongod restart

Next we need to setup each replica in MongoDB.

Configure the servers to include in replica set

Assuming you have 3 machines, with the following hostname

sg.db.hungred.com
us.db.hungred.com
tw.db.hungred.com

Now, head over to the primary MongoDB server that you would like it to be primary (in my case, us.db.hungred.com) and enter to mongodb using the command below,

mongo
rs.initiate

then paste the following

rs.reconfig({ _id : "testDB", members : [ {_id : 0, host : "sg.db.hungred.com:27017", priority: 5}, {_id : 1, host : "us.db.hungred.com:27017", priority: 4}, {_id : 2, host : "tw.db.hungred.com:27017", priority: 3 } ] })

take note of the priority i have given it and make sure this is one liner (yeah its messy but that's how i copy and paste it in one piece), then check your conf

rs.conf()

and status at

rs.status()

and you got yourself a 3 location replica set of MongoDB!

***** UPDATE *****

Adding Security Authentication

If you want to add authentication into your setup, you will need to visit /etc/mongod.conf and add the following

security:
  keyFile: /data/mongodb-keyfile

on all of your primary and secondary Mongodb server. The file will need to generate this way,

openssl rand -base64 741 > /data/mongodb-keyfile
chmod 600 mongodb-keyfile

This is to ensure all replica set can communicate with each other. Once you have generated the file above on the primary MongoDB server, copy the same file to other secondary server and update the /etc/mongod.conf on each secondary server along with it.

Share

How To Install Aide Intrusion Detection System on Ubuntu

Aide stands for Advanced Intrusion Detection Environment which is one of the most popular tools for monitoring changes to a Unix or Linux system. Here i will list out how i am going to set this baby up on some of my server to secure on system.

Updating and Installing Aide

sudo apt-get update -y

Once you have update your repo, simply install Aide using the following command

sudo apt-get install aide

And aide is installed in your machine!

Configuring and Test out Aide

Next we are going to configure this baby. Initial the database with the command below,

sudo aideinit

It will take a while and once you have initial the database, Verify that the new aide database has been created

cd /var/lib/aide
ls -lt

And you should see something like this

AIDE 0.16a2-19-g16ed855 initialized AIDE database at /var/lib/aide/aide.db.new
Start timestamp: 2016-05-12 10:17:20 -0400
Verbose level: 6

Number of entries:	66800

---------------------------------------------------
The attributes of the (uncompressed) database(s):
---------------------------------------------------

/var/lib/aide/aide.db.new
  RMD160   : BOdplDoXDH0ws73WkoYe11+WIhM=
  TIGER    : tJ8xmXCDo9N9e8cJZBuqQSW/yl/ArSnJ
  SHA256   : E+Pc3ae0PDDxfRV9PcZZ8Fq+NsJZBLbo
             SQQ+i6xQ2I0=
  SHA512   : WHHce2bdDPzP1NgMSr9afReWcIvGbW+p
             D09ShUO3kT6EJpFWhqTR0RI60LmYW/sR
             76QTqqOOnIK+Cknc8mKXRA==
  CRC32    : OqKLPA==
  HAVAL    : zT+SY0Ee5SuFaXb7Kjo3gU7NpnH+QIyA
             buxyjH8AedM=
  GOST     : 4cW9q/3KpRawsNsRc2HtdjGgF70fsaI5
             8eRaLnsDlmo=


End timestamp: 2016-05-12 10:24:58 -0400 (run time: 7m 38s)

Move the new file to the new database using the following command,

mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db

Now, let's test this baby out with the following command,

aide.wrapper --check

and you will get something like this

[email protected]:~# aide.wrapper --check
AIDE 0.16a2-19-g16ed855 found differences between database and filesystem!!
Start timestamp: 2016-05-12 10:29:51 -0400
Verbose level: 6

Summary:
  Total number of entries:	66801
  Added entries:		1
  Removed entries:		1
  Changed entries:		3

---------------------------------------------------
Added entries:
---------------------------------------------------

f++++++++++++++++: /var/lib/aide/aide.db

---------------------------------------------------
Removed entries:
---------------------------------------------------

f----------------: /var/lib/aide/aide.db.new

---------------------------------------------------
Changed entries:
---------------------------------------------------

d =.... mc.. .. .: /var/lib/mongodb/diagnostic.data
f >b... mc..C.. .: /var/lib/mongodb/diagnostic.data/metrics.2016-05-12T08-52-09Z-00000
f >.... mci.C.. .: /var/lib/mongodb/diagnostic.data/metrics.interim

---------------------------------------------------
Detailed information about changes:
---------------------------------------------------

Directory: /var/lib/mongodb/diagnostic.data
  Mtime    : 2016-05-12 10:24:39 -0400        | 2016-05-12 10:36:39 -0400
  Ctime    : 2016-05-12 10:24:39 -0400        | 2016-05-12 10:36:39 -0400

File: /var/lib/mongodb/diagnostic.data/metrics.2016-05-12T08-52-09Z-00000
  Size     : 361980                           | 372957
  Bcount   : 720                              | 744
  Mtime    : 2016-05-12 10:22:08 -0400        | 2016-05-12 10:32:08 -0400
  Ctime    : 2016-05-12 10:22:08 -0400        | 2016-05-12 10:32:08 -0400
  RMD160   : czpo/fk+iRIEKUBjlc2+wELg/Wo=     | wEQV9cj/KyiGQmfGSLbzo9B44Gs=
  TIGER    : 2wLpFPWq3lxfxXyHpAMkVXUjDtZ08W8z | x8IbKbindr6NVwNbaUt0J5jWq9Y1cWmv
  SHA256   : lVRtuDTLDD7DYajbBEYoMSPpdrtxdJNA | 3J4B2ToLfGmBbHOQas/hKGj8HXe4zihW
             rxL5xH8A0kA=                     | 0OLKtXC4fqo=
  SHA512   : axlztAMc56xIGz7JnsOq8dAgZfCLmT83 | 49Fex6rPE24SnoOaLc+T/hIiTLEEyOmk
             gFZS6MB2zmT5aPxK4FmOSnEC9W5mtUNJ | YGeLF1W/fxZuRYk3FuwgpFlKA2qrmi2f
             AIaoa5bK736BAXwMcsA+NA==         | xNij3UG21mAiX+Tx2pRw+A==
  CRC32    : drkWXw==                         | rtCgKQ==
  HAVAL    : SR2yfai80zpN2Xw+8sUFSM/kTQBGAHsl | xIk6ByhAZN5C2eU2bTJzZ0oZcJeqsIiz
             71FSIVFT4qA=                     | AMbC0DPcNhg=
  GOST     : bE/NiblzIQRPzFx8jVymvvkEA+NO6on0 | txFhbK566EUxlQk6c36TfqgvYBttntcm
             k3XlP3vO2LA=                     | qyMIxjG3zK8=

File: /var/lib/mongodb/diagnostic.data/metrics.interim
  Size     : 5279                             | 5397
  Mtime    : 2016-05-12 10:24:39 -0400        | 2016-05-12 10:36:39 -0400
  Ctime    : 2016-05-12 10:24:39 -0400        | 2016-05-12 10:36:39 -0400
  Inode    : 1180042                          | 1179903
  RMD160   : Uch+G7OlOobiM/VjjdNHYSdCZUY=     | OnSReGX+lqQuCQURBBxkfHC9U5o=
  TIGER    : bB0QmZYYNl2SKSfz4MlNrpwYKwCS3Evf | ktNDR+97gTAK7catLGoOhEFJu6IfQZwi
  SHA256   : h0s1leYNb7/RxTi86z+nHhe7DChFJtSo | KIlG5ePVgwG/+DopSTPHo6VqnGzdnQMj
             TUZXyOwKKYw=                     | m97NR3Gifhk=
  SHA512   : 8PrN5C6RJgYHIuM7DjL3vjx9/5fRbnsr | QLXQngP8ouoc8bvs580De+Vh7bGR0Lq8
             MDpk+PcTAxLV3AUbkWP9Xq0hTzro7mlM | +2tXCfVed02e1DVRgxeG3LbKxqhofP76
             nT96+O95DnPZRmuD5OAPZA==         | 6Mz99D/w7u9eabdbsYmmOw==
  CRC32    : sTX43A==                         | Ta6Udw==
  HAVAL    : ZDpLBirCqbUqz/jym+FFjv2IvY9T4k+g | qTpVXVypYnzMGQZF4SMw7Wjg/jKkptpw
             hhcWR0kK/ZE=                     | PEqS+lI8g84=
  GOST     : 7yJZnGdeAM8slovcFTD0Ftcec5KT8weQ | gVW46Bk3upRekyxDI5sPP6N1xk7b6gX5
             yPYlQqSMkf4=                     | CJTybT2VVKQ=


---------------------------------------------------
The attributes of the (uncompressed) database(s):
---------------------------------------------------

/var/lib/aide/aide.db
  RMD160   : BOdplDoXDH0ws73WkoYe11+WIhM=
  TIGER    : tJ8xmXCDo9N9e8cJZBuqQSW/yl/ArSnJ
  SHA256   : E+Pc3ae0PDDxfRV9PcZZ8Fq+NsJZBLbo
             SQQ+i6xQ2I0=
  SHA512   : WHHce2bdDPzP1NgMSr9afReWcIvGbW+p
             D09ShUO3kT6EJpFWhqTR0RI60LmYW/sR
             76QTqqOOnIK+Cknc8mKXRA==
  CRC32    : OqKLPA==
  HAVAL    : zT+SY0Ee5SuFaXb7Kjo3gU7NpnH+QIyA
             buxyjH8AedM=
  GOST     : 4cW9q/3KpRawsNsRc2HtdjGgF70fsaI5
             8eRaLnsDlmo=


End timestamp: 2016-05-12 10:37:13 -0400 (run time: 7m 22s)

see the file that we just added and updated? Yeah, that's the one that its reporting.

Crontab Aide

Now we dont want to do this every single day manually, so let's setup a crontab.

vi aide.sh

with the following code

#! /bin/sh
MYDATE=`date +%Y-%m-%d`
MYFILENAME="Aide-"$MYDATE.txt
/bin/echo "Aide check !! `date`" > /tmp/$MYFILENAME
/usr/bin/aide.wrapper --check > /tmp/myAide.txt
/bin/cat /tmp/myAide.txt|/bin/grep -v failed >> /tmp/$MYFILENAME
/bin/echo "**************************************" >> /tmp/$MYFILENAME
/usr/bin/tail -20 /tmp/myAide.txt >> /tmp/$MYFILENAME
/bin/echo "****************DONE******************" >> /tmp/$MYFILENAME
/usr/bin/mail -s"$MYFILENAME `date`" [email protected] < /tmp/$MYFILENAME

now make it executable

chmod +x aide.sh

open up crontab

crontab -e

add the following crontab into it

06 01 * * 0-6 /root/aide.sh

And we are good to go! Simple as that!

Share

Installing avconv from source in Ubuntu

This is pretty interesting as avconv is a pretty sensitive fellow. If you do this wrongly, your avconv is pretty much not workable no matter how you are going to spin this. Hence, i do myself a favour and write this up instead. Now, before i began, let me explain what version of Ubuntu i am using. I am on Ubuntu 12.04.5 LTS and if i do the following from repo

sudo apt-get install ffmpeg libav-tools

I will be getting myself avcon version 8.x.x since i am on a older version of Ubuntu. Hence, the installed of avcon from source.

Removing older avconv or ffmpeg

Before we do anything at all, we need to remove the older version of avcon.

apt-get remove --purge libav-tools

and this will remove the built in avconv that you might have installed

Installing prerequisite files

Now, fire the below command to install everything you may need for your avconv compilation

apt-get install build-essential
apt-get install yasm
apt-get install libfdk-aac-dev
apt-get install libx264-dev

Once you have done that, we are good to go!

Installing avconv from source

I am going to install avconv version 11 from libav.org.

wget https://libav.org/releases/libav-11.6.tar.gz  --no-check-certificate
tar zxvf libav-11.6.tar.gz
cd libav-11.6/
./configure --enable-libvo-aacenc --enable-gpl --enable-nonfree --enable-version3 --enable-libx264 --enable-static --disable-shared
make && make install

After you have finished compilation (which takes some time), you are pretty much installed with the newer version of avconv! Not that tough right?

Share