Setup HA + Floating IP + BGP on Vultr on Ubuntu 17.10

ok, recently i am trying to setup a low cost HA on the cloud. And i decide to go with Vultr since they provide $2.5 machine for me to test and play around (although i can only spam 2 of such machine) Here i will drop down how i did it so i won't forget and you can copy it if you want.

Initial Cloud VPS Setup

ok here, you will need at least 3 machines, i'll call them the following with the spec i've used to create this setup

  • Ubuntu 17.10 x64 1024 MB Server - 192.168.23.10 [Worker1]
  • Ubuntu 17.10 x64 512 MB Server - 192.168.23.11 [Worker2]
  • Ubuntu 17.10 x64 512 MB Server - 192.168.23.11 [Worker3]
  • Floating IP - 192.168.10.10

i name them worker1, worker2 and worker3 as show above. Now, the reason why i choose Ubuntu 17 instead of CoreOS is purely because CoreOS is stuck at Docker 1.12 the time i am writing and you need to install its Alpha version to get Docker 17.09-ce

Note: You should not attach the floating IP to any particular instance via your control panel. If an IP is attached via the control panel, high availability will not function properly.

Installing lastest
Docker on Ubuntu 17.10

Fire the following comment on each machine,

sudo apt-get install     apt-transport-https     ca-certificates     curl     software-properties-common -y
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository    "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"
sudo apt-get update
sudo apt-get install docker-ce -y

and if it doesn't work, most likely docker doesn't support your newest Ubuntu and you have to switch it to a lower one as show below,

vi /etc/apt/sources.list

change artful to zesty

deb [arch=amd64] https://download.docker.com/linux/ubuntu zesty stable

then go ahead and install again.
Once its done, check the version. You should get something like this below,

[email protected]:/etc/bird# docker -v
Docker version 17.09.0-ce, build afdb6d4

once this is done, we are going to install blue for BGP

Installing and Configuring BGP using Bird in Ubuntu 17.10

First, do yourself a favor and add its repo

sudo add-apt-repository ppa:cz.nic-labs/bird
apt-get update -y

once this is done, you'll get an error because it doesn't support Ubuntu 17.10 yet. Hence, gonna go downwards again.

vi /etc/apt/sources.list.d/cz_nic-labs-ubuntu-bird-artful.list

change it to the below,

deb http://ppa.launchpad.net/cz.nic-labs/bird/ubuntu xenial main

save it and try to install bird below,

sudo apt-get install bird

And we have bird installed. Now to configure it, we will paste the following into /etc/bird/bird.conf

log "/var/log/bird" all;

router id 192.168.23.10; # instance ip

protocol device
{
scan time 60;
}

protocol direct
{
interface "dummy1";
}

protocol bgp vultr
{
local as 11512; #instances ASN number
source address 192.168.23.10; #instance ip
import none;
export all;
graceful restart on;
next hop self;
multihop 2;
neighbor 169.254.169.252 as 64019; #vultr IP and ASN number
password "xC4c2EGaXuP7a74!"; #vulr password
}

then proceed to restart

service bird restart
service bird status
● bird.service - BIRD Internet Routing Daemon (IPv4)
   Loaded: loaded (/lib/systemd/system/bird.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2017-10-30 10:30:51 UTC; 19min ago
  Process: 5140 ExecStartPre=/usr/sbin/bird -p (code=exited, status=0/SUCCESS)
  Process: 5130 ExecStartPre=/usr/lib/bird/prepare-environment (code=exited, status=0/SUCCESS)
 Main PID: 5145 (bird)
    Tasks: 1 (limit: 4915)
   Memory: 640.0K
      CPU: 53ms
   CGroup: /system.slice/bird.service
           └─5145 /usr/sbin/bird -f -u bird -g bird

Oct 30 10:30:51 worker_a systemd[1]: Starting BIRD Internet Routing Daemon (IPv4)...
Oct 30 10:30:51 worker_a systemd[1]: Started BIRD Internet Routing Daemon (IPv4).

now make sure you have the above, if not, its most likely because your log permission is set as root, if that's the case, try the below,

chown bird:bird /var/log/bird

and restart bird again. Ensure your firewall isn't blocking port 179 for BGP to communicate with your machine. Once this is done, you can test it out externally via telnet

telnet 192.168.23.10 179
Trying 192.168.23.10...
Connected to 192.168.23.10.vultr.com.
Escape character is '^]'.
Connection closed by foreign host.

you should get something like this. Now, the most important part if not you will get the following

birdc show proto all vultr
BIRD 1.6.3 ready.
name     proto    table    state  since       info
vultr    BGP      master   start  13:49:30    Idle          Received: Bad peer AS
  Preference:     100
  Input filter:   REJECT
  Output filter:  ACCEPT
  Routes:         0 imported, 0 exported, 0 preferred
  Route change stats:     received   rejected   filtered    ignored   accepted
    Import updates:              0          0          0          0          0
    Import withdraws:            0          0        ---          0          0
    Export updates:              0          0          0        ---          0
    Export withdraws:            0        ---        ---        ---          0
  BGP state:          Idle
    Neighbor address: 169.254.169.254
    Neighbor AS:      64515
    Error wait:       10/300
    Last error:       Received: Bad peer AS

Setup Dummy IP Configuration

run the following to create a dummy IP configuration for your Bird to forward to

ip link add dev dummy1 type dummy
ip link set dummy1 up
ip addr add dev dummy1 192.168.10.10

where 192.168.10.10 is the floating IP. Now you can test whether the above setup is correct with the following

ip addr show dev dummy1
6: dummy1: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 4e:79:76:36:6d:b0 brd ff:ff:ff:ff:ff:ff
    inet 45.77.110.78/32 scope global dummy1
       valid_lft forever preferred_lft forever
    inet6 fe80::4c79:76ff:fe36:6db0/64 scope link
       valid_lft forever preferred_lft forever

Now, restart your Bird so that it gets the above dummy. To set it persistently, do the following

vi /etc/rc.local
ip link add dev dummy1 type dummy
ip link set dummy1 up
ip addr add dev dummy1 192.168.10.10/32

then restart or reboot your machine

service bird restart

Once this is done, you should get the following,

birdc show proto all vultr
BIRD 1.6.3 ready.
name     proto    table    state  since       info
vultr    BGP      master   up     14:44:15    Established
  Preference:     100
  Input filter:   REJECT
  Output filter:  ACCEPT
  Routes:         0 imported, 1 exported, 0 preferred
  Route change stats:     received   rejected   filtered    ignored   accepted
    Import updates:              0          0          0          0          0
    Import withdraws:            0          0        ---          0          0
    Export updates:              1          0          0        ---          1
    Export withdraws:            0        ---        ---        ---          0
  BGP state:          Established
    Neighbor address: 169.254.169.254
    Neighbor AS:      64515
    Neighbor ID:      208.167.241.119
    Neighbor caps:    refresh restart-aware AS4 add-path-rx
    Session:          external multihop AS4
    Source address:   45.77.78.111
    Hold timer:       132/180
    Keepalive timer:  29/60

Once you get this, all should be good. Try it out.

Share

Setup MongoDB 3.2 Geographical Replica Set in Ubuntu 15.10

Interestingly, i needed to setup a Replica Set on Ubuntu 15.10 for MongoDB 3.2 which is the latest Ubuntu and MongoDB version. This also serve as a tutorial for anyone who is interested in setting up a MongoDB Replica Set as well.

Import the public key used by the package management system.

Login to your server as root, we will need to import the public key use by the package manager for mongodb, just fire the following command,

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10

And we are good here.

Create a Source list file for MongoDB and Installation

Next, we need to add the source list for MongoDB. However, since MongoDB did not support 15.10 at this time, we will use debian ones

echo "deb http://repo.mongodb.org/apt/debian wheezy/mongodb-org/3.0 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list

Now, we will need to update the server and install mongodb

sudo apt-get update
sudo apt-get install -y mongodb-org

And after everything finished running, you should have your mongodb running.

sudo service mongod start

if no error is given, meaning your MongoDB has successfully installed.

Setup Replica Set

Now, assuming you did the above on 3 machines, you will need to setup each replica with the following steps,

head over to /etc/mongod.conf and replace your config with the one show below,

# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

storage:
    dbPath: "/data/mongo"
    directoryPerDB: false
    journal:
        enabled: true
    engine: "wiredTiger"
    wiredTiger:
        engineConfig:
            cacheSizeGB: 1
        collectionConfig:
            blockCompressor: snappy
systemLog:
    destination: file
    path: "/var/log/mongodb.log"
    logAppend: true
    logRotate: reopen
    timeStampFormat: iso8601-utc
net:
  port: 27017
  bindIp: 0.0.0.0

replication:
   oplogSizeMB: 500
   replSetName: dstTest

Next, create the folder for MongoDB data,

mkdir -p /data/mongo
chown -R mongodb:mongodb /data

Once you have done that, restart MongoDB and make sure there is no error.

sudo service mongod restart

Next we need to setup each replica in MongoDB.

Configure the servers to include in replica set

Assuming you have 3 machines, with the following hostname

sg.db.hungred.com
us.db.hungred.com
tw.db.hungred.com

Now, head over to the primary MongoDB server that you would like it to be primary (in my case, us.db.hungred.com) and enter to mongodb using the command below,

mongo
rs.initiate

then paste the following

rs.reconfig({ _id : "testDB", members : [ {_id : 0, host : "sg.db.hungred.com:27017", priority: 5}, {_id : 1, host : "us.db.hungred.com:27017", priority: 4}, {_id : 2, host : "tw.db.hungred.com:27017", priority: 3 } ] })

take note of the priority i have given it and make sure this is one liner (yeah its messy but that's how i copy and paste it in one piece), then check your conf

rs.conf()

and status at

rs.status()

and you got yourself a 3 location replica set of MongoDB!

***** UPDATE *****

Adding Security Authentication

If you want to add authentication into your setup, you will need to visit /etc/mongod.conf and add the following

security:
  keyFile: /data/mongodb-keyfile

on all of your primary and secondary Mongodb server. The file will need to generate this way,

openssl rand -base64 741 > /data/mongodb-keyfile
chmod 600 mongodb-keyfile

This is to ensure all replica set can communicate with each other. Once you have generated the file above on the primary MongoDB server, copy the same file to other secondary server and update the /etc/mongod.conf on each secondary server along with it.

Share

How To Install Aide Intrusion Detection System on Ubuntu

Aide stands for Advanced Intrusion Detection Environment which is one of the most popular tools for monitoring changes to a Unix or Linux system. Here i will list out how i am going to set this baby up on some of my server to secure on system.

Updating and Installing Aide

sudo apt-get update -y

Once you have update your repo, simply install Aide using the following command

sudo apt-get install aide

And aide is installed in your machine!

Configuring and Test out Aide

Next we are going to configure this baby. Initial the database with the command below,

sudo aideinit

It will take a while and once you have initial the database, Verify that the new aide database has been created

cd /var/lib/aide
ls -lt

And you should see something like this

AIDE 0.16a2-19-g16ed855 initialized AIDE database at /var/lib/aide/aide.db.new
Start timestamp: 2016-05-12 10:17:20 -0400
Verbose level: 6

Number of entries:	66800

---------------------------------------------------
The attributes of the (uncompressed) database(s):
---------------------------------------------------

/var/lib/aide/aide.db.new
  RMD160   : BOdplDoXDH0ws73WkoYe11+WIhM=
  TIGER    : tJ8xmXCDo9N9e8cJZBuqQSW/yl/ArSnJ
  SHA256   : E+Pc3ae0PDDxfRV9PcZZ8Fq+NsJZBLbo
             SQQ+i6xQ2I0=
  SHA512   : WHHce2bdDPzP1NgMSr9afReWcIvGbW+p
             D09ShUO3kT6EJpFWhqTR0RI60LmYW/sR
             76QTqqOOnIK+Cknc8mKXRA==
  CRC32    : OqKLPA==
  HAVAL    : zT+SY0Ee5SuFaXb7Kjo3gU7NpnH+QIyA
             buxyjH8AedM=
  GOST     : 4cW9q/3KpRawsNsRc2HtdjGgF70fsaI5
             8eRaLnsDlmo=


End timestamp: 2016-05-12 10:24:58 -0400 (run time: 7m 38s)

Move the new file to the new database using the following command,

mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db

Now, let's test this baby out with the following command,

aide.wrapper --check

and you will get something like this

[email protected]:~# aide.wrapper --check
AIDE 0.16a2-19-g16ed855 found differences between database and filesystem!!
Start timestamp: 2016-05-12 10:29:51 -0400
Verbose level: 6

Summary:
  Total number of entries:	66801
  Added entries:		1
  Removed entries:		1
  Changed entries:		3

---------------------------------------------------
Added entries:
---------------------------------------------------

f++++++++++++++++: /var/lib/aide/aide.db

---------------------------------------------------
Removed entries:
---------------------------------------------------

f----------------: /var/lib/aide/aide.db.new

---------------------------------------------------
Changed entries:
---------------------------------------------------

d =.... mc.. .. .: /var/lib/mongodb/diagnostic.data
f >b... mc..C.. .: /var/lib/mongodb/diagnostic.data/metrics.2016-05-12T08-52-09Z-00000
f >.... mci.C.. .: /var/lib/mongodb/diagnostic.data/metrics.interim

---------------------------------------------------
Detailed information about changes:
---------------------------------------------------

Directory: /var/lib/mongodb/diagnostic.data
  Mtime    : 2016-05-12 10:24:39 -0400        | 2016-05-12 10:36:39 -0400
  Ctime    : 2016-05-12 10:24:39 -0400        | 2016-05-12 10:36:39 -0400

File: /var/lib/mongodb/diagnostic.data/metrics.2016-05-12T08-52-09Z-00000
  Size     : 361980                           | 372957
  Bcount   : 720                              | 744
  Mtime    : 2016-05-12 10:22:08 -0400        | 2016-05-12 10:32:08 -0400
  Ctime    : 2016-05-12 10:22:08 -0400        | 2016-05-12 10:32:08 -0400
  RMD160   : czpo/fk+iRIEKUBjlc2+wELg/Wo=     | wEQV9cj/KyiGQmfGSLbzo9B44Gs=
  TIGER    : 2wLpFPWq3lxfxXyHpAMkVXUjDtZ08W8z | x8IbKbindr6NVwNbaUt0J5jWq9Y1cWmv
  SHA256   : lVRtuDTLDD7DYajbBEYoMSPpdrtxdJNA | 3J4B2ToLfGmBbHOQas/hKGj8HXe4zihW
             rxL5xH8A0kA=                     | 0OLKtXC4fqo=
  SHA512   : axlztAMc56xIGz7JnsOq8dAgZfCLmT83 | 49Fex6rPE24SnoOaLc+T/hIiTLEEyOmk
             gFZS6MB2zmT5aPxK4FmOSnEC9W5mtUNJ | YGeLF1W/fxZuRYk3FuwgpFlKA2qrmi2f
             AIaoa5bK736BAXwMcsA+NA==         | xNij3UG21mAiX+Tx2pRw+A==
  CRC32    : drkWXw==                         | rtCgKQ==
  HAVAL    : SR2yfai80zpN2Xw+8sUFSM/kTQBGAHsl | xIk6ByhAZN5C2eU2bTJzZ0oZcJeqsIiz
             71FSIVFT4qA=                     | AMbC0DPcNhg=
  GOST     : bE/NiblzIQRPzFx8jVymvvkEA+NO6on0 | txFhbK566EUxlQk6c36TfqgvYBttntcm
             k3XlP3vO2LA=                     | qyMIxjG3zK8=

File: /var/lib/mongodb/diagnostic.data/metrics.interim
  Size     : 5279                             | 5397
  Mtime    : 2016-05-12 10:24:39 -0400        | 2016-05-12 10:36:39 -0400
  Ctime    : 2016-05-12 10:24:39 -0400        | 2016-05-12 10:36:39 -0400
  Inode    : 1180042                          | 1179903
  RMD160   : Uch+G7OlOobiM/VjjdNHYSdCZUY=     | OnSReGX+lqQuCQURBBxkfHC9U5o=
  TIGER    : bB0QmZYYNl2SKSfz4MlNrpwYKwCS3Evf | ktNDR+97gTAK7catLGoOhEFJu6IfQZwi
  SHA256   : h0s1leYNb7/RxTi86z+nHhe7DChFJtSo | KIlG5ePVgwG/+DopSTPHo6VqnGzdnQMj
             TUZXyOwKKYw=                     | m97NR3Gifhk=
  SHA512   : 8PrN5C6RJgYHIuM7DjL3vjx9/5fRbnsr | QLXQngP8ouoc8bvs580De+Vh7bGR0Lq8
             MDpk+PcTAxLV3AUbkWP9Xq0hTzro7mlM | +2tXCfVed02e1DVRgxeG3LbKxqhofP76
             nT96+O95DnPZRmuD5OAPZA==         | 6Mz99D/w7u9eabdbsYmmOw==
  CRC32    : sTX43A==                         | Ta6Udw==
  HAVAL    : ZDpLBirCqbUqz/jym+FFjv2IvY9T4k+g | qTpVXVypYnzMGQZF4SMw7Wjg/jKkptpw
             hhcWR0kK/ZE=                     | PEqS+lI8g84=
  GOST     : 7yJZnGdeAM8slovcFTD0Ftcec5KT8weQ | gVW46Bk3upRekyxDI5sPP6N1xk7b6gX5
             yPYlQqSMkf4=                     | CJTybT2VVKQ=


---------------------------------------------------
The attributes of the (uncompressed) database(s):
---------------------------------------------------

/var/lib/aide/aide.db
  RMD160   : BOdplDoXDH0ws73WkoYe11+WIhM=
  TIGER    : tJ8xmXCDo9N9e8cJZBuqQSW/yl/ArSnJ
  SHA256   : E+Pc3ae0PDDxfRV9PcZZ8Fq+NsJZBLbo
             SQQ+i6xQ2I0=
  SHA512   : WHHce2bdDPzP1NgMSr9afReWcIvGbW+p
             D09ShUO3kT6EJpFWhqTR0RI60LmYW/sR
             76QTqqOOnIK+Cknc8mKXRA==
  CRC32    : OqKLPA==
  HAVAL    : zT+SY0Ee5SuFaXb7Kjo3gU7NpnH+QIyA
             buxyjH8AedM=
  GOST     : 4cW9q/3KpRawsNsRc2HtdjGgF70fsaI5
             8eRaLnsDlmo=


End timestamp: 2016-05-12 10:37:13 -0400 (run time: 7m 22s)

see the file that we just added and updated? Yeah, that's the one that its reporting.

Crontab Aide

Now we dont want to do this every single day manually, so let's setup a crontab.

vi aide.sh

with the following code

#! /bin/sh
MYDATE=`date +%Y-%m-%d`
MYFILENAME="Aide-"$MYDATE.txt
/bin/echo "Aide check !! `date`" > /tmp/$MYFILENAME
/usr/bin/aide.wrapper --check > /tmp/myAide.txt
/bin/cat /tmp/myAide.txt|/bin/grep -v failed >> /tmp/$MYFILENAME
/bin/echo "**************************************" >> /tmp/$MYFILENAME
/usr/bin/tail -20 /tmp/myAide.txt >> /tmp/$MYFILENAME
/bin/echo "****************DONE******************" >> /tmp/$MYFILENAME
/usr/bin/mail -s"$MYFILENAME `date`" [email protected] < /tmp/$MYFILENAME

now make it executable

chmod +x aide.sh

open up crontab

crontab -e

add the following crontab into it

06 01 * * 0-6 /root/aide.sh

And we are good to go! Simple as that!

Share

Installing avconv from source in Ubuntu

This is pretty interesting as avconv is a pretty sensitive fellow. If you do this wrongly, your avconv is pretty much not workable no matter how you are going to spin this. Hence, i do myself a favour and write this up instead. Now, before i began, let me explain what version of Ubuntu i am using. I am on Ubuntu 12.04.5 LTS and if i do the following from repo

sudo apt-get install ffmpeg libav-tools

I will be getting myself avcon version 8.x.x since i am on a older version of Ubuntu. Hence, the installed of avcon from source.

Removing older avconv or ffmpeg

Before we do anything at all, we need to remove the older version of avcon.

apt-get remove --purge libav-tools

and this will remove the built in avconv that you might have installed

Installing prerequisite files

Now, fire the below command to install everything you may need for your avconv compilation

apt-get install build-essential
apt-get install yasm
apt-get install libfdk-aac-dev
apt-get install libx264-dev

Once you have done that, we are good to go!

Installing avconv from source

I am going to install avconv version 11 from libav.org.

wget https://libav.org/releases/libav-11.6.tar.gz  --no-check-certificate
tar zxvf libav-11.6.tar.gz
cd libav-11.6/
./configure --enable-libvo-aacenc --enable-gpl --enable-nonfree --enable-version3 --enable-libx264 --enable-static --disable-shared
make && make install

After you have finished compilation (which takes some time), you are pretty much installed with the newer version of avconv! Not that tough right?

Share