Install OpenFlow Network (OpenVSwich, Floodlight, VirtualBox)

>>> Purpose
This document shows how to deploy a SDN(Software Defined Network) environment in a Linux computer.The SDN is implemented based on OpenFlow protocol. 

We deploy followings on a Ubuntu 12.04 Linux machine. 
 - an OpenVSwitch(OpenFlow Switch), 
 - a Floodlight(OpenFlow Controller), 
 - a VirtualBox(hypervisor) and several virtual machines(Fedora 18 OS).

>>> Environment
- I use root id

>>> System Topology
  vm1 vm2 vm3
   |        |      |
   *       *      *    
    \       |     / 
   OpenVSwitch  --------- Floodlight (Openflow Controller)

*** vm1,vm2,vm3 run on VirtualBox hypervisor
*** ovs(OpenVSwitch) and floodlight are installed on Ubuntu   and connected. 

>>> Preparation (on Ubuntu)
- change root password
 > sudo passwd -u root
 > sudo passwd root

- remove network-manager
root@server:/etc/network# apt-get purge network-manager
:

- set ip address (for Internet and for private network)

In this example, I will use two interfaces(network cards). 
One(eth0) is used for SDN, and the other(eth1) is used to access for management. 
eth1 can be dynamic IP address or static IP address. (For me, I used a static IP address)

root@server:~# vi /etc/network/interfaces
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
  address 192.168.1.20
  netmask 255.255.255.0

auto eth1
iface eth1 inet static
  address X.X.X.X
  netmask Y.Y.Y.Y
  gateway Z.Z.Z.Z
  dns-nameservers A.A.A.A

*** If we use dynamic IP address for eth1, remove "auto eth1" section. 

- install sshd
For access to manage, we install SSH server. 
  > apt-get install openssh-server

>>> Install and run OVS
- install dependencies
** Please be careful to use ` (just left to number 1 on keyboard) rather than '(single quotation mark)

> apt-get update

> apt-get install -y git python-simplejson python-qt4 python-twisted-conch automake autoconf gcc uml-utilities libtool build-essential git pkg-config linux-headers-`uname -r`

- download OpenVSwitch

> git clone git://openvswitch.org/openvswitch
> cd openvswitch

- install OpenVSwitch
root@server:~/openvswitch#  ./boot.sh
root@server:~/openvswitch# ./configure --with-linux=/lib/modules/`uname -r`/build
root@server:~/openvswitch#  make && make install

- load openvswitch module into kernel
root@server:~/openvswitch# pwd
/root/openvswitch
root@server:~/openvswitch# cd datapath/linux
root@server:~/openvswitch/datapath/linux# modprobe openvswitch
root@server:~/openvswitch/datapath/linux# lsmod | grep openvswitch
openvswitch            47849  0  

  cf) we use "modprobe" rather than "insmod" 
      : insmod datapath/linux/openvswitch.ko

- needed file and directory
> touch /usr/local/etc/ovs-vswitchd.conf
> mkdir -p /usr/local/etc/openvswitch

- create conf.db (at openvswitch directory)
root@server:~/openvswitch/datapath/linux# cd ../..
root@server:~/openvswitch# ovsdb-tool create /usr/local/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema

- make openvswitch.sh (at home directory)
root@server:~/openvswitch# cd
root@server:~# vi openvswitch.sh
ovsdb-server /usr/local/etc/openvswitch/conf.db \
--remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,Open_vSwitch,manager_options \
--private-key=db:Open_vSwitch,SSL,private_key \
--certificate=db:Open_vSwitch,SSL,certificate \
--bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach --log-file

ovs-vsctl --no-wait init
ovs-vswitchd --pidfile --detach
ovs-vsctl show

- change permission and run the openvswitch.sh
root@server:~# chmod 755 openvswitch.sh
root@server:~# ls -l
total 12
:
-rwxr-xr-x  1 root root  409 Oct 18 12:26 openvswitch.sh
root@server:~# ./openvswitch.sh
2013-10-18T17:26:51Z|00001|vlog|INFO|opened log file /usr/local/var/log/openvswitch/ovsdb-server.log
2013-10-18T17:26:51Z|00001|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: connecting...
2013-10-18T17:26:51Z|00002|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: connected
25a1e20e-8902-43f7-8800-154ef462202b

- check the status of openvswitch
root@server:~# ovs-vsctl show
02e70590-d0ad-417a-bbb1-0636bdff30d0

root@server1:~# ovs-vsctl --version
ovs-vsctl (Open vSwitch) 2.0.90
Compiled Sep 17 2013 16:11:27

root@server:~# ps -ea | grep ovs
20035 ?        00:00:00 ovsdb-server
20039 ?        00:00:00 ovs-vswitchd

*** expected error1
If we do not create db (vswitch.ovsschema), we get following error message. 
root@server:~/openvswitch# ovsdb-server /usr/local/etc/openvswitch/conf.db \
 --remote=punix:/usr/local/var/run/openvswitch/db.sock \
 --remote=db:Open_vSwitch,manager_options \
 --private-key=db:SSL,private_key \
 --certificate=db:SSL,certificate \
 --bootstrap-ca-cert=db:SSL,ca_cert --pidfile --detach --log-file

2013-09-17T21:43:10Z|00001|vlog|INFO|opened log file /usr/local/var/log/openvswitch/ovsdb-server.log
ovsdb-server: I/O error: open: /usr/local/etc/openvswitch/db.sock failed (No such file or directory)

>>> Connect physcial interface to virtual bridge
- create a bridge and connect it to ethernet port
  : ip address of the ethernet port should be set to 0, and  set an ip address(including netmask) of the bridge
root@server:~# ovs-vsctl add-br br0
root@server:~# ovs-vsctl add-port br0 eth0
root@server:~# ifconfig eth0 0 up
root@server:~# ifconfig br0 192.168.1.20 netmask 255.255.255.0 up

- check 
  : bridge has an ip address, but eth0 does not have an ip address. 
  : eth0 is connected to br0
root@server:~# ifconfig
br0     Link encap:Ethernet  HWaddr 
          inet addr:192.168.1.20  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:468 (468.0 B)
eth0      Link encap:Ethernet  HWaddr 
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:20 Memory:e1600000-e1620000 
:

>>> Install Floodlight and attach openvSwitch
- install dependencies
root@server:~# apt-get install build-essential default-jdk ant python-dev eclipse git
root@server:~# git clone git://github.com/floodlight/floodlight.git
Cloning into 'floodlight'...
remote: Counting objects: 25370, done.
remote: Compressing objects: 100% (10126/10126), done.
remote: Total 25370 (delta 12934), reused 24232 (delta 12079)
Receiving objects: 100% (25370/25370), 26.94 MiB | 1.90 MiB/s, done.
Resolving deltas: 100% (12934/12934), done.

root@server:~# ls
floodlight  openvswitch  openvswitch.sh

root@server:~# cd floodlight
root@server:~/floodlight# ls
apps       debian   findbugs-exclude.xml  floodlight_style_settings.xml  LICENSE.txt  Makefile    README.md         src
build.xml  example  floodlight.sh         lib                            logback.xml  NOTICE.txt  setup-eclipse.sh

root@server:~/floodlight# ant
:
Buildfile: /root/floodlight/build.xml

init:
    [mkdir] Created dir: /root/floodlight/target/bin
    [mkdir] Created dir: /root/floodlight/target/bin-test
    [mkdir] Created dir: /root/floodlight/target/lib
    [mkdir] Created dir: /root/floodlight/target/test
compile:
    [javac] Compiling 579 source files to /root/floodlight/target/bin
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
     [copy] Copying 54 files to /root/floodlight/target/bin
compile-test:
    [javac] Compiling 113 source files to /root/floodlight/target/bin-test
dist:
      [jar] Building jar: /root/floodlight/target/floodlight.jar
      [jar] Building jar: /root/floodlight/target/floodlight-test.jar
BUILD SUCCESSFUL

Total time: 10 seconds

root@server1:~/floodlight# java -jar target/floodlight.jar
19:57:39.899 INFO [n.f.c.m.FloodlightModuleLoader:main] Loading default modules
:
19:57:52.037 INFO [n.f.c.i.Controller:main] Listening for switch connections on 0.0.0.0/0.0.0.0:6633
:

>>> Attach openvswitch to floodlight
- set "secure mode"
root@server:~# ovs-vsctl set-fail-mode br0 secure

With setting "secure mode", OVS cannot route without floodlight(or other remote contollers). 
An explanation of the fail-mode is below, taken from [1]:

The OpenFlow controller is responsible for setting up all flows on the switch, which means that when the controller is not running there should be no packet switching at all. Depending on the setup of yournetwork, such a behavior might not be desired. It might be best that when the controller is down, the switch should default back in being a learning layer 2 switch. In other circumstances however this might beundesirable. In OVS this is a tunable parameter, called fail-safe-mode
which can be set to the following parameters:

*standalone [default]: in which case OVS will take responsibility for forwarding the packets if the controller fails 

*secure: in which case only the controller is responsible for forwarding packets, and if the controller is down all packets are going to be dropped.

[1] http://groups.geni.net/geni/wiki/GENIExperimenter/Tutorials/OpenFlowOVS/Execute

- show ovs status (br0 -- eth0)
root@server:~# ovs-vsctl show
02e70590-d0ad-417a-bbb1-0636bdff30d0
    Bridge "br0"
        fail_mode: secure
        Port "eth0"
            Interface "eth0"
        Port "br0"
            Interface "br0"
                type: internal

- attach ovs to an controller (br0 -- controller)
root@server:~# ovs-vsctl set-controller br0 tcp:192.168.1.20:6633
root@server:~# ovs-vsctl show
02e70590-d0ad-417a-bbb1-0636bdff30d0
    Bridge "br0"
        Controller "tcp:192.168.1.20:6633"
            is_connected: true
        fail_mode: secure
        Port "eth0"
            Interface "eth0"
        Port "br0"
            Interface "br0"
                type: internal

- check
root@server:~# ifconfig
br0       Link encap:Ethernet  HWaddr 
          inet addr:192.168.1.20  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:468 (468.0 B)

eth0      Link encap:Ethernet  HWaddr 
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:20 Memory:e1600000-e1620000 

- in floodlight console, we see following message.
:
20:07:42.012 WARN [n.f.c.i.C.s.notification:main] Switch XXXXXXXXXXX connected

>>> Install Virtualbox (4.3 on Ubuntu 12.04)
- install virtualbox
root@server:~# wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc -O- | sudo apt-key add -
OK
root@server:~# sh -c 'echo "deb http://download.virtualbox.org/virtualbox/debian precise contrib" >> /etc/apt/sources.list'
root@server:~# apt-get install dkms
oot@server:~# apt-get update && apt-get install virtualbox-4.3
:

*** We see "Oracle VM Virtual Box" when you type "vir" at Dash Home. 

 

>>> Virtual Machine : preparation (per each vm)
*** the virtual machine port to be connected to a bridge(br0) is vnetX (like vnet1, vnet2,...)

- openVSwitch configuration (vnetX)
root@server:~# ip tuntap add mode tap vnet1
root@server:~# ip link set vnet1 up
root@server:~# ovs-vsctl add-port br0 vnet1
root@server:~# ifconfig br0 up

root@server:~# ovs-vsctl show
02e70590-d0ad-417a-bbb1-0636bdff30d0
    Bridge "br0"
        Controller "tcp:192.168.1.20:6633"
            is_connected: true
        fail_mode: secure
        Port "eth0"
            Interface "eth0"
        Port "vnet1"
            Interface "vnet1"
        Port "br0"
            Interface "br0"
                type: internal

- check if the datapath or bridge and interfaces are up
root@server:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
2: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
3: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
5: br0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
6: vnet1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 500

root@server:~# ifconfig
br0       Link encap:Ethernet  HWaddr 
          inet addr:192.168.1.20  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:468 (468.0 B)
eth0      Link encap:Ethernet  HWaddr 
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:20 Memory:e1600000-e1620000 
eth3      Link encap:Ethernet  HWaddr 
          inet addr:X.X.X.X  Bcast:B.B.B.B  Mask:Y.Y.Y.Y
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:139052 errors:0 dropped:1864 overruns:0 frame:0
          TX packets:18223 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:87098297 (87.0 MB)  TX bytes:1225005 (1.2 MB)
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:14650 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14650 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:4302790 (4.3 MB)  TX bytes:4302790 (4.3 MB)
vnet1     Link encap:Ethernet  HWaddr 
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

root@server:~# ovs-vsctl show
02e70590-d0ad-417a-bbb1-0636bdff30d0
    Bridge "br0"
         "tcp:192.168.1.20:6633"
            is_connected: true
        fail_mode: secure
        Port "eth0"
            Interface "eth0"
        Port "vnet1"
            Interface "vnet1"
        Port "br0"
            Interface "br0"
                type: internal

>>> Virtual Machine Configuration (Install Fedora 18 on each VM)

** I use Fedora 18 as an Operating System for VM, but you can use Ubuntu 12.04 TLS (It's up to you)However, screen capture is based on Fedora 18. 

- run VirtualBox
- "new" - Name (vmX), type(Linux), Version(Fedora(64bit)) - Memory Size (2048 MB) - Hard Drive (Create a virtual hard drive now) - Hard drive file type(VDI) - Storage on physical hard drive(Dynamically allocated) - File location and size (danny\VirtualBox VMs\vm1, 50 GB)

- setting
  : "Network" - Adapter_1 (Attached to - Bridged Adapter, name - vnet1)

 

- start - select start-up disk (put fedora disk into cdrom) - "start"
- install Fedora 18
- Network setting (/etc/sysconfig/network-scripts/ifcfg-XXX)
  :
  BOOTPROTO=none
  :
  IPADDR=192.168.1.XX
  NETMASK=255.255.255.0

> service NetworkManager stop
> chkconfig NetworkManager off
> service network restart

- check ping from VM(Fedora) to Host(Ubuntu)
  : ping 192.168.1.XXX (IP address of host(Ubuntu 12.04))

- check ping from Host(Ubuntu) to (Fedora)
  : ping 192.168.1.XXX (IP address of vmX)

- To make access to VM easy, run sshd into VM
  > service sshd start
  > chkconfig sshd on

>>> Rebooting steps
Follow these steps whenever reboot due to network reconfiguration.

- up virtual network interface
root@server:~# ip tuntap add mode tap vnet1
root@server:~# ip link set vnet1 up

(repeat for other virtual network interfaces, for example of vnet2)
root@server:~# ip tuntap add mode tap vnet2
root@server:~# ip link set vnet2 up

- up bridge and bridged interface
root@server:~# ifconfig eth0 0 up
root@server:~# cd ./openvswitch/datapath/linux; modprobe openvswitch;cd ~;/root/openvswitch.sh   
root@server:~# ifconfig br0 192.168.1.20 netmask 255.255.255.0 up

- run floodlight
root@server:~# cd /root/floodlight; java -jar target/floodlight.jar

- run Virtual Machines & up interface
root@server:~# ifup ifcfg-<interface>   // <interface> --> e.g. p2p1

- connection test (between VMs, among switches) using ping command

 

 

 

 

 

 

 

 

 

 

출처 : http://dannykim.me/danny/openflow/57620