Oracle 12c RAC ASM Flex Cluster on LXC Linux Containers Ubuntu 15.04
This blog post is intended to be a complete, comprehensive step-by-step how-to for installing Oracle 12c RAC GNS ASM Flex Cluster on Ubuntu 15.04 using Oracle Enterprise Linux 6.5 LXC Linux Containers and OpenvSwitch. This is an update to the similar guide for Ubuntu 14.10 here. The guide for 14.10 was copied to this blog page, and then the procedure for doing the whole setup for OpenvSwitch and Oracle RAC on Ubuntu 15.04 was documented in this blog by editing and updating the steps where necessary.
Updated Begin for Ubuntu 15.04 2015-05-16
I recently upgraded my laptop to Ubuntu 15.04 and there were a few issues that had to be resolved which had to do with SCST. The cpu_mask issue is a reported bug in SCST and was probably the cause. Bart Van Assche responded back to my SCST list server enquiry with the following information.
Until June 2014 there was a bug in the SCST code for parsing cpu masks. You may have hit that bug. Updating to SCST trunk r5596 or later or to the latest version of the SCST 3.0 branch should resolve the issue related to parsing cpu masks.
The guide that is being created here now is based on a fresh install of Ubuntu 15.04 Vivid Vervet 64-bit desktop edition running on a Lenovo W520 mobile workstation. Below there is a note about SCST. The way I did the LUN presentation to Oracle previously was to build a KVM Oracle Enterprise Linux (OEL) 6.5 guest and then compile SCST into the kernel of that KVM guest. Since SCST has to be compiled into the kernel, installing it on the base Ubuntu 15.04 OS presents the problem that the kernel will have to be recompiled every time there is an update, and with Ubuntu that is typically going to be very frequently. The downside is that KVM and SCST do not provide all the great of IO relatively speaking to what one can get in a Linux Container, and so all Oracle IO to the LUNs on the KVM guest running SCST has to go through KVM which will degrade IO performance to the Oracle DB measurably. A less than optimal solution from that point of view. I haven't got a particularly good solution at this point for a better mousetrap for presenting the LUNs other than using a KVM guest, because, the nice thing about KVM is that I can set the logical and physical sector size on the LUNs with KVM, and SCST also has a manual setting for 4K physical sector size LUNs, so this KVM-SCST combo allows this lab system to present both 512/512 and 4096/4096 and 512/4096 LUN configurations to Oracle so that the setup can be used to play with Oracle using 4K native LUNs and/or 4K native LUNs with 512 emulation mode.
Update End for Ubuntu 15.04 2015-05-16
This guide is also a de-facto recipe for creating an OpenvSwitch-based Ubuntu Linux [ desktop | laptop ] networking environment that accomplishes a number of key goals as enumerated below.
Design Assumptions
This set of procedures has been tested and built on a fresh install of the following Ubuntu distributions. Installation on matching Ubuntu installations which have been running for a few months or years and have been customized could possibly introduce variations of existing configuration that could cause the steps described in this blog to have results different from the desired and expected results, YMMV. This blog to reiterate assumes a fresh install of the following distributions, and has been built and tested successfully on all of the following distributions of Ubuntu Linux.
Ubuntu 14.04 64-bit desktop edition
Ubuntu 14.10 64-bit desktop edition
Ubuntu 15.04 64-bit desktop edition
Note, if installing Ubuntu 14.04 or 14.10 or 15.04 fresh for this work, do NOT accept the "download updates during install" option. The reason is because these updates contain a number of CVE kernel security updates, and this procedure uses the post install Ubuntu CVE updates to check and verify correct operation of Oracle Ksplice Uptrack, so, in other words, Oracle Ksplice will handle the post-install CVE kernel security updates. Once KSplice operation has been so verified, Ubuntu Software Updater can be used for updates going forward as normal. Oracle KSplice can be used for urgent CVE kernel security updates with zero downtime as needed.
This particular blog is for Ubuntu 15.04 64-bit desktop Vivid Vervet.
Definitions
DEU
Density and elasticity unit. What formerly was called a "VM" but which now, with the advent of Linux Containers, requires imho a new term which includes density and elasticity solutions such as LXC which do not use hardware virtualization and as such are not "machines" at all. So in this blog, a "DEU" will be used as the all-inclusive term to describe a single "VM" or "LXC Linux Container".
VM
A type of DEU which uses a hypervisor to virtualize (emulate using software) the hardware layers of an actual physical machine and provides what is often called a "guest" operating system.
LXC
The term LXC will be used interchangeably according to context to either mean the software called LXC which is a Linux Container software in the same category as OpenVZ and others, and in other contexts, LXC may be used to denote a specific single LXC DEU which is running on LXC software.
Design Features
No changes to Ubuntu dnsmasq-base default network, i.e is a pure add-on networking overlay;
Does not require any changes to Ubuntu NetworkManager or to default Ubuntu networking;
Does not require any removal of Linux Bridge software;
No physical interfaces directly to the OpenvSwitch (iptables/NAT/masq for external address resolution);
Allows external interface switching on LXC containers are running with NO loss of www DNS resolution;
Automatic internet-connected interface (eth0, wlan0, bnep0) detection and connection to OpenvSwitch;
Uses OpenvSwitch as the networking solution for LXC, KVM and VirtualBox DEUs;
Uses bind9 for OpenvSwitch DNS;
Uses isc-dhcp-server for OpenvSwitch DHCP services;
Integrates DNS and DHCP to update DNS automatically when new DEUs are added;
Uses the built-in Ubuntu dnsmasq-base for Ubuntu default networking.
Because it's LXC, VT-d and VT-x are NOT needed so older laptops and desktops without VT-d/x should be able to run LXC
The installation of bind9 is constructed so as not to interfere with the default Ubuntu desktop; that is, both bind9 and dnsmasq-base coexist peacefully and successfully so that even with the OpenvSwitch and bind9 installed, all the features of Ubuntu NetworkManager are preserved, including use of wifi network manager, VPN manager, bluetooth, and in general ALL default and configurable features of Ubuntu Network Manager.
Install Synaptic Package Manager (optional)
Synaptic Package Manager is not required, but has some very nice features and since old-timers to Ubuntu sometimes like it, so it can be installed. It was the GUI for package management in Ubuntu Linux desktop prior to the introduction of the "Ubuntu Software Manager" and so some folks like to have it in addtion to the Software Manager. Both tools can coexist, and synaptic can sometimes be very useful for some tasks.
gstanden@u1504:~$ sudo apt-get install synaptic
[sudo] password for gstanden:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
docbook-xml libept1.4.12 librarian0 libvte-2.90-9 libvte-2.90-common rarian-compat sgml-data
Suggested packages:
docbook docbook-dsssl docbook-xsl docbook-defguide perlsgml w3-recs opensp libxml2-utils dwww menu deborphan tasksel
The following NEW packages will be installed:
docbook-xml libept1.4.12 librarian0 libvte-2.90-9 libvte-2.90-common rarian-compat sgml-data synaptic
0 upgraded, 8 newly installed, 0 to remove and 53 not upgraded.
Need to get 2,413 kB of archives.
After this operation, 14.2 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://us.archive.ubuntu.com/ubuntu/ vivid/main sgml-data all 2.0.10 [173 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ vivid/main docbook-xml all 4.5-7.2 [336 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ vivid/main libept1.4.12 amd64 1.0.12.1 [125 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu/ vivid/main libvte-2.90-common all 1:0.36.3-1ubuntu2 [24.5 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu/ vivid/main libvte-2.90-9 amd64 1:0.36.3-1ubuntu2 [278 kB]
Get:6 http://us.archive.ubuntu.com/ubuntu/ vivid/main librarian0 amd64 0.8.1-6 [50.4 kB]
Get:7 http://us.archive.ubuntu.com/ubuntu/ vivid/main rarian-compat amd64 0.8.1-6 [55.7 kB]
Get:8 http://us.archive.ubuntu.com/ubuntu/ vivid/universe synaptic amd64 0.81.3 [1,370 kB]
Fetched 2,413 kB in 6s (355 kB/s)
Selecting previously unselected package sgml-data.
(Reading database ... 173095 files and directories currently installed.)
Preparing to unpack .../sgml-data_2.0.10_all.deb ...
Unpacking sgml-data (2.0.10) ...
Selecting previously unselected package docbook-xml.
Preparing to unpack .../docbook-xml_4.5-7.2_all.deb ...
Unpacking docbook-xml (4.5-7.2) ...
Selecting previously unselected package libept1.4.12:amd64.
Preparing to unpack .../libept1.4.12_1.0.12.1_amd64.deb ...
Unpacking libept1.4.12:amd64 (1.0.12.1) ...
Selecting previously unselected package libvte-2.90-common.
Preparing to unpack .../libvte-2.90-common_1%3a0.36.3-1ubuntu2_all.deb ...
Unpacking libvte-2.90-common (1:0.36.3-1ubuntu2) ...
Selecting previously unselected package libvte-2.90-9.
Preparing to unpack .../libvte-2.90-9_1%3a0.36.3-1ubuntu2_amd64.deb ...
Unpacking libvte-2.90-9 (1:0.36.3-1ubuntu2) ...
Selecting previously unselected package librarian0.
Preparing to unpack .../librarian0_0.8.1-6_amd64.deb ...
Unpacking librarian0 (0.8.1-6) ...
Selecting previously unselected package rarian-compat.
Preparing to unpack .../rarian-compat_0.8.1-6_amd64.deb ...
Unpacking rarian-compat (0.8.1-6) ...
Selecting previously unselected package synaptic.
Preparing to unpack .../synaptic_0.81.3_amd64.deb ...
Unpacking synaptic (0.81.3) ...
Processing triggers for sgml-base (1.26+nmu4ubuntu1) ...
Processing triggers for doc-base (0.10.6) ...
Scrollkeeper was installed, forcing re-registration of all documents.
Unregistering 35 doc-base files, re-registering 35 doc-base files...
Registering documents with scrollkeeper...
Processing triggers for gnome-menus (3.10.1-0ubuntu5) ...
Processing triggers for desktop-file-utils (0.22-1ubuntu3) ...
Processing triggers for bamfdaemon (0.5.1+15.04.20150202-0ubuntu1) ...
Rebuilding /usr/share/applications/bamf-2.index...
Processing triggers for mime-support (3.58ubuntu1) ...
Processing triggers for hicolor-icon-theme (0.14-0ubuntu1) ...
Processing triggers for man-db (2.7.0.2-5) ...
Setting up sgml-data (2.0.10) ...
Setting up libept1.4.12:amd64 (1.0.12.1) ...
Setting up libvte-2.90-common (1:0.36.3-1ubuntu2) ...
Setting up libvte-2.90-9 (1:0.36.3-1ubuntu2) ...
Setting up librarian0 (0.8.1-6) ...
Setting up synaptic (0.81.3) ...
Processing triggers for sgml-base (1.26+nmu4ubuntu1) ...
Setting up docbook-xml (4.5-7.2) ...
Processing triggers for sgml-base (1.26+nmu4ubuntu1) ...
Setting up rarian-compat (0.8.1-6) ...
Processing triggers for libc-bin (2.21-0ubuntu4) ...
gstanden@u1504:~$
Install Oracle Ksplice (optional)
Oracle Ksplice is a free product for Ubuntu Linux which allows updates of kernel software with no downtime. Oracle Ksplice is not required for this project, but as Wim Coaekerts has pointed out here, Oracle Ksplice is likely to play a big role for organizations using Linux Containers, because Ksplice allows zero downtime kernel security patching. So it is included in this blog as part of this build, especially since Ksplice is free to install and use the service for Ubuntu Linux.
Oracle Ksplice will be a key part of any LXC Linux Container deployment because it allows all LXC Containers to continue running during security updates to the single kernel used by all LXC containers on the system.
Download Oracle KSplice Uptrack
The Oracle KSplice website already has support for Ubuntu 15.04 64-bit deskop Vivid Vervet. Not only that, but the install instructions at the KSplice website worked perfectly and were much simpler than what had to be done (in my experience anyway) before when I did this install on 14.04 and 14.10. The instructions for installing Oracle KSplice onto Ubuntu 15.04 are here at Oracle KSplice Website and I followed the instructions there verbatim and the install went perfectly according to those instructions no glitches. Click on the red text "Show Ubuntu Installation Instructions" and just follow them exactly.
My install screen shots for Oracle KSplice were as shown below. After clicking on the "Download" red button in the screenshot above, you get the download dialog on the Ubuntu desktop. Choose the "open with Ubuntu Software Center (default)" option as shown below.
The "Ubuntu Software Center" opens. Click the "Install" button at the upper right hand side of the screen below the black bar. The install of KSplice begins as shown below.
A dialog box opens to authenticate the installation, type in your password for your linux user, and then another box opens to accept the KSplice Uptrack Subscription Agreement. This is a free tool for Ubuntu Desktop users, so tick the agreement, and click "Forward" as shown below.
If any dialogs open up to authenticate, just use the Linux password that was set for your linux userid when Ubuntu was installed. The KSplice Uptrack Manager runs as shown below advising of security updates to the kernel that need to be applied. Click on "Install All Updates" as shown below.
Type in the user password for your Ubuntu linux account to authenticate KSplice Uptrack Manager as shown below.
Oracle KSplice Uptrack runs and then displays that all security kernel updates have been applied as shown below.
Run Oracle KSplice Uptrack regularly to check for kernel security updates to be applied with zero downtime (no reboot needed). This is the functionality that KSplice Uptrack technology provides: zero-downtime kernel security updates.
Update Fresh Installation (optional)
If this is a fresh install, then update Ubuntu software to get all most recent updates using the Software Updater app or "sudo apt-get update". If this is not a fresh install, optionally check for updates using the same steps.
Install LXC, OpenvSwitch, and UML-Utilities Packages
Now install other required packages as shown below.
gstanden@u1504:~$ sudo apt-get install lxc uml-utilities openvswitch-switch
[sudo] password for gstanden:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
bridge-utils cloud-image-utils debootstrap distro-info distro-info-data euca2ools libaio1 libboost-thread1.55.0 liblxc1 librados2 librbd1 libseccomp2 lxc-templates
lxcfs openvswitch-common python-ndg-httpsclient python-requestbuilder python-requests python-setuptools python-urllib3 python3-lxc qemu-utils sharutils uidmap
Suggested packages:
shunit2 btrfs-tools lxctl qemu-user-static bsd-mailx mailx user-mode-linux
The following NEW packages will be installed:
bridge-utils cloud-image-utils debootstrap distro-info distro-info-data euca2ools libaio1 libboost-thread1.55.0 liblxc1 librados2 librbd1 libseccomp2 lxc
lxc-templates lxcfs openvswitch-common openvswitch-switch python-ndg-httpsclient python-requestbuilder python-requests python-setuptools python-urllib3 python3-lxc
qemu-utils sharutils uidmap uml-utilities
0 upgraded, 27 newly installed, 0 to remove and 0 not upgraded.
Need to get 7,180 kB of archives.
After this operation, 29.9 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://us.archive.ubuntu.com/ubuntu/ vivid/main libaio1 amd64 0.3.110-1 [6,454 B]
Get:2 http://us.archive.ubuntu.com/ubuntu/ vivid/main libboost-thread1.55.0 amd64 1.55.0+dfsg-3ubuntu2 [31.5 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ vivid/main libseccomp2 amd64 2.1.1-1 [26.3 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu/ vivid/main liblxc1 amd64 1.1.2-0ubuntu3 [175 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu/ vivid/main librados2 amd64 0.94.1-0ubuntu1 [1,550 kB]
Get:6 http://us.archive.ubuntu.com/ubuntu/ vivid/main librbd1 amd64 0.94.1-0ubuntu1 [1,684 kB]
Get:7 http://us.archive.ubuntu.com/ubuntu/ vivid/main python-urllib3 all 1.9.1-3 [49.1 kB]
Get:8 http://us.archive.ubuntu.com/ubuntu/ vivid/main python-requests all 2.4.3-6 [193 kB]
Get:9 http://us.archive.ubuntu.com/ubuntu/ vivid/main python-requestbuilder all 0.2.3-1 [27.2 kB]
Get:10 http://us.archive.ubuntu.com/ubuntu/ vivid/main bridge-utils amd64 1.5-7ubuntu1 [29.1 kB]
Get:11 http://us.archive.ubuntu.com/ubuntu/ vivid-updates/main distro-info-data all 0.26ubuntu0.1 [4,056 B]
Get:12 http://us.archive.ubuntu.com/ubuntu/ vivid/main distro-info amd64 0.14 [20.1 kB]
Get:13 http://us.archive.ubuntu.com/ubuntu/ vivid/main python-setuptools all 12.2-1 [197 kB]
Get:14 http://us.archive.ubuntu.com/ubuntu/ vivid/main euca2ools all 3.1.0-1 [270 kB] Get:15 http://us.archive.ubuntu.com/ubuntu/ vivid/main python3-lxc amd64 1.1.2-0ubuntu3 [20.5 kB] Get:16 http://us.archive.ubuntu.com/ubuntu/ vivid/main lxc amd64 1.1.2-0ubuntu3 [520 kB] Get:17 http://us.archive.ubuntu.com/ubuntu/ vivid/main lxc-templates amd64 1.1.2-0ubuntu3 [63.8 kB] Get:18 http://us.archive.ubuntu.com/ubuntu/ vivid/main lxcfs amd64 0.7-0ubuntu4 [22.5 kB] Get:19 http://us.archive.ubuntu.com/ubuntu/ vivid/main python-ndg-httpsclient all 0.3.2-1ubuntu4 [20.8 kB] Get:20 http://us.archive.ubuntu.com/ubuntu/ vivid-updates/main qemu-utils amd64 1:2.2+dfsg-5expubuntu9.1 [445 kB] Get:21 http://us.archive.ubuntu.com/ubuntu/ vivid/main sharutils amd64 1:4.14-2 [146 kB] Get:22 http://us.archive.ubuntu.com/ubuntu/ vivid/main uidmap amd64 1:4.1.5.1-1.1ubuntu4 [63.3 kB] Get:23 http://us.archive.ubuntu.com/ubuntu/ vivid/main cloud-image-utils all 0.27-0ubuntu15 [25.5 kB] Get:24 http://us.archive.ubuntu.com/ubuntu/ vivid-updates/main debootstrap all 1.0.67ubuntu0.1 [35.5 kB] Get:25 http://us.archive.ubuntu.com/ubuntu/ vivid/main openvswitch-common amd64 2.3.1-0ubuntu1 [520 kB] Get:26 http://us.archive.ubuntu.com/ubuntu/ vivid/main openvswitch-switch amd64 2.3.1-0ubuntu1 [985 kB] Get:27 http://us.archive.ubuntu.com/ubuntu/ vivid/universe uml-utilities amd64 20070815-1.4 [49.4 kB] Fetched 7,180 kB in 9s (749 kB/s) Selecting previously unselected package libaio1:amd64.
(Reading database ... 203959 files and directories currently installed.)
Preparing to unpack .../libaio1_0.3.110-1_amd64.deb ...
Unpacking libaio1:amd64 (0.3.110-1) ...
Selecting previously unselected package libboost-thread1.55.0:amd64.
Preparing to unpack .../libboost-thread1.55.0_1.55.0+dfsg-3ubuntu2_amd64.deb ...
Unpacking libboost-thread1.55.0:amd64 (1.55.0+dfsg-3ubuntu2) ...
Selecting previously unselected package libseccomp2:amd64.
Preparing to unpack .../libseccomp2_2.1.1-1_amd64.deb ...
Unpacking libseccomp2:amd64 (2.1.1-1) ...
Selecting previously unselected package liblxc1.
Preparing to unpack .../liblxc1_1.1.2-0ubuntu3_amd64.deb ...
Unpacking liblxc1 (1.1.2-0ubuntu3) ...
Selecting previously unselected package librados2.
Preparing to unpack .../librados2_0.94.1-0ubuntu1_amd64.deb ...
Unpacking librados2 (0.94.1-0ubuntu1) ...
Selecting previously unselected package librbd1.
Preparing to unpack .../librbd1_0.94.1-0ubuntu1_amd64.deb ...
Unpacking librbd1 (0.94.1-0ubuntu1) ...
Selecting previously unselected package python-urllib3.
Preparing to unpack .../python-urllib3_1.9.1-3_all.deb ...
Unpacking python-urllib3 (1.9.1-3) ...
Selecting previously unselected package python-requests.
Preparing to unpack .../python-requests_2.4.3-6_all.deb ...
Unpacking python-requests (2.4.3-6) ...
Selecting previously unselected package python-requestbuilder.
Preparing to unpack .../python-requestbuilder_0.2.3-1_all.deb ...
Unpacking python-requestbuilder (0.2.3-1) ...
Selecting previously unselected package bridge-utils.
Preparing to unpack .../bridge-utils_1.5-7ubuntu1_amd64.deb ...
Unpacking bridge-utils (1.5-7ubuntu1) ...
Selecting previously unselected package distro-info-data.
Preparing to unpack .../distro-info-data_0.26ubuntu0.1_all.deb ...
Unpacking distro-info-data (0.26ubuntu0.1) ...
Selecting previously unselected package distro-info.
Preparing to unpack .../distro-info_0.14_amd64.deb ...
Unpacking distro-info (0.14) ...
Selecting previously unselected package python-setuptools.
Preparing to unpack .../python-setuptools_12.2-1_all.deb ...
Unpacking python-setuptools (12.2-1) ...
Selecting previously unselected package euca2ools.
Preparing to unpack .../euca2ools_3.1.0-1_all.deb ...
Unpacking euca2ools (3.1.0-1) ...
Selecting previously unselected package python3-lxc.
Preparing to unpack .../python3-lxc_1.1.2-0ubuntu3_amd64.deb ...
Unpacking python3-lxc (1.1.2-0ubuntu3) ...
Selecting previously unselected package lxc.
Preparing to unpack .../lxc_1.1.2-0ubuntu3_amd64.deb ...
Unpacking lxc (1.1.2-0ubuntu3) ...
Selecting previously unselected package lxc-templates.
Preparing to unpack .../lxc-templates_1.1.2-0ubuntu3_amd64.deb ...
Unpacking lxc-templates (1.1.2-0ubuntu3) ...
Selecting previously unselected package lxcfs.
Preparing to unpack .../lxcfs_0.7-0ubuntu4_amd64.deb ...
Unpacking lxcfs (0.7-0ubuntu4) ...
Selecting previously unselected package python-ndg-httpsclient.
Preparing to unpack .../python-ndg-httpsclient_0.3.2-1ubuntu4_all.deb ...
Unpacking python-ndg-httpsclient (0.3.2-1ubuntu4) ...
Selecting previously unselected package qemu-utils.
Preparing to unpack .../qemu-utils_1%3a2.2+dfsg-5expubuntu9.1_amd64.deb ...
Unpacking qemu-utils (1:2.2+dfsg-5expubuntu9.1) ...
Selecting previously unselected package sharutils.
Preparing to unpack .../sharutils_1%3a4.14-2_amd64.deb ...
Unpacking sharutils (1:4.14-2) ...
Selecting previously unselected package uidmap.
Preparing to unpack .../uidmap_1%3a4.1.5.1-1.1ubuntu4_amd64.deb ...
Unpacking uidmap (1:4.1.5.1-1.1ubuntu4) ...
Selecting previously unselected package cloud-image-utils.
Preparing to unpack .../cloud-image-utils_0.27-0ubuntu15_all.deb ...
Unpacking cloud-image-utils (0.27-0ubuntu15) ...
Selecting previously unselected package debootstrap.
Preparing to unpack .../debootstrap_1.0.67ubuntu0.1_all.deb ...
Unpacking debootstrap (1.0.67ubuntu0.1) ...
Selecting previously unselected package openvswitch-common.
Preparing to unpack .../openvswitch-common_2.3.1-0ubuntu1_amd64.deb ...
Unpacking openvswitch-common (2.3.1-0ubuntu1) ...
Selecting previously unselected package openvswitch-switch.
Preparing to unpack .../openvswitch-switch_2.3.1-0ubuntu1_amd64.deb ...
Unpacking openvswitch-switch (2.3.1-0ubuntu1) ...
Selecting previously unselected package uml-utilities.
Preparing to unpack .../uml-utilities_20070815-1.4_amd64.deb ...
Unpacking uml-utilities (20070815-1.4) ...
Processing triggers for man-db (2.7.0.2-5) ...
Processing triggers for ureadahead (0.100.0-19) ...
ureadahead will be reprofiled on next reboot
Processing triggers for systemd (219-7ubuntu5) ...
Processing triggers for install-info (5.2.0.dfsg.1-6) ...
Setting up libaio1:amd64 (0.3.110-1) ...
Setting up libboost-thread1.55.0:amd64 (1.55.0+dfsg-3ubuntu2) ...
Setting up libseccomp2:amd64 (2.1.1-1) ...
Setting up liblxc1 (1.1.2-0ubuntu3) ...
Setting up librados2 (0.94.1-0ubuntu1) ...
Setting up librbd1 (0.94.1-0ubuntu1) ...
Setting up python-urllib3 (1.9.1-3) ...
Setting up python-requests (2.4.3-6) ...
Setting up python-requestbuilder (0.2.3-1) ...
Setting up bridge-utils (1.5-7ubuntu1) ...
Setting up distro-info-data (0.26ubuntu0.1) ...
Setting up distro-info (0.14) ...
Setting up python-setuptools (12.2-1) ...
Setting up euca2ools (3.1.0-1) ...
Setting up python3-lxc (1.1.2-0ubuntu3) ...
Setting up lxc (1.1.2-0ubuntu3) ...
Setting up lxc dnsmasq configuration.
Setting up lxc-templates (1.1.2-0ubuntu3) ...
Setting up lxcfs (0.7-0ubuntu4) ...
Setting up python-ndg-httpsclient (0.3.2-1ubuntu4) ...
Setting up qemu-utils (1:2.2+dfsg-5expubuntu9.1) ...
Setting up sharutils (1:4.14-2) ...
Setting up uidmap (1:4.1.5.1-1.1ubuntu4) ...
Setting up cloud-image-utils (0.27-0ubuntu15) ...
Setting up debootstrap (1.0.67ubuntu0.1) ...
Setting up openvswitch-common (2.3.1-0ubuntu1) ...
Setting up openvswitch-switch (2.3.1-0ubuntu1) ...
openvswitch-nonetwork.service is a disabled or a static unit, not starting it.
Setting up uml-utilities (20070815-1.4) ...
Processing triggers for libc-bin (2.21-0ubuntu4) ...
Processing triggers for ureadahead (0.100.0-19) ...
Processing triggers for systemd (219-7ubuntu5) ...
gstanden@u1504:~$
Install DNS and DHCP packages
Next install bind9 and isc-dhcp-utils as shown below.
gstanden@u1504:~$ sudo apt-get install bind9 bind9utils isc-dhcp-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
bind9-doc isc-dhcp-server-ldap
The following NEW packages will be installed:
bind9 bind9utils isc-dhcp-server
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 804 kB of archives.
After this operation, 2,663 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu/ vivid/main bind9utils amd64 1:9.9.5.dfsg-9 [147 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ vivid/main bind9 amd64 1:9.9.5.dfsg-9 [294 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ vivid-updates/main isc-dhcp-server amd64 4.3.1-5ubuntu2.1 [363 kB]
Fetched 804 kB in 2s (335 kB/s)
Preconfiguring packages ...
Selecting previously unselected package bind9utils.
(Reading database ... 205485 files and directories currently installed.)
Preparing to unpack .../bind9utils_1%3a9.9.5.dfsg-9_amd64.deb ...
Unpacking bind9utils (1:9.9.5.dfsg-9) ...
Selecting previously unselected package bind9.
Preparing to unpack .../bind9_1%3a9.9.5.dfsg-9_amd64.deb ...
Unpacking bind9 (1:9.9.5.dfsg-9) ...
Selecting previously unselected package isc-dhcp-server.
Preparing to unpack .../isc-dhcp-server_4.3.1-5ubuntu2.1_amd64.deb ...
Unpacking isc-dhcp-server (4.3.1-5ubuntu2.1) ...
Processing triggers for man-db (2.7.0.2-5) ...
Processing triggers for ufw (0.34~rc-0ubuntu5) ...
Processing triggers for systemd (219-7ubuntu5) ...
Processing triggers for ureadahead (0.100.0-19) ...
Setting up bind9utils (1:9.9.5.dfsg-9) ...
Setting up bind9 (1:9.9.5.dfsg-9) ...
Adding group `bind' (GID 134) ...
Done.
Adding system user `bind' (UID 122) ...
Adding new user `bind' (UID 122) with group `bind' ...
Not creating home directory `/var/cache/bind'.
wrote key file "/etc/bind/rndc.key"
#
Setting up isc-dhcp-server (4.3.1-5ubuntu2.1) ...
Generating /etc/default/isc-dhcp-server...
Processing triggers for systemd (219-7ubuntu5) ...
Processing triggers for ureadahead (0.100.0-19) ...
Processing triggers for ufw (0.34~rc-0ubuntu5) ...
gstanden@u1504:~$
Install OpenvSwitch Configuration Scripts
These scripts create and configure the OpenvSwitch switches. The "crt_ovs_sw*.sh" scripts and the /etc/init/my-network-up script and reboot to verify openvswitch is creating switches and ports correctly. The scripts are attached to this blog (see end of blog) and can be downloaded directly. They will be downloaded by default to "/home/username/Downloads" directory.
Once the scripts are downloaded, create a directory "/home/username/OpenvSwitch" and install the "crt_ovs_sw*.sh" scripts to that directory and set correct permissions as shown below.
gstanden@u1504:~$ pwd
/home/gstanden
gstanden@u1504:~$ mkdir OpenvSwitch
gstanden@u1504:~$ cd Downloads
gstanden@u1504:~/Downloads$ ls -lrt
total 40
-rw-rw-r-- 1 gstanden gstanden 2153 May 16 21:05 crt_ovs_sw1.sh
-rw-rw-r-- 1 gstanden gstanden 1016 May 16 21:05 crt_ovs_sw2.sh
-rw-rw-r-- 1 gstanden gstanden 1019 May 16 21:05 crt_ovs_sw3.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:05 crt_ovs_sw4.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:05 crt_ovs_sw5.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:05 crt_ovs_sw6.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:06 crt_ovs_sw7.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:06 crt_ovs_sw8.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:06 crt_ovs_sw9.sh
-rw-rw-r-- 1 gstanden gstanden 1935 May 16 21:06 crt_ovs_sx1.sh
gstanden@u1504:~/Downloads$ mv crt_ovs_s*.sh ../OpenvSwitch/.
gstanden@u1504:~/Downloads$ ls -lrt
total 0
gstanden@u1504:~/Downloads$ cd ..
gstanden@u1504:~$ cd OpenvSwitch/
gstanden@u1504:~/OpenvSwitch$ ls -lrt
total 40
-rw-rw-r-- 1 gstanden gstanden 2153 May 16 21:05 crt_ovs_sw1.sh
-rw-rw-r-- 1 gstanden gstanden 1016 May 16 21:05 crt_ovs_sw2.sh
-rw-rw-r-- 1 gstanden gstanden 1019 May 16 21:05 crt_ovs_sw3.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:05 crt_ovs_sw4.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:05 crt_ovs_sw5.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:05 crt_ovs_sw6.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:06 crt_ovs_sw7.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:06 crt_ovs_sw8.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:06 crt_ovs_sw9.sh
-rw-rw-r-- 1 gstanden gstanden 1935 May 16 21:06 crt_ovs_sx1.sh
gstanden@u1504:~/OpenvSwitch$
Also download the "my-network-up.sh" script from the end of this blog. This script is used to ensure that the OpenvSwitch "crt_ovs_sw*.sh" scripts run only after networking has started. The idea of using the "my-network-up.sh" script for starting up the OpenvSwitches at boot after the network interfaces are up is thanks to Cheesehead here.
The "my-network-up.sh" script will be downloaded to /home/username/Downloads". Move the script to "/etc/init" as shown below and ensure it has correct permissions as shown below. After moving the file into "/etc/init" create a soft link to the file in the "/home/username/OpenvSwitch" directory to reference where this file is located for future configuration updates to find the file easily as shown below.
gstanden@u1504:~$ cd Downloads
gstanden@u1504:~/Downloads$ ls -lrt
total 4
-rw-rw-r-- 1 gstanden gstanden 1119 May 16 21:08 my-network-up.sh
gstanden@u1504:~/Downloads$ cd ..
gstanden@u1504:~$ cd Downloads
gstanden@u1504:~/Downloads$ ls -lrt
total 4
-rw-rw-r-- 1 gstanden gstanden 1119 May 16 21:08 my-network-up.sh
gstanden@u1504:~/Downloads$ more my-network-up.sh
# 'my-network-up.conf' - My custom upstart events
#
# These are the scripts that run when a network appears.
description "My custom upstart events"
start on net-device-up
task
script
/home/gstanden/OpenvSwitch/crt_ovs_sw1.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw1.log
/home/gstanden/OpenvSwitch/crt_ovs_sx1.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sx1.log
/home/gstanden/OpenvSwitch/crt_ovs_sw2.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw2.log
/home/gstanden/OpenvSwitch/crt_ovs_sw3.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw3.log
/home/gstanden/OpenvSwitch/crt_ovs_sw4.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw4.log
/home/gstanden/OpenvSwitch/crt_ovs_sw5.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw5.log
/home/gstanden/OpenvSwitch/crt_ovs_sw6.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw6.log
/home/gstanden/OpenvSwitch/crt_ovs_sw7.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw7.log
/home/gstanden/OpenvSwitch/crt_ovs_sw8.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw8.log
/home/gstanden/OpenvSwitch/crt_ovs_sw9.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw9.log
end script
gstanden@u1504:~/Downloads$ sudo mv my-network-up.sh /etc/init/.
gstanden@u1504:~/Downloads$ cd ..
gstanden@u1504:~$ cd OpenvSwitch
gstanden@u1504:~/OpenvSwitch$ sudo ln -s /etc/init/my-network-up.sh .
gstanden@u1504:~/OpenvSwitch$ ls -lrt
total 40
-rw-rw-r-- 1 gstanden gstanden 2153 May 16 21:05 crt_ovs_sw1.sh
-rw-rw-r-- 1 gstanden gstanden 1016 May 16 21:05 crt_ovs_sw2.sh
-rw-rw-r-- 1 gstanden gstanden 1019 May 16 21:05 crt_ovs_sw3.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:05 crt_ovs_sw4.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:05 crt_ovs_sw5.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:05 crt_ovs_sw6.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:06 crt_ovs_sw7.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:06 crt_ovs_sw8.sh
-rw-rw-r-- 1 gstanden gstanden 704 May 16 21:06 crt_ovs_sw9.sh
-rw-rw-r-- 1 gstanden gstanden 1935 May 16 21:06 crt_ovs_sx1.sh
lrwxrwxrwx 1 root root 26 May 16 21:11 my-network-up.sh -> /etc/init/my-network-up.sh
gstanden@u1504:~/OpenvSwitch$
gstanden@u1504:~/OpenvSwitch$ sudo chmod 755 /etc/init/my-network-up.sh
Update Ubuntu 15.04 2015-05-16
The my-network-up.sh script doesn't seem to be working anymore in Ubuntu 15.04 but it's (possibly ?) some mistake on my part. In any case, in the meantime, I had to add entries to /etc/rc.local script to get my OpenvSwitch network to startup ok at boot, so currently I still have the /etc/init/my-network-up.sh script in place, but also now the following in /etc/rc.local as well as shown below.
gstanden@u1504:~$ cat /etc/rc.local
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
/home/gstanden/OpenvSwitch/crt_ovs_sx1.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sx1.log
/home/gstanden/OpenvSwitch/crt_ovs_sw1.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw1.log
/home/gstanden/OpenvSwitch/crt_ovs_sw2.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw2.log
/home/gstanden/OpenvSwitch/crt_ovs_sw3.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw3.log
/home/gstanden/OpenvSwitch/crt_ovs_sw4.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw4.log
/home/gstanden/OpenvSwitch/crt_ovs_sw5.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw5.log
/home/gstanden/OpenvSwitch/crt_ovs_sw6.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw6.log
/home/gstanden/OpenvSwitch/crt_ovs_sw7.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw7.log
/home/gstanden/OpenvSwitch/crt_ovs_sw8.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw8.log
/home/gstanden/OpenvSwitch/crt_ovs_sw9.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw9.log
exit 0
gstanden@u1504:~$
Note also that crt_ovs_sx1.sh has been added. This is the addition of a new network subnet 10.207.29.1 and comes with a new set of bind9 and dhcp configurations. Basically, this update to the guide here shows how you add additional networks to the DNS-DHCP bind9/isc-dhcp-server setup on the laptop.
Update End Ubuntu 15.04 2015-05-16
Once the "crt_ovs_sw*.sh" scripts are downloaded, create a directory for them as shown below, and move the scripts to the "/home/username/OpenvSwitch" directory as shown below. Ensure that the "crt_ovs_sw*.sh" scripts have the correct ownership, permissions, and are installed in the directory as shown below. The log files do not need to be created. They are written automatically during each bootup of the laptop or desktop when the OpenvSwitch switches are created during bootup.
Create a soft link in the "/home/username/OpenvSwitch" directory as a reminder of where this important script is installed for future maintenance or reference. The idea is that all scripts directly relevant to the OpenvSwitch configuration are linked in the "/home/username/OpenvSwitch" directory for quick reference and accessability.
gstanden@u1504:~/OpenvSwitch$ chmod 755 crt_ovs_s*.sh
gstanden@u1504:~/OpenvSwitch$ ls -lrt
total 40
-rwxr-xr-x 1 gstanden gstanden 2153 May 16 21:05 crt_ovs_sw1.sh
-rwxr-xr-x 1 gstanden gstanden 1016 May 16 21:05 crt_ovs_sw2.sh
-rwxr-xr-x 1 gstanden gstanden 1019 May 16 21:05 crt_ovs_sw3.sh
-rwxr-xr-x 1 gstanden gstanden 704 May 16 21:05 crt_ovs_sw4.sh
-rwxr-xr-x 1 gstanden gstanden 704 May 16 21:05 crt_ovs_sw5.sh
-rwxr-xr-x 1 gstanden gstanden 704 May 16 21:05 crt_ovs_sw6.sh
-rwxr-xr-x 1 gstanden gstanden 704 May 16 21:06 crt_ovs_sw7.sh
-rwxr-xr-x 1 gstanden gstanden 704 May 16 21:06 crt_ovs_sw8.sh
-rwxr-xr-x 1 gstanden gstanden 704 May 16 21:06 crt_ovs_sw9.sh
-rwxr-xr-x 1 gstanden gstanden 1935 May 16 21:06 crt_ovs_sx1.sh
lrwxrwxrwx 1 root root 26 May 16 21:11 my-network-up.sh -> /etc/init/my-network-up.sh
gstanden@u1504:~/OpenvSwitch$
The OpenvSwitch switch scripts are also listed below for reference.
gstanden@u1504:~/OpenvSwitch$ more crt_ovs_s*.sh
::::::::::::::
crt_ovs_sw1.sh
::::::::::::::
#!/bin/bash
# Requires use of Upstart Script /etc/init/my-network-up.conf to ensure interfaces are up before running.
tunctl -t s1
tunctl -t s2
tunctl -t s3
tunctl -t s4
tunctl -t s5
#tunctl -t s6
ip link set s1 up
ip link set s2 up
ip link set s3 up
ip link set s4 up
ip link set s5 up
#ip link set s6 up
ovs-vsctl add-br sw1
ovs-vsctl add-port sw1 s1
ovs-vsctl add-port sw1 s2
ovs-vsctl add-port sw1 s3
ovs-vsctl add-port sw1 s4
ovs-vsctl add-port sw1 s5
#ovs-vsctl add-port sw1 s6
ip link set up dev sw1
ip addr add 10.207.39.1/24 dev sw1
ip route replace 10.207.39.0/24 dev sw1
ovs-vsctl set port sw1 trunks=10
ovs-vsctl set port sw1 tag=10
# GLS 20140825 Get active external interface dynamically at boot. Tested & works with {wlan0, eth0, bnep0} on Ubuntu 14.04.1 Desktop x86_64.
# GLS 20140825 Interface "bnep0" is Blackberry Z30 OS10 Bluetooth Tether.
### BEGIN Get Active EXTIF Dynamcially. ###
function GetInterface
{
ifconfig | egrep -B1 'inet addr' | egrep -A1 'wlan|eth|bnep' | sed '$!N;s/\n/ /' | sed 's/ */ /g' | cut -f1,7 -d' ' | sed 's/ addr//' | head -1 | cut -f1 -d':'
}
function GetIP
{
ifconfig | egrep -B1 'inet addr' | egrep -A1 'wlan|eth|bnep' | sed '$!N;s/\n/ /' | sed 's/ */ /g' | cut -f1,7 -d' ' | sed 's/ addr//' | head -1 | cut -f2 -d':'
}
### END Get Active EXTIF Dynamically. ###
echo ' IP: '$(GetIP)
echo 'Interface: '$(GetInterface)
INTIF="sw1"
EXTIF=$(GetInterface)
# EXTIF="wlan0"
echo 1 > /proc/sys/net/ipv4/ip_forward
# clear existing iptable rules, set a default policy
iptables -P INPUT ACCEPT
iptables -F INPUT
iptables -P OUTPUT ACCEPT
iptables -F OUTPUT
iptables -P FORWARD DROP
iptables -F FORWARD
iptables -t nat -F
# set forwarding and nat rules
# iptables -A FORWARD -i $EXTIF -o $INTIF -j ACCEPT
# iptables -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT
# iptables -t nat -A POSTROUTING -o $EXTIF -j MASQUERADE
iptables -A FORWARD -s 10.207.39.0/24 -o $EXTIF -j ACCEPT
iptables -A FORWARD -d 10.207.39.0/24 -m state --state ESTABLISHED,RELATED -i $EXTIF -j ACCEPT
iptables -t nat -A POSTROUTING -s 10.207.39.0/24 -o $EXTIF -j MASQUERADE
service isc-dhcp-server start
service bind9 restart
::::::::::::::
crt_ovs_sw2.sh
::::::::::::::
#!/bin/bash
tunctl -t t1
tunctl -t t2
tunctl -t t3
tunctl -t t4
tunctl -t t5
ip link set t1 up
ip link set t2 up
ip link set t3 up
ip link set t4 up
ip link set t5 up
ovs-vsctl add-br sw2
ovs-vsctl add-port sw2 t1
ovs-vsctl add-port sw2 t2
ovs-vsctl add-port sw2 t3
ovs-vsctl add-port sw2 t4
ovs-vsctl add-port sw2 t5
ip link set up dev sw2
ip addr add 10.207.40.1/24 dev sw2
ip route replace 10.207.40.0/24 dev sw2
ifconfig sw2 10.207.40.1 netmask 255.255.255.0
ovs-vsctl set port sw2 tag=80
# INTIF="sw2"
# EXTIF="wlan0"
# echo 1 > /proc/sys/net/ipv4/ip_forward
# clear existing iptable rules, set a default policy
# iptables -P INPUT ACCEPT
# iptables -F INPUT
# iptables -P OUTPUT ACCEPT
# iptables -F OUTPUT
# iptables -P FORWARD DROP
# iptables -F FORWARD
# iptables -t nat -F
# set forwarding and nat rules
# iptables -A FORWARD -i $EXTIF -o $INTIF -j ACCEPT
# iptables -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT
# iptables -t nat -A POSTROUTING -o $EXTIF -j MASQUERADE
# service isc-dhcp-server start
::::::::::::::
crt_ovs_sw3.sh
::::::::::::::
#!/bin/bash
tunctl -t w1
tunctl -t w2
tunctl -t w3
tunctl -t w4
tunctl -t w5
ip link set w1 up
ip link set w2 up
ip link set w3 up
ip link set w4 up
ip link set w5 up
ovs-vsctl add-br sw3
ovs-vsctl add-port sw3 w1
ovs-vsctl add-port sw3 w2
ovs-vsctl add-port sw3 w3
ovs-vsctl add-port sw3 w4
ovs-vsctl add-port sw3 w5
ip link set up dev sw3
ip addr add 10.207.41.1/24 dev sw3
ip route replace 10.207.41.0/24 dev sw3
ifconfig sw3 10.207.41.1 netmask 255.255.255.0
ovs-vsctl set port sw3 tag=90
# INTIF="sw3"
# EXTIF="wlan0"
# echo 1 > /proc/sys/net/ipv4/ip_forward
# clear existing iptable rules, set a default policy
# iptables -P INPUT ACCEPT
# iptables -F INPUT
# iptables -P OUTPUT ACCEPT
# iptables -F OUTPUT
# iptables -P FORWARD DROP
# iptables -F FORWARD
# iptables -t nat -F
# set forwarding and nat rules
# iptables -A FORWARD -i $EXTIF -o $INTIF -j ACCEPT
# iptables -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT
# iptables -t nat -A POSTROUTING -o $EXTIF -j MASQUERADE
# service isc-dhcp-server start
::::::::::::::
crt_ovs_sw4.sh
::::::::::::::
#!/bin/bash
ovs-vsctl add-br sw4
ip link set up dev sw4
ip addr add 192.210.39.1/24 dev sw4
ip route replace 192.210.39.0/24 dev sw4
ifconfig sw4 192.210.39.1 netmask 255.255.255.0
# INTIF="sw3"
# EXTIF="wlan0"
# echo 1 > /proc/sys/net/ipv4/ip_forward
# clear existing iptable rules, set a default policy
# iptables -P INPUT ACCEPT
# iptables -F INPUT
# iptables -P OUTPUT ACCEPT
# iptables -F OUTPUT
# iptables -P FORWARD DROP
# iptables -F FORWARD
# iptables -t nat -F
# set forwarding and nat rules
# iptables -A FORWARD -i $EXTIF -o $INTIF -j ACCEPT
# iptables -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT
# iptables -t nat -A POSTROUTING -o $EXTIF -j MASQUERADE
# service isc-dhcp-server start
::::::::::::::
crt_ovs_sw5.sh
::::::::::::::
#!/bin/bash
ovs-vsctl add-br sw5
ip link set up dev sw5
ip addr add 192.211.39.1/24 dev sw5
ip route replace 192.211.39.0/24 dev sw5
ifconfig sw5 192.211.39.1 netmask 255.255.255.0
# INTIF="sw3"
# EXTIF="wlan0"
# echo 1 > /proc/sys/net/ipv4/ip_forward
# clear existing iptable rules, set a default policy
# iptables -P INPUT ACCEPT
# iptables -F INPUT
# iptables -P OUTPUT ACCEPT
# iptables -F OUTPUT
# iptables -P FORWARD DROP
# iptables -F FORWARD
# iptables -t nat -F
# set forwarding and nat rules
# iptables -A FORWARD -i $EXTIF -o $INTIF -j ACCEPT
# iptables -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT
# iptables -t nat -A POSTROUTING -o $EXTIF -j MASQUERADE
# service isc-dhcp-server start
::::::::::::::
crt_ovs_sw6.sh
::::::::::::::
#!/bin/bash
ovs-vsctl add-br sw6
ip link set up dev sw6
ip addr add 192.212.39.1/24 dev sw6
ip route replace 192.212.39.0/24 dev sw6
ifconfig sw6 192.212.39.1 netmask 255.255.255.0
# INTIF="sw3"
# EXTIF="wlan0"
# echo 1 > /proc/sys/net/ipv4/ip_forward
# clear existing iptable rules, set a default policy
# iptables -P INPUT ACCEPT
# iptables -F INPUT
# iptables -P OUTPUT ACCEPT
# iptables -F OUTPUT
# iptables -P FORWARD DROP
# iptables -F FORWARD
# iptables -t nat -F
# set forwarding and nat rules
# iptables -A FORWARD -i $EXTIF -o $INTIF -j ACCEPT
# iptables -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT
# iptables -t nat -A POSTROUTING -o $EXTIF -j MASQUERADE
# service isc-dhcp-server start
::::::::::::::
crt_ovs_sw7.sh
::::::::::::::
#!/bin/bash
ovs-vsctl add-br sw7
ip link set up dev sw7
ip addr add 192.213.39.1/24 dev sw7
ip route replace 192.213.39.0/24 dev sw7
ifconfig sw7 192.213.39.1 netmask 255.255.255.0
# INTIF="sw3"
# EXTIF="wlan0"
# echo 1 > /proc/sys/net/ipv4/ip_forward
# clear existing iptable rules, set a default policy
# iptables -P INPUT ACCEPT
# iptables -F INPUT
# iptables -P OUTPUT ACCEPT
# iptables -F OUTPUT
# iptables -P FORWARD DROP
# iptables -F FORWARD
# iptables -t nat -F
# set forwarding and nat rules
# iptables -A FORWARD -i $EXTIF -o $INTIF -j ACCEPT
# iptables -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT
# iptables -t nat -A POSTROUTING -o $EXTIF -j MASQUERADE
# service isc-dhcp-server start
::::::::::::::
crt_ovs_sw8.sh
::::::::::::::
#!/bin/bash
ovs-vsctl add-br sw8
ip link set up dev sw8
ip addr add 172.220.40.1/24 dev sw8
ip route replace 172.220.40.0/24 dev sw8
ifconfig sw8 172.220.40.1 netmask 255.255.255.0
# INTIF="sw3"
# EXTIF="wlan0"
# echo 1 > /proc/sys/net/ipv4/ip_forward
# clear existing iptable rules, set a default policy
# iptables -P INPUT ACCEPT
# iptables -F INPUT
# iptables -P OUTPUT ACCEPT
# iptables -F OUTPUT
# iptables -P FORWARD DROP
# iptables -F FORWARD
# iptables -t nat -F
# set forwarding and nat rules
# iptables -A FORWARD -i $EXTIF -o $INTIF -j ACCEPT
# iptables -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT
# iptables -t nat -A POSTROUTING -o $EXTIF -j MASQUERADE
# service isc-dhcp-server start
::::::::::::::
crt_ovs_sw9.sh
::::::::::::::
#!/bin/bash
ovs-vsctl add-br sw9
ip link set up dev sw9
ip addr add 172.221.40.1/24 dev sw9
ip route replace 172.221.40.0/24 dev sw9
ifconfig sw9 172.221.40.1 netmask 255.255.255.0
# INTIF="sw3"
# EXTIF="wlan0"
# echo 1 > /proc/sys/net/ipv4/ip_forward
# clear existing iptable rules, set a default policy
# iptables -P INPUT ACCEPT
# iptables -F INPUT
# iptables -P OUTPUT ACCEPT
# iptables -F OUTPUT
# iptables -P FORWARD DROP
# iptables -F FORWARD
# iptables -t nat -F
# set forwarding and nat rules
# iptables -A FORWARD -i $EXTIF -o $INTIF -j ACCEPT
# iptables -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT
# iptables -t nat -A POSTROUTING -o $EXTIF -j MASQUERADE
# service isc-dhcp-server start
::::::::::::::
crt_ovs_sx1.sh
::::::::::::::
#!/bin/bash
# Requires use of Upstart Script /etc/init/my-network-up.conf to ensure interfaces are up before running.
sleep 10
tunctl -t a1
tunctl -t a2
tunctl -t a3
tunctl -t a4
tunctl -t a5
#tunctl -t a6
ip link set a1 up
ip link set a2 up
ip link set a3 up
ip link set a4 up
ip link set a5 up
#ip link set a6 up
ovs-vsctl add-br sx1
ovs-vsctl add-port sx1 a1
ovs-vsctl add-port sx1 a2
ovs-vsctl add-port sx1 a3
ovs-vsctl add-port sx1 a4
ovs-vsctl add-port sx1 a5
#ovs-vsctl add-port sx1 a6
ip link set up dev sx1
ip addr add 10.207.29.1/24 dev sx1
ip route replace 10.207.29.0/24 dev sx1
ovs-vsctl set port sx1 trunks=10
ovs-vsctl set port sx1 tag=10
# GLS 20140825 Get active external interface dynamically at boot. Tested & works with {wlan0, eth0, bnep0} on Ubuntu 14.04.1 Desktop x86_64.
# GLS 20140825 Interface "bnep0" is Blackberry Z30 OS10 Bluetooth Tether.
### BEGIN Get Active EXTIF Dynamcially. ###
function GetInterface
{
ifconfig | egrep -B1 'inet addr' | egrep -A1 'wlan|eth|bnep' | sed '$!N;s/\n/ /' | sed 's/ */ /g' | cut -f1,7 -d' ' | sed 's/ addr//' | head -1 | cut -f1 -d':'
}
function GetIP
{
ifconfig | egrep -B1 'inet addr' | egrep -A1 'wlan|eth|bnep' | sed '$!N;s/\n/ /' | sed 's/ */ /g' | cut -f1,7 -d' ' | sed 's/ addr//' | head -1 | cut -f2 -d':'
}
### END Get Active EXTIF Dynamically. ###
echo ' IP: '$(GetIP)
echo 'Interface: '$(GetInterface)
INTIF="sx1"
EXTIF=$(GetInterface)
# EXTIF="wlan0"
echo 1 > /proc/sys/net/ipv4/ip_forward
# clear existing iptable rules, set a default policy
iptables -P INPUT ACCEPT
iptables -F INPUT
iptables -P OUTPUT ACCEPT
iptables -F OUTPUT
iptables -P FORWARD DROP
iptables -F FORWARD
iptables -t nat -F
# set forwarding and nat rules
iptables -A FORWARD -i $EXTIF -o $INTIF -j ACCEPT
iptables -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT
iptables -t nat -A POSTROUTING -o $EXTIF -j MASQUERADE
# service isc-dhcp-server restart
# service bind9 restart
gstanden@u1504:~/OpenvSwitch$
Setup DNS and DHCP for System
The DNS and DHCP are setup in this blog so that DHCP can provide addresses to LXC containers and VMs on the OpenvSwitch, and also add those DHCP assignments to DNS automatically so that as containers and KVM guests are created, they get handed out IP addresses, and those IP addresses get automatically added to the fwd and reverse lookup files without having to edit them manually at all.
Backup DNS Default Configuration Files and Install Downloaded Versions
First backup the default configuration files as shown below. Then download from the end of this blog this files "named.conf.options" and "named.conf.local" and move to "/etc/bind" directory as shown below.
gstanden@u1504:~$ sudo su -
root@u1504:~# cd /etc/bind
root@u1504:/etc/bind# cp -p named.conf.local named.conf.local.original.install.bak
root@u1504:/etc/bind# cp -p named.conf.options named.conf.options.original.install.bak
root@u1504:/etc/bind# cp -p rndc.key rndc.key.original.install.bak
root@u1504:/etc/bind# ls -lrt
total 64
-rw-r--r-- 1 root root 1317 Feb 19 11:07 zones.rfc1918
-rw-r--r-- 1 root bind 165 Feb 19 11:07 named.conf.local.original.install.bak
-rw-r--r-- 1 root bind 165 Feb 19 11:07 named.conf.local
-rw-r--r-- 1 root bind 490 Feb 19 11:07 named.conf.default-zones
-rw-r--r-- 1 root bind 463 Feb 19 11:07 named.conf
-rw-r--r-- 1 root root 3048 Feb 19 11:07 db.root
-rw-r--r-- 1 root root 270 Feb 19 11:07 db.local
-rw-r--r-- 1 root root 353 Feb 19 11:07 db.empty
-rw-r--r-- 1 root root 237 Feb 19 11:07 db.255
-rw-r--r-- 1 root root 271 Feb 19 11:07 db.127
-rw-r--r-- 1 root root 237 Feb 19 11:07 db.0
-rw-r--r-- 1 root root 2389 Feb 19 11:07 bind.keys
-rw-r----- 1 bind bind 77 May 16 21:03 rndc.key.original.install.bak
-rw-r----- 1 bind bind 77 May 16 21:03 rndc.key
-rw-r--r-- 1 root bind 890 May 16 21:03 named.conf.options.original.install.bak
-rw-r--r-- 1 root bind 890 May 16 21:03 named.conf.options
root@u1504:/etc/bind# cd
root@u1504:~# exit
logout
gstanden@u1504:~$ cd Downloads
gstanden@u1504:~/Downloads$ ls -lrt
total 8
-rw-rw-r-- 1 gstanden gstanden 435 May 16 21:33 named.conf.local
-rw-rw-r-- 1 gstanden gstanden 299 May 16 21:33 named.conf.options
gstanden@u1504:~/Downloads$ sudo mv named.* /etc/bind/.
gstanden@u1504:~/Downloads$
The new named.conf.local file is shown below.
gstanden@u1504:~/Downloads$ more named.conf.local
//
// Do any local configuration here
//
// Consider adding the 1918 zones here, if they are not used in your
// organization
include "/etc/bind/zones.rfc1918";
include "/etc/bind/rndc.key";
zone "vmem.org" {
type master;
file "/var/lib/bind/fwd.vmem.org";
allow-update { key rndc-key; };
};
zone "mccc.org" {
type master;
file "/var/lib/bind/fwd.mccc.org";
allow-update { key rndc-key; };
};
zone "39.207.10.in-addr.arpa" {
type master;
file "/var/lib/bind/rev.vmem.org";
allow-update { key rndc-key; };
};
zone "29.207.10.in-addr.arpa" {
type master;
file "/var/lib/bind/rev.mccc.org";
allow-update { key rndc-key; };
};
gstanden@u1504:~/Downloads$
The named.conf.options file is shown below. Download it from this blog and move it to /etc/bind directory. The dhcp files are also shown here.
gstanden@u1504:~/Downloads$ sudo mv named.conf.local /etc/bind/.
gstanden@u1504:~/Downloads$ sudo mv named.conf.options /etc/bind/.
gstanden@u1504:~/Downloads$ cat /etc/dhcp/dhclient.conf
# Configuration file for /sbin/dhclient, which is included in Debian's
# dhcp3-client package.
#
# This is a sample configuration file for dhclient. See dhclient.conf's
# man page for more information about the syntax of this file
# and a more comprehensive list of the parameters understood by
# dhclient.
#
# Normally, if the DHCP server provides reasonable information and does
# not leave anything out (like the domain name, for example), then
# few changes must be made to this file, if any.
#
option rfc3442-classless-static-routes code 121 = array of unsigned integer 8;
send host-name = gethostname();
prepend domain-name-servers 127.0.0.1;
# Oracle GNS and additional domains
append domain-name " mccc.org";
request subnet-mask, broadcast-address, time-offset, routers,
domain-name, domain-name-servers, domain-search, host-name,
dhcp6.name-servers, dhcp6.domain-search,
netbios-name-servers, netbios-scope, interface-mtu,
rfc3442-classless-static-routes, ntp-servers,
dhcp6.fqdn, dhcp6.sntp-servers;
gstanden@u1504:~/Downloads$ cat /etc/dhcp/dhcpd.conf
#
# Configuration file for ISC dhcpd for Ubuntu 14.04
# Configuration file for ISC dhcpd for Ubuntu 15.04 GLS Tested on 15.04 2015.05.02
# Added a second subnet configuration (10.207.29.1/24) for mediacomcorp.org domain
#
ddns-updates on;
ddns-update-style interim;
update-static-leases on;
authoritative;
key rndc-key { algorithm hmac-md5; secret "5rcWKMkWCP6RBUeu9tjIZg==";}
allow unknown-clients;
use-host-decl-names on;
default-lease-time 1814400;
max-lease-time 1814400;
log-facility local7;
zone vmem.org. {
primary 10.207.39.1;
key rndc-key;
}
zone mccc.org. {
primary 10.207.29.1;
key rndc-key;
}
zone 39.207.10.in-addr.arpa. {
primary 10.207.39.1;
key rndc-key;
}
zone 29.207.10.in-addr.arpa. {
primary 10.207.29.1;
key rndc-key;
}
subnet 10.207.39.0 netmask 255.255.255.0 {
# --- default gateway
option routers 10.207.39.1;
option subnet-mask 255.255.255.0;
option domain-name "vmem.org";
option domain-name-servers 10.207.39.1;
# option ntp-servers 10.207.39.1;
ddns-domainname "vmem.org.";
ddns-rev-domainname "in-addr.arpa.";
range 10.207.39.70 10.207.39.254;
default-lease-time 1814400;
max-lease-time 1814400;
}
subnet 10.207.29.0 netmask 255.255.255.0 {
# --- default gateway
option routers 10.207.29.1;
option subnet-mask 255.255.255.0;
option domain-name "mccc.org";
option domain-name-servers 10.207.29.1;
# option ntp-servers 10.207.29.1;
ddns-domainname "mccc.org.";
ddns-rev-domainname "in-addr.arpa.";
range 10.207.29.70 10.207.29.254;
default-lease-time 1814400;
max-lease-time 1814400;
}
gstanden@u1504:~/Downloads$
Note that these are the updated files to support multiple domains and multiple networks on the desktop. Also, the /etc/NetworkManager/dnsmasq.d/local file is shown below. This file tells dnsmasq not to listen on these subnets and allow named to manage these subnets exclusively.
gstanden@u1504:~/dnsmasq.d$ more local
server=/vmem.org/10.207.39.1
server=/39.207.10.in-addr.arpa/10.207.39.1
server=/mccc.org/10.207.29.1
server=/29.207.10.in-addr.arpa/10.207.29.1
gstanden@u1504:~/dnsmasq.d$
Backup DHCP Default Configuration Files
Now backup the isc-dhcp-server (DHCP) original install configuration files as shown below. With the addition of extra subnets and domains, dhclient.conf has some changes from default, because it has been used to add search domains to /etc/resolv.conf at boot, so both dhclient.conf and dhcpd.conf are changed from default as shown below.
gstanden@u1504:~/Downloads$ sudo su -
root@u1504:~# cd /etc/dhcp
root@u1504:/etc/dhcp# ls -lrt
total 20
-rw-r--r-- 1 root root 3641 Nov 5 2014 dhcpd.conf
-rw-r--r-- 1 root root 1830 Nov 5 2014 dhclient.conf
drwxr-xr-x 2 root root 4096 May 16 20:51 dhclient-enter-hooks.d
drwxr-xr-x 2 root root 4096 May 16 20:51 dhclient-exit-hooks.d
drwxr-x--- 2 root dhcpd 4096 May 16 21:04 ddns-keys
root@u1504:/etc/dhcp# cp -p dhcpd.conf dhcpd.conf.original.install.bak
root@u1504:/etc/dhcp# cp -p dhclient.conf dhclient.conf.original.install.bak
root@u1504:/etc/dhcp# exit
logout
gstanden@u1504:~/Downloads$ ls -lrt
total 8
-rw-rw-r-- 1 gstanden gstanden 1019 May 16 21:39 dhclient.conf
-rw-rw-r-- 1 gstanden gstanden 1596 May 16 21:39 dhcpd.conf
gstanden@u1504:~/Downloads$ sudo cp -p dh*.conf /etc/dhcp/.
gstanden@u1504:~/Downloads$ cd /etc/dhcp
gstanden@u1504:/etc/dhcp$ ls -lrt
total 28
-rw-r--r-- 1 root root 3641 Nov 5 2014 dhcpd.conf.original.install.bak
-rw-r--r-- 1 root root 1830 Nov 5 2014 dhclient.conf.original.install.bak
drwxr-xr-x 2 root root 4096 May 16 20:51 dhclient-enter-hooks.d
drwxr-xr-x 2 root root 4096 May 16 20:51 dhclient-exit-hooks.d
drwxr-x--- 2 root dhcpd 4096 May 16 21:04 ddns-keys
-rw-rw-r-- 1 gstanden gstanden 1019 May 16 21:39 dhclient.conf
-rw-rw-r-- 1 gstanden gstanden 1596 May 16 21:39 dhcpd.conf
gstanden@u1504:/etc/dhcp$
Update Begin Ubuntu 15.04 2015-05-16
There is a change to the /etc/dhcp/dhclient.conf file for the added mccc.org domain as shown below. This is set now in /etc/network/interfaces file, not here anymore.
gstanden@u1504:/etc/dhcp$ cat dhclient.conf
# Configuration file for /sbin/dhclient, which is included in Debian's
# dhcp3-client package.
#
# This is a sample configuration file for dhclient. See dhclient.conf's
# man page for more information about the syntax of this file
# and a more comprehensive list of the parameters understood by
# dhclient.
#
# Normally, if the DHCP server provides reasonable information and does
# not leave anything out (like the domain name, for example), then
# few changes must be made to this file, if any.
#
option rfc3442-classless-static-routes code 121 = array of unsigned integer 8;
send host-name = gethostname();
prepend domain-name-servers 127.0.0.1;
# Oracle GNS and additional domains
# append domain-name " mccc.org"; <-- this is set in /etc/network/interfaces now, not using this line anymore.
request subnet-mask, broadcast-address, time-offset, routers,
domain-name, domain-name-servers, domain-search, host-name,
dhcp6.name-servers, dhcp6.domain-search,
netbios-name-servers, netbios-scope, interface-mtu,
rfc3442-classless-static-routes, ntp-servers,
dhcp6.fqdn, dhcp6.sntp-servers;
gstanden@u1504:/etc/dhcp$
Install Downloaded DHCP Files
Move the downloaded "dhcpd.conf" to "/etc/dhcp" directory as shown below and ensure it has the correct value from /etc/bind/rndc.key file as shown below..
gstanden@u1504:/etc/init$ sudo cat /etc/bind/rndc.key
[sudo] password for gstanden:
key "rndc-key" {
algorithm hmac-md5;
secret "FLf9C0hIeYhEeE23lfN1Mw=="; <-- Be sure to copy this to /etc/dhcp/dhcpd.conf file as shown below.
};
gstanden@u1504:/etc/init$ cat /etc/dhcp/dhcpd.conf
#
# Configuration file for ISC dhcpd for Ubuntu 14.04
# Configuration file for ISC dhcpd for Ubuntu 15.04 GLS Tested on 15.04 2015.05.02
# Added a second subnet configuration (10.207.29.1/24) for mediacomcorp.org domain
#
ddns-updates on;
ddns-update-style interim;
update-static-leases on;
authoritative;
key rndc-key { algorithm hmac-md5; secret "FLf9C0hIeYhEeE23lfN1Mw==";}
allow unknown-clients;
use-host-decl-names on;
default-lease-time 1814400;
max-lease-time 1814400;
log-facility local7;
zone vmem.org. {
primary 10.207.39.1;
key rndc-key;
}
zone mccc.org. {
primary 10.207.29.1;
key rndc-key;
}
zone 39.207.10.in-addr.arpa. {
primary 10.207.39.1;
key rndc-key;
}
zone 29.207.10.in-addr.arpa. {
primary 10.207.29.1;
key rndc-key;
}
subnet 10.207.39.0 netmask 255.255.255.0 {
# --- default gateway
option routers 10.207.39.1;
option subnet-mask 255.255.255.0;
option domain-name "vmem.org";
option domain-name-servers 10.207.39.1;
# option ntp-servers 10.207.39.1;
ddns-domainname "vmem.org.";
ddns-rev-domainname "in-addr.arpa.";
range 10.207.39.70 10.207.39.254;
default-lease-time 1814400;
max-lease-time 1814400;
}
subnet 10.207.29.0 netmask 255.255.255.0 {
# --- default gateway
option routers 10.207.29.1;
option subnet-mask 255.255.255.0;
option domain-name "mccc.org";
option domain-name-servers 10.207.29.1;
# option ntp-servers 10.207.29.1;
ddns-domainname "mccc.org.";
ddns-rev-domainname "in-addr.arpa.";
range 10.207.29.70 10.207.29.254;
default-lease-time 1814400;
max-lease-time 1814400;
}
gstanden@u1504:/etc/init$
Set Ownership and Permissions of DHCP Configuration Files
Change ownership of dhcp configuration files to root as shown below.
root@W520:/etc/dhcp# chown root:root dhclient.conf dhcpd.conf
root@W520:/etc/dhcp# ls -lrt *.conf
total 28
-rw-rw-r-- 1 root root 940 Dec 26 10:15 dhclient.conf
-rw-rw-r-- 1 root root 890 Dec 26 10:25 dhcpd.conf
root@W520:/etc/dhcp#
Install DNS Forward and Reverse Zone Configuration Files
Download the "fwd.vmem.org" and the "rev.vmem.org" zone files from the downloadable files at the end of this blog, and also the fwd.mccc.org and the rev.mccc.org files. Install the downloaded zone files into correct location as shown below. Download the "named.conf.options" and the "named.conf.local" file from the downloadable files at the end of the blog and install.
gstanden@u1504:~$ cd Downloads
gstanden@u1504:~/Downloads$ ls -lrt
total 264
-rw-rw-r-- 1 gstanden gstanden 250832 Dec 25 22:54 rev.vmem.org
-rw-rw-r-- 1 gstanden gstanden 1096 Dec 26 10:15 fwd.vmem.org
-rw-rw-r-- 1 gstanden gstanden 435 Dec 26 10:15 fwd.mccc.org
-rw-rw-r-- 1 gstanden gstanden 299 Dec 26 10:15 rev.mccc.org
gstanden@u1504:~/Downloads$ sudo mv fwd.*.org /var/lib/bind/.
gstanden@u1504:~/Downloads$ sudo mv rev.*.org /var/lib/bind/.
Edit DNS Forward and Reverse Zone Files
Setup the forward and reverse zone files. Change the domain name information if desired. Be careful when editing not to remove any of the "." characters in the file, they matter! Also, verify the hostname as shown and make sure to update the forward and reverse zone files with the name of your desktop host (in this example the desktop hostname is "u1504"). When installing the downloaded forward and reverse lookup files "fwd.vmem.org" and "rev.vmem.org" it will be necessary to change the "u1504" to the name of the actual Ubuntu desktop system. Also, the domain can be changed from "vmem.org" to some-other-domain.[org|com|net|biz|...]" if desired. Note that if it is changed to something else, changes will need to be made in all the other corresponding files, such as "/etc/network/interfaces", "named.conf.local" etc.
gstanden@u1504:~/Downloads$ sudo su -
root@u1504:~# cd /var/lib/bind
root@u1504:/var/lib/bind# ls -lrt
total 20
-rw-r--r-- 1 root root 53 May 16 21:03 bind9-default.md5sum
-rw-rw-r-- 1 gstanden gstanden 394 May 17 21:18 fwd.mccc.org
-rw-rw-r-- 1 gstanden gstanden 342 May 17 21:18 rev.mccc.org
-rw-rw-r-- 1 gstanden gstanden 506 May 17 21:20 fwd.vmem.org
-rw-rw-r-- 1 gstanden gstanden 456 May 17 21:21 rev.vmem.org
root@u1504:/var/lib/bind# named-checkzone mccc.org fwd.mccc.org
zone mccc.org/IN: loaded serial 1505021309
OK
root@u1504:/var/lib/bind# named-checkzone 29.207.10.in-addr.arpa rev.mccc.org
zone 29.207.10.in-addr.arpa/IN: loaded serial 1411021420
OK
root@u1504:/var/lib/bind# named-checkzone 39.207.10.in-addr.arpa rev.vmem.org
zone 39.207.10.in-addr.arpa/IN: loaded serial 1412261100
OK
root@u1504:/var/lib/bind# named-checkzone vmem.org fwd.vmem.org
zone vmem.org/IN: loaded serial 1412261100
OK
root@u1504:/var/lib/bind# cat fwd.vmem.org
$ORIGIN .
$TTL 86400 ; time-to-live in seconds
vmem.org IN SOA u1504.vmem.org. postmaster.vmem.org. (
1412261100 ; serial
60 ; refresh (1 minute)
1800 ; retry (30 minutes)
604800 ; expire (1 week)
86400 ; minimum (1 day)
)
NS u1504.vmem.org.
$ORIGIN vmem.org.
_sflow._udp TXT "txtvers=1" "polling=20" "sampling=512"
SRV 0 0 6343 u1504
u1504 A 10.207.39.1
lxc1-gns-vip.vmem.org. A 10.207.39.3
$ORIGIN gns1.vmem.org.
@ IN NS lxc1-gns-vip.vmem.org.
root@u1504:/var/lib/bind# cat rev.vmem.org
$ORIGIN .
$TTL 86400 ; time-to-live in seconds
39.207.10.in-addr.arpa IN SOA u1504.vmem.org. postmaster.vmem.org. (
1412261100 ; serial
60 ; refresh (1 hour)
1800 ; retry (30 minutes)
604800 ; expire (1 week)
86400 ; minimum (1 day)
)
NS u1504.vmem.org.
$ORIGIN 39.207.10.in-addr.arpa.
1 PTR u1504.vmem.org.
3 PTR lxc1-gns-vip.vmem.org.
39.207.10.in-addr.arpa NS u1504.vmem.org.
W520 A 10.207.39.1
root@u1504:/var/lib/bind# cat fwd.mccc.org
$ORIGIN .
$TTL 86400 ; 1 day
mccc.org IN SOA mccc1.mccc.org. postmaster.mccc.org. (
1505021309 ; serial
60 ; refresh (1 minute)
1800 ; retry (30 minutes)
604800 ; expire (1 week)
86400 ; minimum (1 day)
)
NS mccc1.mccc.org.
$ORIGIN mccc.org.
_sflow._udp TXT "txtvers=1" "polling=20" "sampling=512"
SRV 0 0 6343 mccc1
mccc1 A 10.207.29.1
root@u1504:/var/lib/bind# cat rev.mccc.org
$ORIGIN .
$TTL 86400 ; 1 day
29.207.10.in-addr.arpa IN SOA mccc1.mccc.org. postmaster.mccc.org. (
1411021420 ; serial
3600 ; refresh (1 hour)
1800 ; retry (30 minutes)
604800 ; expire (1 week)
86400 ; minimum (1 day)
)
NS mccc1.mccc.org.
$ORIGIN 29.207.10.in-addr.arpa.
1 PTR mccc1.mccc.org.
root@u1504:/var/lib/bind#
Configure dnsmasq-base
Now configure dnsmasq-base ("NetworkManager" component) to IGNORE DNS requests for the local-only vmem.org domain as shown below. In other words, this step puts the local domain "vmem.org" under the control of bind9, and tells dnsmasq-base to ignore this domain, and also to ignore mccc.org domain as well. The additonal lines in bold are for the added mccc.org domain, and additonal domains can be added with similar two-line additions of the same form.
root@u1504:/var/lib/bind# cat /etc/NetworkManager/dnsmasq.d/local
server=/vmem.org/10.207.39.1
server=/39.207.10.in-addr.arpa/10.207.39.1
server=/mccc.org/10.207.29.1
server=/29.207.10.in-addr.arpa/10.207.29.1
root@u1504:/var/lib/bind#
Configure /etc/network/interfaces
Ensure that "/etc/network/interfaces" file has the following three lines added after "iface" line. These lines are responsible for configuring the "/etc/resolv.conf" file. Note that "mccc.org" is added to /etc/resolv.conf here. In earlier versions it had been added at /etc/dhcp/dhclient.conf file, but that method is not used here; all domains are added here at /etc/network/interfaces file.
gstanden@u1504:~$ cat /etc/network/interfaces
# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback
dns-domain vmem.org
dns-search mccc.org gns1.vmem.org
dns-nameserver 127.0.0.1
gstanden@u1504:~$ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1
search vmem.org mccc.org gns1.vmem.org <-- /etc/network/interfaces above produces this /etc/resolv.conf file
gstanden@u1504:~$
At this point, the DNS-DHCP should be configured. Verify by restarting bind9 and isc-dhcp-server as shown below and testing lookups on both domains and subnets, both forward and reverse lookups, and verify that /etc/resolv.conf file has the desired search and domain settings. Note that "vmem.org" is the domain for the desktop, and that "mccc.org" is an added "search domain". There would typically be some LXC Linux Containers, or KVM guests that might be on this "mccc.org" domain, but that the domain of the desktop Ubuntu system is "vmem.org".
root@u1504:/var/lib/bind# service bind9 restart
root@u1504:/var/lib/bind# service bind9 status
â—Ź bind9.service - BIND Domain Name Server
Loaded: loaded (/lib/systemd/system/bind9.service; enabled; vendor preset: enabled)
Drop-In: /run/systemd/generator/bind9.service.d
└─50-insserv.conf-$named.conf
Active: active (running) since Sun 2015-05-17 21:38:34 CDT; 3s ago
Docs: man:named(8)
Process: 5353 ExecStop=/usr/sbin/rndc stop (code=exited, status=0/SUCCESS)
Main PID: 5358 (named)
Memory: 24.2M
CGroup: /system.slice/bind9.service
└─5358 /usr/sbin/named -f -u bind
May 17 21:38:34 u1504 named[5358]: zone 21.172.in-addr.arpa/IN: loaded serial 1
May 17 21:38:34 u1504 named[5358]: zone 22.172.in-addr.arpa/IN: loaded serial 1
May 17 21:38:34 u1504 named[5358]: zone localhost/IN: loaded serial 2
May 17 21:38:34 u1504 named[5358]: zone 127.in-addr.arpa/IN: loaded serial 1
May 17 21:38:34 u1504 named[5358]: zone 27.172.in-addr.arpa/IN: loaded serial 1
May 17 21:38:34 u1504 named[5358]: zone 16.172.in-addr.arpa/IN: loaded serial 1
May 17 21:38:34 u1504 named[5358]: zone 24.172.in-addr.arpa/IN: loaded serial 1
May 17 21:38:34 u1504 named[5358]: zone mccc.org/IN: loaded serial 1505021309
May 17 21:38:34 u1504 named[5358]: all zones loaded
May 17 21:38:34 u1504 named[5358]: running
root@u1504:/var/lib/bind# service isc-dhcp-server restart
root@u1504:/var/lib/bind# service isc-dhcp-server status
â—Ź isc-dhcp-server.service - ISC DHCP IPv4 server
Loaded: loaded (/lib/systemd/system/isc-dhcp-server.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2015-05-17 21:38:45 CDT; 20s ago
Docs: man:dhcpd(8)
Process: 5387 ExecStartPre=/bin/chown dhcpd:dhcpd /run/dhcp-server (code=exited, status=0/SUCCESS)
Main PID: 5390 (dhcpd)
Memory: 10.3M
CGroup: /system.slice/isc-dhcp-server.service
└─5390 dhcpd -user dhcpd -group dhcpd -f -4 -pf /run/dhcp-server/dhcpd.pid -cf /etc/dhcp/dhcpd.conf
May 17 21:38:45 u1504 sh[5390]: you want, please write a subnet declaration
May 17 21:38:45 u1504 sh[5390]: in your dhcpd.conf file for the network segment
May 17 21:38:45 u1504 sh[5390]: to which interface wlan0 is attached. **
May 17 21:38:45 u1504 sh[5390]: No subnet declaration for eth0 (no IPv4 addresses).
May 17 21:38:45 u1504 sh[5390]: ** Ignoring requests on eth0. If this is not what
May 17 21:38:45 u1504 sh[5390]: you want, please write a subnet declaration
May 17 21:38:45 u1504 sh[5390]: in your dhcpd.conf file for the network segment
May 17 21:38:45 u1504 sh[5390]: to which interface eth0 is attached. **
May 17 21:38:45 u1504 sh[5390]: Sending on Socket/fallback/fallback-net
May 17 21:38:45 u1504 sh[5390]: Can't create PID file /run/dhcp-server/dhcpd.pid: Permission denied.
root@u1504:/var/lib/bind# nslookup u1504
Server: 127.0.1.1
Address: 127.0.1.1#53
Name: u1504.vmem.org
Address: 10.207.39.1
root@u1504:/var/lib/bind# nslookup mccc1
Server: 127.0.1.1
Address: 127.0.1.1#53
Name: mccc1.mccc.org
Address: 10.207.29.1
root@u1504:/var/lib/bind# cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.1.1
search vmem.org mccc.org gns1.vmem.org <-- Note until reboot/network restart, may have to add domains manually to resolv.conf
root@u1504:/var/lib/bind#
root@u1504:/var/lib/bind# nslookup 10.207.39.1
Server: 127.0.1.1
Address: 127.0.1.1#53
1.39.207.10.in-addr.arpa name = u1504.vmem.org.
root@u1504:/var/lib/bind# nslookup 10.207.29.1
Server: 127.0.1.1
Address: 127.0.1.1#53
1.29.207.10.in-addr.arpa name = mccc1.mccc.org.
root@u1504:/var/lib/bind#
Configure /etc/sysctl.conf
Make the settings values changes to existing parameters in the /etc/sysctl.conf as shown below thanks to the blogpost from Venu Murthy here. Additional reference material and full-resolution schematics of OpenvSwitch OpenStack networking can be found at OpenStack documentation.
gstanden@u1504:~/Downloads$ cat /etc/sysctl.conf | egrep -B1 'rp_filter|ip_forward'
# GLS 20141226 http://thenewstack.io/solving-a-common-beginners-problem-when-pinging-from-an-openstack-instance/
net.ipv4.conf.default.rp_filter=0
net.ipv4.conf.all.rp_filter=0
--
# GLS 20141226 http://thenewstack.io/solving-a-common-beginners-problem-when-pinging-from-an-openstack-instance/
net.ipv4.ip_forward=1
gstanden@u1504:~/Downloads$
Install and Configure Required Apparmor Package
Next install apparmor-utils package as shown below.
gstanden@u1504:~$ sudo apt-get install apparmor-utils
gstanden@u1504:~/Downloads$ sudo apt-get install apparmor-utils
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
python3-apparmor python3-libapparmor
Suggested packages:
apparmor-docs vim-addon-manager
The following NEW packages will be installed:
apparmor-utils python3-apparmor python3-libapparmor
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 141 kB of archives.
After this operation, 911 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://us.archive.ubuntu.com/ubuntu/ vivid/main python3-libapparmor amd64 2.9.1-0ubuntu9 [24.9 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ vivid/main python3-apparmor amd64 2.9.1-0ubuntu9 [63.2 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ vivid/main apparmor-utils amd64 2.9.1-0ubuntu9 [53.4 kB]
Fetched 141 kB in 0s (364 kB/s)
Selecting previously unselected package python3-libapparmor.
(Reading database ... 205593 files and directories currently installed.)
Preparing to unpack .../python3-libapparmor_2.9.1-0ubuntu9_amd64.deb ...
Unpacking python3-libapparmor (2.9.1-0ubuntu9) ...
Selecting previously unselected package python3-apparmor.
Preparing to unpack .../python3-apparmor_2.9.1-0ubuntu9_amd64.deb ...
Unpacking python3-apparmor (2.9.1-0ubuntu9) ...
Selecting previously unselected package apparmor-utils.
Preparing to unpack .../apparmor-utils_2.9.1-0ubuntu9_amd64.deb ...
Unpacking apparmor-utils (2.9.1-0ubuntu9) ...
Processing triggers for man-db (2.7.0.2-5) ...
Setting up python3-libapparmor (2.9.1-0ubuntu9) ...
Setting up python3-apparmor (2.9.1-0ubuntu9) ...
Setting up apparmor-utils (2.9.1-0ubuntu9) ...
gstanden@u1504:~/Downloads$
Set the apparmor profile for lxc-start to "complain" mode as shown below. Currently, this is considered by the author to be the Ubuntu Linux equivalent of setting selinux to "permissive" but not absolutely sure of that. This then would mean that this is the equivalent step in Venu's blog post for setting selinux to permissive mode in the apparmor paradigm.
gstanden@u1504:~/Downloads$ sudo aa-complain /usr/bin/lxc-start
Setting /usr/bin/lxc-start to complain mode.
gstanden@u1504:~/Downloads$
Install and configure openssh-server
The openssh-server package is needed so that ssh from the container to the Ubuntu host OS will work. Install the openssh-server package as shown below.
[root@lxcora01 ~]# ssh gstanden@u1504.vmem.org
ssh: connect to host u1504.vmem.org port 22: Connection refused
[root@lxcora01 ~]# exit
logout
Connection to lxcora01 closed.
gstanden@u1504:~$ sudo apt-get install openssh-server
[sudo] password for gstanden:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
linux-headers-3.19.0-15 linux-headers-3.19.0-15-generic linux-image-3.19.0-15-generic linux-image-extra-3.19.0-15-generic
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
ncurses-term openssh-sftp-server ssh-import-id
Suggested packages:
rssh molly-guard monkeysphere
The following NEW packages will be installed:
ncurses-term openssh-server openssh-sftp-server ssh-import-id
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 630 kB of archives.
After this operation, 3,441 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://us.archive.ubuntu.com/ubuntu/ vivid/main ncurses-term all 5.9+20140712-2ubuntu2 [248 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ vivid/main openssh-sftp-server amd64 1:6.7p1-5ubuntu1 [38.3 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ vivid/main openssh-server amd64 1:6.7p1-5ubuntu1 [334 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu/ vivid/main ssh-import-id all 4.1-0ubuntu1 [9,344 B]
Fetched 630 kB in 0s (631 kB/s)
Preconfiguring packages ...
Selecting previously unselected package ncurses-term.
(Reading database ... 236321 files and directories currently installed.)
Preparing to unpack .../ncurses-term_5.9+20140712-2ubuntu2_all.deb ...
Unpacking ncurses-term (5.9+20140712-2ubuntu2) ...
Selecting previously unselected package openssh-sftp-server.
Preparing to unpack .../openssh-sftp-server_1%3a6.7p1-5ubuntu1_amd64.deb ...
Unpacking openssh-sftp-server (1:6.7p1-5ubuntu1) ...
Selecting previously unselected package openssh-server.
Preparing to unpack .../openssh-server_1%3a6.7p1-5ubuntu1_amd64.deb ...
Unpacking openssh-server (1:6.7p1-5ubuntu1) ...
Selecting previously unselected package ssh-import-id.
Preparing to unpack .../ssh-import-id_4.1-0ubuntu1_all.deb ...
Unpacking ssh-import-id (4.1-0ubuntu1) ...
Processing triggers for man-db (2.7.0.2-5) ...
Processing triggers for systemd (219-7ubuntu5) ...
Processing triggers for ureadahead (0.100.0-19) ...
ureadahead will be reprofiled on next reboot
Processing triggers for ufw (0.34~rc-0ubuntu5) ...
Setting up ncurses-term (5.9+20140712-2ubuntu2) ...
Setting up openssh-sftp-server (1:6.7p1-5ubuntu1) ...
Setting up openssh-server (1:6.7p1-5ubuntu1) ...
Creating SSH2 RSA key; this may take some time ...
2048 dc:07:ad:02:9a:59:e6:a4:ec:12:85:b1:86:14:2b:48 /etc/ssh/ssh_host_rsa_key.pub (RSA)
Creating SSH2 DSA key; this may take some time ...
1024 93:ef:1b:01:ef:c2:b4:d1:ca:30:db:4e:b2:9b:f6:3a /etc/ssh/ssh_host_dsa_key.pub (DSA)
Creating SSH2 ECDSA key; this may take some time ...
256 d7:9f:a8:e2:b6:7d:71:93:ae:f8:7d:d5:70:54:26:c8 /etc/ssh/ssh_host_ecdsa_key.pub (ECDSA)
Creating SSH2 ED25519 key; this may take some time ...
256 9b:86:f9:41:a6:7a:67:8f:08:ed:95:fd:13:69:75:18 /etc/ssh/ssh_host_ed25519_key.pub (ED25519)
Setting up ssh-import-id (4.1-0ubuntu1) ...
Processing triggers for systemd (219-7ubuntu5) ...
Processing triggers for ureadahead (0.100.0-19) ...
Processing triggers for ufw (0.34~rc-0ubuntu5) ...
gstanden@u1504:~$
Add Settings to /etc/sysctl.conf File
Next add the following settings at the end of /etc/sysctl.conf for Oracle as shown below. Hugepages value is optional and can be adjusted downward depending on whether Hugepages are used or not and depending on what size of Oracle SGA.
# Oracle
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 6815744
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
vm.nr_hugepages = 2060
kernel.panic_on_oops = 1
The final /etc/sysctl.conf file after all above edits should be similar to the one shown below. Sections edited and added are shown in bold.
gstanden@u1504:~/Downloads$ cat /etc/sysctl.conf
# /etc/sysctl.conf - Configuration file for setting system variables
# See /etc/sysctl.d/ for additional system variables.
# See sysctl.conf (5) for information.
#
#kernel.domainname = example.com
# Uncomment the following to stop low-level messages on console
#kernel.printk = 3 4 1 3
##############################################################3
# Functions previously found in netbase
#
# Uncomment the next two lines to enable Spoof protection (reverse-path filter)
# Turn on Source Address Verification in all interfaces to
# prevent some spoofing attacks
# GLS 20141226 http://thenewstack.io/solving-a-common-beginners-problem-when-pinging-from-an-openstack-instance/
net.ipv4.conf.default.rp_filter=0
net.ipv4.conf.all.rp_filter=0
# Uncomment the next line to enable TCP/IP SYN cookies
# See http://lwn.net/Articles/277146/
# Note: This may impact IPv6 TCP sessions too
#net.ipv4.tcp_syncookies=1
# Uncomment the next line to enable packet forwarding for IPv4
# GLS 20141226 http://thenewstack.io/solving-a-common-beginners-problem-when-pinging-from-an-openstack-instance/
net.ipv4.ip_forward=1
# Uncomment the next line to enable packet forwarding for IPv6
# Enabling this option disables Stateless Address Autoconfiguration
# based on Router Advertisements for this host
#net.ipv6.conf.all.forwarding=1
###################################################################
# Additional settings - these settings can improve the network
# security of the host and prevent against some network attacks
# including spoofing attacks and man in the middle attacks through
# redirection. Some network environments, however, require that these
# settings are disabled so review and enable them as needed.
#
# Do not accept ICMP redirects (prevent MITM attacks)
#net.ipv4.conf.all.accept_redirects = 0
#net.ipv6.conf.all.accept_redirects = 0
# _or_
# Accept ICMP redirects only for gateways listed in our default
# gateway list (enabled by default)
# net.ipv4.conf.all.secure_redirects = 1
#
# Do not send ICMP redirects (we are not a router)
#net.ipv4.conf.all.send_redirects = 0
#
# Do not accept IP source route packets (we are not a router)
#net.ipv4.conf.all.accept_source_route = 0
#net.ipv6.conf.all.accept_source_route = 0
#
# Log Martian Packets
#net.ipv4.conf.all.log_martians = 1
#
# Oracle
#
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 6815744
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
vm.nr_hugepages = 2060
kernel.panic_on_oops = 1
gstanden@u1504:~/Downloads$
Apply the changes to the system using the command as shown below and verify that all new settings were accepted and applied.
gstanden@u1504:~/Downloads$ sudo sysctl -p
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.all.rp_filter = 0
net.ipv4.ip_forward = 1
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 6815744
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
vm.nr_hugepages = 2060
kernel.panic_on_oops = 1
gstanden@u1504:~/Downloads$
Restart DNS and DHCP to Verify Configuration
Reboot the desktop host to reset the "/etc/resolv.conf" file to the new settings, and verify DNS and DHCP configuration is correct by doing an nslookup on "w520" and "w520.vmem.org". Both should return the local IP address for that server. Run the tests as shown below to:
Verify DNS is working;
Verify that "named" is monitoring 10.207.39.1
Verify that "named" in monitoring 10.207.29.1 (new network for mccc.org) GLS 2015-05-16
Verify that "dnsmasq" is monitoring 10.0.3.1
Verify that the OpenvSwitch switches are created
Verify that "/etc/resolv.conf" has the correct settings
These tests are shown below with correct expected output.
gstanden@u1504:~/Downloads$ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1
search vmem.org mccc.org gns1.vmem.org
gstanden@u1504:~/Downloads$ nslookup u1504
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: u1504.vmem.org
Address: 10.207.39.1
gstanden@u1504:~/Downloads$ nslookup u1504.vmem.org
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: u1504.vmem.org
Address: 10.207.39.1
gstanden@u1504:~/Downloads$ nslookup mccc1
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: mccc1.mccc.org
Address: 10.207.29.1
gstanden@u1504:~/Downloads$ nslookup mccc1.mccc.org
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: mccc1.mccc.org
Address: 10.207.29.1
gstanden@u1504:~/Downloads$ nslookup 10.207.39.1
Server: 127.0.0.1
Address: 127.0.0.1#53
1.39.207.10.in-addr.arpa name = u1504.vmem.org.
gstanden@u1504:~/Downloads$ nslookup 10.207.29.1
Server: 127.0.0.1
Address: 127.0.0.1#53
1.29.207.10.in-addr.arpa name = mccc1.mccc.org.
gstanden@u1504:~/Downloads$ sudo netstat -ulnp | grep 53
udp 0 0 0.0.0.0:5353 0.0.0.0:* 861/avahi-daemon: r
udp 0 0 10.207.29.1:53 0.0.0.0:* 2861/named
udp 0 0 10.207.39.1:53 0.0.0.0:* 2861/named
udp 0 0 127.0.0.1:53 0.0.0.0:* 2861/named
udp 0 0 127.0.1.1:53 0.0.0.0:* 1444/dnsmasq
udp 0 0 10.0.3.1:53 0.0.0.0:* 1196/dnsmasq
udp6 0 0 :::53974 :::* 861/avahi-daemon: r
udp6 0 0 :::5353 :::* 861/avahi-daemon: r
udp6 0 0 :::53 :::* 2861/named
udp6 0 0 fe80::a4d0:6dff:fe09:53 :::* 1196/dnsmasq
gstanden@u1504:~/Downloads$ sudo ovs-vsctl show
684bc501-621d-4083-a692-697fccf1e0e6
Bridge "sw6"
Port "sw6"
Interface "sw6"
type: internal
Bridge "sw4"
Port "sw4"
Interface "sw4"
type: internal
Bridge "sx1"
Port "a4"
Interface "a4"
Port "sx1"
tag: 10
trunks: [10]
Interface "sx1"
type: internal
Port "a3"
Interface "a3"
Port "a5"
Interface "a5"
Port "a1"
Interface "a1"
Port "a2"
Interface "a2"
Bridge "sw7"
Port "sw7"
Interface "sw7"
type: internal
Bridge "sw1"
Port "s5"
Interface "s5"
Port "s3"
Interface "s3"
Port "s2"
Interface "s2"
Port "sw1"
tag: 10
trunks: [10]
Interface "sw1"
type: internal
Port "s4"
Interface "s4"
Port "s1"
Interface "s1"
Bridge "sw3"
Port "w5"
Interface "w5"
Port "sw3"
tag: 90
Interface "sw3"
type: internal
Port "w4"
Interface "w4"
Port "w2"
Interface "w2"
Port "w1"
Interface "w1"
Port "w3"
Interface "w3"
Bridge "sw8"
Port "sw8"
Interface "sw8"
type: internal
Bridge "sw2"
Port "t3"
Interface "t3"
Port "sw2"
tag: 80
Interface "sw2"
type: internal
Port "t4"
Interface "t4"
Port "t1"
Interface "t1"
Port "t5"
Interface "t5"
Port "t2"
Interface "t2"
Bridge "sw5"
Port "sw5"
Interface "sw5"
type: internal
Bridge "sw9"
Port "sw9"
Interface "sw9"
type: internal
ovs_version: "2.3.1"
gstanden@u1504:~/Downloads$
Create Oracle Enteprise Linux 6.5 LXC Linux Container
Install Ubuntu rpm and yum Packages
Create an Oracle Enteprise Linux (OEL) 6.5 LXC Linux Container as shown below. Notice that the "rpm" and "yum" deb packages must be installed first. As shown below, LXC container creation is not possible without the Ubuntu versions of "rpm" and "yum" installed first.
gstanden@u1504:~/Downloads$ sudo apt-get install rpm yum
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
debugedit librpm3 librpmbuild3 librpmio3 librpmsign1 libsqlite0 python-libxml2 python-lzma python-rpm python-sqlite python-sqlitecachec python-urlgrabber rpm-common
rpm2cpio
Suggested packages:
rpm-i18n python-sqlite-dbg alien elfutils rpmlint rpm2html
The following NEW packages will be installed:
debugedit librpm3 librpmbuild3 librpmio3 librpmsign1 libsqlite0 python-libxml2 python-lzma python-rpm python-sqlite python-sqlitecachec python-urlgrabber rpm
rpm-common rpm2cpio yum
0 upgraded, 16 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,372 kB of archives.
After this operation, 7,560 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://us.archive.ubuntu.com/ubuntu/ vivid/main librpmio3 amd64 4.11.3-1.1 [69.3 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ vivid/main debugedit amd64 4.11.3-1.1 [15.8 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ vivid/main librpm3 amd64 4.11.3-1.1 [155 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu/ vivid/main librpmbuild3 amd64 4.11.3-1.1 [58.0 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu/ vivid/main librpmsign1 amd64 4.11.3-1.1 [8,062 B]
Get:6 http://us.archive.ubuntu.com/ubuntu/ vivid/universe libsqlite0 amd64 2.8.17-12fakesync1 [139 kB]
Get:7 http://us.archive.ubuntu.com/ubuntu/ vivid/main python-libxml2 amd64 2.9.2+dfsg1-3 [143 kB]
Get:8 http://us.archive.ubuntu.com/ubuntu/ vivid/universe python-lzma amd64 0.5.3-2build1 [32.2 kB]
Get:9 http://us.archive.ubuntu.com/ubuntu/ vivid/universe python-sqlite amd64 1.0.1-11 [20.7 kB]
Get:10 http://us.archive.ubuntu.com/ubuntu/ vivid/main python-urlgrabber all 3.9.1-4ubuntu5 [42.4 kB]
Get:11 http://us.archive.ubuntu.com/ubuntu/ vivid/main rpm-common amd64 4.11.3-1.1 [26.3 kB]
Get:12 http://us.archive.ubuntu.com/ubuntu/ vivid/main rpm2cpio amd64 4.11.3-1.1 [5,228 B]
Get:13 http://us.archive.ubuntu.com/ubuntu/ vivid/main rpm amd64 4.11.3-1.1 [115 kB]
Get:14 http://us.archive.ubuntu.com/ubuntu/ vivid/universe python-rpm amd64 4.11.3-1.1 [33.6 kB]
Get:15 http://us.archive.ubuntu.com/ubuntu/ vivid/universe python-sqlitecachec amd64 1.1.4-1 [21.4 kB]
Get:16 http://us.archive.ubuntu.com/ubuntu/ vivid/universe yum all 3.4.3-3 [486 kB]
Fetched 1,372 kB in 2s (684 kB/s)
Selecting previously unselected package librpmio3.
(Reading database ... 205658 files and directories currently installed.)
Preparing to unpack .../librpmio3_4.11.3-1.1_amd64.deb ...
Unpacking librpmio3 (4.11.3-1.1) ...
Selecting previously unselected package debugedit.
Preparing to unpack .../debugedit_4.11.3-1.1_amd64.deb ...
Unpacking debugedit (4.11.3-1.1) ...
Selecting previously unselected package librpm3.
Preparing to unpack .../librpm3_4.11.3-1.1_amd64.deb ...
Unpacking librpm3 (4.11.3-1.1) ...
Selecting previously unselected package librpmbuild3.
Preparing to unpack .../librpmbuild3_4.11.3-1.1_amd64.deb ...
Unpacking librpmbuild3 (4.11.3-1.1) ...
Selecting previously unselected package librpmsign1.
Preparing to unpack .../librpmsign1_4.11.3-1.1_amd64.deb ...
Unpacking librpmsign1 (4.11.3-1.1) ...
Selecting previously unselected package libsqlite0.
Preparing to unpack .../libsqlite0_2.8.17-12fakesync1_amd64.deb ...
Unpacking libsqlite0 (2.8.17-12fakesync1) ...
Selecting previously unselected package python-libxml2.
Preparing to unpack .../python-libxml2_2.9.2+dfsg1-3_amd64.deb ...
Unpacking python-libxml2 (2.9.2+dfsg1-3) ...
Selecting previously unselected package python-lzma.
Preparing to unpack .../python-lzma_0.5.3-2build1_amd64.deb ...
Unpacking python-lzma (0.5.3-2build1) ...
Selecting previously unselected package python-sqlite.
Preparing to unpack .../python-sqlite_1.0.1-11_amd64.deb ...
Unpacking python-sqlite (1.0.1-11) ...
Selecting previously unselected package python-urlgrabber.
Preparing to unpack .../python-urlgrabber_3.9.1-4ubuntu5_all.deb ...
Unpacking python-urlgrabber (3.9.1-4ubuntu5) ...
Selecting previously unselected package rpm-common.
Preparing to unpack .../rpm-common_4.11.3-1.1_amd64.deb ...
Unpacking rpm-common (4.11.3-1.1) ...
Selecting previously unselected package rpm2cpio.
Preparing to unpack .../rpm2cpio_4.11.3-1.1_amd64.deb ...
Unpacking rpm2cpio (4.11.3-1.1) ...
Selecting previously unselected package rpm.
Preparing to unpack .../rpm_4.11.3-1.1_amd64.deb ...
Unpacking rpm (4.11.3-1.1) ...
Selecting previously unselected package python-rpm.
Preparing to unpack .../python-rpm_4.11.3-1.1_amd64.deb ...
Unpacking python-rpm (4.11.3-1.1) ...
Selecting previously unselected package python-sqlitecachec.
Preparing to unpack .../python-sqlitecachec_1.1.4-1_amd64.deb ...
Unpacking python-sqlitecachec (1.1.4-1) ...
Selecting previously unselected package yum.
Preparing to unpack .../archives/yum_3.4.3-3_all.deb ...
Unpacking yum (3.4.3-3) ...
Processing triggers for man-db (2.7.0.2-5) ...
Setting up librpmio3 (4.11.3-1.1) ...
Setting up debugedit (4.11.3-1.1) ...
Setting up librpm3 (4.11.3-1.1) ...
Setting up librpmbuild3 (4.11.3-1.1) ...
Setting up librpmsign1 (4.11.3-1.1) ...
Setting up libsqlite0 (2.8.17-12fakesync1) ...
Setting up python-libxml2 (2.9.2+dfsg1-3) ...
Setting up python-lzma (0.5.3-2build1) ...
Setting up python-sqlite (1.0.1-11) ...
Setting up python-urlgrabber (3.9.1-4ubuntu5) ...
Setting up rpm-common (4.11.3-1.1) ...
Setting up rpm2cpio (4.11.3-1.1) ...
Setting up rpm (4.11.3-1.1) ...
Setting up python-rpm (4.11.3-1.1) ...
Setting up python-sqlitecachec (1.1.4-1) ...
Setting up yum (3.4.3-3) ...
Processing triggers for libc-bin (2.21-0ubuntu4) ...
Processing triggers for python-support (1.0.15) ...
gstanden@u1504:~/Downloads$
Create LXC Container
Now install the OEL 6.5 LXC Linux Container as shown below. The full log of the installation is attached to the blog as "lxcora01.log". Abridged information from the creation of the LXC is shown below.
gstanden@u1504:~/Downloads$ sudo lxc-create -t oracle -n lxcora01
Host is Ubuntu 15.04
No release specified with -R, defaulting to 6.5
Create configuration file /var/lib/lxc/lxcora01/config
Yum installing release 6.5 for x86_64
ol6_u5_base | 1.4 kB 00:00
ol6_u5_base/primary | 3.2 MB 00:02
ol6_u5_base 8573/8573
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package chkconfig.x86_64 0:1.3.49.3-2.el6_4.1 will be installed
--> Processing Dependency: libc.so.6(GLIBC_2.2.5)(64bit) for package: chkconfig-1.3.49.3-2.el6_4.1.x86_64
--> Processing Dependency: libc.so.6(GLIBC_2.8)(64bit) for package: chkconfig-1.3.49.3-2.el6_4.1.x86_64
--> Processing Dependency: libpopt.so.0(LIBPOPT_0)(64bit) for package: chkconfig-1.3.49.3-2.el6_4.1.x86_64
--> Processing Dependency: rtld(GNU_HASH) for package: chkconfig-1.3.49.3-2.el6_4.1.x86_64
--> Processing Dependency: libc.so.6(GLIBC_2.3.4)(64bit) for package: chkconfig-1.3.49.3-2.el6_4.1.x86_64
--> Processing Dependency: libc.so.6(GLIBC_2.4)(64bit) for package: chkconfig-1.3.49.3-2.el6_4.1.x86_64
--> Processing Dependency: libc.so.6(GLIBC_2.3)(64bit) for package: chkconfig-1.3.49.3-2.el6_4.1.x86_64
--> Processing Dependency: libpopt.so.0()(64bit) for package: chkconfig-1.3.49.3-2.el6_4.1.x86_64
--> Processing Dependency: libc.so.6()(64bit) for package: chkconfig-1.3.49.3-2.el6_4.1.x86_64
---> Package dhclient.x86_64 12:4.1.1-38.P1.0.1.el6 will be installed
--> Processing Dependency: dhcp-common = 12:4.1.1-38.P1.0.1.el6 for package: 12:dhclient-4.1.1-38.P1.0.1.el6.x86_64
--> Processing Dependency: grep for package: 12:dhclient-4.1.1-38.P1.0.1.el6.x86_64
--> Processing Dependency: coreutils for package: 12:dhclient-4.1.1-38.P1.0.1.el6.x86_64
--> Processing Dependency: /bin/bash for package: 12:dhclient-4.1.1-38.P1.0.1.el6.x86_64
--> Processing Dependency: /bin/sh for package: 12:dhclient-4.1.1-38.P1.0.1.el6.x86_64
--> Processing Dependency: libcap-ng.so.0()(64bit) for package: 12:dhclient-4.1.1-38.P1.0.1.el6.x86_64
---> Package initscripts.x86_64 0:9.03.40-2.0.1.el6 will be installed
--> Processing Dependency: upstart >= 0.6.0 for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: udev >= 125-1 for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: util-linux-ng >= 2.16 for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: sysvinit-tools >= 2.87 for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: ethtool >= 1.8-2 for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: /sbin/fuser for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: /sbin/arping for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: cpio for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: /usr/sbin/groupadd for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: /bin/find for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: /sbin/ip for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: /sbin/blkid for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: /bin/sed for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: net-tools for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: /sbin/sysctl for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: /bin/awk for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: module-init-tools for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: oracle-logos for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: mingetty for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: /sbin/pidof for package: initscripts-9.03.40-2.0.1.el6.x86_64
--> Processing Dependency: libglib-2.0.so.0()(64bit) for package: initscripts-9.03.40-2.0.1.el6.x86_64
---> Package openssh-clients.x86_64 0:5.3p1-94.el6 will be installed
--> Processing Dependency: openssh = 5.3p1-94.el6 for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libgssapi_krb5.so.2(gssapi_krb5_2_MIT)(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libnss3.so(NSS_3.4)(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libnss3.so(NSS_3.9.3)(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libnss3.so(NSS_3.5)(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libnss3.so(NSS_3.9)(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libcrypto.so.10(libcrypto.so.10)(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libcrypto.so.10(OPENSSL_1.0.1)(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libnss3.so(NSS_3.2)(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libnss3.so(NSS_3.6)(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libnss3.so(NSS_3.3)(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libnss3.so(NSS_3.9.2)(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libfipscheck.so.1()(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libk5crypto.so.3()(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libkrb5.so.3()(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libselinux.so.1()(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libedit.so.0()(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libcom_err.so.2()(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libplc4.so()(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libgssapi_krb5.so.2()(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libz.so.1()(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libcrypto.so.10()(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libnss3.so()(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
--> Processing Dependency: libncurses.so.5()(64bit) for package: openssh-clients-5.3p1-94.el6.x86_64
---> Package openssh-server.x86_64 0:5.3p1-94.el6 will be installed
--> Processing Dependency: pam >= 1.0.1-3 for package: openssh-server-5.3p1-94.el6.x86_64
--> Processing Dependency: libpam.so.0(LIBPAM_1.0)(64bit) for package: openssh-server-5.3p1-94.el6.x86_64
--> Processing Dependency: libaudit.so.1()(64bit) for package: openssh-server-5.3p1-94.el6.x86_64
--> Processing Dependency: libwrap.so.0()(64bit) for package: openssh-server-5.3p1-94.el6.x86_64
--> Processing Dependency: libpam.so.0()(64bit) for package: openssh-server-5.3p1-94.el6.x86_64
---> Package oraclelinux-release.x86_64 6:6Server-5.0.2 will be installed
--> Processing Dependency: redhat-release-server for package: 6:oraclelinux-release-6Server-5.0.2.x86_64
---> Package passwd.x86_64 0:0.77-4.el6_2.2 will be installed
--> Processing Dependency: libuser.so.1()(64bit) for package: passwd-0.77-4.el6_2.2.x86_64
---> Package policycoreutils.x86_64 0:2.0.83-19.39.0.1.el6 will be installed
--> Processing Dependency: libselinux-utils >= 2.0.90-3 for package: policycoreutils-2.0.83-19.39.0.1.el6.x86_64
--> Processing Dependency: libsepol >= 2.0.41-3 for package: policycoreutils-2.0.83-19.39.0.1.el6.x86_64
--> Processing Dependency: checkpolicy for package: policycoreutils-2.0.83-19.39.0.1.el6.x86_64
--> Processing Dependency: libsemanage.so.1(LIBSEMANAGE_1.0)(64bit) for package: policycoreutils-2.0.83-19.39.0.1.el6.x86_64
--> Processing Dependency: /usr/bin/diff for package: policycoreutils-2.0.83-19.39.0.1.el6.x86_64
--> Processing Dependency: rpm for package: policycoreutils-2.0.83-19.39.0.1.el6.x86_64
--> Processing Dependency: libdbus-1.so.3()(64bit) for package: policycoreutils-2.0.83-19.39.0.1.el6.x86_64
--> Processing Dependency: libdbus-glib-1.so.2()(64bit) for package: policycoreutils-2.0.83-19.39.0.1.el6.x86_64
--> Processing Dependency: libsemanage.so.1()(64bit) for package: policycoreutils-2.0.83-19.39.0.1.el6.x86_64
--> Processing Dependency: libsepol.so.1()(64bit) for package: policycoreutils-2.0.83-19.39.0.1.el6.x86_64
---> Package rootfiles.noarch 0:8.1-6.1.el6 will be installed
---> Package rsyslog.x86_64 0:5.8.10-8.0.1.el6 will be installed
--> Processing Dependency: logrotate >= 3.5.2 for package: rsyslog-5.8.10-8.0.1.el6.x86_64
--> Processing Dependency: libgcc_s.so.1(GCC_3.0)(64bit) for package: rsyslog-5.8.10-8.0.1.el6.x86_64
--> Processing Dependency: libgcc_s.so.1(GCC_3.3.1)(64bit) for package: rsyslog-5.8.10-8.0.1.el6.x86_64
--> Processing Dependency: libgcc_s.so.1()(64bit) for package: rsyslog-5.8.10-8.0.1.el6.x86_64
---> Package vim-minimal.x86_64 2:7.2.411-1.8.el6 will be installed
--> Processing Dependency: libacl.so.1(ACL_1.0)(64bit) for package: 2:vim-minimal-7.2.411-1.8.el6.x86_64
--> Processing Dependency: libacl.so.1()(64bit) for package: 2:vim-minimal-7.2.411-1.8.el6.x86_64
---> Package yum.noarch 0:3.2.29-40.0.1.el6 will be installed
--> Processing Dependency: python(abi) = 2.6 for package: yum-3.2.29-40.0.1.el6.noarch
--> Processing Dependency: python-urlgrabber >= 3.9.0-8 for package: yum-3.2.29-40.0.1.el6.noarch
--> Processing Dependency: python >= 2.4 for package: yum-3.2.29-40.0.1.el6.noarch
--> Processing Dependency: yum-metadata-parser >= 1.1.0 for package: yum-3.2.29-40.0.1.el6.noarch
--> Processing Dependency: python-iniparse for package: yum-3.2.29-40.0.1.el6.noarch
--> Processing Dependency: python-sqlite for package: yum-3.2.29-40.0.1.el6.noarch
--> Processing Dependency: pygpgme for package: yum-3.2.29-40.0.1.el6.noarch
--> Processing Dependency: rpm-python for package: yum-3.2.29-40.0.1.el6.noarch
--> Processing Dependency: /usr/bin/python for package: yum-3.2.29-40.0.1.el6.noarch
--> Running transaction check
---> Package audit-libs.x86_64 0:2.2-2.el6 will be installed
---> Package bash.x86_64 0:4.1.2-15.el6_4 will be installed
---> Package checkpolicy.x86_64 0:2.0.22-1.el6 will be installed
---> Package coreutils.x86_64 0:8.4-31.0.1.el6 will be installed
--> Processing Dependency: coreutils-libs = 8.4-31.0.1.el6 for package: coreutils-8.4-31.0.1.el6.x86_64
--> Processing Dependency: libattr for package: coreutils-8.4-31.0.1.el6.x86_64
--> Processing Dependency: libattr.so.1(ATTR_1.1)(64bit) for package: coreutils-8.4-31.0.1.el6.x86_64
--> Processing Dependency: ncurses for package: coreutils-8.4-31.0.1.el6.x86_64
--> Processing Dependency: gmp for package: coreutils-8.4-31.0.1.el6.x86_64
--> Processing Dependency: libcap for package: coreutils-8.4-31.0.1.el6.x86_64
--> Processing Dependency: /sbin/install-info for package: coreutils-8.4-31.0.1.el6.x86_64
--> Processing Dependency: libcap.so.2()(64bit) for package: coreutils-8.4-31.0.1.el6.x86_64
--> Processing Dependency: libattr.so.1()(64bit) for package: coreutils-8.4-31.0.1.el6.x86_64
--> Processing Dependency: libgmp.so.3()(64bit) for package: coreutils-8.4-31.0.1.el6.x86_64
---> Package cpio.x86_64 0:2.10-11.el6_3 will be installed
---> Package dbus-glib.x86_64 0:0.86-6.el6_4 will be installed
--> Processing Dependency: libexpat.so.1()(64bit) for package: dbus-glib-0.86-6.el6_4.x86_64
---> Package dbus-libs.x86_64 1:1.2.24-7.0.1.el6_3 will be installed
---> Package dhcp-common.x86_64 12:4.1.1-38.P1.0.1.el6 will be installed
---> Package diffutils.x86_64 0:2.8.1-28.el6 will be installed
---> Package ethtool.x86_64 2:3.5-1.el6 will be installed
---> Package findutils.x86_64 1:4.4.2-6.el6 will be installed
---> Package fipscheck-lib.x86_64 0:1.2.0-7.el6 will be installed
--> Processing Dependency: /usr/bin/fipscheck for package: fipscheck-lib-1.2.0-7.el6.x86_64
---> Package gawk.x86_64 0:3.1.7-10.el6 will be installed
---> Package glib2.x86_64 0:2.26.1-3.el6 will be installed
--> Processing Dependency: shared-mime-info for package: glib2-2.26.1-3.el6.x86_64
--> Processing Dependency: libfam.so.0()(64bit) for package: glib2-2.26.1-3.el6.x86_64
---> Package glibc.x86_64 0:2.12-1.132.el6 will be installed
--> Processing Dependency: glibc-common = 2.12-1.132.el6 for package: glibc-2.12-1.132.el6.x86_64
--> Processing Dependency: libfreebl3.so(NSSRAWHASH_3.12.3)(64bit) for package: glibc-2.12-1.132.el6.x86_64
--> Processing Dependency: basesystem for package: glibc-2.12-1.132.el6.x86_64
--> Processing Dependency: libfreebl3.so()(64bit) for package: glibc-2.12-1.132.el6.x86_64
---> Package grep.x86_64 0:2.6.3-4.el6 will be installed
--> Processing Dependency: libpcre.so.0()(64bit) for package: grep-2.6.3-4.el6.x86_64
---> Package iproute.x86_64 0:2.6.32-31.el6 will be installed
--> Processing Dependency: iptables >= 1.4.5 for package: iproute-2.6.32-31.el6.x86_64
--> Processing Dependency: libdb-4.7.so()(64bit) for package: iproute-2.6.32-31.el6.x86_64
--> Processing Dependency: libxtables.so.4()(64bit) for package: iproute-2.6.32-31.el6.x86_64
---> Package iputils.x86_64 0:20071127-17.el6_4.2 will be installed
--> Processing Dependency: libidn.so.11(LIBIDN_1.0)(64bit) for package: iputils-20071127-17.el6_4.2.x86_64
--> Processing Dependency: libidn.so.11()(64bit) for package: iputils-20071127-17.el6_4.2.x86_64
---> Package krb5-libs.x86_64 0:1.10.3-10.el6_4.6 will be installed
--> Processing Dependency: libkeyutils.so.1(KEYUTILS_0.3)(64bit) for package: krb5-libs-1.10.3-10.el6_4.6.x86_64
--> Processing Dependency: libkeyutils.so.1()(64bit) for package: krb5-libs-1.10.3-10.el6_4.6.x86_64
---> Package libacl.x86_64 0:2.2.49-6.el6 will be installed
---> Package libcap-ng.x86_64 0:0.6.4-3.el6_0.1 will be installed
---> Package libcom_err.x86_64 0:1.42.8-1.0.1.el6 will be installed
---> Package libedit.x86_64 0:2.11-4.20080712cvs.1.el6 will be installed
---> Package libgcc.x86_64 0:4.4.7-4.el6 will be installed
---> Package libselinux.x86_64 0:2.0.94-5.3.el6_4.1 will be installed
---> Package libselinux-utils.x86_64 0:2.0.94-5.3.el6_4.1 will be installed
---> Package libsemanage.x86_64 0:2.0.43-4.2.el6 will be installed
--> Processing Dependency: bzip2-libs for package: libsemanage-2.0.43-4.2.el6.x86_64
--> Processing Dependency: libustr-1.0.so.1(USTR_1.0)(64bit) for package: libsemanage-2.0.43-4.2.el6.x86_64
--> Processing Dependency: libustr-1.0.so.1(USTR_1.0.1)(64bit) for package: libsemanage-2.0.43-4.2.el6.x86_64
--> Processing Dependency: libustr-1.0.so.1()(64bit) for package: libsemanage-2.0.43-4.2.el6.x86_64
--> Processing Dependency: libbz2.so.1()(64bit) for package: libsemanage-2.0.43-4.2.el6.x86_64
---> Package libsepol.x86_64 0:2.0.41-4.el6 will be installed
---> Package libuser.x86_64 0:0.56.13-5.el6 will be installed
--> Processing Dependency: libldap-2.4.so.2()(64bit) for package: libuser-0.56.13-5.el6.x86_64
---> Package logrotate.x86_64 0:3.7.8-17.el6 will be installed
---> Package mingetty.x86_64 0:1.08-5.el6 will be installed
---> Package module-init-tools.x86_64 0:3.9-21.0.1.el6_4 will be installed
--> Processing Dependency: binutils for package: module-init-tools-3.9-21.0.1.el6_4.x86_64
---> Package ncurses-libs.x86_64 0:5.7-3.20090208.el6 will be installed
--> Processing Dependency: ncurses-base = 5.7-3.20090208.el6 for package: ncurses-libs-5.7-3.20090208.el6.x86_64
---> Package net-tools.x86_64 0:1.60-110.el6_2 will be installed
---> Package nspr.x86_64 0:4.10.0-1.el6 will be installed
---> Package nss.x86_64 0:3.15.1-15.0.1.el6 will be installed
--> Processing Dependency: nss-softokn(x86-64) >= 3.14.3 for package: nss-3.15.1-15.0.1.el6.x86_64
--> Processing Dependency: nss-util >= 3.15.1 for package: nss-3.15.1-15.0.1.el6.x86_64
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.15)(64bit) for package: nss-3.15.1-15.0.1.el6.x86_64
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.12.3)(64bit) for package: nss-3.15.1-15.0.1.el6.x86_64
--> Processing Dependency: nss-system-init for package: nss-3.15.1-15.0.1.el6.x86_64
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.14)(64bit) for package: nss-3.15.1-15.0.1.el6.x86_64
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.12)(64bit) for package: nss-3.15.1-15.0.1.el6.x86_64
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.13)(64bit) for package: nss-3.15.1-15.0.1.el6.x86_64
--> Processing Dependency: libnssutil3.so(NSSUTIL_3.12.5)(64bit) for package: nss-3.15.1-15.0.1.el6.x86_64
--> Processing Dependency: libnssutil3.so()(64bit) for package: nss-3.15.1-15.0.1.el6.x86_64
--> Processing Dependency: libsoftokn3.so()(64bit) for package: nss-3.15.1-15.0.1.el6.x86_64
---> Package openssh.x86_64 0:5.3p1-94.el6 will be installed
---> Package openssl.x86_64 0:1.0.1e-15.el6 will be installed
--> Processing Dependency: ca-certificates >= 2008-5 for package: openssl-1.0.1e-15.el6.x86_64
--> Processing Dependency: make for package: openssl-1.0.1e-15.el6.x86_64
---> Package oracle-logos.noarch 0:60.0.14-1.0.1.el6 will be installed
---> Package pam.x86_64 0:1.1.1-17.el6 will be installed
--> Processing Dependency: cracklib-dicts >= 2.8 for package: pam-1.1.1-17.el6.x86_64
--> Processing Dependency: cracklib for package: pam-1.1.1-17.el6.x86_64
--> Processing Dependency: libcrack.so.2()(64bit) for package: pam-1.1.1-17.el6.x86_64
---> Package popt.x86_64 0:1.13-7.el6 will be installed
---> Package procps.x86_64 0:3.2.8-25.el6 will be installed
---> Package psmisc.x86_64 0:22.6-15.el6_0.1 will be installed
---> Package pygpgme.x86_64 0:0.1-18.20090824bzr68.el6 will be installed
--> Processing Dependency: libgpgme.so.11(GPGME_1.1)(64bit) for package: pygpgme-0.1-18.20090824bzr68.el6.x86_64
--> Processing Dependency: libgpgme.so.11(GPGME_1.0)(64bit) for package: pygpgme-0.1-18.20090824bzr68.el6.x86_64
--> Processing Dependency: libgpgme.so.11()(64bit) for package: pygpgme-0.1-18.20090824bzr68.el6.x86_64
--> Processing Dependency: libpython2.6.so.1.0()(64bit) for package: pygpgme-0.1-18.20090824bzr68.el6.x86_64
---> Package python.x86_64 0:2.6.6-51.el6 will be installed
---> Package python-iniparse.noarch 0:0.3.1-2.1.el6 will be installed
---> Package python-urlgrabber.noarch 0:3.9.1-9.el6 will be installed
--> Processing Dependency: python-pycurl for package: python-urlgrabber-3.9.1-9.el6.noarch
---> Package redhat-release-server.x86_64 1:6Server-6.5.0.1.0.1.el6 will be installed
---> Package rpm.x86_64 0:4.8.0-37.el6 will be installed
--> Processing Dependency: curl for package: rpm-4.8.0-37.el6.x86_64
--> Processing Dependency: db4-utils for package: rpm-4.8.0-37.el6.x86_64
--> Processing Dependency: librpm.so.1()(64bit) for package: rpm-4.8.0-37.el6.x86_64
--> Processing Dependency: liblua-5.1.so()(64bit) for package: rpm-4.8.0-37.el6.x86_64
--> Processing Dependency: libelf.so.1()(64bit) for package: rpm-4.8.0-37.el6.x86_64
--> Processing Dependency: librpmbuild.so.1()(64bit) for package: rpm-4.8.0-37.el6.x86_64
--> Processing Dependency: librpmio.so.1()(64bit) for package: rpm-4.8.0-37.el6.x86_64
--> Processing Dependency: liblzma.so.0()(64bit) for package: rpm-4.8.0-37.el6.x86_64
--> Processing Dependency: libmagic.so.1()(64bit) for package: rpm-4.8.0-37.el6.x86_64
---> Package rpm-python.x86_64 0:4.8.0-37.el6 will be installed
---> Package sed.x86_64 0:4.2.1-10.el6 will be installed
---> Package shadow-utils.x86_64 2:4.1.4.2-13.el6 will be installed
--> Processing Dependency: setup for package: 2:shadow-utils-4.1.4.2-13.el6.x86_64
---> Package sysvinit-tools.x86_64 0:2.87-5.dsf.el6 will be installed
---> Package tcp_wrappers-libs.x86_64 0:7.6-57.el6 will be installed
---> Package udev.x86_64 0:147-2.51.0.1.el6 will be installed
--> Processing Dependency: MAKEDEV >= 3.11 for package: udev-147-2.51.0.1.el6.x86_64
--> Processing Dependency: hwdata for package: udev-147-2.51.0.1.el6.x86_64
--> Processing Dependency: libusb-0.1.so.4()(64bit) for package: udev-147-2.51.0.1.el6.x86_64
---> Package upstart.x86_64 0:0.6.5-12.el6_4.1 will be installed
--> Processing Dependency: libnih-dbus.so.1(LIBNIH_DBUS_1_0)(64bit) for package: upstart-0.6.5-12.el6_4.1.x86_64
--> Processing Dependency: libnih.so.1(LIBNIH_1_0)(64bit) for package: upstart-0.6.5-12.el6_4.1.x86_64
--> Processing Dependency: libnih.so.1()(64bit) for package: upstart-0.6.5-12.el6_4.1.x86_64
--> Processing Dependency: libnih-dbus.so.1()(64bit) for package: upstart-0.6.5-12.el6_4.1.x86_64
---> Package util-linux-ng.x86_64 0:2.17.2-12.14.el6 will be installed
--> Processing Dependency: libuuid = 2.17.2-12.14.el6 for package: util-linux-ng-2.17.2-12.14.el6.x86_64
--> Processing Dependency: libblkid = 2.17.2-12.14.el6 for package: util-linux-ng-2.17.2-12.14.el6.x86_64
--> Processing Dependency: libuuid.so.1(UUID_1.0)(64bit) for package: util-linux-ng-2.17.2-12.14.el6.x86_64
--> Processing Dependency: libblkid.so.1(BLKID_1.0)(64bit) for package: util-linux-ng-2.17.2-12.14.el6.x86_64
--> Processing Dependency: libutempter.so.0(UTEMPTER_1.1)(64bit) for package: util-linux-ng-2.17.2-12.14.el6.x86_64
--> Processing Dependency: libblkid.so.1(BLKID_2.15)(64bit) for package: util-linux-ng-2.17.2-12.14.el6.x86_64
--> Processing Dependency: libblkid.so.1(BLKID_2.17)(64bit) for package: util-linux-ng-2.17.2-12.14.el6.x86_64
--> Processing Dependency: libutempter.so.0()(64bit) for package: util-linux-ng-2.17.2-12.14.el6.x86_64
--> Processing Dependency: libblkid.so.1()(64bit) for package: util-linux-ng-2.17.2-12.14.el6.x86_64
--> Processing Dependency: libuuid.so.1()(64bit) for package: util-linux-ng-2.17.2-12.14.el6.x86_64
---> Package yum-metadata-parser.x86_64 0:1.1.2-16.el6 will be installed
--> Processing Dependency: libxml2.so.2(LIBXML2_2.4.30)(64bit) for package: yum-metadata-parser-1.1.2-16.el6.x86_64
--> Processing Dependency: libxml2.so.2()(64bit) for package: yum-metadata-parser-1.1.2-16.el6.x86_64
--> Processing Dependency: libsqlite3.so.0()(64bit) for package: yum-metadata-parser-1.1.2-16.el6.x86_64
---> Package zlib.x86_64 0:1.2.3-29.el6 will be installed
--> Running transaction check
---> Package MAKEDEV.x86_64 0:3.24-6.el6 will be installed
---> Package basesystem.noarch 0:10.0-4.0.1.el6 will be installed
--> Processing Dependency: filesystem for package: basesystem-10.0-4.0.1.el6.noarch
---> Package binutils.x86_64 0:2.20.51.0.2-5.36.el6 will be installed
---> Package bzip2-libs.x86_64 0:1.0.5-7.el6_0 will be installed
---> Package ca-certificates.noarch 0:2013.1.94-65.0.el6 will be installed
--> Processing Dependency: p11-kit-trust >= 0.18.4-2 for package: ca-certificates-2013.1.94-65.0.el6.noarch
--> Processing Dependency: p11-kit >= 0.18.4-2 for package: ca-certificates-2013.1.94-65.0.el6.noarch
---> Package coreutils-libs.x86_64 0:8.4-31.0.1.el6 will be installed
---> Package cracklib.x86_64 0:2.8.16-4.el6 will be installed
--> Processing Dependency: gzip for package: cracklib-2.8.16-4.el6.x86_64
---> Package cracklib-dicts.x86_64 0:2.8.16-4.el6 will be installed
---> Package curl.x86_64 0:7.19.7-37.el6_4 will be installed
--> Processing Dependency: libcurl = 7.19.7-37.el6_4 for package: curl-7.19.7-37.el6_4.x86_64
--> Processing Dependency: libssh2.so.1()(64bit) for package: curl-7.19.7-37.el6_4.x86_64
--> Processing Dependency: libcurl.so.4()(64bit) for package: curl-7.19.7-37.el6_4.x86_64
---> Package db4.x86_64 0:4.7.25-18.el6_4 will be installed
---> Package db4-utils.x86_64 0:4.7.25-18.el6_4 will be installed
---> Package elfutils-libelf.x86_64 0:0.152-1.el6 will be installed
---> Package expat.x86_64 0:2.0.1-11.el6_2 will be installed
---> Package file-libs.x86_64 0:5.04-15.el6 will be installed
---> Package fipscheck.x86_64 0:1.2.0-7.el6 will be installed
---> Package gamin.x86_64 0:0.1.10-9.el6 will be installed
---> Package glibc-common.x86_64 0:2.12-1.132.el6 will be installed
--> Processing Dependency: tzdata >= 2003a for package: glibc-common-2.12-1.132.el6.x86_64
---> Package gmp.x86_64 0:4.3.1-7.el6_2.2 will be installed
--> Processing Dependency: libstdc++.so.6(CXXABI_1.3)(64bit) for package: gmp-4.3.1-7.el6_2.2.x86_64
--> Processing Dependency: libstdc++.so.6(GLIBCXX_3.4.11)(64bit) for package: gmp-4.3.1-7.el6_2.2.x86_64
--> Processing Dependency: libstdc++.so.6(GLIBCXX_3.4)(64bit) for package: gmp-4.3.1-7.el6_2.2.x86_64
--> Processing Dependency: libstdc++.so.6()(64bit) for package: gmp-4.3.1-7.el6_2.2.x86_64
---> Package gpgme.x86_64 0:1.1.8-3.el6 will be installed
--> Processing Dependency: gnupg2 for package: gpgme-1.1.8-3.el6.x86_64
--> Processing Dependency: libpth.so.20()(64bit) for package: gpgme-1.1.8-3.el6.x86_64
--> Processing Dependency: libgpg-error.so.0()(64bit) for package: gpgme-1.1.8-3.el6.x86_64
---> Package hwdata.noarch 0:0.233-9.1.el6 will be installed
---> Package info.x86_64 0:4.13a-8.el6 will be installed
---> Package iptables.x86_64 0:1.4.7-11.0.1.el6 will be installed
---> Package keyutils-libs.x86_64 0:1.4-4.el6 will be installed
---> Package libattr.x86_64 0:2.4.44-7.el6 will be installed
---> Package libblkid.x86_64 0:2.17.2-12.14.el6 will be installed
---> Package libcap.x86_64 0:2.16-5.5.el6 will be installed
---> Package libidn.x86_64 0:1.18-2.el6 will be installed
---> Package libnih.x86_64 0:1.0.1-7.el6 will be installed
---> Package libusb.x86_64 0:0.1.12-23.el6 will be installed
---> Package libutempter.x86_64 0:1.1.5-4.1.el6 will be installed
---> Package libuuid.x86_64 0:2.17.2-12.14.el6 will be installed
---> Package libxml2.x86_64 0:2.7.6-14.0.1.el6 will be installed
---> Package lua.x86_64 0:5.1.4-4.1.el6 will be installed
--> Processing Dependency: libreadline.so.6()(64bit) for package: lua-5.1.4-4.1.el6.x86_64
---> Package make.x86_64 1:3.81-20.el6 will be installed
---> Package ncurses.x86_64 0:5.7-3.20090208.el6 will be installed
---> Package ncurses-base.x86_64 0:5.7-3.20090208.el6 will be installed
---> Package nss-softokn.x86_64 0:3.14.3-9.el6 will be installed
---> Package nss-softokn-freebl.x86_64 0:3.14.3-9.el6 will be installed
---> Package nss-sysinit.x86_64 0:3.15.1-15.0.1.el6 will be installed
---> Package nss-util.x86_64 0:3.15.1-3.el6 will be installed
---> Package openldap.x86_64 0:2.4.23-32.el6_4.1 will be installed
--> Processing Dependency: nss-tools for package: openldap-2.4.23-32.el6_4.1.x86_64
--> Processing Dependency: libsasl2.so.2()(64bit) for package: openldap-2.4.23-32.el6_4.1.x86_64
---> Package pcre.x86_64 0:7.8-6.el6 will be installed
---> Package python-libs.x86_64 0:2.6.6-51.el6 will be installed
--> Processing Dependency: libffi.so.5()(64bit) for package: python-libs-2.6.6-51.el6.x86_64
--> Processing Dependency: libgdbm.so.2()(64bit) for package: python-libs-2.6.6-51.el6.x86_64
---> Package python-pycurl.x86_64 0:7.19.0-8.el6 will be installed
---> Package rpm-libs.x86_64 0:4.8.0-37.el6 will be installed
---> Package setup.noarch 0:2.8.14-20.el6_4.1 will be installed
---> Package shared-mime-info.x86_64 0:0.70-4.el6 will be installed
--> Processing Dependency: /usr/bin/pkg-config for package: shared-mime-info-0.70-4.el6.x86_64
--> Processing Dependency: pkgconfig for package: shared-mime-info-0.70-4.el6.x86_64
---> Package sqlite.x86_64 0:3.6.20-1.el6 will be installed
---> Package ustr.x86_64 0:1.0.4-9.1.el6 will be installed
---> Package xz-libs.x86_64 0:4.999.9-0.3.beta.20091007git.el6 will be installed
--> Running transaction check
---> Package cyrus-sasl-lib.x86_64 0:2.1.23-13.el6_3.1 will be installed
---> Package filesystem.x86_64 0:2.4.30-3.el6 will be installed
---> Package gdbm.x86_64 0:1.8.0-36.el6 will be installed
---> Package gnupg2.x86_64 0:2.0.14-6.el6_4 will be installed
--> Processing Dependency: libgcrypt.so.11(GCRYPT_1.2)(64bit) for package: gnupg2-2.0.14-6.el6_4.x86_64
--> Processing Dependency: pinentry for package: gnupg2-2.0.14-6.el6_4.x86_64
--> Processing Dependency: libgcrypt.so.11()(64bit) for package: gnupg2-2.0.14-6.el6_4.x86_64
---> Package gzip.x86_64 0:1.3.12-19.el6_4 will be installed
--> Processing Dependency: less for package: gzip-1.3.12-19.el6_4.x86_64
---> Package libcurl.x86_64 0:7.19.7-37.el6_4 will be installed
---> Package libffi.x86_64 0:3.0.5-3.2.el6 will be installed
---> Package libgpg-error.x86_64 0:1.7-4.el6 will be installed
---> Package libssh2.x86_64 0:1.4.2-1.el6 will be installed
---> Package libstdc++.x86_64 0:4.4.7-4.el6 will be installed
---> Package nss-tools.x86_64 0:3.15.1-15.0.1.el6 will be installed
---> Package p11-kit.x86_64 0:0.18.5-2.el6 will be installed
--> Processing Dependency: libtasn1.so.3(LIBTASN1_0_3)(64bit) for package: p11-kit-0.18.5-2.el6.x86_64
--> Processing Dependency: libtasn1.so.3()(64bit) for package: p11-kit-0.18.5-2.el6.x86_64
---> Package p11-kit-trust.x86_64 0:0.18.5-2.el6 will be installed
---> Package pkgconfig.x86_64 1:0.23-9.1.el6 will be installed
---> Package pth.x86_64 0:2.0.7-9.3.el6 will be installed
---> Package readline.x86_64 0:6.0-4.el6 will be installed
---> Package tzdata.noarch 0:2013g-1.el6 will be installed
--> Running transaction check
---> Package less.x86_64 0:436-10.el6 will be installed
--> Processing Dependency: groff for package: less-436-10.el6.x86_64
---> Package libgcrypt.x86_64 0:1.4.5-11.el6_4 will be installed
---> Package libtasn1.x86_64 0:2.3-3.el6_2.1 will be installed
---> Package pinentry.x86_64 0:0.7.6-6.el6 will be installed
--> Running transaction check
---> Package groff.x86_64 0:1.18.1.4-21.el6 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
==========================================================================================================================================================================
Package Arch Version Repository Size
==========================================================================================================================================================================
Installing:
chkconfig x86_64 1.3.49.3-2.el6_4.1 ol6_u5_base 158 k
dhclient x86_64 12:4.1.1-38.P1.0.1.el6 ol6_u5_base 317 k
initscripts x86_64 9.03.40-2.0.1.el6 ol6_u5_base 940 k
openssh-clients x86_64 5.3p1-94.el6 ol6_u5_base 401 k
openssh-server x86_64 5.3p1-94.el6 ol6_u5_base 311 k
oraclelinux-release x86_64 6:6Server-5.0.2 ol6_u5_base 22 k
passwd x86_64 0.77-4.el6_2.2 ol6_u5_base 89 k
policycoreutils x86_64 2.0.83-19.39.0.1.el6 ol6_u5_base 648 k
rootfiles noarch 8.1-6.1.el6 ol6_u5_base 6.3 k
rsyslog x86_64 5.8.10-8.0.1.el6 ol6_u5_base 649 k
vim-minimal x86_64 2:7.2.411-1.8.el6 ol6_u5_base 363 k
yum noarch 3.2.29-40.0.1.el6 ol6_u5_base 995 k
Installing for dependencies:
MAKEDEV x86_64 3.24-6.el6 ol6_u5_base 88 k
audit-libs x86_64 2.2-2.el6 ol6_u5_base 60 k
basesystem noarch 10.0-4.0.1.el6 ol6_u5_base 4.3 k
bash x86_64 4.1.2-15.el6_4 ol6_u5_base 904 k
binutils x86_64 2.20.51.0.2-5.36.el6 ol6_u5_base 2.8 M
bzip2-libs x86_64 1.0.5-7.el6_0 ol6_u5_base 36 k
ca-certificates noarch 2013.1.94-65.0.el6 ol6_u5_base 1.1 M
checkpolicy x86_64 2.0.22-1.el6 ol6_u5_base 189 k
coreutils x86_64 8.4-31.0.1.el6 ol6_u5_base 3.0 M
coreutils-libs x86_64 8.4-31.0.1.el6 ol6_u5_base 49 k
cpio x86_64 2.10-11.el6_3 ol6_u5_base 191 k
cracklib x86_64 2.8.16-4.el6 ol6_u5_base 70 k
cracklib-dicts x86_64 2.8.16-4.el6 ol6_u5_base 3.6 M
curl x86_64 7.19.7-37.el6_4 ol6_u5_base 193 k
cyrus-sasl-lib x86_64 2.1.23-13.el6_3.1 ol6_u5_base 135 k
db4 x86_64 4.7.25-18.el6_4 ol6_u5_base 562 k
db4-utils x86_64 4.7.25-18.el6_4 ol6_u5_base 130 k
dbus-glib x86_64 0.86-6.el6_4 ol6_u5_base 169 k
dbus-libs x86_64 1:1.2.24-7.0.1.el6_3 ol6_u5_base 126 k
dhcp-common x86_64 12:4.1.1-38.P1.0.1.el6 ol6_u5_base 141 k
diffutils x86_64 2.8.1-28.el6 ol6_u5_base 198 k
elfutils-libelf x86_64 0.152-1.el6 ol6_u5_base 169 k
ethtool x86_64 2:3.5-1.el6 ol6_u5_base 100 k
expat x86_64 2.0.1-11.el6_2 ol6_u5_base 76 k
file-libs x86_64 5.04-15.el6 ol6_u5_base 310 k
filesystem x86_64 2.4.30-3.el6 ol6_u5_base 1.0 M
findutils x86_64 1:4.4.2-6.el6 ol6_u5_base 447 k
fipscheck x86_64 1.2.0-7.el6 ol6_u5_base 14 k
fipscheck-lib x86_64 1.2.0-7.el6 ol6_u5_base 7.8 k
gamin x86_64 0.1.10-9.el6 ol6_u5_base 122 k
gawk x86_64 3.1.7-10.el6 ol6_u5_base 774 k
gdbm x86_64 1.8.0-36.el6 ol6_u5_base 28 k
glib2 x86_64 2.26.1-3.el6 ol6_u5_base 1.6 M
glibc x86_64 2.12-1.132.el6 ol6_u5_base 3.8 M
glibc-common x86_64 2.12-1.132.el6 ol6_u5_base 14 M
gmp x86_64 4.3.1-7.el6_2.2 ol6_u5_base 206 k
gnupg2 x86_64 2.0.14-6.el6_4 ol6_u5_base 1.6 M
gpgme x86_64 1.1.8-3.el6 ol6_u5_base 144 k
grep x86_64 2.6.3-4.el6 ol6_u5_base 228 k
groff x86_64 1.18.1.4-21.el6 ol6_u5_base 1.5 M
gzip x86_64 1.3.12-19.el6_4 ol6_u5_base 115 k
hwdata noarch 0.233-9.1.el6 ol6_u5_base 1.2 M
info x86_64 4.13a-8.el6 ol6_u5_base 177 k
iproute x86_64 2.6.32-31.el6 ol6_u5_base 364 k
iptables x86_64 1.4.7-11.0.1.el6 ol6_u5_base 252 k
iputils x86_64 20071127-17.el6_4.2 ol6_u5_base 120 k
keyutils-libs x86_64 1.4-4.el6 ol6_u5_base 19 k
krb5-libs x86_64 1.10.3-10.el6_4.6 ol6_u5_base 760 k
less x86_64 436-10.el6 ol6_u5_base 106 k
libacl x86_64 2.2.49-6.el6 ol6_u5_base 23 k
libattr x86_64 2.4.44-7.el6 ol6_u5_base 15 k
libblkid x86_64 2.17.2-12.14.el6 ol6_u5_base 114 k
libcap x86_64 2.16-5.5.el6 ol6_u5_base 31 k
libcap-ng x86_64 0.6.4-3.el6_0.1 ol6_u5_base 21 k
libcom_err x86_64 1.42.8-1.0.1.el6 ol6_u5_base 36 k
libcurl x86_64 7.19.7-37.el6_4 ol6_u5_base 164 k
libedit x86_64 2.11-4.20080712cvs.1.el6 ol6_u5_base 74 k
libffi x86_64 3.0.5-3.2.el6 ol6_u5_base 25 k
libgcc x86_64 4.4.7-4.el6 ol6_u5_base 100 k
libgcrypt x86_64 1.4.5-11.el6_4 ol6_u5_base 228 k
libgpg-error x86_64 1.7-4.el6 ol6_u5_base 59 k
libidn x86_64 1.18-2.el6 ol6_u5_base 205 k
libnih x86_64 1.0.1-7.el6 ol6_u5_base 137 k
libselinux x86_64 2.0.94-5.3.el6_4.1 ol6_u5_base 107 k
libselinux-utils x86_64 2.0.94-5.3.el6_4.1 ol6_u5_base 80 k
libsemanage x86_64 2.0.43-4.2.el6 ol6_u5_base 103 k
libsepol x86_64 2.0.41-4.el6 ol6_u5_base 128 k
libssh2 x86_64 1.4.2-1.el6 ol6_u5_base 122 k
libstdc++ x86_64 4.4.7-4.el6 ol6_u5_base 292 k
libtasn1 x86_64 2.3-3.el6_2.1 ol6_u5_base 237 k
libusb x86_64 0.1.12-23.el6 ol6_u5_base 27 k
libuser x86_64 0.56.13-5.el6 ol6_u5_base 368 k
libutempter x86_64 1.1.5-4.1.el6 ol6_u5_base 22 k
libuuid x86_64 2.17.2-12.14.el6 ol6_u5_base 67 k
libxml2 x86_64 2.7.6-14.0.1.el6 ol6_u5_base 799 k
logrotate x86_64 3.7.8-17.el6 ol6_u5_base 55 k
lua x86_64 5.1.4-4.1.el6 ol6_u5_base 184 k
make x86_64 1:3.81-20.el6 ol6_u5_base 388 k
mingetty x86_64 1.08-5.el6 ol6_u5_base 20 k
module-init-tools x86_64 3.9-21.0.1.el6_4 ol6_u5_base 463 k
ncurses x86_64 5.7-3.20090208.el6 ol6_u5_base 267 k
ncurses-base x86_64 5.7-3.20090208.el6 ol6_u5_base 60 k
ncurses-libs x86_64 5.7-3.20090208.el6 ol6_u5_base 245 k
net-tools x86_64 1.60-110.el6_2 ol6_u5_base 268 k
nspr x86_64 4.10.0-1.el6 ol6_u5_base 112 k
nss x86_64 3.15.1-15.0.1.el6 ol6_u5_base 820 k
nss-softokn x86_64 3.14.3-9.el6 ol6_u5_base 264 k
nss-softokn-freebl x86_64 3.14.3-9.el6 ol6_u5_base 157 k
nss-sysinit x86_64 3.15.1-15.0.1.el6 ol6_u5_base 39 k
nss-tools x86_64 3.15.1-15.0.1.el6 ol6_u5_base 356 k
nss-util x86_64 3.15.1-3.el6 ol6_u5_base 63 k
openldap x86_64 2.4.23-32.el6_4.1 ol6_u5_base 265 k
openssh x86_64 5.3p1-94.el6 ol6_u5_base 257 k
openssl x86_64 1.0.1e-15.el6 ol6_u5_base 1.5 M
oracle-logos noarch 60.0.14-1.0.1.el6 ol6_u5_base 12 M
p11-kit x86_64 0.18.5-2.el6 ol6_u5_base 93 k
p11-kit-trust x86_64 0.18.5-2.el6 ol6_u5_base 71 k
pam x86_64 1.1.1-17.el6 ol6_u5_base 658 k
pcre x86_64 7.8-6.el6 ol6_u5_base 194 k
pinentry x86_64 0.7.6-6.el6 ol6_u5_base 65 k
pkgconfig x86_64 1:0.23-9.1.el6 ol6_u5_base 70 k
popt x86_64 1.13-7.el6 ol6_u5_base 38 k
procps x86_64 3.2.8-25.el6 ol6_u5_base 211 k
psmisc x86_64 22.6-15.el6_0.1 ol6_u5_base 80 k
pth x86_64 2.0.7-9.3.el6 ol6_u5_base 85 k
pygpgme x86_64 0.1-18.20090824bzr68.el6 ol6_u5_base 70 k
python x86_64 2.6.6-51.el6 ol6_u5_base 73 k
python-iniparse noarch 0.3.1-2.1.el6 ol6_u5_base 36 k
python-libs x86_64 2.6.6-51.el6 ol6_u5_base 5.3 M
python-pycurl x86_64 7.19.0-8.el6 ol6_u5_base 76 k
python-urlgrabber noarch 3.9.1-9.el6 ol6_u5_base 85 k
readline x86_64 6.0-4.el6 ol6_u5_base 178 k
redhat-release-server x86_64 1:6Server-6.5.0.1.0.1.el6 ol6_u5_base 2.6 k
rpm x86_64 4.8.0-37.el6 ol6_u5_base 901 k
rpm-libs x86_64 4.8.0-37.el6 ol6_u5_base 312 k
rpm-python x86_64 4.8.0-37.el6 ol6_u5_base 56 k
sed x86_64 4.2.1-10.el6 ol6_u5_base 215 k
setup noarch 2.8.14-20.el6_4.1 ol6_u5_base 150 k
shadow-utils x86_64 2:4.1.4.2-13.el6 ol6_u5_base 896 k
shared-mime-info x86_64 0.70-4.el6 ol6_u5_base 208 k
sqlite x86_64 3.6.20-1.el6 ol6_u5_base 301 k
sysvinit-tools x86_64 2.87-5.dsf.el6 ol6_u5_base 59 k
tcp_wrappers-libs x86_64 7.6-57.el6 ol6_u5_base 62 k
tzdata noarch 2013g-1.el6 ol6_u5_base 457 k
udev x86_64 147-2.51.0.1.el6 ol6_u5_base 348 k
upstart x86_64 0.6.5-12.el6_4.1 ol6_u5_base 176 k
ustr x86_64 1.0.4-9.1.el6 ol6_u5_base 85 k
util-linux-ng x86_64 2.17.2-12.14.el6 ol6_u5_base 1.5 M
xz-libs x86_64 4.999.9-0.3.beta.20091007git.el6 ol6_u5_base 89 k
yum-metadata-parser x86_64 1.1.2-16.el6 ol6_u5_base 26 k
zlib x86_64 1.2.3-29.el6 ol6_u5_base 72 k
Transaction Summary
==========================================================================================================================================================================
Install 143 Packages
Total download size: 82 M
Installed size: 303 M
Downloading Packages:
(1/143): MAKEDEV-3.24-6.el6.x86_64.rpm | 88 kB 00:00
(2/143): audit-libs-2.2-2.el6.x86_64.rpm | 60 kB 00:00
(3/143): basesystem-10.0-4.0.1.el6.noarch.rpm | 4.3 kB 00:00
(4/143): bash-4.1.2-15.el6_4.x86_64.rpm | 904 kB 00:00
(5/143): binutils-2.20.51.0.2-5.36.el6.x86_64.rpm | 2.8 MB 00:01
(6/143): bzip2-libs-1.0.5-7.el6_0.x86_64.rpm | 36 kB 00:00
(7/143): ca-certificates-2013.1.94-65.0.el6.noarch.rpm | 1.1 MB 00:00
(8/143): checkpolicy-2.0.22-1.el6.x86_64.rpm | 189 kB 00:00
(9/143): chkconfig-1.3.49.3-2.el6_4.1.x86_64.rpm | 158 kB 00:00
(10/143): coreutils-8.4-31.0.1.el6.x86_64.rpm | 3.0 MB 00:02
(11/143): coreutils-libs-8.4-31.0.1.el6.x86_64.rpm | 49 kB 00:00
(12/143): cpio-2.10-11.el6_3.x86_64.rpm | 191 kB 00:00
(13/143): cracklib-2.8.16-4.el6.x86_64.rpm | 70 kB 00:00
(14/143): cracklib-dicts-2.8.16-4.el6.x86_64.rpm | 3.6 MB 00:03
(15/143): curl-7.19.7-37.el6_4.x86_64.rpm | 193 kB 00:00
(16/143): cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64.rpm | 135 kB 00:00
(17/143): db4-4.7.25-18.el6_4.x86_64.rpm | 562 kB 00:00
(18/143): db4-utils-4.7.25-18.el6_4.x86_64.rpm | 130 kB 00:00
(19/143): dbus-glib-0.86-6.el6_4.x86_64.rpm | 169 kB 00:00
(20/143): dbus-libs-1.2.24-7.0.1.el6_3.x86_64.rpm | 126 kB 00:00
(21/143): dhclient-4.1.1-38.P1.0.1.el6.x86_64.rpm | 317 kB 00:00
(22/143): dhcp-common-4.1.1-38.P1.0.1.el6.x86_64.rpm | 141 kB 00:00
(23/143): diffutils-2.8.1-28.el6.x86_64.rpm | 198 kB 00:00
(24/143): elfutils-libelf-0.152-1.el6.x86_64.rpm | 169 kB 00:00
(25/143): ethtool-3.5-1.el6.x86_64.rpm | 100 kB 00:00
(26/143): expat-2.0.1-11.el6_2.x86_64.rpm | 76 kB 00:00
(27/143): file-libs-5.04-15.el6.x86_64.rpm | 310 kB 00:00
(28/143): filesystem-2.4.30-3.el6.x86_64.rpm | 1.0 MB 00:00
(29/143): findutils-4.4.2-6.el6.x86_64.rpm | 447 kB 00:00
(30/143): fipscheck-1.2.0-7.el6.x86_64.rpm | 14 kB 00:00
(31/143): fipscheck-lib-1.2.0-7.el6.x86_64.rpm | 7.8 kB 00:00
(32/143): gamin-0.1.10-9.el6.x86_64.rpm | 122 kB 00:00
(33/143): gawk-3.1.7-10.el6.x86_64.rpm | 774 kB 00:00
(34/143): gdbm-1.8.0-36.el6.x86_64.rpm | 28 kB 00:00
(35/143): glib2-2.26.1-3.el6.x86_64.rpm | 1.6 MB 00:01
(36/143): glibc-2.12-1.132.el6.x86_64.rpm | 3.8 MB 00:02
(37/143): glibc-common-2.12-1.132.el6.x86_64.rpm | 14 MB 00:09
(38/143): gmp-4.3.1-7.el6_2.2.x86_64.rpm | 206 kB 00:00
(39/143): gnupg2-2.0.14-6.el6_4.x86_64.rpm | 1.6 MB 00:01
(40/143): gpgme-1.1.8-3.el6.x86_64.rpm | 144 kB 00:00
(41/143): grep-2.6.3-4.el6.x86_64.rpm | 228 kB 00:00
(42/143): groff-1.18.1.4-21.el6.x86_64.rpm | 1.5 MB 00:01
(43/143): gzip-1.3.12-19.el6_4.x86_64.rpm | 115 kB 00:00
(44/143): hwdata-0.233-9.1.el6.noarch.rpm | 1.2 MB 00:01
(45/143): info-4.13a-8.el6.x86_64.rpm | 177 kB 00:00
(46/143): initscripts-9.03.40-2.0.1.el6.x86_64.rpm | 940 kB 00:01
(47/143): iproute-2.6.32-31.el6.x86_64.rpm | 364 kB 00:00
(48/143): iptables-1.4.7-11.0.1.el6.x86_64.rpm | 252 kB 00:00
(49/143): iputils-20071127-17.el6_4.2.x86_64.rpm | 120 kB 00:00
(50/143): keyutils-libs-1.4-4.el6.x86_64.rpm | 19 kB 00:00
(51/143): krb5-libs-1.10.3-10.el6_4.6.x86_64.rpm | 760 kB 00:01
(52/143): less-436-10.el6.x86_64.rpm | 106 kB 00:00
(53/143): libacl-2.2.49-6.el6.x86_64.rpm | 23 kB 00:00
(54/143): libattr-2.4.44-7.el6.x86_64.rpm | 15 kB 00:00
(55/143): libblkid-2.17.2-12.14.el6.x86_64.rpm | 114 kB 00:00
(56/143): libcap-2.16-5.5.el6.x86_64.rpm | 31 kB 00:00
(57/143): libcap-ng-0.6.4-3.el6_0.1.x86_64.rpm | 21 kB 00:00
(58/143): libcom_err-1.42.8-1.0.1.el6.x86_64.rpm | 36 kB 00:00
(59/143): libcurl-7.19.7-37.el6_4.x86_64.rpm | 164 kB 00:00
(60/143): libedit-2.11-4.20080712cvs.1.el6.x86_64.rpm | 74 kB 00:00
(61/143): libffi-3.0.5-3.2.el6.x86_64.rpm | 25 kB 00:00
(62/143): libgcc-4.4.7-4.el6.x86_64.rpm | 100 kB 00:00
(63/143): libgcrypt-1.4.5-11.el6_4.x86_64.rpm | 228 kB 00:00
(64/143): libgpg-error-1.7-4.el6.x86_64.rpm | 59 kB 00:00
(65/143): libidn-1.18-2.el6.x86_64.rpm | 205 kB 00:00
(66/143): libnih-1.0.1-7.el6.x86_64.rpm | 137 kB 00:00
(67/143): libselinux-2.0.94-5.3.el6_4.1.x86_64.rpm | 107 kB 00:00
(68/143): libselinux-utils-2.0.94-5.3.el6_4.1.x86_64.rpm | 80 kB 00:00
(69/143): libsemanage-2.0.43-4.2.el6.x86_64.rpm | 103 kB 00:00
(70/143): libsepol-2.0.41-4.el6.x86_64.rpm | 128 kB 00:00
(71/143): libssh2-1.4.2-1.el6.x86_64.rpm | 122 kB 00:00
(72/143): libstdc++-4.4.7-4.el6.x86_64.rpm | 292 kB 00:00
(73/143): libtasn1-2.3-3.el6_2.1.x86_64.rpm | 237 kB 00:00
(74/143): libusb-0.1.12-23.el6.x86_64.rpm | 27 kB 00:00
(75/143): libuser-0.56.13-5.el6.x86_64.rpm | 368 kB 00:00
(76/143): libutempter-1.1.5-4.1.el6.x86_64.rpm | 22 kB 00:00
(77/143): libuuid-2.17.2-12.14.el6.x86_64.rpm | 67 kB 00:00
(78/143): libxml2-2.7.6-14.0.1.el6.x86_64.rpm | 799 kB 00:00
(79/143): logrotate-3.7.8-17.el6.x86_64.rpm | 55 kB 00:00
(80/143): lua-5.1.4-4.1.el6.x86_64.rpm | 184 kB 00:00
(81/143): make-3.81-20.el6.x86_64.rpm | 388 kB 00:00
(82/143): mingetty-1.08-5.el6.x86_64.rpm | 20 kB 00:00
(83/143): module-init-tools-3.9-21.0.1.el6_4.x86_64.rpm | 463 kB 00:00
(84/143): ncurses-5.7-3.20090208.el6.x86_64.rpm | 267 kB 00:00
(85/143): ncurses-base-5.7-3.20090208.el6.x86_64.rpm | 60 kB 00:00
(86/143): ncurses-libs-5.7-3.20090208.el6.x86_64.rpm | 245 kB 00:00
(87/143): net-tools-1.60-110.el6_2.x86_64.rpm | 268 kB 00:00
(88/143): nspr-4.10.0-1.el6.x86_64.rpm | 112 kB 00:00
(89/143): nss-3.15.1-15.0.1.el6.x86_64.rpm | 820 kB 00:00
(90/143): nss-softokn-3.14.3-9.el6.x86_64.rpm | 264 kB 00:00
(91/143): nss-softokn-freebl-3.14.3-9.el6.x86_64.rpm | 157 kB 00:00
(92/143): nss-sysinit-3.15.1-15.0.1.el6.x86_64.rpm | 39 kB 00:00
(93/143): nss-tools-3.15.1-15.0.1.el6.x86_64.rpm | 356 kB 00:00
(94/143): nss-util-3.15.1-3.el6.x86_64.rpm | 63 kB 00:00
(95/143): openldap-2.4.23-32.el6_4.1.x86_64.rpm | 265 kB 00:00
(96/143): openssh-5.3p1-94.el6.x86_64.rpm | 257 kB 00:00
(97/143): openssh-clients-5.3p1-94.el6.x86_64.rpm | 401 kB 00:00
(98/143): openssh-server-5.3p1-94.el6.x86_64.rpm | 311 kB 00:00
(99/143): openssl-1.0.1e-15.el6.x86_64.rpm | 1.5 MB 00:00
(100/143): oracle-logos-60.0.14-1.0.1.el6.noarch.rpm | 12 MB 00:08
(101/143): oraclelinux-release-6Server-5.0.2.x86_64.rpm | 22 kB 00:00
(102/143): p11-kit-0.18.5-2.el6.x86_64.rpm | 93 kB 00:00
(103/143): p11-kit-trust-0.18.5-2.el6.x86_64.rpm | 71 kB 00:00
(104/143): pam-1.1.1-17.el6.x86_64.rpm | 658 kB 00:00
(105/143): passwd-0.77-4.el6_2.2.x86_64.rpm | 89 kB 00:00
(106/143): pcre-7.8-6.el6.x86_64.rpm | 194 kB 00:00
(107/143): pinentry-0.7.6-6.el6.x86_64.rpm | 65 kB 00:00
(108/143): pkgconfig-0.23-9.1.el6.x86_64.rpm | 70 kB 00:00
(109/143): policycoreutils-2.0.83-19.39.0.1.el6.x86_64.rpm | 648 kB 00:00
(110/143): popt-1.13-7.el6.x86_64.rpm | 38 kB 00:00
(111/143): procps-3.2.8-25.el6.x86_64.rpm | 211 kB 00:00
(112/143): psmisc-22.6-15.el6_0.1.x86_64.rpm | 80 kB 00:00
(113/143): pth-2.0.7-9.3.el6.x86_64.rpm | 85 kB 00:00
(114/143): pygpgme-0.1-18.20090824bzr68.el6.x86_64.rpm | 70 kB 00:00
(115/143): python-2.6.6-51.el6.x86_64.rpm | 73 kB 00:00
(116/143): python-iniparse-0.3.1-2.1.el6.noarch.rpm | 36 kB 00:00
(117/143): python-libs-2.6.6-51.el6.x86_64.rpm | 5.3 MB 00:04
(118/143): python-pycurl-7.19.0-8.el6.x86_64.rpm | 76 kB 00:00
(119/143): python-urlgrabber-3.9.1-9.el6.noarch.rpm | 85 kB 00:00
(120/143): readline-6.0-4.el6.x86_64.rpm | 178 kB 00:00
(121/143): redhat-release-server-6Server-6.5.0.1.0.1.el6.x86_64.rpm | 2.6 kB 00:00
(122/143): rootfiles-8.1-6.1.el6.noarch.rpm | 6.3 kB 00:00
(123/143): rpm-4.8.0-37.el6.x86_64.rpm | 901 kB 00:00
(124/143): rpm-libs-4.8.0-37.el6.x86_64.rpm | 312 kB 00:00
(125/143): rpm-python-4.8.0-37.el6.x86_64.rpm | 56 kB 00:00
(126/143): rsyslog-5.8.10-8.0.1.el6.x86_64.rpm | 649 kB 00:00
(127/143): sed-4.2.1-10.el6.x86_64.rpm | 215 kB 00:00
(128/143): setup-2.8.14-20.el6_4.1.noarch.rpm | 150 kB 00:00
(129/143): shadow-utils-4.1.4.2-13.el6.x86_64.rpm | 896 kB 00:00
(130/143): shared-mime-info-0.70-4.el6.x86_64.rpm | 208 kB 00:00
(131/143): sqlite-3.6.20-1.el6.x86_64.rpm | 301 kB 00:00
(132/143): sysvinit-tools-2.87-5.dsf.el6.x86_64.rpm | 59 kB 00:00
(133/143): tcp_wrappers-libs-7.6-57.el6.x86_64.rpm | 62 kB 00:00
(134/143): tzdata-2013g-1.el6.noarch.rpm | 457 kB 00:00
(135/143): udev-147-2.51.0.1.el6.x86_64.rpm | 348 kB 00:00
(136/143): upstart-0.6.5-12.el6_4.1.x86_64.rpm | 176 kB 00:00
(137/143): ustr-1.0.4-9.1.el6.x86_64.rpm | 85 kB 00:00
(138/143): util-linux-ng-2.17.2-12.14.el6.x86_64.rpm | 1.5 MB 00:00
(139/143): vim-minimal-7.2.411-1.8.el6.x86_64.rpm | 363 kB 00:00
(140/143): xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64.rpm | 89 kB 00:00
(141/143): yum-3.2.29-40.0.1.el6.noarch.rpm | 995 kB 00:00
(142/143): yum-metadata-parser-1.1.2-16.el6.x86_64.rpm | 26 kB 00:00
(143/143): zlib-1.2.3-29.el6.x86_64.rpm | 72 kB 00:00
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 798 kB/s | 82 MB 01:45
Running Transaction Check
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : libgcc-4.4.7-4.el6.x86_64 1/143
Installing : setup-2.8.14-20.el6_4.1.noarch 2/143
Installing : filesystem-2.4.30-3.el6.x86_64 3/143
Installing : basesystem-10.0-4.0.1.el6.noarch 4/143
Installing : tzdata-2013g-1.el6.noarch 5/143
Installing : 1:redhat-release-server-6Server-6.5.0.1.0.1.el6.x86_64 6/143
Installing : 12:dhcp-common-4.1.1-38.P1.0.1.el6.x86_64 7/143
Installing : ncurses-base-5.7-3.20090208.el6.x86_64 8/143
Installing : nss-softokn-freebl-3.14.3-9.el6.x86_64 9/143
Installing : glibc-common-2.12-1.132.el6.x86_64 10/143
Installing : glibc-2.12-1.132.el6.x86_64 11/143
Installing : ncurses-libs-5.7-3.20090208.el6.x86_64 12/143
Installing : bash-4.1.2-15.el6_4.x86_64 13/143
Installing : libattr-2.4.44-7.el6.x86_64 14/143
Installing : libcap-2.16-5.5.el6.x86_64 15/143
Installing : zlib-1.2.3-29.el6.x86_64 16/143
Installing : info-4.13a-8.el6.x86_64 17/143
Installing : nspr-4.10.0-1.el6.x86_64 18/143
Installing : popt-1.13-7.el6.x86_64 19/143
Installing : chkconfig-1.3.49.3-2.el6_4.1.x86_64 20/143
Installing : libacl-2.2.49-6.el6.x86_64 21/143
Installing : db4-4.7.25-18.el6_4.x86_64 22/143
Installing : audit-libs-2.2-2.el6.x86_64 23/143
Installing : nss-util-3.15.1-3.el6.x86_64 24/143
Installing : libcom_err-1.42.8-1.0.1.el6.x86_64 25/143
Installing : libsepol-2.0.41-4.el6.x86_64 26/143
Installing : libselinux-2.0.94-5.3.el6_4.1.x86_64 27/143
Installing : bzip2-libs-1.0.5-7.el6_0.x86_64 28/143
Installing : sed-4.2.1-10.el6.x86_64 29/143
Installing : 2:shadow-utils-4.1.4.2-13.el6.x86_64 30/143
Installing : readline-6.0-4.el6.x86_64 31/143
Installing : 1:dbus-libs-1.2.24-7.0.1.el6_3.x86_64 32/143
Installing : libstdc++-4.4.7-4.el6.x86_64 33/143
Installing : lua-5.1.4-4.1.el6.x86_64 34/143
Installing : sqlite-3.6.20-1.el6.x86_64 35/143
Installing : libidn-1.18-2.el6.x86_64 36/143
Installing : file-libs-5.04-15.el6.x86_64 37/143
Installing : xz-libs-4.999.9-0.3.beta.20091007git.el6.x86_64 38/143
Installing : libgpg-error-1.7-4.el6.x86_64 39/143
Installing : elfutils-libelf-0.152-1.el6.x86_64 40/143
Installing : pcre-7.8-6.el6.x86_64 41/143
Installing : grep-2.6.3-4.el6.x86_64 42/143
Installing : gawk-3.1.7-10.el6.x86_64 43/143
Installing : libxml2-2.7.6-14.0.1.el6.x86_64 44/143
Installing : pth-2.0.7-9.3.el6.x86_64 45/143
Installing : libtasn1-2.3-3.el6_2.1.x86_64 46/143
Installing : p11-kit-0.18.5-2.el6.x86_64 47/143
Installing : libuuid-2.17.2-12.14.el6.x86_64 48/143
Installing : expat-2.0.1-11.el6_2.x86_64 49/143
Installing : sysvinit-tools-2.87-5.dsf.el6.x86_64 50/143
Installing : libblkid-2.17.2-12.14.el6.x86_64 51/143
Installing : p11-kit-trust-0.18.5-2.el6.x86_64 52/143
Installing : ca-certificates-2013.1.94-65.0.el6.noarch 53/143
Installing : libgcrypt-1.4.5-11.el6_4.x86_64 54/143
Installing : nss-softokn-3.14.3-9.el6.x86_64 55/143
Installing : gmp-4.3.1-7.el6_2.2.x86_64 56/143
Installing : libusb-0.1.12-23.el6.x86_64 57/143
Installing : libnih-1.0.1-7.el6.x86_64 58/143
Installing : upstart-0.6.5-12.el6_4.1.x86_64 59/143
Installing : MAKEDEV-3.24-6.el6.x86_64 60/143
Installing : libutempter-1.1.5-4.1.el6.x86_64 61/143
Installing : procps-3.2.8-25.el6.x86_64 62/143
Installing : libselinux-utils-2.0.94-5.3.el6_4.1.x86_64 63/143
Installing : net-tools-1.60-110.el6_2.x86_64 64/143
Installing : psmisc-22.6-15.el6_0.1.x86_64 65/143
Installing : checkpolicy-2.0.22-1.el6.x86_64 66/143
Installing : 1:findutils-4.4.2-6.el6.x86_64 67/143
Installing : cyrus-sasl-lib-2.1.23-13.el6_3.1.x86_64 68/143
Installing : db4-utils-4.7.25-18.el6_4.x86_64 69/143
Installing : pinentry-0.7.6-6.el6.x86_64 70/143
Installing : diffutils-2.8.1-28.el6.x86_64 71/143
Installing : binutils-2.20.51.0.2-5.36.el6.x86_64 72/143
Installing : 1:make-3.81-20.el6.x86_64 73/143
Installing : cpio-2.10-11.el6_3.x86_64 74/143
Installing : 6:oraclelinux-release-6Server-5.0.2.x86_64 75/143
Installing : ncurses-5.7-3.20090208.el6.x86_64 76/143
Installing : groff-1.18.1.4-21.el6.x86_64 77/143
Installing : less-436-10.el6.x86_64 78/143
Installing : gzip-1.3.12-19.el6_4.x86_64 79/143
Installing : cracklib-2.8.16-4.el6.x86_64 80/143
Installing : cracklib-dicts-2.8.16-4.el6.x86_64 81/143
Installing : coreutils-libs-8.4-31.0.1.el6.x86_64 82/143
Installing : coreutils-8.4-31.0.1.el6.x86_64 83/143
Installing : pam-1.1.1-17.el6.x86_64 84/143
Installing : module-init-tools-3.9-21.0.1.el6_4.x86_64 85/143
Installing : hwdata-0.233-9.1.el6.noarch 86/143
Installing : oracle-logos-60.0.14-1.0.1.el6.noarch 87/143
Installing : logrotate-3.7.8-17.el6.x86_64 88/143
Installing : nss-3.15.1-15.0.1.el6.x86_64 89/143
Installing : nss-sysinit-3.15.1-15.0.1.el6.x86_64 90/143
Installing : nss-tools-3.15.1-15.0.1.el6.x86_64 91/143
Installing : openldap-2.4.23-32.el6_4.1.x86_64 92/143
Installing : libedit-2.11-4.20080712cvs.1.el6.x86_64 93/143
Installing : ustr-1.0.4-9.1.el6.x86_64 94/143
Installing : libsemanage-2.0.43-4.2.el6.x86_64 95/143
Installing : libffi-3.0.5-3.2.el6.x86_64 96/143
Installing : keyutils-libs-1.4-4.el6.x86_64 97/143
Installing : krb5-libs-1.10.3-10.el6_4.6.x86_64 98/143
Installing : openssl-1.0.1e-15.el6.x86_64 99/143
Installing : libssh2-1.4.2-1.el6.x86_64 100/143
Installing : libcurl-7.19.7-37.el6_4.x86_64 101/143
Installing : curl-7.19.7-37.el6_4.x86_64 102/143
Installing : rpm-libs-4.8.0-37.el6.x86_64 103/143
Installing : rpm-4.8.0-37.el6.x86_64 104/143
Installing : gnupg2-2.0.14-6.el6_4.x86_64 105/143
Installing : gpgme-1.1.8-3.el6.x86_64 106/143
Installing : fipscheck-lib-1.2.0-7.el6.x86_64 107/143
Installing : fipscheck-1.2.0-7.el6.x86_64 108/143
Installing : 1:pkgconfig-0.23-9.1.el6.x86_64 109/143
Installing : shared-mime-info-0.70-4.el6.x86_64 110/143
Non-fatal POSTIN scriptlet failure in rpm package shared-mime-info-0.70-4.el6.x86_64
warning: %post(shared-mime-info-0.70-4.el6.x86_64) scriptlet failed, exit status 127
Installing : glib2-2.26.1-3.el6.x86_64 111/143
Installing : gamin-0.1.10-9.el6.x86_64 112/143
Installing : dbus-glib-0.86-6.el6_4.x86_64 113/143
Installing : libuser-0.56.13-5.el6.x86_64 114/143
Installing : libcap-ng-0.6.4-3.el6_0.1.x86_64 115/143
Installing : 2:ethtool-3.5-1.el6.x86_64 116/143
Installing : gdbm-1.8.0-36.el6.x86_64 117/143
Installing : python-2.6.6-51.el6.x86_64 118/143
Installing : python-libs-2.6.6-51.el6.x86_64 119/143
Installing : rpm-python-4.8.0-37.el6.x86_64 120/143
Installing : yum-metadata-parser-1.1.2-16.el6.x86_64 121/143
Installing : python-pycurl-7.19.0-8.el6.x86_64 122/143
Installing : python-urlgrabber-3.9.1-9.el6.noarch 123/143
Installing : pygpgme-0.1-18.20090824bzr68.el6.x86_64 124/143
Installing : python-iniparse-0.3.1-2.1.el6.noarch 125/143
Installing : tcp_wrappers-libs-7.6-57.el6.x86_64 126/143
Installing : mingetty-1.08-5.el6.x86_64 127/143
Installing : policycoreutils-2.0.83-19.39.0.1.el6.x86_64 128/143
Installing : iptables-1.4.7-11.0.1.el6.x86_64 129/143
Installing : iproute-2.6.32-31.el6.x86_64 130/143
Installing : iputils-20071127-17.el6_4.2.x86_64 131/143
Installing : initscripts-9.03.40-2.0.1.el6.x86_64 132/143
Installing : util-linux-ng-2.17.2-12.14.el6.x86_64 133/143
Installing : udev-147-2.51.0.1.el6.x86_64 134/143
Installing : openssh-5.3p1-94.el6.x86_64 135/143
Installing : openssh-clients-5.3p1-94.el6.x86_64 136/143
Installing : openssh-server-5.3p1-94.el6.x86_64 137/143
Installing : 12:dhclient-4.1.1-38.P1.0.1.el6.x86_64 138/143
Installing : rsyslog-5.8.10-8.0.1.el6.x86_64 139/143
Installing : yum-3.2.29-40.0.1.el6.noarch 140/143
Installing : passwd-0.77-4.el6_2.2.x86_64 141/143
Installing : 2:vim-minimal-7.2.411-1.8.el6.x86_64 142/143
Installing : rootfiles-8.1-6.1.el6.noarch 143/143
Installed:
chkconfig.x86_64 0:1.3.49.3-2.el6_4.1 dhclient.x86_64 12:4.1.1-38.P1.0.1.el6 initscripts.x86_64 0:9.03.40-2.0.1.el6 openssh-clients.x86_64 0:5.3p1-94.el6
openssh-server.x86_64 0:5.3p1-94.el6 oraclelinux-release.x86_64 6:6Server-5.0.2 passwd.x86_64 0:0.77-4.el6_2.2 policycoreutils.x86_64 0:2.0.83-19.39.0.1.el6
rootfiles.noarch 0:8.1-6.1.el6 rsyslog.x86_64 0:5.8.10-8.0.1.el6 vim-minimal.x86_64 2:7.2.411-1.8.el6 yum.noarch 0:3.2.29-40.0.1.el6
Dependency Installed:
MAKEDEV.x86_64 0:3.24-6.el6 audit-libs.x86_64 0:2.2-2.el6 basesystem.noarch 0:10.0-4.0.1.el6
bash.x86_64 0:4.1.2-15.el6_4 binutils.x86_64 0:2.20.51.0.2-5.36.el6 bzip2-libs.x86_64 0:1.0.5-7.el6_0
ca-certificates.noarch 0:2013.1.94-65.0.el6 checkpolicy.x86_64 0:2.0.22-1.el6 coreutils.x86_64 0:8.4-31.0.1.el6
coreutils-libs.x86_64 0:8.4-31.0.1.el6 cpio.x86_64 0:2.10-11.el6_3 cracklib.x86_64 0:2.8.16-4.el6
cracklib-dicts.x86_64 0:2.8.16-4.el6 curl.x86_64 0:7.19.7-37.el6_4 cyrus-sasl-lib.x86_64 0:2.1.23-13.el6_3.1
db4.x86_64 0:4.7.25-18.el6_4 db4-utils.x86_64 0:4.7.25-18.el6_4 dbus-glib.x86_64 0:0.86-6.el6_4
dbus-libs.x86_64 1:1.2.24-7.0.1.el6_3 dhcp-common.x86_64 12:4.1.1-38.P1.0.1.el6 diffutils.x86_64 0:2.8.1-28.el6
elfutils-libelf.x86_64 0:0.152-1.el6 ethtool.x86_64 2:3.5-1.el6 expat.x86_64 0:2.0.1-11.el6_2
file-libs.x86_64 0:5.04-15.el6 filesystem.x86_64 0:2.4.30-3.el6 findutils.x86_64 1:4.4.2-6.el6
fipscheck.x86_64 0:1.2.0-7.el6 fipscheck-lib.x86_64 0:1.2.0-7.el6 gamin.x86_64 0:0.1.10-9.el6
gawk.x86_64 0:3.1.7-10.el6 gdbm.x86_64 0:1.8.0-36.el6 glib2.x86_64 0:2.26.1-3.el6
glibc.x86_64 0:2.12-1.132.el6 glibc-common.x86_64 0:2.12-1.132.el6 gmp.x86_64 0:4.3.1-7.el6_2.2
gnupg2.x86_64 0:2.0.14-6.el6_4 gpgme.x86_64 0:1.1.8-3.el6 grep.x86_64 0:2.6.3-4.el6
groff.x86_64 0:1.18.1.4-21.el6 gzip.x86_64 0:1.3.12-19.el6_4 hwdata.noarch 0:0.233-9.1.el6
info.x86_64 0:4.13a-8.el6 iproute.x86_64 0:2.6.32-31.el6 iptables.x86_64 0:1.4.7-11.0.1.el6
iputils.x86_64 0:20071127-17.el6_4.2 keyutils-libs.x86_64 0:1.4-4.el6 krb5-libs.x86_64 0:1.10.3-10.el6_4.6
less.x86_64 0:436-10.el6 libacl.x86_64 0:2.2.49-6.el6 libattr.x86_64 0:2.4.44-7.el6
libblkid.x86_64 0:2.17.2-12.14.el6 libcap.x86_64 0:2.16-5.5.el6 libcap-ng.x86_64 0:0.6.4-3.el6_0.1
libcom_err.x86_64 0:1.42.8-1.0.1.el6 libcurl.x86_64 0:7.19.7-37.el6_4 libedit.x86_64 0:2.11-4.20080712cvs.1.el6
libffi.x86_64 0:3.0.5-3.2.el6 libgcc.x86_64 0:4.4.7-4.el6 libgcrypt.x86_64 0:1.4.5-11.el6_4
libgpg-error.x86_64 0:1.7-4.el6 libidn.x86_64 0:1.18-2.el6 libnih.x86_64 0:1.0.1-7.el6
libselinux.x86_64 0:2.0.94-5.3.el6_4.1 libselinux-utils.x86_64 0:2.0.94-5.3.el6_4.1 libsemanage.x86_64 0:2.0.43-4.2.el6
libsepol.x86_64 0:2.0.41-4.el6 libssh2.x86_64 0:1.4.2-1.el6 libstdc++.x86_64 0:4.4.7-4.el6
libtasn1.x86_64 0:2.3-3.el6_2.1 libusb.x86_64 0:0.1.12-23.el6 libuser.x86_64 0:0.56.13-5.el6
libutempter.x86_64 0:1.1.5-4.1.el6 libuuid.x86_64 0:2.17.2-12.14.el6 libxml2.x86_64 0:2.7.6-14.0.1.el6
logrotate.x86_64 0:3.7.8-17.el6 lua.x86_64 0:5.1.4-4.1.el6 make.x86_64 1:3.81-20.el6
mingetty.x86_64 0:1.08-5.el6 module-init-tools.x86_64 0:3.9-21.0.1.el6_4 ncurses.x86_64 0:5.7-3.20090208.el6
ncurses-base.x86_64 0:5.7-3.20090208.el6 ncurses-libs.x86_64 0:5.7-3.20090208.el6 net-tools.x86_64 0:1.60-110.el6_2
nspr.x86_64 0:4.10.0-1.el6 nss.x86_64 0:3.15.1-15.0.1.el6 nss-softokn.x86_64 0:3.14.3-9.el6
nss-softokn-freebl.x86_64 0:3.14.3-9.el6 nss-sysinit.x86_64 0:3.15.1-15.0.1.el6 nss-tools.x86_64 0:3.15.1-15.0.1.el6
nss-util.x86_64 0:3.15.1-3.el6 openldap.x86_64 0:2.4.23-32.el6_4.1 openssh.x86_64 0:5.3p1-94.el6
openssl.x86_64 0:1.0.1e-15.el6 oracle-logos.noarch 0:60.0.14-1.0.1.el6 p11-kit.x86_64 0:0.18.5-2.el6
p11-kit-trust.x86_64 0:0.18.5-2.el6 pam.x86_64 0:1.1.1-17.el6 pcre.x86_64 0:7.8-6.el6
pinentry.x86_64 0:0.7.6-6.el6 pkgconfig.x86_64 1:0.23-9.1.el6 popt.x86_64 0:1.13-7.el6
procps.x86_64 0:3.2.8-25.el6 psmisc.x86_64 0:22.6-15.el6_0.1 pth.x86_64 0:2.0.7-9.3.el6
pygpgme.x86_64 0:0.1-18.20090824bzr68.el6 python.x86_64 0:2.6.6-51.el6 python-iniparse.noarch 0:0.3.1-2.1.el6
python-libs.x86_64 0:2.6.6-51.el6 python-pycurl.x86_64 0:7.19.0-8.el6 python-urlgrabber.noarch 0:3.9.1-9.el6
readline.x86_64 0:6.0-4.el6 redhat-release-server.x86_64 1:6Server-6.5.0.1.0.1.el6 rpm.x86_64 0:4.8.0-37.el6
rpm-libs.x86_64 0:4.8.0-37.el6 rpm-python.x86_64 0:4.8.0-37.el6 sed.x86_64 0:4.2.1-10.el6
setup.noarch 0:2.8.14-20.el6_4.1 shadow-utils.x86_64 2:4.1.4.2-13.el6 shared-mime-info.x86_64 0:0.70-4.el6
sqlite.x86_64 0:3.6.20-1.el6 sysvinit-tools.x86_64 0:2.87-5.dsf.el6 tcp_wrappers-libs.x86_64 0:7.6-57.el6
tzdata.noarch 0:2013g-1.el6 udev.x86_64 0:147-2.51.0.1.el6 upstart.x86_64 0:0.6.5-12.el6_4.1
ustr.x86_64 0:1.0.4-9.1.el6 util-linux-ng.x86_64 0:2.17.2-12.14.el6 xz-libs.x86_64 0:4.999.9-0.3.beta.20091007git.el6
yum-metadata-parser.x86_64 0:1.1.2-16.el6 zlib.x86_64 0:1.2.3-29.el6
Complete!
Rebuilding rpm database
Patching container rootfs /var/lib/lxc/lxcora01/rootfs for Oracle Linux 6.5
Configuring container for Oracle Linux 6.5
Added container user:oracle password:oracle
Added container user:root password:root
Container : /var/lib/lxc/lxcora01/rootfs
Config : /var/lib/lxc/lxcora01/config
Network : eth0 (veth) on virbr0
gstanden@u1504:~/Downloads$ sudo lxc-ls -f
NAME STATE IPV4 IPV6 GROUPS AUTOSTART
------------------------------------------------
lxcora01 STOPPED - - - NO
gstanden@u1504:~/Downloads$
Start LXC Linux Container lxcora01 in foreground mode ("-F") so that console will connect on current startup session as shown below. Notice that DHCP provided by default by dnsmasq-base listening on lxcbr0 (10.0.3.1) has provided a dhcp-issued IP address for the container. This is the default networking provided by the standard template. After verifying, shutdown the container using "shutdown -h now" as shown below.
Also, note the MAC address ("HWAddr") shown in bold below which will be used when editing the config file for the container for OpenvSwitch networking.
gstanden@u1504:~/Downloads$ sudo lxc-start -n lxcora01 -F
/etc/rc.d/rc.sysinit: line 30: 1mount: ambiguous redirect
Welcome to Oracle Linux Server
Setting hostname lxcora01: [ OK ]
Checking filesystems
[ OK ]
Mounting local filesystems: [ OK ]
Enabling /etc/fstab swaps: [ OK ]
Entering non-interactive startup
Bringing up loopback interface: [ OK ]
Bringing up interface eth0:
Determining IP information for eth0... done.
[ OK ]
Starting system logger: [ OK ]
Mounting filesystems: [ OK ]
Generating SSH1 RSA host key: [ OK ]
Generating SSH2 RSA host key: [ OK ]
Generating SSH2 DSA host key: [ OK ]
Starting sshd: [ OK ]
Oracle Linux Server release 6.5
Kernel 3.19.0-16-generic on an x86_64
lxcora01 login: root
Password:
[root@lxcora01 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr FE:86:AB:24:50:25 <-- Same value in /var/lib/lxc/lxcora01/config file
inet addr:10.0.3.169 Bcast:10.0.3.255 Mask:255.255.255.0 <-- 10.0.3.169 is provided by dnsmasq
inet6 addr: fe80::fc86:abff:fe24:5025/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:29 errors:0 dropped:0 overruns:0 frame:0
TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4579 (4.4 KiB) TX bytes:1458 (1.4 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
[root@lxcora01 ~]# shutdown -h now
Configure LXC Container for OpenvSwitch Networking
Make a backup of the original install LXC config file as shown below.
gstanden@u1504:~$ sudo su -
root@u1504:~# cd /var/lib/lxc/lxcora01
root@u1504:/var/lib/lxc/lxcora01# ls -lrt
total 12
-rw-r--r-- 1 root root 984 May 18 22:19 config
dr-xr-xr-x 21 root root 4096 May 18 22:23 rootfs
root@u1504:/var/lib/lxc/lxcora01# cp -p config config.original.install.bak
Edit LXC Container config File for OpenvSwitch
The original "/var/lib/lxc/lxcora01/config" file is shown below. In the next step, the config is edited to put the container on the OpenvSwitch network., The lines shown in bold will be commented out in the next step to remove the LXC container from Linux Bridge networking, and the file will be rearranged slightly for clarity and readability.
root@u1504:/var/lib/lxc/lxcora01# cat config.original.install.bak
# Template used to create this container: /usr/share/lxc/templates/lxc-oracle
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)
lxc.network.type = veth
lxc.network.link = lxcbr0
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:2e:aa:66
lxc.rootfs = /var/lib/lxc/lxcora01/rootfs
# Common configuration
lxc.include = /usr/share/lxc/config/oracle.common.conf
# Container configuration for Oracle Linux 6.5
lxc.arch = x86_64
lxc.utsname = lxcora01
lxc.cap.drop = sys_resource
lxc.cap.drop = setfcap setpcap
# Networking
lxc.network.name = eth0
lxc.network.mtu = 1500
lxc.network.hwaddr = fe:86:ab:24:50:25
root@u1504:/var/lib/lxc/lxcora01#
The edited version of the "/var/lib/lxc/lxcora01/config" for use with OpenvSwitch networking is shown below. TheLinux Bridge networking lines have been removed, and the file has also been rearranged into sections for better readability and clarity.
root@u1504:/var/lib/lxc/lxcora01# cat config
# Template used to create this container: /usr/share/lxc/templates/lxc-oracle
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)
# Filesystem
lxc.rootfs = /var/lib/lxc/lxcora01/rootfs
# Common configuration
lxc.include = /usr/share/lxc/config/oracle.common.conf
# Container configuration for Oracle Linux 6.5
lxc.arch = x86_64
lxc.utsname = lxcora01
lxc.cap.drop = sys_resource
lxc.cap.drop = setfcap setpcap
# OpenvSwitch Networking
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-pub-ifup-sw1
lxc.network.script.down = /etc/network/if-down.d/lxcora01-pub-ifdown-sw1
lxc.network.veth.pair = lxcora01-pub
lxc.network.name = eth0
lxc.network.mtu = 1500
lxc.network.hwaddr = fe:86:ab:24:50:25
# Linux Bridge Networking
# lxc.network.type = veth
# lxc.network.link = lxcbr0
# lxc.network.flags = up
# lxc.network.hwaddr = 00:16:3e:2e:aa:66
# lxc.network.name = eth0
# lxc.network.mtu = 1500
root@u1504:/var/lib/lxc/lxcora01#
Create Additional Required Networking Files
Next the two files "lxcora01-pub-ifup-sw1" and "lxcora01-pub-ifdown-sw1" must be installed and configured as shown below.
root@W520:/etc/network/if-up.d# chmod 744 lxcora01-pub-ifup-sw1
root@W520:/etc/network/if-up.d# cd ..
root@W520:/etc/network# cd if-down.d
root@W520:/etc/network/if-down.d# chmod 744 lxcora01-pub-ifdown-sw1
root@W520:/etc/network/if-down.d#
root@W520:/etc/network/if-up.d# cat lxcora01-pub-ifup-sw1
#!/bin/bash
ovsBr='sw1'
ovs-vsctl add-port ${ovsBr} $5
ovs-vsctl set port $5 tag=10 <-- Note the "tag=10" which is the VLAN needed
root@W520:/etc/network/if-up.d# cd ..
root@W520:/etc/network# cd if-down.d/
root@u1504:/var/lib/lxc/lxcora01# cat /etc/network/if-down.d/lxcora01-pub-ifdown-sw1
#!/bin/bash
ovsBr='sw1'
sudo ovs-vsctl del-port ${ovsBr} $5
function CheckPortExists
{
sudo ovs-vsctl show | grep \"$5\" | grep Port | sed 's/^[ \t]*//;s/[ \t]*$//' | cut -f2 -d' ' | sed 's/"//g'
}
PortExists=$(CheckPortExists)
if [ "$PortExists" = "$5" ]
then
sudo ovs-vsctl del-port ${ovsBr} $5
fi
root@u1504:/var/lib/lxc/lxcora01#
Configure dhclient.conf File
Next edit the "/var/lib/lxc/lxcora01/rootfs/etc/dhcp/dhclient.conf" file for the LXC container as shown below. This file configures the "/etc/resolv.conf" file of the LXC Linux Container at boot time ensuring that it always has the correct set of nameserver values and domain name values added dynamically at boot. Note that the "vmem.org" domain name is not included because it is added to the container /etc/resolv.conf by DHCP of the host. Optionally, it could be included here, but typically this would result in "vmem.org" appearing twice in the resolv.conf file of the LXC container.
root@u1504:/var/lib/lxc/lxcora01# cat rootfs/etc/dhcp/dhclient.conf
append domain-name-servers 8.8.8.8, 10.207.39.3;
append domain-name " gns1.vmem.org";
root@u1504:/var/lib/lxc/lxcora01#
The "/var/lib/lxc/lxcora01/rootfs/etc/sysconfig/network-scripts/ifcfg-eth0" file can be listed and checked as well as shown below just to verify it is correct. It should be similar to the one shown below.
root@u1504:/var/lib/lxc/lxcora01# cat rootfs/etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
HOSTNAME=lxcora01
DHCP_HOSTNAME=lxcora01
NM_CONTROLLED=no
TYPE=Ethernet
root@u1504:/var/lib/lxc/lxcora01#
Verify Container Operation on DHCP and DNS with OpenvSwitch
Start the container (or reboot if it is already running) as shown below. IP assignment ot eth0 should be successful.
gstanden@W520:~$ sudo lxc-start -n lxcora01 -F
Welcome to Oracle Linux Server
Setting hostname lxcora01: [ OK ]
Checking filesystems
[ OK ]
Mounting local filesystems: [ OK ]
Enabling /etc/fstab swaps: [ OK ]
Entering non-interactive startup
Bringing up loopback interface: [ OK ]
Bringing up interface eth0:
Determining IP information for eth0... done.
[ OK ]
Starting system logger: [ OK ]
Mounting filesystems: [ OK ]
Starting sshd: [ OK ]
Oracle Linux Server release 6.5
Kernel 3.16.0-28-generic on an x86_64
lxcora01 login:
Verify Container on OpenvSwitch Network
Login to the container and verify that the container is now on the OpenvSwitch network as shown below. In this case the first container gets ".70" as it's IP because this was set as the lowest IP in the DHCP reserved range. IP addresses below 70 are reserved for static assignment on the 10.207.39.x subnet.
[root@lxcora01 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr FE:71:FA:7E:CB:AF
inet addr:10.207.39.70 Bcast:10.207.39.255 Mask:255.255.255.0
inet6 addr: fe80::fc71:faff:fe7e:cbaf/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:124 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:24086 (23.5 KiB) TX bytes:1074 (1.0 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Verify Container /etc/resolv.conf Correct Dynamic Formatting
Check that the /etc/resolv.conf has the correct entries assigned at startup by dhclient.conf file as shown below.
[root@lxcora01 ~]# cat /etc/resolv.conf
; generated by /sbin/dhclient-script
search vmem.org gns1.vmem.org
nameserver 10.207.39.1
nameserver 8.8.8.8
nameserver 10.207.39.3
[root@lxcora01 ~]#
Install Package "bind-utils" into LXC Container
Install the "bind-utils" package inside the LXC container using "yum" as shown below, and check then check that "nslookup" can resolv "lxcora01" the hostname of the LXC container as shown below. The bind-utils package is needed to obtain "nslookup".
[root@lxcora01 ~]# yum install bind-utils
Loaded plugins: lxc-patch
ol6_u5_base | 1.4 kB 00:00
ol6_u5_base/primary | 3.2 MB 00:01
ol6_u5_base 8573/8573
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package bind-utils.x86_64 32:9.8.2-0.17.rc1.0.2.el6_4.6 will be installed
--> Processing Dependency: bind-libs = 32:9.8.2-0.17.rc1.0.2.el6_4.6 for package: 32:bind-utils-9.8.2-0.17.rc1.0.2.el6_4.6.x86_64
--> Processing Dependency: libdns.so.81()(64bit) for package: 32:bind-utils-9.8.2-0.17.rc1.0.2.el6_4.6.x86_64
--> Processing Dependency: libbind9.so.80()(64bit) for package: 32:bind-utils-9.8.2-0.17.rc1.0.2.el6_4.6.x86_64
--> Processing Dependency: libisc.so.83()(64bit) for package: 32:bind-utils-9.8.2-0.17.rc1.0.2.el6_4.6.x86_64
--> Processing Dependency: libisccc.so.80()(64bit) for package: 32:bind-utils-9.8.2-0.17.rc1.0.2.el6_4.6.x86_64
--> Processing Dependency: liblwres.so.80()(64bit) for package: 32:bind-utils-9.8.2-0.17.rc1.0.2.el6_4.6.x86_64
--> Processing Dependency: libisccfg.so.82()(64bit) for package: 32:bind-utils-9.8.2-0.17.rc1.0.2.el6_4.6.x86_64
--> Running transaction check
---> Package bind-libs.x86_64 32:9.8.2-0.17.rc1.0.2.el6_4.6 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
==========================================================================================================================================================================
Package Arch Version Repository Size
==========================================================================================================================================================================
Installing:
bind-utils x86_64 32:9.8.2-0.17.rc1.0.2.el6_4.6 ol6_u5_base 182 k
Installing for dependencies:
bind-libs x86_64 32:9.8.2-0.17.rc1.0.2.el6_4.6 ol6_u5_base 878 k
Transaction Summary
==========================================================================================================================================================================
Install 2 Package(s)
Total download size: 1.0 M
Installed size: 2.6 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): bind-libs-9.8.2-0.17.rc1.0.2.el6_4.6.x86_64.rpm | 878 kB 00:30
(2/2): bind-utils-9.8.2-0.17.rc1.0.2.el6_4.6.x86_64.rpm | 182 kB 00:02
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 29 kB/s | 1.0 MB 00:36
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Importing GPG key 0xEC551F03:
Userid : Oracle OSS group (Open Source Software group) <build@oss.oracle.com>
Package: 6:oraclelinux-release-6Server-5.0.2.x86_64 (@ol6_u5_base/$releasever)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Is this ok [y/N]: y
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : 32:bind-libs-9.8.2-0.17.rc1.0.2.el6_4.6.x86_64 1/2
Installing : 32:bind-utils-9.8.2-0.17.rc1.0.2.el6_4.6.x86_64 2/2
lxc-patch: checking if updated pkgs need patching...
Verifying : 32:bind-libs-9.8.2-0.17.rc1.0.2.el6_4.6.x86_64 1/2
Verifying : 32:bind-utils-9.8.2-0.17.rc1.0.2.el6_4.6.x86_64 2/2
Installed:
bind-utils.x86_64 32:9.8.2-0.17.rc1.0.2.el6_4.6
Dependency Installed:
bind-libs.x86_64 32:9.8.2-0.17.rc1.0.2.el6_4.6
Complete!
[root@lxcora01 ~]#
Run nslookup Tests in LXC Container to Verify DNS Resolutions
Now run nslookup as shown below to verify automatic DNS registration of DHCP-issued IP addresses and to test that various configured dynamic and static forward and reverse lookups are working correctly from the LXC container.
[root@lxcora01 ~]# nslookup lxcora01
Server: 10.207.39.1
Address: 10.207.39.1#53
Name: lxcora01.vmem.org
Address: 10.207.39.70
[root@lxcora01 ~]# nslookup 10.207.39.70
Server: 10.207.39.1
Address: 10.207.39.1#53
70.39.207.10.in-addr.arpa name = lxcora01.vmem.org.
[root@lxcora01 ~]# nslookup w520
Server: 10.207.39.1
Address: 10.207.39.1#53
Name: W520.vmem.org
Address: 10.207.39.1
[root@lxcora01 ~]# nslookup 10.207.39.1
Server: 10.207.39.1
Address: 10.207.39.1#53
1.39.207.10.in-addr.arpa name = W520.vmem.org.
[root@lxcora01 ~]# nslookup lxc1-gns-vip
Server: 10.207.39.1
Address: 10.207.39.1#53
Name: lxc1-gns-vip.vmem.org
Address: 10.207.39.3
[root@lxcora01 ~]# nslookup 10.207.39.3
Server: 10.207.39.1
Address: 10.207.39.1#53
3.39.207.10.in-addr.arpa name = lxc1-gns-vip.vmem.org.
Run Various Status Commands on Container
The LXC container is now on the OpenvSwitch network and ready for configuration of pre-requisites for Oracle Enterprise Database product installations. Various checks on the running container can be run as shown below.
gstanden@w520:~$ sudo lxc-info -n lxcora01
[sudo] password for gstanden:
Name: lxcora01
State: RUNNING
PID: 5288
IP: 10.207.39.70
CPU use: 4.99 seconds
BlkIO use: 2.23 MiB
Memory use: 33.65 MiB
KMem use: 0 bytes
Link: lxcora01-pub
TX bytes: 107.54 KiB
RX bytes: 4.54 MiB
Total bytes: 4.65 MiB
gstanden@w520:~$ sudo lxc-ls -f
NAME STATE IPV4 IPV6 GROUPS AUTOSTART
--------------------------------------------------------
lxcora01 RUNNING 10.207.39.70 - - NO
gstanden@w520:~$
Create File Management Links
Several files located in various directories are used to configure and manage this environment. It is useful to build links off of the "/home/username" directory to act as pointers to various required files for centralized managment and control. Use the "ln -s" command to create links as shown below. Note that links to files inside the rootfs of the LXC container will show as red due to access permissions when connected as non-root account, but are useful nevertheless for reminding location of these files.
root@vmem1:/home/gstanden/Networking# cat crt_links.sh
ln -s /etc/dhcp/dhcpd.conf .
ln -s /etc/dhcp/dhclient.conf .
ln -s /etc/init/my-network-up.sh .
ln -s /var/lib/bind/fwd.vmem.org .
ln -s /var/lib/bind/rev.vmem.org .
ln -s /var/lib/bind/fwd.mccc.org .
ln -s /var/lib/bind/rev.mccc.org .
ln -s /etc/network/if-up.d/lxc-ifup-sw1 .
ln -s /etc/network/if-down.d/lxc-ifdown-sw1 .
ln -s /etc/bind/named.conf.options .
ln -s /etc/bind/named.conf.local .
ln -s /etc/bind/rndc.key .
ln -s /var/lib/lxc/lxcora02/config lxcora02-config
ln -s /var/lib/lxc/lxcora03/config lxcora03-config
ln -s /etc/NetworkManager/dnsmasq.d/local .
root@vmem1:/home/gstanden/Networking#
Contents of the /home/gstanden/OpenvSwitch directory shown below.
gstanden@w520:~$ cd OpenvSwitch
gstanden@w520:~/OpenvSwitch$ ls -lrt
total 48
-rwxr-xr-x 1 gstanden gstanden 2153 Dec 30 19:34 crt_ovs_sw1.sh
-rwxr-xr-x 1 gstanden gstanden 1016 Dec 30 19:34 crt_ovs_sw2.sh
-rwxr-xr-x 1 gstanden gstanden 1019 Dec 30 19:34 crt_ovs_sw3.sh
-rwxr-xr-x 1 gstanden gstanden 704 Dec 30 19:34 crt_ovs_sw4.sh
-rwxr-xr-x 1 gstanden gstanden 704 Dec 30 19:34 crt_ovs_sw5.sh
-rwxr-xr-x 1 gstanden gstanden 704 Dec 30 19:34 crt_ovs_sw6.sh
-rwxr-xr-x 1 gstanden gstanden 704 Dec 30 19:34 crt_ovs_sw7.sh
-rwxr-xr-x 1 gstanden gstanden 704 Dec 30 19:34 crt_ovs_sw8.sh
-rwxr-xr-x 1 gstanden gstanden 704 Dec 30 19:34 crt_ovs_sw9.sh
lrwxrwxrwx 1 gstanden gstanden 28 Dec 30 19:38 my-network-up.conf -> /etc/init/my-network-up.conf
-rw-r--r-- 1 gstanden gstanden 414 Dec 30 20:50 crt_ovs_sw1.log
-rw-r--r-- 1 gstanden gstanden 195 Dec 30 20:50 crt_ovs_sw2.log
-rw-r--r-- 1 gstanden gstanden 195 Dec 30 20:50 crt_ovs_sw3.log
-rw-r--r-- 1 gstanden gstanden 0 Dec 30 20:50 crt_ovs_sw4.log
-rw-r--r-- 1 gstanden gstanden 0 Dec 30 20:50 crt_ovs_sw5.log
-rw-r--r-- 1 gstanden gstanden 0 Dec 30 20:50 crt_ovs_sw6.log
-rw-r--r-- 1 gstanden gstanden 0 Dec 30 20:50 crt_ovs_sw7.log
-rw-r--r-- 1 gstanden gstanden 0 Dec 30 20:50 crt_ovs_sw8.log
-rw-r--r-- 1 gstanden gstanden 0 Dec 30 20:50 crt_ovs_sw9.log
gstanden@w520:~/OpenvSwitch$ cd ..
gstanden@w520:~$ cd Networking
root@vmem1:/home/gstanden/Networking# ls -lrt
total 4
lrwxrwxrwx 1 root root 20 May 16 17:38 dhcpd.conf -> /etc/dhcp/dhcpd.conf
lrwxrwxrwx 1 root root 23 May 16 17:38 dhclient.conf -> /etc/dhcp/dhclient.conf
lrwxrwxrwx 1 root root 26 May 16 17:38 rev.vmem.org -> /var/lib/bind/rev.vmem.org
lrwxrwxrwx 1 root root 26 May 16 17:38 fwd.vmem.org -> /var/lib/bind/fwd.vmem.org
lrwxrwxrwx 1 root root 26 May 16 17:38 rev.mccc.org -> /var/lib/bind/rev.mccc.org
lrwxrwxrwx 1 root root 26 May 16 17:38 fwd.mccc.org -> /var/lib/bind/fwd.mccc.org
lrwxrwxrwx 1 root root 33 May 16 17:38 lxc-ifup-sw1 -> /etc/network/if-up.d/lxc-ifup-sw1
lrwxrwxrwx 1 root root 37 May 16 17:38 lxc-ifdown-sw1 -> /etc/network/if-down.d/lxc-ifdown-sw1
lrwxrwxrwx 1 root root 18 May 16 17:38 rndc.key -> /etc/bind/rndc.key
lrwxrwxrwx 1 root root 28 May 16 17:38 named.conf.options -> /etc/bind/named.conf.options
lrwxrwxrwx 1 root root 26 May 16 17:38 named.conf.local -> /etc/bind/named.conf.local
lrwxrwxrwx 1 root root 35 May 16 17:38 local -> /etc/NetworkManager/dnsmasq.d/local
lrwxrwxrwx 1 root root 28 May 16 17:39 lxcora02-config -> /var/lib/lxc/lxcora02/config
lrwxrwxrwx 1 root root 28 May 16 17:39 lxcora03-config -> /var/lib/lxc/lxcora03/config
-rwxr-xr-x 1 gstanden gstanden 569 May 16 17:40 crt_links.sh
lrwxrwxrwx 1 root root 26 May 16 17:40 my-network-up.sh -> /etc/init/my-network-up.sh
root@vmem1:/home/gstanden/Networking#
gstanden@w520:~/Networking$ cd ..
gstanden@w520:~$ cd Templates
gstanden@w520:~/Templates$ ls -lrt
total 0
lrwxrwxrwx 1 gstanden gstanden 24 Dec 30 21:18 templates -> /usr/share/lxc/templates
gstanden@w520:~/Templates$
Add HAIP and ASM Networks to Container Configuation File
Additional network adapters are needed in the LXC Oracle Linux 6.5 containers. Add those containers by adding additional network adapters to the /var/lib/lxc/lxcora01/config file. The updated config file is shown below with the added code shown in bold. Note that these additional adapters are for HAIP using four subnets and for the ASM flex cluster network using two subnets. Note that the additional files needed in "/etc/network/if-up.d" and in "/etc/network/if-down.d" are found at the end of this blog in the downloadable files section for the lxcora01 container.
root@u1504:/var/lib/lxc/lxcora01# cat config
# Template used to create this container: /usr/share/lxc/templates/lxc-oracle
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)
# Filesystem
lxc.rootfs = /var/lib/lxc/lxcora01/rootfs
# Common configuration
lxc.include = /usr/share/lxc/config/oracle.common.conf
# Container configuration for Oracle Linux 6.5
lxc.arch = x86_64
lxc.utsname = lxcora01
lxc.cap.drop = sys_resource
lxc.cap.drop = setfcap setpcap
# OpenvSwitch Networking
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-pub-ifup-sw1
lxc.network.script.down = /etc/network/if-down.d/lxcora01-pub-ifdown-sw1
lxc.network.veth.pair = lxcora01-pub
lxc.network.name = eth0
lxc.network.mtu = 1500
lxc.network.hwaddr = fe:86:ab:24:50:25
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-priv1-ifup-sw4
lxc.network.script.down = /etc/network/if-down.d/lxcora01-priv1-ifdown-sw4
lxc.network.veth.pair = lxcora01-priv1
lxc.network.name = eth1
lxc.network.mtu = 9000
lxc.network.ipv4 = 192.210.39.10
lxc.network.hwaddr = 00:16:3e:28:e4:88
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-priv2-ifup-sw5
lxc.network.script.down = /etc/network/if-down.d/lxcora01-priv2-ifdown-sw5
lxc.network.veth.pair = lxcora01-priv2
lxc.network.name = eth2
lxc.network.mtu = 9000
lxc.network.ipv4 = 192.211.39.10
lxc.network.hwaddr = 00:16:3e:34:05:3f
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-priv3-ifup-sw6
lxc.network.script.down = /etc/network/if-down.d/lxcora01-priv3-ifdown-sw6
lxc.network.veth.pair = lxcora01-priv3
lxc.network.name = eth3
lxc.network.mtu = 9000
lxc.network.ipv4 = 192.212.39.10
lxc.network.hwaddr = 00:16:3e:38:4e:36
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-priv4-ifup-sw7
lxc.network.script.down = /etc/network/if-down.d/lxcora01-priv4-ifdown-sw7
lxc.network.veth.pair = lxcora01-priv4
lxc.network.name = eth4
lxc.network.mtu = 9000
lxc.network.ipv4 = 192.213.39.10
lxc.network.hwaddr = 00:16:3e:68:e8:35
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-asm1-ifup-sw8
lxc.network.script.down = /etc/network/if-down.d/lxcora01-asm1-ifdown-sw8
lxc.network.veth.pair = lxcora01-asm1
lxc.network.name = eth5
lxc.network.mtu = 9000
lxc.network.ipv4 = 172.220.40.10
lxc.network.hwaddr = 00:16:3e:50:81:28
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-asm2-ifup-sw9
lxc.network.script.down = /etc/network/if-down.d/lxcora01-asm2-ifdown-sw9
lxc.network.veth.pair = lxcora01-asm2
lxc.network.name = eth6
lxc.network.mtu = 9000
lxc.network.ipv4 = 172.221.40.10
lxc.network.hwaddr = 00:16:3e:24:c8:4b
# Linux Bridge Networking
# lxc.network.type = veth
# lxc.network.link = lxcbr0
# lxc.network.flags = up
# lxc.network.hwaddr = 00:16:3e:2e:aa:66
# lxc.network.name = eth0
# lxc.network.mtu = 1500
lxc.cgroup.devices.allow = c 10:236 rwm
lxc.cgroup.devices.allow = b 252:* rwm
# lxc.mount.entry = /dev/mapper /var/lib/lxc/lxcora10/rootfs/dev/mapper none defaults,bind,create=dir 0 0
lxc.mount.auto = proc sys cgroup
lxc.mount.auto = proc:rw sys:rw cgroup-full:rw
root@u1504:/var/lib/lxc/lxcora01#
Note that the lxc.network.script.up and the lxc.network.script.down require additional files to be installed under /etc/network/if-up.d and under /etc/network/if-down.d as shown above in the config file. Those files are attached to this blog and can be downloaded. A couple of examples of the files are shown below for reference.
root@u1504:/var/lib/lxc/lxcora01# cat /etc/network/if-up.d/lxcora01-asm2-ifup-sw9
#!/bin/bash
ovsBr='sw9'
sudo ovs-vsctl add-port ${ovsBr} $5
sudo ovs-vsctl set port lxcora01-asm2 tag=70
sudo ovs-vsctl set port ${ovsBr} tag=70
sudo ovs-vsctl set port ${ovsBr} trunks=60,70
root@u1504:/var/lib/lxc/lxcora01# cat /etc/network/if-down.d/lxcora01-asm2-ifdown-sw9
#!/bin/bash
ovsBr='sw9'
sudo ovs-vsctl del-port ${ovsBr} $5
function CheckPortExists
{
sudo ovs-vsctl show | grep \"$5\" | grep Port | sed 's/^[ \t]*//;s/[ \t]*$//' | cut -f2 -d' ' | sed 's/"//g'
}
PortExists=$(CheckPortExists)
if [ "$PortExists" = "$5" ]
then
sudo ovs-vsctl del-port ${ovsBr} $5
fi
root@u1504:/var/lib/lxc/lxcora01#
Disable Transparent HugePages on Ubuntu Linux
Disable transparent huge pages as shown below.
gstanden@u1504:~$ sudo apt-get install hugepages
[sudo] password for gstanden:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
linux-headers-3.19.0-15 linux-headers-3.19.0-15-generic linux-image-3.19.0-15-generic linux-image-extra-3.19.0-15-generic
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
libhugetlbfs0
Suggested packages:
libhugetlbfs-tests
The following NEW packages will be installed:
hugepages libhugetlbfs0
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 104 kB of archives.
After this operation, 348 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://us.archive.ubuntu.com/ubuntu/ vivid/universe libhugetlbfs0 amd64 2.17-0ubuntu2 [55.4 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ vivid/universe hugepages amd64 2.17-0ubuntu2 [48.4 kB]
Fetched 104 kB in 0s (292 kB/s)
Selecting previously unselected package libhugetlbfs0.
(Reading database ... 271909 files and directories currently installed.)
Preparing to unpack .../libhugetlbfs0_2.17-0ubuntu2_amd64.deb ...
Unpacking libhugetlbfs0 (2.17-0ubuntu2) ...
Selecting previously unselected package hugepages.
Preparing to unpack .../hugepages_2.17-0ubuntu2_amd64.deb ...
Unpacking hugepages (2.17-0ubuntu2) ...
Processing triggers for man-db (2.7.0.2-5) ...
Setting up libhugetlbfs0 (2.17-0ubuntu2) ...
Setting up hugepages (2.17-0ubuntu2) ...
Processing triggers for libc-bin (2.21-0ubuntu4) ...
/sbin/ldconfig.real: /usr/lib/libbrcolm2.so.1 is not a symbolic link
/sbin/ldconfig.real: /usr/lib/libbrscandec2.so.1 is not a symbolic link
gstanden@u1504:~$ hugeadm
hugeadm [options]
options:
--list-all-mounts List all current hugetlbfs mount points
--pool-list List all pools
--hard specified with --pool-pages-min to make
multiple attempts at adjusting the pool size to the
specified count on failure
--pool-pages-min <size|DEFAULT>:[+|-]<pagecount|memsize<G|M|K>>
Adjust pool 'size' lower bound
--obey-mempolicy Obey the NUMA memory policy when
adjusting the pool 'size' lower bound
--thp-always Enable transparent huge pages always
--thp-madvise Enable transparent huge pages with madvise
--thp-never Disable transparent huge pages
--thp-khugepaged-pages <pages to scan> Number of pages that khugepaged
should scan on each pass
--thp-khugepaged-scan-sleep <milliseconds> Time in ms to sleep between
khugepaged passes
--thp-khugepages-alloc-sleep <milliseconds> Time in ms for khugepaged
to wait if there was a huge page allocation failure
--pool-pages-max <size|DEFAULT>:[+|-]<pagecount|memsize<G|M|K>>
Adjust pool 'size' upper bound
--set-recommended-min_free_kbytes
Sets min_free_kbytes to a recommended value to improve availability of
huge pages at runtime
--set-recommended-shmmax Sets shmmax to a recommended value to
maximise the size possible for shared memory pools
--set-shm-group <gid|groupname> Sets hugetlb_shm_group to the
specified group, which has permission to use hugetlb shared memory pools
--add-temp-swap[=count] Specified with --pool-pages-min to create
temporary swap space for the duration of the pool resize. Default swap
size is 5 huge pages. Optional arg sets size to 'count' huge pages
--add-ramdisk-swap Specified with --pool-pages-min to create
swap space on ramdisks. By default, swap is removed after the resize.
--persist Specified with --add-temp-swap or --add-ramdisk-swap
options to make swap space persist after the resize.
--enable-zone-movable Use ZONE_MOVABLE for huge pages
--disable-zone-movable Do not use ZONE_MOVABLE for huge pages
--create-mounts Creates a mount point for each available
huge page size on this system under /var/lib/hugetlbfs
--create-user-mounts <user>
Creates a mount point for each available huge
page size under /var/lib/hugetlbfs/<user>
usable by user <user>
--create-group-mounts <group>
Creates a mount point for each available huge
page size under /var/lib/hugetlbfs/<group>
usable by group <group>
--create-global-mounts
Creates a mount point for each available huge
page size under /var/lib/hugetlbfs/global
usable by anyone
--max-size <size<G|M|K>> Limit the filesystem size of a new mount point
--max-inodes <number> Limit the number of inodes on a new mount point
--page-sizes Display page sizes that a configured pool
--page-sizes-all Display page sizes support by the hardware
--dry-run Print the equivalent shell commands for what
the specified options would have done without
taking any action
--explain Gives a overview of the status of the system
with respect to huge page availability
--verbose <level>, -v Increases/sets tracing levels
--help, -h Prints this message
gstanden@u1504:~$ cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never
gstanden@u1504:~$ sudo hugeadm --thp-never
gstanden@u1504:~$ cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
gstanden@u1504:~$
Configure Users and Packages In LXC Container
Run the following scripts as shown below. The create_users.sh script installs the users and creates some directories. The packages.sh script installs all packages required for Oracle RDBMS RAC. Run these scripts inside the LXC container.
[root@lxcora01 ~]# ls -lrt
total 12
-rwxr-xr-x 1 root root 416 Sep 18 2014 create_users.sh
-rwxr-xr-x 1 root root 1273 Sep 18 2014 packages.sh
-rwxr-xr-x 1 root root 1176 Sep 21 2014 hugepages_setting.sh
[root@lxcora01 ~]# pwd
/root
[root@lxcora01 ~]#
Add the Following Lines to the end of /etc/security/limits.conf File
Add the lines below to /etc/security/limits.conf as shown below.
# Oracle Grid Infrastructure
grid hard nofile 65536
grid soft nofile 65536
grid soft nproc 2047
# Oracle Database
# * soft memlock 9873408
# * hard memlock 9873408
Add Oracle Configurations to /etc/sysctl.conf
Add the following lines to /etc/sysctl.conf as shown below (new lines are in bold).
[root@lxcora01 ~]# cat /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.
# Controls IP packet forwarding
net.ipv4.ip_forward = 0
# Controls source route verification
net.ipv4.conf.default.rp_filter = 1
# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0
# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0
# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
kernel.core_uses_pid = 1
# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1
# Disable netfilter on bridges.
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
# Controls the default maxmimum size of a mesage queue
kernel.msgmnb = 65536
# Controls the maximum size of a message, in bytes
kernel.msgmax = 65536
# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736
# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296
# Oracle
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 6815744
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
kernel.panic_on_oops = 1
net.ipv4.conf.eth1.rp_filter = 2
net.ipv4.conf.eth2.rp_filter = 2
net.ipv4.conf.eth3.rp_filter = 2
net.ipv4.conf.eth4.rp_filter = 2
net.ipv4.conf.eth5.rp_filter = 2
net.ipv4.conf.eth6.rp_filter = 2
[root@lxcora01 ~]#
Apply Updates to "/etc/sysctl.conf"
Apply the updates to the /etc/sysctl.conf file as shown below.
[root@lxcora01 ~]# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
error: permission denied on key 'kernel.sysrq'
error: permission denied on key 'kernel.core_uses_pid'
error: "net.ipv4.tcp_syncookies" is an unknown key
error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key
error: "net.bridge.bridge-nf-call-iptables" is an unknown key
error: "net.bridge.bridge-nf-call-arptables" is an unknown key
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
error: permission denied on key 'fs.file-max'
error: permission denied on key 'fs.aio-max-nr'
net.ipv4.ip_local_port_range = 9000 65500
error: "net.core.rmem_default" is an unknown key
error: "net.core.rmem_max" is an unknown key
error: "net.core.wmem_default" is an unknown key
error: "net.core.wmem_max" is an unknown key
error: permission denied on key 'kernel.panic_on_oops'
net.ipv4.conf.eth1.rp_filter = 2
net.ipv4.conf.eth2.rp_filter = 2
net.ipv4.conf.eth3.rp_filter = 2
net.ipv4.conf.eth4.rp_filter = 2
net.ipv4.conf.eth5.rp_filter = 2
net.ipv4.conf.eth6.rp_filter = 2
[root@lxcora01 ~]#
Set umask for "oracle" and "grid" Users
Set the umask to 022 as shown below in the .bashrc files for "oracle" and "grid" users.
[oracle@lxcora01 ~]$ cat .bashrc
# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# User specific aliases and functions
umask 022
[oracle@lxcora01 ~]$ exit
logout
[root@lxcora01 ~]# su - grid
[grid@lxcora01 ~]$ cat .bashrc
# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# User specific aliases and functions
umask 022
[grid@lxcora01 ~]$
Set Password for "grid" Linux User
As the root user, set the password for the grid user as shown below.
[root@lxcora01 ~]# passwd grid
Changing password for user grid.
New password:
BAD PASSWORD: it is too short
BAD PASSWORD: is too simple
Retype new password:
passwd: all authentication tokens updated successfully.
[root@lxcora01 ~]#
Edit "/etc/rc.local" File
Add the following lines to the /etc/rc.local file in the LXC containers as shown below and configure the /etc/sysconfig/ntpd file as shown below.
# Oracle ASM Storage
chown grid:asmadmin /dev/mapper/asm_disk*
chmod 0660 /dev/mapper/asm_disk*
# NTP
ntpd
Configure "/etc/sysconfig/ntpd" File
Configure the /etc/sysconfig/ntpd file as shown below.
[root@lxcora01 ~]# cat /etc/sysconfig/ntpd
# Drop root to id 'ntp:ntp' by default.
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid -g"
[root@lxcora01 ~]#
Install NFS Server on Ubuntu Base OS Host
Install nfs server on the Ubuntu host as shown below.
gstanden@u1504:~$ sudo apt-get install nfs-kernel-server nfs-common portmap
[sudo] password for gstanden:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'rpcbind' instead of 'portmap'
The following packages were automatically installed and are no longer required:
linux-headers-3.19.0-15 linux-headers-3.19.0-15-generic linux-image-3.19.0-15-generic linux-image-extra-3.19.0-15-generic
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
keyutils libnfsidmap2 libtirpc1
Suggested packages:
open-iscsi watchdog
The following NEW packages will be installed:
keyutils libnfsidmap2 libtirpc1 nfs-common nfs-kernel-server rpcbind
0 upgraded, 6 newly installed, 0 to remove and 0 not upgraded.
Need to get 468 kB of archives.
After this operation, 2,066 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://us.archive.ubuntu.com/ubuntu/ vivid/main libnfsidmap2 amd64 0.25-5 [32.2 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ vivid/main libtirpc1 amd64 0.2.5-1 [75.6 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ vivid/main keyutils amd64 1.5.9-5ubuntu1 [47.9 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu/ vivid/main rpcbind amd64 0.2.1-6ubuntu3 [38.3 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu/ vivid-updates/main nfs-common amd64 1:1.2.8-9ubuntu8.1 [186 kB]
Get:6 http://us.archive.ubuntu.com/ubuntu/ vivid-updates/main nfs-kernel-server amd64 1:1.2.8-9ubuntu8.1 [88.0 kB]
Fetched 468 kB in 0s (818 kB/s)
Selecting previously unselected package libnfsidmap2:amd64.
(Reading database ... 271938 files and directories currently installed.)
Preparing to unpack .../libnfsidmap2_0.25-5_amd64.deb ...
Unpacking libnfsidmap2:amd64 (0.25-5) ...
Selecting previously unselected package libtirpc1:amd64.
Preparing to unpack .../libtirpc1_0.2.5-1_amd64.deb ...
Unpacking libtirpc1:amd64 (0.2.5-1) ...
Selecting previously unselected package keyutils.
Preparing to unpack .../keyutils_1.5.9-5ubuntu1_amd64.deb ...
Unpacking keyutils (1.5.9-5ubuntu1) ...
Selecting previously unselected package rpcbind.
Preparing to unpack .../rpcbind_0.2.1-6ubuntu3_amd64.deb ...
Unpacking rpcbind (0.2.1-6ubuntu3) ...
Selecting previously unselected package nfs-common.
Preparing to unpack .../nfs-common_1%3a1.2.8-9ubuntu8.1_amd64.deb ...
Unpacking nfs-common (1:1.2.8-9ubuntu8.1) ...
Selecting previously unselected package nfs-kernel-server.
Preparing to unpack .../nfs-kernel-server_1%3a1.2.8-9ubuntu8.1_amd64.deb ...
Unpacking nfs-kernel-server (1:1.2.8-9ubuntu8.1) ...
Processing triggers for man-db (2.7.0.2-5) ...
Processing triggers for systemd (219-7ubuntu6) ...
Processing triggers for ureadahead (0.100.0-19) ...
ureadahead will be reprofiled on next reboot
Setting up libnfsidmap2:amd64 (0.25-5) ...
Setting up libtirpc1:amd64 (0.2.5-1) ...
Setting up keyutils (1.5.9-5ubuntu1) ...
Setting up rpcbind (0.2.1-6ubuntu3) ...
Setting up nfs-common (1:1.2.8-9ubuntu8.1) ...
Creating config file /etc/idmapd.conf with new version
Creating config file /etc/default/nfs-common with new version
Adding system user `statd' (UID 126) ...
Adding new user `statd' (UID 126) with group `nogroup' ...
Not creating home directory `/var/lib/nfs'.
invoke-rc.d: gssd.service doesn't exist but the upstart job does. Nothing to start or stop until a systemd or init job is present.
invoke-rc.d: idmapd.service doesn't exist but the upstart job does. Nothing to start or stop until a systemd or init job is present.
nfs-utils.service is a disabled or a static unit, not starting it.
Setting up nfs-kernel-server (1:1.2.8-9ubuntu8.1) ...
Creating config file /etc/exports with new version
Creating config file /etc/default/nfs-kernel-server with new version
Processing triggers for libc-bin (2.21-0ubuntu4) ...
/sbin/ldconfig.real: /usr/lib/libbrcolm2.so.1 is not a symbolic link
/sbin/ldconfig.real: /usr/lib/libbrscandec2.so.1 is not a symbolic link
Processing triggers for systemd (219-7ubuntu6) ...
Processing triggers for ureadahead (0.100.0-19) ...
gstanden@u1504:~$ cat /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
# to NFS clients. See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check)
#
gstanden@u1504:~$
Create ASM LUNs for Oracle Grid Infrastructure
Create ASM LUNs for the Oracle grid infrastructure install as shown below.
gstanden@u1504:~$ sudo su -
root@u1504:~# cd /root
root@u1504:~# ls -lrt
total 4
-rwxr-xr-x 1 root root 219 Jun 27 21:55 create-oracle-luns.sh
root@u1504:~# cat create-oracle-luns.sh
dd if=/dev/zero of=/oracle_data/AsmSystem_01.img bs=512 count=0 seek=15M
dd if=/dev/zero of=/oracle_data/AsmSystem_02.img bs=512 count=0 seek=15M
dd if=/dev/zero of=/oracle_data/AsmSystem_03.img bs=512 count=0 seek=15M
root@u1504:~# ls -lrt /oracle_data/
total 0
-rw-r--r-- 1 root root 8053063680 Jun 27 21:55 AsmSystem_01.img
-rw-r--r-- 1 root root 8053063680 Jun 27 21:55 AsmSystem_03.img
-rw-r--r-- 1 root root 8053063680 Jun 27 21:55 AsmSystem_02.img
root@u1504:~#
Configure "/etc/exports" File on Ubuntu Base OS Host
Configure the /etc/exports file on the Ubuntu host as shown below.
gstanden@u1504:~$ cat /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
# to NFS clients. See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check)
#
/oracle_data *(rw,sync,no_wdelay,insecure_locks,no_root_squash)
gstanden@u1504:~$
Edit "/etc/apparmor.d/lxc/lxc-default" File for NFS
Add configuration lines to allow NFS to operate under the apparmor regime as shown below. Then restart apparmor as shown below.
gstanden@u1504:~$ cat /etc/apparmor.d/lxc/lxc-default
# Do not load this file. Rather, load /etc/apparmor.d/lxc-containers, which
# will source all profiles under /etc/apparmor.d/lxc
profile lxc-container-default flags=(attach_disconnected,mediate_deleted) {
#include <abstractions/lxc/container-base>
# the container may never be allowed to mount devpts. If it does, it
# will remount the host's devpts. We could allow it to do it with
# the newinstance option (but, right now, we don't).
deny mount fstype=devpts,
mount fstype=nfs,
mount fstype=nfs4,
mount fstype=rpc_pipefs,
}
gstanden@u1504:~$ sudo /etc/init.d/apparmor restart
[ ok ] Restarting apparmor (via systemctl): apparmor.service.
gstanden@u1504:~$
Configure /dev/shm in LXC Containers
The /dev/shm is configured using an entry in the /var/lib/lxc/lxcora01/config file (and similar line in the lxcora02/config file) as shown below.
lxc.mount.entry = shm dev/shm tmpfs size=3500m,nosuid,nodev,noexec,create=dir 0 0
Edit "/etc/fstab" File
Also, an entry is needed in the /etc/fstab inside each LXC containers as shown below.
[root@lxcora01 ~]# cat /etc/fstab
tmpfs /dev/shm tmpfs rw,exec,size=3500m 0 0
u1504:/oracle_data /oradata nfs rw,bg,hard,nointr,tcp,vers=3,timeo=600,rsize=32768,wsize=32768,actimeo=0 0 0
[root@lxcora01 ~]#
Install cvuqdisk RPM
Install the cvuqdisk rpm as root, go to /home/grid/grid/rpm and install the rpm on the LXC container.
[root@lxcora01 ~]# cd /home/grid/grid/rpm
[root@lxcora01 rpm]# rpm -Uvh cvuqdisk-1.0.9-1.rpm
Configure Additional Linux Groups for Oracle 12c
Configure the additional groups for the oracle user on both LXC containers as shown below.
[root@lxcora02 ~]# ./create_users_12c_addon.sh
[root@lxcora02 ~]# id oracle
uid=500(oracle) gid=500(oracle) groups=500(oracle),1000(oinstall),1200(dba),1300(asmdba),1201(oper),1401(backupdba),1501(dgdba),1601(kmdba)
[root@lxcora02 ~]# cat create_users_12c_addon.sh
groupadd -g 1401 backupdba
groupadd -g 1501 dgdba
groupadd -g 1601 kmdba
groupadd -g 1701 osacfs
groupadd -g 1801 osaudit
usermod -a -G backupdba,dgdba,kmdba oracle
[root@lxcora02 ~]#
Configure ZEROCONF
Configure ZEROCONF as shown below on both LXC containers.
[root@lxcora02 ~]# cat /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=lxcora02
NOZEROCONF=yes
[root@lxcora02 ~]#
Verify SELINUX
Verify that SELINUX is set to disabled in both LXC containers as shown below.
[root@lxcora02 ~]# cat /etc/selinux/config
SELINUX=disabled
[root@lxcora02 ~]#
Clone the Container to Create Second RAC Node
Shutdown the LXC container and then clone it to create the second RAC node as shown below.
[root@lxcora01 ~]# shutdown -h now
Broadcast message from root@lxcora01
(/dev/lxc/console) at 18:55 ...
The system is going down for halt NOW!
[root@lxcora01 ~]# Stopping sshd: [ OK ]
Shutting down system logger: [ OK ]
Shutting down interface eth0: [ OK ]
Shutting down loopback interface: [ OK ]
Sending all processes the TERM signal... [ OK ]
Saving random seed: [ OK ]
Unmounting file systems: [ OK ]
umount2: Permission denied
umount: /proc/uptime: block devices not permitted on fs
umount2: Permission denied
umount: /proc/sys/fs/binfmt_misc: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /proc/stat: block devices not permitted on fs
umount2: Permission denied
umount: /proc/meminfo: block devices not permitted on fs
umount2: Permission denied
umount: /proc/diskstats: block devices not permitted on fs
umount2: Permission denied
umount: /proc/cpuinfo: block devices not permitted on fs
umount2: Permission denied
umount: /dev/pts: block devices not permitted on fs
init: Re-executing /sbin/init
mount: you must specify the filesystem type
Halting system...
gstanden@u1504:~$ clear
Note on Startup and Shutdown of LXC Oracle Linux 6.5 Containers
There are some error and warning messages that occur during startup and shutdown of the OEL 6.5 LXC Containers that can be safely ignored for the purposes of installing Oracle Clusterware and RDBMS. The startup and shutdown output of the LXC containers is shown below with the acceptable warnings and messages highlighted in bold. These messages are due to known issues with the startup and shutdown of LXC containers currently. These messages and warnings highlighted in bold can be safely ignored.
gstanden@u1504:~$ sudo lxc-start -n lxcora02 -F
[sudo] password for gstanden:
/etc/rc.d/rc.sysinit: line 30: 1mount: ambiguous redirect <-- This message can be safely ignored
Welcome to Oracle Linux Server
Setting hostname lxcora02: [ OK ]
Checking filesystems
[ OK ]
Mounting local filesystems: [ OK ]
Enabling /etc/fstab swaps: [ OK ]
Entering non-interactive startup
Calling the system activity data collector (sadc)...
Bringing up loopback interface: [ OK ]
Bringing up interface eth0:
Determining IP information for eth0... done.
[ OK ]
Starting system logger: [ OK ]
Starting rpcbind: [ OK ]
Starting NFS statd: [ OK ]
NFS filesystems queued to be mounted
Mounting filesystems: [ OK ]
Starting sshd: [ OK ]
Starting crond: [ OK ]
Oracle Linux Server release 6.5
Kernel 3.19.0-21-generic on an x86_64
lxcora02 login: root
Password:
Last login: Sun Jun 28 12:53:51 on lxc/console
[root@lxcora02 ~]# shutdown -h now
Broadcast message from root@lxcora02
(/dev/lxc/console) at 14:07 ...
The system is going down for halt NOW!
[root@lxcora02 ~]# Stopping sshd: [ OK ]
Stopping crond: [ OK ]
Unmounting NFS filesystems: [ OK ]
Stopping rpcbind: [ OK ]
Shutting down system logger: [ OK ]
Shutting down interface eth0: [ OK ]
Shutting down loopback interface: [ OK ]
Sending all processes the TERM signal... [ OK ]
Sending all processes the KILL signal... [ OK ]
Saving random seed: [ OK ]
Unmounting file systems: [ OK ]
umount2: Permission denied <-- All mount and umount errors can be ignored
umount: /proc/uptime: block devices not permitted on fs
umount2: Permission denied
umount: /proc/sys/fs/binfmt_misc: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /proc/stat: block devices not permitted on fs
umount2: Permission denied
umount: /proc/meminfo: block devices not permitted on fs
umount2: Permission denied
umount: /proc/diskstats: block devices not permitted on fs
umount2: Permission denied
umount: /proc/cpuinfo: block devices not permitted on fs
umount2: Permission denied
umount: /dev/shm: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /dev/pts: block devices not permitted on fs
init: Re-executing /sbin/init
mount: you must specify the filesystem type
mount: you must specify the filesystem type
Halting system...
gstanden@u1504:~$
Verify that the LXC container is stopped as shown below.
gstanden@u1504:~$ sudo lxc-ls -f
NAME STATE IPV4 IPV6 GROUPS AUTOSTART
------------------------------------------------
lxcora01 STOPPED - - - NO
gstanden@u1504:~$
Clone the LXC container as shown below.
gstanden@u1504:~$ sudo lxc-clone -o lxcora01 -n lxcora02
Created container lxcora02 as copy of lxcora01
gstanden@u1504:~$
This will create a config file for the cloned container as shown below. It is necessary to change the references in the OpenvSwitch stanzas from "lxcora01" to "lxcora02" because the lxc-clone does not know about OpenvSwitch networking by default.
root@u1504:/var/lib/lxc/lxcora02# cat config
# Template used to create this container: /usr/share/lxc/templates/lxc-oracle
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)
# Filesystem
# Common configuration
lxc.include = /usr/share/lxc/config/oracle.common.conf
# Container configuration for Oracle Linux 6.5
lxc.arch = x86_64
lxc.cap.drop = sys_resource
lxc.cap.drop = setfcap setpcap
# OpenvSwitch Networking
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxc-ifup-sw1
lxc.network.script.down = /etc/network/if-down.d/lxc-ifdown-sw1
lxc.network.veth.pair = lxcora01-pub <-- Change to "lxcora01-pub" and similar for all other similar references
lxc.network.name = eth0
lxc.network.mtu = 1500
lxc.network.hwaddr = 00:16:3e:73:1e:71
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-priv1-ifup-sw4
lxc.network.script.down = /etc/network/if-down.d/lxcora01-priv1-ifdown-sw4
lxc.network.veth.pair = lxcora01-priv1
lxc.network.name = eth1
lxc.network.mtu = 9000
lxc.network.ipv4 = 192.210.39.10 <-- Change all ".10" static IP addresses to ".11" static IP addresses
lxc.network.hwaddr = 00:16:3e:f6:30:ca
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-priv2-ifup-sw5
lxc.network.script.down = /etc/network/if-down.d/lxcora01-priv2-ifdown-sw5
lxc.network.veth.pair = lxcora01-priv2
lxc.network.name = eth2
lxc.network.mtu = 9000
lxc.network.ipv4 = 192.211.39.10
lxc.network.hwaddr = 00:16:3e:51:53:30
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-priv3-ifup-sw6
lxc.network.script.down = /etc/network/if-down.d/lxcora01-priv3-ifdown-sw6
lxc.network.veth.pair = lxcora01-priv3
lxc.network.name = eth3
lxc.network.mtu = 9000
lxc.network.ipv4 = 192.212.39.10
lxc.network.hwaddr = 00:16:3e:38:9c:d7
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-priv4-ifup-sw7
lxc.network.script.down = /etc/network/if-down.d/lxcora01-priv4-ifdown-sw7
lxc.network.veth.pair = lxcora01-priv4
lxc.network.name = eth4
lxc.network.mtu = 9000
lxc.network.ipv4 = 192.213.39.10
lxc.network.hwaddr = 00:16:3e:b5:65:3f
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-asm1-ifup-sw8
lxc.network.script.down = /etc/network/if-down.d/lxcora01-asm1-ifdown-sw8
lxc.network.veth.pair = lxcora01-asm1
lxc.network.name = eth5
lxc.network.mtu = 9000
lxc.network.ipv4 = 172.220.40.10
lxc.network.hwaddr = 00:16:3e:d4:a0:55
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora01-asm2-ifup-sw9
lxc.network.script.down = /etc/network/if-down.d/lxcora01-asm2-ifdown-sw9
lxc.network.veth.pair = lxcora01-asm2
lxc.network.name = eth6
lxc.network.mtu = 9000
lxc.network.ipv4 = 172.221.40.10
lxc.network.hwaddr = 00:16:3e:36:22:62
# Linux Bridge Networking
# lxc.network.type = veth
# lxc.network.link = lxcbr0
# lxc.network.flags = up
# lxc.network.hwaddr = 00:16:3e:2e:aa:66
# lxc.network.name = eth0
# lxc.network.mtu = 1500
lxc.cgroup.devices.allow = c 10:236 rwm
lxc.cgroup.devices.allow = b 252:* rwm
# lxc.mount.entry = /dev/mapper /var/lib/lxc/lxcora10/rootfs/dev/mapper none defaults,bind,create=dir 0 0
lxc.mount.auto = proc sys cgroup
lxc.mount.auto = proc:rw sys:rw cgroup-full:rw
lxc.rootfs = /var/lib/lxc/lxcora02/rootfs
lxc.utsname = lxcora02
root@u1504:/var/lib/lxc/lxcora02#
The edited config file for the cloned LXC container is shown below.
root@u1504:/var/lib/lxc/lxcora02# pwd
/var/lib/lxc/lxcora02
root@u1504:/var/lib/lxc/lxcora02# cat config
# Template used to create this container: /usr/share/lxc/templates/lxc-oracle
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)
# Filesystem
# Common configuration
lxc.include = /usr/share/lxc/config/oracle.common.conf
# Container configuration for Oracle Linux 6.5
lxc.arch = x86_64
lxc.cap.drop = sys_resource
lxc.cap.drop = setfcap setpcap
# OpenvSwitch Networking
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxc-ifup-sw1
lxc.network.script.down = /etc/network/if-down.d/lxc-ifdown-sw1
lxc.network.veth.pair = lxcora02-pub
lxc.network.name = eth0
lxc.network.mtu = 1500
lxc.network.hwaddr = 00:16:3e:73:1e:71
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora02-priv1-ifup-sw4
lxc.network.script.down = /etc/network/if-down.d/lxcora02-priv1-ifdown-sw4
lxc.network.veth.pair = lxcora02-priv1
lxc.network.name = eth1
lxc.network.mtu = 9000
lxc.network.ipv4 = 192.210.39.11
lxc.network.hwaddr = 00:16:3e:f6:30:ca
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora02-priv2-ifup-sw5
lxc.network.script.down = /etc/network/if-down.d/lxcora02-priv2-ifdown-sw5
lxc.network.veth.pair = lxcora02-priv2
lxc.network.name = eth2
lxc.network.mtu = 9000
lxc.network.ipv4 = 192.211.39.11
lxc.network.hwaddr = 00:16:3e:51:53:30
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora02-priv3-ifup-sw6
lxc.network.script.down = /etc/network/if-down.d/lxcora02-priv3-ifdown-sw6
lxc.network.veth.pair = lxcora02-priv3
lxc.network.name = eth3
lxc.network.mtu = 9000
lxc.network.ipv4 = 192.212.39.11
lxc.network.hwaddr = 00:16:3e:38:9c:d7
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora02-priv4-ifup-sw7
lxc.network.script.down = /etc/network/if-down.d/lxcora02-priv4-ifdown-sw7
lxc.network.veth.pair = lxcora02-priv4
lxc.network.name = eth4
lxc.network.mtu = 9000
lxc.network.ipv4 = 192.213.39.11
lxc.network.hwaddr = 00:16:3e:b5:65:3f
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora02-asm1-ifup-sw8
lxc.network.script.down = /etc/network/if-down.d/lxcora02-asm1-ifdown-sw8
lxc.network.veth.pair = lxcora02-asm1
lxc.network.name = eth5
lxc.network.mtu = 9000
lxc.network.ipv4 = 172.220.40.11
lxc.network.hwaddr = 00:16:3e:d4:a0:55
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/lxcora02-asm2-ifup-sw9
lxc.network.script.down = /etc/network/if-down.d/lxcora02-asm2-ifdown-sw9
lxc.network.veth.pair = lxcora02-asm2
lxc.network.name = eth6
lxc.network.mtu = 9000
lxc.network.ipv4 = 172.221.40.11
lxc.network.hwaddr = 00:16:3e:36:22:62
# Linux Bridge Networking
# lxc.network.type = veth
# lxc.network.link = lxcbr0
# lxc.network.flags = up
# lxc.network.hwaddr = 00:16:3e:2e:aa:66
# lxc.network.name = eth0
# lxc.network.mtu = 1500
lxc.cgroup.devices.allow = c 10:236 rwm
lxc.cgroup.devices.allow = b 252:* rwm
# lxc.mount.entry = /dev/mapper /var/lib/lxc/lxcora10/rootfs/dev/mapper none defaults,bind,create=dir 0 0
lxc.mount.auto = proc sys cgroup
lxc.mount.auto = proc:rw sys:rw cgroup-full:rw
lxc.rootfs = /var/lib/lxc/lxcora02/rootfs
lxc.utsname = lxcora02
root@u1504:/var/lib/lxc/lxcora02#
Next it is necessary to create the required files to create and destroy the OpenvSwitch ports as shown below. To create the additional files needed in the "/etc/network/if-up.d" and in the "/etc/network/if-down.d" directories a script is used, as shown below.
To use the script, as shown below, it is necessary to decide how many Oracle RAC nodes will be a part of this deployment. In this blog, a 6-node RAC ASM Flex Cluster will be created. Therefore, all the networking files needed for all six nodes will now be created, even though for now only nodes 1 (lxcora01) and node 2 (lxcora02) will be worked with. Later on, farther down the blog, the additional nodes lxcora03, lxcora04, lxcora05, and lxcora06 will be cloned from lxcora02 and configured for the 6-node RAC cluster. Note that the blog is written so that a 2-node RAC can be created and the section later in the blog to create the addtional 4 nodes to make a 6-node RAC can be skipped. Now, below, the networking files are created using the script, as shown below.
root@u1504:/etc/network/if-up.d# ls -lrt lxcora01*
-rwxr-xr-x 1 root root 84 May 18 23:23 lxcora01-pub-ifup-sw1
-rwxr-xr-x 1 root root 180 Jun 28 17:32 lxcora01-asm1-ifup-sw8
-rwxr-xr-x 1 root root 180 Jun 28 17:32 lxcora01-asm2-ifup-sw9
-rwxr-xr-x 1 root root 186 Jun 28 17:33 lxcora01-priv1-ifup-sw4
-rwxr-xr-x 1 root root 186 Jun 28 17:33 lxcora01-priv2-ifup-sw5
-rwxr-xr-x 1 root root 186 Jun 28 17:34 lxcora01-priv3-ifup-sw6
-rwxr-xr-x 1 root root 186 Jun 28 17:34 lxcora01-priv4-ifup-sw7
root@u1504:/etc/network/if-up.d# cat create-ovs-sw-files-v2.sh
#! /bin/bash
# Usage
# sudo ./create-ovs-sw-files.sh ContainerBaseName ContainerCount
# Example
# sudo ./create-ovs-sw-files.sh lxcora0 6
# Above example would create networking files for 6 LXC containers with names of the form {lxcora01, lxcora02, lxcora03,lxcora04 lxcora05,lxcora06}
# Notes
# The ContainerBaseName must be passed in at the command line and is the prefix of the LXC container names
# The ContainerCount is the total number of LXC containers to be created
export DirectoryName='/etc/network/if-up.d /etc/network/if-down.d'
export ContainerBaseName=$1
export ContainerCount=$2
for j in $DirectoryName
do
cd $j
function GetFileNames {
ls "$ContainerBaseName"1* | more | grep "$1" | sed 's/$/ /' | tr -d '\n'
}
FileName=$(GetFileNames)
let n=2
for i in $FileName
do
while [ $n -le $ContainerCount ]
do
function NewFileNames {
echo $i | sed "s/"$ContainerBaseName"1/$ContainerBaseName$n/"
}
NewFile=$(NewFileNames)
cp $i $NewFile
((n=n+1))
done
let n=2
done
done
root@u1504:/etc/network/if-up.d# ./create-ovs-sw-files-v2.sh lxcora0 6
In the above script, "lxcora0" is the naming prefix of the LXC container RAC nodes, and "6" is the number of RAC nodes to be deployed. If using a different naming scheme for the LXC RAC node containers, such as {racnode01, racnode01} then change the parameter from "lxcora0" to "racnode0", and similarly, if building a 4-node RAC, then change the "6" parameter to a "4" parameter. Running this script will create all the files needed for both the "/etc/network/if-up.d" and the "/etc/network/if-down.d" directories for all 6 of the RAC nodes. The results of running the script are shown below.
root@u1504:/etc/network/if-up.d# ls -lo lxcora0*
-rwxr-xr-x 1 root 180 Jun 28 17:32 lxcora01-asm1-ifup-sw8
-rwxr-xr-x 1 root 180 Jun 28 17:32 lxcora01-asm2-ifup-sw9
-rwxr-xr-x 1 root 186 Jun 28 17:33 lxcora01-priv1-ifup-sw4
-rwxr-xr-x 1 root 186 Jun 28 17:33 lxcora01-priv2-ifup-sw5
-rwxr-xr-x 1 root 186 Jun 28 17:34 lxcora01-priv3-ifup-sw6
-rwxr-xr-x 1 root 186 Jun 28 17:34 lxcora01-priv4-ifup-sw7
-rwxr-xr-x 1 root 84 May 18 23:23 lxcora01-pub-ifup-sw1
-rwxr-xr-x 1 root 180 Jun 28 18:04 lxcora02-asm1-ifup-sw8
-rwxr-xr-x 1 root 180 Jun 28 18:04 lxcora02-asm2-ifup-sw9
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora02-priv1-ifup-sw4
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora02-priv2-ifup-sw5
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora02-priv3-ifup-sw6
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora02-priv4-ifup-sw7
-rwxr-xr-x 1 root 84 Jun 28 18:04 lxcora02-pub-ifup-sw1
-rwxr-xr-x 1 root 180 Jun 28 18:04 lxcora03-asm1-ifup-sw8
-rwxr-xr-x 1 root 180 Jun 28 18:04 lxcora03-asm2-ifup-sw9
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora03-priv1-ifup-sw4
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora03-priv2-ifup-sw5
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora03-priv3-ifup-sw6
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora03-priv4-ifup-sw7
-rwxr-xr-x 1 root 84 Jun 28 18:04 lxcora03-pub-ifup-sw1
-rwxr-xr-x 1 root 180 Jun 28 18:04 lxcora04-asm1-ifup-sw8
-rwxr-xr-x 1 root 180 Jun 28 18:04 lxcora04-asm2-ifup-sw9
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora04-priv1-ifup-sw4
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora04-priv2-ifup-sw5
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora04-priv3-ifup-sw6
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora04-priv4-ifup-sw7
-rwxr-xr-x 1 root 84 Jun 28 18:04 lxcora04-pub-ifup-sw1
-rwxr-xr-x 1 root 180 Jun 28 18:04 lxcora05-asm1-ifup-sw8
-rwxr-xr-x 1 root 180 Jun 28 18:04 lxcora05-asm2-ifup-sw9
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora05-priv1-ifup-sw4
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora05-priv2-ifup-sw5
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora05-priv3-ifup-sw6
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora05-priv4-ifup-sw7
-rwxr-xr-x 1 root 84 Jun 28 18:04 lxcora05-pub-ifup-sw1
-rwxr-xr-x 1 root 180 Jun 28 18:04 lxcora06-asm1-ifup-sw8
-rwxr-xr-x 1 root 180 Jun 28 18:04 lxcora06-asm2-ifup-sw9
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora06-priv1-ifup-sw4
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora06-priv2-ifup-sw5
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora06-priv3-ifup-sw6
-rwxr-xr-x 1 root 186 Jun 28 18:04 lxcora06-priv4-ifup-sw7
-rwxr-xr-x 1 root 84 Jun 28 18:04 lxcora06-pub-ifup-sw1
root@u1504:/etc/network/if-up.d# cd ..
root@u1504:/etc/network# cd if-down.d
root@u1504:/etc/network/if-down.d# ls -lo lxcora0*
-rwxr-xr-x 1 root 301 Jun 18 21:25 lxcora01-asm1-ifdown-sw8
-rwxr-xr-x 1 root 301 Jun 18 21:26 lxcora01-asm2-ifdown-sw9
-rwxr-xr-x 1 root 301 Jun 18 21:25 lxcora01-priv1-ifdown-sw4
-rwxr-xr-x 1 root 301 Jun 18 21:25 lxcora01-priv2-ifdown-sw5
-rwxr-xr-x 1 root 301 Jun 18 21:25 lxcora01-priv3-ifdown-sw6
-rwxr-xr-x 1 root 301 Jun 18 21:24 lxcora01-priv4-ifdown-sw7
-rwxr-xr-x 1 root 301 May 18 23:00 lxcora01-pub-ifdown-sw1
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora02-asm1-ifdown-sw8
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora02-asm2-ifdown-sw9
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora02-priv1-ifdown-sw4
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora02-priv2-ifdown-sw5
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora02-priv3-ifdown-sw6
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora02-priv4-ifdown-sw7
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora02-pub-ifdown-sw1
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora03-asm1-ifdown-sw8
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora03-asm2-ifdown-sw9
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora03-priv1-ifdown-sw4
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora03-priv2-ifdown-sw5
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora03-priv3-ifdown-sw6
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora03-priv4-ifdown-sw7
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora03-pub-ifdown-sw1
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora04-asm1-ifdown-sw8
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora04-asm2-ifdown-sw9
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora04-priv1-ifdown-sw4
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora04-priv2-ifdown-sw5
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora04-priv3-ifdown-sw6
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora04-priv4-ifdown-sw7
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora04-pub-ifdown-sw1
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora05-asm1-ifdown-sw8
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora05-asm2-ifdown-sw9
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora05-priv1-ifdown-sw4
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora05-priv2-ifdown-sw5
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora05-priv3-ifdown-sw6
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora05-priv4-ifdown-sw7
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora05-pub-ifdown-sw1
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora06-asm1-ifdown-sw8
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora06-asm2-ifdown-sw9
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora06-priv1-ifdown-sw4
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora06-priv2-ifdown-sw5
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora06-priv3-ifdown-sw6
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora06-priv4-ifdown-sw7
-rwxr-xr-x 1 root 301 Jun 28 18:04 lxcora06-pub-ifdown-sw1
root@u1504:/etc/network/if-down.d#
Note that the ports, such as "lxcora01-pub", are referenced by an environmental variable that is available ("$5") and so the actual names of the switches to not need to be specified in the files, where "$5" is the value of "lxc.network.veth.pair" from the LXC /var/lib/lxc/lxcora01/config file.
Note that only the scripts for lxcora01 for the ip-up.d and the if-down.d directories are attached at the end of this blog, since all the other files can be generated as needed by the script. The script is also attached at the end of the blog.
Now start up the container and see what starts up leaving the original container lxcora01 down until verification has been done that the actual hostname and networking etc are correct (non-duplicate). Since this is a first for the author too, all these steps will be carefully documented.
gstanden@u1504:~$ sudo lxc-ls -f
[sudo] password for gstanden:
NAME STATE IPV4 IPV6 GROUPS AUTOSTART
------------------------------------------------
lxcora01 STOPPED - - - NO
lxcora02 STOPPED - - - NO
gstanden@u1504:~$
Verify that the container starts up correctly and note that the IP address assigned to eth0 is different from the IP assigned to eth0 on the original LXC container from which the clone copy was made, as shown below.
gstanden@u1504:~$ sudo lxc-start -n lxcora02 -F
/etc/rc.d/rc.sysinit: line 30: 1mount: ambiguous redirect
Welcome to Oracle Linux Server
Setting hostname lxcora02: [ OK ]
Checking filesystems
[ OK ]
Mounting local filesystems: [ OK ]
Enabling /etc/fstab swaps: [ OK ]
Entering non-interactive startup
Calling the system activity data collector (sadc)...
Bringing up loopback interface: [ OK ]
Bringing up interface eth0:
Determining IP information for eth0... done.
[ OK ]
Starting system logger: [ OK ]
Starting rpcbind: [ OK ]
Starting NFS statd: [ OK ]
Mounting filesystems: [ OK ]
Starting sshd: [ OK ]
Starting crond: [ OK ]
Oracle Linux Server release 6.5
Kernel 3.19.0-21-generic on an x86_64
lxcora02 login: root
Password:
Last login: Sat Jun 27 18:55:55 on lxc/console
[root@lxcora02 ~]# ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:16:3E:73:1E:71
inet addr:10.207.39.71 Bcast:10.207.39.255 Mask:255.255.255.0
inet6 addr: fe80::216:3eff:fe73:1e71/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:32 errors:0 dropped:0 overruns:0 frame:0
TX packets:15 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6119 (5.9 KiB) TX bytes:1938 (1.8 KiB)
[root@lxcora02 ~]# uname -a
Linux lxcora02 3.19.0-21-generic #21-Ubuntu SMP Sun Jun 14 18:31:11 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@lxcora02 ~]#
Check that all other ethX are configured with correct static IP addresses and have correct MTU 9000 settings as shown below.
[root@lxcora02 ~]# ifconfig | grep -A2 eth
eth0 Link encap:Ethernet HWaddr 00:16:3E:73:1E:71
inet addr:10.207.39.71 Bcast:10.207.39.255 Mask:255.255.255.0
inet6 addr: fe80::216:3eff:fe73:1e71/64 Scope:Link
--
eth1 Link encap:Ethernet HWaddr 00:16:3E:F6:30:CA
inet addr:192.210.39.11 Bcast:192.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::216:3eff:fef6:30ca/64 Scope:Link
--
eth2 Link encap:Ethernet HWaddr 00:16:3E:51:53:30
inet addr:192.211.39.11 Bcast:192.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::216:3eff:fe51:5330/64 Scope:Link
--
eth3 Link encap:Ethernet HWaddr 00:16:3E:38:9C:D7
inet addr:192.212.39.11 Bcast:192.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::216:3eff:fe38:9cd7/64 Scope:Link
--
eth4 Link encap:Ethernet HWaddr 00:16:3E:B5:65:3F
inet addr:192.213.39.11 Bcast:192.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::216:3eff:feb5:653f/64 Scope:Link
--
eth5 Link encap:Ethernet HWaddr 00:16:3E:D4:A0:55
inet addr:172.220.40.11 Bcast:172.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::216:3eff:fed4:a055/64 Scope:Link
--
eth6 Link encap:Ethernet HWaddr 00:16:3E:36:22:62
inet addr:172.221.40.11 Bcast:172.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::216:3eff:fe36:2262/64 Scope:Link
[root@lxcora02 ~]# ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:16:3E:F6:30:CA
inet addr:192.210.39.11 Bcast:192.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::216:3eff:fef6:30ca/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:116 errors:0 dropped:0 overruns:0 frame:0
TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:24599 (24.0 KiB) TX bytes:828 (828.0 b)
[root@lxcora02 ~]#
Startup the original LXC container (lxcora01) and run ping tests between the various ethX.
Copy the Oracle Grid install media into the /var/lib/lxc/lxcora01/rootfs/home/grid directory as shown below.
root@u1504:/home/gstanden/Downloads# cp -p linuxamd64_12102_grid_*.zip /var/lib/lxc/lxcora01/rootfs/home/grid/.
root@u1504:/home/gstanden/Downloads# ls -lrt linuxamd64_12102_grid*.zip
-rw-r--r-- 1 gstanden gstanden 646972897 May 16 23:26 linuxamd64_12102_grid_2of2.zip
-rw-r--r-- 1 gstanden gstanden 1747043545 May 16 23:50 linuxamd64_12102_grid_1of2.zip
root@u1504:/home/gstanden/Downloads#
Set the correct permissions on the files as shown below.
[root@lxcora01 grid]# ls -al
total 2337944
drwx------ 2 grid oinstall 4096 Jun 27 20:28 .
drwxr-xr-x 5 root root 4096 Jun 27 18:58 ..
-rw------- 1 grid oinstall 73 Jun 27 20:40 .bash_history
-rw-r--r-- 1 grid oinstall 18 Jul 18 2013 .bash_logout
-rw-r--r-- 1 grid oinstall 176 Jul 18 2013 .bash_profile
-rw-r--r-- 1 grid oinstall 135 Jun 27 19:30 .bashrc
-rw-r--r-- 1 1000 oinstall 1747043545 May 16 23:50 linuxamd64_12102_grid_1of2.zip
-rw-r--r-- 1 1000 oinstall 646972897 May 16 23:26 linuxamd64_12102_grid_2of2.zip
[root@lxcora01 grid]# chown grid:oinstall *.zip
[root@lxcora01 grid]# ls -lrt
total 2337920
-rw-r--r-- 1 grid oinstall 646972897 May 16 23:26 linuxamd64_12102_grid_2of2.zip
-rw-r--r-- 1 grid oinstall 1747043545 May 16 23:50 linuxamd64_12102_grid_1of2.zip
[root@lxcora01 grid]#
As the "grid" linux user, unzip the files in order (1of2 then 2of2) as shown below.
[grid@lxcora01 ~]$ unzip linuxamd64_12102_grid_1of2.zip
Do a pre-check of the GNS configuration as shown below.
[grid@lxcora01 grid]$ pwd
/home/grid/grid
[grid@lxcora01 grid]$
[grid@lxcora01 grid]$ ./runcluvfy.sh comp gns -precrsinst -domain vmem.org -vip 10.207.39.3 -verbose -n lxcora01
Verifying GNS integrity
Checking GNS integrity...
Checking if the GNS subdomain name is valid...
The GNS subdomain name "vmem.org" is a valid domain name
Checking if the GNS VIP is a valid address...
GNS VIP "10.207.39.3" resolves to a valid IP address
Checking the status of GNS VIP...
GNS integrity check passed
Verification of GNS integrity was successful.
[grid@lxcora01 grid]$
Verify that lxcora01 and lxcora02 are resolved by DNS as shown below.
[grid@lxcora01 grid]$ nslookup lxcora02
Server: 10.207.39.1
Address: 10.207.39.1#53
Name: lxcora02.vmem.org
Address: 10.207.39.71
[grid@lxcora01 grid]$ nslookup lxcora01
Server: 10.207.39.1
Address: 10.207.39.1#53
Name: lxcora01.vmem.org
Address: 10.207.39.70
[grid@lxcora01 grid]$
Create no-password ssh between the nodes for the grid user as shown below.
[grid@lxcora01 grid]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/grid/.ssh/id_rsa):
Created directory '/home/grid/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/grid/.ssh/id_rsa.
Your public key has been saved in /home/grid/.ssh/id_rsa.pub.
The key fingerprint is:
ec:a0:2f:54:58:77:b1:19:94:37:d1:8c:48:06:97:92 grid@lxcora01
The key's randomart image is:
+--[ RSA 2048]----+
| .=B=.= |
| . Eo=++ o |
| o . oo. . |
| . .. |
| .. S |
| .. o |
| .. . |
| .. |
| .. |
+-----------------+
[grid@lxcora01 grid]$
As the root user, set the password for the grid user as shown below.
[root@lxcora01 ~]# passwd grid
Changing password for user grid.
New password:
BAD PASSWORD: it is too short
BAD PASSWORD: is too simple
Retype new password:
passwd: all authentication tokens updated successfully.
[root@lxcora01 ~]#
Add the following lines to the /etc/rc.local file in the LXC containers as shown below and configure the /etc/sysconfig/ntpd file as shown below.
# Oracle ASM Storage
chown grid:asmadmin /dev/mapper/asm_disk*
chmod 0660 /dev/mapper/asm_disk*
# NTP
ntpd
Configure the /etc/sysconfig/ntpd file as shown below.
[root@lxcora01 ~]# cat /etc/sysconfig/ntpd
# Drop root to id 'ntp:ntp' by default.
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid -g"
[root@lxcora01 ~]#
Install nfs server on the Ubuntu host as shown below.
gstanden@u1504:~$ sudo apt-get install nfs-kernel-server nfs-common portmap
[sudo] password for gstanden:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'rpcbind' instead of 'portmap'
The following packages were automatically installed and are no longer required:
linux-headers-3.19.0-15 linux-headers-3.19.0-15-generic linux-image-3.19.0-15-generic linux-image-extra-3.19.0-15-generic
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
keyutils libnfsidmap2 libtirpc1
Suggested packages:
open-iscsi watchdog
The following NEW packages will be installed:
keyutils libnfsidmap2 libtirpc1 nfs-common nfs-kernel-server rpcbind
0 upgraded, 6 newly installed, 0 to remove and 0 not upgraded.
Need to get 468 kB of archives.
After this operation, 2,066 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://us.archive.ubuntu.com/ubuntu/ vivid/main libnfsidmap2 amd64 0.25-5 [32.2 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ vivid/main libtirpc1 amd64 0.2.5-1 [75.6 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ vivid/main keyutils amd64 1.5.9-5ubuntu1 [47.9 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu/ vivid/main rpcbind amd64 0.2.1-6ubuntu3 [38.3 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu/ vivid-updates/main nfs-common amd64 1:1.2.8-9ubuntu8.1 [186 kB]
Get:6 http://us.archive.ubuntu.com/ubuntu/ vivid-updates/main nfs-kernel-server amd64 1:1.2.8-9ubuntu8.1 [88.0 kB]
Fetched 468 kB in 0s (818 kB/s)
Selecting previously unselected package libnfsidmap2:amd64.
(Reading database ... 271938 files and directories currently installed.)
Preparing to unpack .../libnfsidmap2_0.25-5_amd64.deb ...
Unpacking libnfsidmap2:amd64 (0.25-5) ...
Selecting previously unselected package libtirpc1:amd64.
Preparing to unpack .../libtirpc1_0.2.5-1_amd64.deb ...
Unpacking libtirpc1:amd64 (0.2.5-1) ...
Selecting previously unselected package keyutils.
Preparing to unpack .../keyutils_1.5.9-5ubuntu1_amd64.deb ...
Unpacking keyutils (1.5.9-5ubuntu1) ...
Selecting previously unselected package rpcbind.
Preparing to unpack .../rpcbind_0.2.1-6ubuntu3_amd64.deb ...
Unpacking rpcbind (0.2.1-6ubuntu3) ...
Selecting previously unselected package nfs-common.
Preparing to unpack .../nfs-common_1%3a1.2.8-9ubuntu8.1_amd64.deb ...
Unpacking nfs-common (1:1.2.8-9ubuntu8.1) ...
Selecting previously unselected package nfs-kernel-server.
Preparing to unpack .../nfs-kernel-server_1%3a1.2.8-9ubuntu8.1_amd64.deb ...
Unpacking nfs-kernel-server (1:1.2.8-9ubuntu8.1) ...
Processing triggers for man-db (2.7.0.2-5) ...
Processing triggers for systemd (219-7ubuntu6) ...
Processing triggers for ureadahead (0.100.0-19) ...
ureadahead will be reprofiled on next reboot
Setting up libnfsidmap2:amd64 (0.25-5) ...
Setting up libtirpc1:amd64 (0.2.5-1) ...
Setting up keyutils (1.5.9-5ubuntu1) ...
Setting up rpcbind (0.2.1-6ubuntu3) ...
Setting up nfs-common (1:1.2.8-9ubuntu8.1) ...
Creating config file /etc/idmapd.conf with new version
Creating config file /etc/default/nfs-common with new version
Adding system user `statd' (UID 126) ...
Adding new user `statd' (UID 126) with group `nogroup' ...
Not creating home directory `/var/lib/nfs'.
invoke-rc.d: gssd.service doesn't exist but the upstart job does. Nothing to start or stop until a systemd or init job is present.
invoke-rc.d: idmapd.service doesn't exist but the upstart job does. Nothing to start or stop until a systemd or init job is present.
nfs-utils.service is a disabled or a static unit, not starting it.
Setting up nfs-kernel-server (1:1.2.8-9ubuntu8.1) ...
Creating config file /etc/exports with new version
Creating config file /etc/default/nfs-kernel-server with new version
Processing triggers for libc-bin (2.21-0ubuntu4) ...
/sbin/ldconfig.real: /usr/lib/libbrcolm2.so.1 is not a symbolic link
/sbin/ldconfig.real: /usr/lib/libbrscandec2.so.1 is not a symbolic link
Processing triggers for systemd (219-7ubuntu6) ...
Processing triggers for ureadahead (0.100.0-19) ...
gstanden@u1504:~$ cat /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
# to NFS clients. See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check)
#
gstanden@u1504:~$
Create ASM LUNs for the Oracle grid infrastructure install as shown below.
gstanden@u1504:~$ sudo su -
root@u1504:~# cd /root
root@u1504:~# ls -lrt
total 4
-rwxr-xr-x 1 root root 219 Jun 27 21:55 create-oracle-luns.sh
root@u1504:~# cat create-oracle-luns.sh
dd if=/dev/zero of=/oracle_data/AsmSystem_01.img bs=512 count=0 seek=15M
dd if=/dev/zero of=/oracle_data/AsmSystem_02.img bs=512 count=0 seek=15M
dd if=/dev/zero of=/oracle_data/AsmSystem_03.img bs=512 count=0 seek=15M
root@u1504:~# ls -lrt /oracle_data/
total 0
-rw-r--r-- 1 root root 8053063680 Jun 27 21:55 AsmSystem_01.img
-rw-r--r-- 1 root root 8053063680 Jun 27 21:55 AsmSystem_03.img
-rw-r--r-- 1 root root 8053063680 Jun 27 21:55 AsmSystem_02.img
root@u1504:~#
Configure the /etc/exports file on the Ubuntu host as shown below.
gstanden@u1504:~$ cat /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
# to NFS clients. See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check)
#
/oracle_data *(rw,sync,no_wdelay,insecure_locks,no_root_squash)
gstanden@u1504:~$
Add configuration lines as shown below to allow NFS to operate under the apparmor regime as shown below. Then restart apparmor as shown below.
gstanden@u1504:~$ cat /etc/apparmor.d/lxc/lxc-default
# Do not load this file. Rather, load /etc/apparmor.d/lxc-containers, which
# will source all profiles under /etc/apparmor.d/lxc
profile lxc-container-default flags=(attach_disconnected,mediate_deleted) {
#include <abstractions/lxc/container-base>
# the container may never be allowed to mount devpts. If it does, it
# will remount the host's devpts. We could allow it to do it with
# the newinstance option (but, right now, we don't).
deny mount fstype=devpts,
mount fstype=nfs,
mount fstype=nfs4,
mount fstype=rpc_pipefs,
}
gstanden@u1504:~$ sudo /etc/init.d/apparmor restart
[ ok ] Restarting apparmor (via systemctl): apparmor.service.
gstanden@u1504:~$
Run Cluster Verification Check
The expected correct output should be similar to that shown below. There are a few errors which are acceptable, and are due to known bugs in the CVU or are due to differences in the way that LXC containers return check data to CVU, and those acceptable errors are in bold.
[grid@lxcora01 grid]$ ./runcluvfy.sh stage -pre crsinst -n lxcora01,lxcora02 -r 12.1 -osdba oinstall -asm -asmgrp asmadmin -asmdev /oradata/AsmSystem_01.img,/oradata/AsmSystem_02.img,/oradata/AsmSystem_03.img -verbose | tee report7.txt
[grid@lxcora01 grid]$ more report7.txt
Performing pre-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "lxcora01"
Destination Node Reachable?
------------------------------------ ------------------------
lxcora01 yes
lxcora02 yes
Result: Node reachability check passed from node "lxcora01"
Checking user equivalence...
Check: User equivalence for user "grid"
Node Name Status
------------------------------------ ------------------------
lxcora02 passed
lxcora01 passed
Result: User equivalence check passed for user "grid"
Check: Package existence for "cvuqdisk"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 cvuqdisk-1.0.9-1 cvuqdisk-1.0.9-1 passed
lxcora01 cvuqdisk-1.0.9-1 cvuqdisk-1.0.9-1 passed
Result: Package existence check passed for "cvuqdisk"
Checking node connectivity...
Checking hosts config file...
Node Name Status
------------------------------------ ------------------------
lxcora01 passed
lxcora02 passed
Verification of the hosts config file successful
Interface information for node "lxcora01"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
eth0 10.207.39.70 10.207.39.0 0.0.0.0 10.207.39.1 FE:86:AB:24:50:25 1500
eth1 192.210.39.10 192.0.0.0 0.0.0.0 10.207.39.1 00:16:3E:28:E4:88 9000
eth2 192.211.39.10 192.0.0.0 0.0.0.0 10.207.39.1 00:16:3E:34:05:3F 9000
eth3 192.212.39.10 192.0.0.0 0.0.0.0 10.207.39.1 00:16:3E:38:4E:36 9000
eth4 192.213.39.10 192.0.0.0 0.0.0.0 10.207.39.1 00:16:3E:68:E8:35 9000
eth5 172.220.40.10 172.0.0.0 0.0.0.0 10.207.39.1 00:16:3E:50:81:28 9000
eth6 172.221.40.10 172.0.0.0 0.0.0.0 10.207.39.1 00:16:3E:24:C8:4B 9000
Interface information for node "lxcora02"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
eth0 10.207.39.71 10.207.39.0 0.0.0.0 10.207.39.1 00:16:3E:73:1E:71 1500
eth1 192.210.39.11 192.0.0.0 0.0.0.0 10.207.39.1 00:16:3E:F6:30:CA 9000
eth2 192.211.39.11 192.0.0.0 0.0.0.0 10.207.39.1 00:16:3E:51:53:30 9000
eth3 192.212.39.11 192.0.0.0 0.0.0.0 10.207.39.1 00:16:3E:38:9C:D7 9000
eth4 192.213.39.11 192.0.0.0 0.0.0.0 10.207.39.1 00:16:3E:B5:65:3F 9000
eth5 172.220.40.11 172.0.0.0 0.0.0.0 10.207.39.1 00:16:3E:D4:A0:55 9000
eth6 172.221.40.11 172.0.0.0 0.0.0.0 10.207.39.1 00:16:3E:36:22:62 9000
Check: Node connectivity of subnet "10.207.39.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
lxcora01[10.207.39.70] lxcora02[10.207.39.71] yes
Result: Node connectivity passed for subnet "10.207.39.0" with node(s) lxcora01,lxcora02
Check: TCP connectivity of subnet "10.207.39.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
lxcora01 : 10.207.39.70 lxcora01 : 10.207.39.70 passed
lxcora02 : 10.207.39.71 lxcora01 : 10.207.39.70 passed
lxcora01 : 10.207.39.70 lxcora02 : 10.207.39.71 passed
lxcora02 : 10.207.39.71 lxcora02 : 10.207.39.71 passed
Result: TCP connectivity check passed for subnet "10.207.39.0"
Check: Node connectivity of subnet "192.0.0.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
lxcora01[192.210.39.10] lxcora01[192.211.39.10] yes
lxcora01[192.210.39.10] lxcora01[192.212.39.10] yes
lxcora01[192.210.39.10] lxcora01[192.213.39.10] yes
lxcora01[192.210.39.10] lxcora02[192.210.39.11] yes
lxcora01[192.210.39.10] lxcora02[192.211.39.11] yes
lxcora01[192.210.39.10] lxcora02[192.212.39.11] yes
lxcora01[192.210.39.10] lxcora02[192.213.39.11] yes
lxcora01[192.211.39.10] lxcora01[192.212.39.10] yes
lxcora01[192.211.39.10] lxcora01[192.213.39.10] yes
lxcora01[192.211.39.10] lxcora02[192.210.39.11] yes
lxcora01[192.211.39.10] lxcora02[192.211.39.11] yes
lxcora01[192.211.39.10] lxcora02[192.212.39.11] yes
lxcora01[192.211.39.10] lxcora02[192.213.39.11] yes
lxcora01[192.212.39.10] lxcora01[192.213.39.10] yes
lxcora01[192.212.39.10] lxcora02[192.210.39.11] yes
lxcora01[192.212.39.10] lxcora02[192.211.39.11] yes
lxcora01[192.212.39.10] lxcora02[192.212.39.11] yes
lxcora01[192.212.39.10] lxcora02[192.213.39.11] yes
lxcora01[192.213.39.10] lxcora02[192.210.39.11] yes
lxcora01[192.213.39.10] lxcora02[192.211.39.11] yes
lxcora01[192.213.39.10] lxcora02[192.212.39.11] yes
lxcora01[192.213.39.10] lxcora02[192.213.39.11] yes
lxcora02[192.210.39.11] lxcora02[192.211.39.11] yes
lxcora02[192.210.39.11] lxcora02[192.212.39.11] yes
lxcora02[192.210.39.11] lxcora02[192.213.39.11] yes
lxcora02[192.211.39.11] lxcora02[192.212.39.11] yes
lxcora02[192.211.39.11] lxcora02[192.213.39.11] yes
lxcora02[192.212.39.11] lxcora02[192.213.39.11] yes
Result: Node connectivity passed for subnet "192.0.0.0" with node(s) lxcora01,lxcora02
Check: TCP connectivity of subnet "192.0.0.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
lxcora01 : 192.210.39.10 lxcora01 : 192.210.39.10 passed
lxcora01 : 192.211.39.10 lxcora01 : 192.210.39.10 passed
lxcora01 : 192.212.39.10 lxcora01 : 192.210.39.10 passed
lxcora01 : 192.213.39.10 lxcora01 : 192.210.39.10 passed
lxcora02 : 192.210.39.11 lxcora01 : 192.210.39.10 passed
lxcora02 : 192.211.39.11 lxcora01 : 192.210.39.10 passed
lxcora02 : 192.212.39.11 lxcora01 : 192.210.39.10 passed
lxcora02 : 192.213.39.11 lxcora01 : 192.210.39.10 passed
lxcora01 : 192.210.39.10 lxcora01 : 192.211.39.10 passed
lxcora01 : 192.211.39.10 lxcora01 : 192.211.39.10 passed
lxcora01 : 192.212.39.10 lxcora01 : 192.211.39.10 passed
lxcora01 : 192.213.39.10 lxcora01 : 192.211.39.10 passed
lxcora02 : 192.210.39.11 lxcora01 : 192.211.39.10 passed
lxcora02 : 192.211.39.11 lxcora01 : 192.211.39.10 passed
lxcora02 : 192.212.39.11 lxcora01 : 192.211.39.10 passed
lxcora02 : 192.213.39.11 lxcora01 : 192.211.39.10 passed
lxcora01 : 192.210.39.10 lxcora01 : 192.212.39.10 passed
lxcora01 : 192.211.39.10 lxcora01 : 192.212.39.10 passed
lxcora01 : 192.212.39.10 lxcora01 : 192.212.39.10 passed
lxcora01 : 192.213.39.10 lxcora01 : 192.212.39.10 passed
lxcora02 : 192.210.39.11 lxcora01 : 192.212.39.10 passed
lxcora02 : 192.211.39.11 lxcora01 : 192.212.39.10 passed
lxcora02 : 192.212.39.11 lxcora01 : 192.212.39.10 passed
lxcora02 : 192.213.39.11 lxcora01 : 192.212.39.10 passed
lxcora01 : 192.210.39.10 lxcora01 : 192.213.39.10 passed
lxcora01 : 192.211.39.10 lxcora01 : 192.213.39.10 passed
lxcora01 : 192.212.39.10 lxcora01 : 192.213.39.10 passed
lxcora01 : 192.213.39.10 lxcora01 : 192.213.39.10 passed
lxcora02 : 192.210.39.11 lxcora01 : 192.213.39.10 passed
lxcora02 : 192.211.39.11 lxcora01 : 192.213.39.10 passed
lxcora02 : 192.212.39.11 lxcora01 : 192.213.39.10 passed
lxcora02 : 192.213.39.11 lxcora01 : 192.213.39.10 passed
lxcora01 : 192.210.39.10 lxcora02 : 192.210.39.11 passed
lxcora01 : 192.211.39.10 lxcora02 : 192.210.39.11 passed
lxcora01 : 192.212.39.10 lxcora02 : 192.210.39.11 passed
lxcora01 : 192.213.39.10 lxcora02 : 192.210.39.11 passed
lxcora02 : 192.210.39.11 lxcora02 : 192.210.39.11 passed
lxcora02 : 192.211.39.11 lxcora02 : 192.210.39.11 passed
lxcora02 : 192.212.39.11 lxcora02 : 192.210.39.11 passed
lxcora02 : 192.213.39.11 lxcora02 : 192.210.39.11 passed
lxcora01 : 192.210.39.10 lxcora02 : 192.211.39.11 passed
lxcora01 : 192.211.39.10 lxcora02 : 192.211.39.11 passed
lxcora01 : 192.212.39.10 lxcora02 : 192.211.39.11 passed
lxcora01 : 192.213.39.10 lxcora02 : 192.211.39.11 passed
lxcora02 : 192.210.39.11 lxcora02 : 192.211.39.11 passed
lxcora02 : 192.211.39.11 lxcora02 : 192.211.39.11 passed
lxcora02 : 192.212.39.11 lxcora02 : 192.211.39.11 passed
lxcora02 : 192.213.39.11 lxcora02 : 192.211.39.11 passed
lxcora01 : 192.210.39.10 lxcora02 : 192.212.39.11 passed
lxcora01 : 192.211.39.10 lxcora02 : 192.212.39.11 passed
lxcora01 : 192.212.39.10 lxcora02 : 192.212.39.11 passed
lxcora01 : 192.213.39.10 lxcora02 : 192.212.39.11 passed
lxcora02 : 192.210.39.11 lxcora02 : 192.212.39.11 passed
lxcora02 : 192.211.39.11 lxcora02 : 192.212.39.11 passed
lxcora02 : 192.212.39.11 lxcora02 : 192.212.39.11 passed
lxcora02 : 192.213.39.11 lxcora02 : 192.212.39.11 passed
lxcora01 : 192.210.39.10 lxcora02 : 192.213.39.11 passed
lxcora01 : 192.211.39.10 lxcora02 : 192.213.39.11 passed
lxcora01 : 192.212.39.10 lxcora02 : 192.213.39.11 passed
lxcora01 : 192.213.39.10 lxcora02 : 192.213.39.11 passed
lxcora02 : 192.210.39.11 lxcora02 : 192.213.39.11 passed
lxcora02 : 192.211.39.11 lxcora02 : 192.213.39.11 passed
lxcora02 : 192.212.39.11 lxcora02 : 192.213.39.11 passed
lxcora02 : 192.213.39.11 lxcora02 : 192.213.39.11 passed
Result: TCP connectivity check passed for subnet "192.0.0.0"
Check: Node connectivity of subnet "172.0.0.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
lxcora01[172.220.40.10] lxcora01[172.221.40.10] yes
lxcora01[172.220.40.10] lxcora02[172.220.40.11] yes
lxcora01[172.220.40.10] lxcora02[172.221.40.11] yes
lxcora01[172.221.40.10] lxcora02[172.220.40.11] yes
lxcora01[172.221.40.10] lxcora02[172.221.40.11] yes
lxcora02[172.220.40.11] lxcora02[172.221.40.11] yes
Result: Node connectivity passed for subnet "172.0.0.0" with node(s) lxcora01,lxcora02
Check: TCP connectivity of subnet "172.0.0.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
lxcora01 : 172.220.40.10 lxcora01 : 172.220.40.10 passed
lxcora01 : 172.221.40.10 lxcora01 : 172.220.40.10 passed
lxcora02 : 172.220.40.11 lxcora01 : 172.220.40.10 passed
lxcora02 : 172.221.40.11 lxcora01 : 172.220.40.10 passed
lxcora01 : 172.220.40.10 lxcora01 : 172.221.40.10 passed
lxcora01 : 172.221.40.10 lxcora01 : 172.221.40.10 passed
lxcora02 : 172.220.40.11 lxcora01 : 172.221.40.10 passed
lxcora02 : 172.221.40.11 lxcora01 : 172.221.40.10 passed
lxcora01 : 172.220.40.10 lxcora02 : 172.220.40.11 passed
lxcora01 : 172.221.40.10 lxcora02 : 172.220.40.11 passed
lxcora02 : 172.220.40.11 lxcora02 : 172.220.40.11 passed
lxcora02 : 172.221.40.11 lxcora02 : 172.220.40.11 passed
lxcora01 : 172.220.40.10 lxcora02 : 172.221.40.11 passed
lxcora01 : 172.221.40.10 lxcora02 : 172.221.40.11 passed
lxcora02 : 172.220.40.11 lxcora02 : 172.221.40.11 passed
lxcora02 : 172.221.40.11 lxcora02 : 172.221.40.11 passed
Result: TCP connectivity check passed for subnet "172.0.0.0"
Interfaces found on subnet "10.207.39.0" that are likely candidates for VIP are:
lxcora01 eth0:10.207.39.70
lxcora02 eth0:10.207.39.71
Interfaces found on subnet "192.0.0.0" that are likely candidates for VIP are:
lxcora01 eth1:192.210.39.10
lxcora02 eth1:192.210.39.11
Interfaces found on subnet "192.0.0.0" that are likely candidates for VIP are:
lxcora01 eth2:192.211.39.10
lxcora02 eth2:192.211.39.11
Interfaces found on subnet "192.0.0.0" that are likely candidates for VIP are:
lxcora01 eth3:192.212.39.10
lxcora02 eth3:192.212.39.11
Interfaces found on subnet "192.0.0.0" that are likely candidates for VIP are:
lxcora01 eth4:192.213.39.10
lxcora02 eth4:192.213.39.11
Interfaces found on subnet "172.0.0.0" that are likely candidates for VIP are:
lxcora01 eth5:172.220.40.10
lxcora02 eth5:172.220.40.11
Interfaces found on subnet "172.0.0.0" that are likely candidates for VIP are:
lxcora01 eth6:172.221.40.10
lxcora02 eth6:172.221.40.11
WARNING:
Could not find a suitable set of interfaces for the private interconnect
Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "10.207.39.0".
Subnet mask consistency check passed for subnet "192.0.0.0".
Subnet mask consistency check passed for subnet "172.0.0.0".
Subnet mask consistency check passed.
Result: Node connectivity check passed
Checking multicast communication...
Checking subnet "10.207.39.0" for multicast communication with multicast group "224.0.0.251"...
Check of subnet "10.207.39.0" for multicast communication with multicast group "224.0.0.251" passed.
Check of multicast communication passed.
Check: Total memory
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 30.2745GB (3.174514E7KB) 4GB (4194304.0KB) passed
lxcora01 30.2745GB (3.174514E7KB) 4GB (4194304.0KB) passed
Result: Total memory check passed
Check: Available memory
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 30.2627GB (3.1732696E7KB) 50MB (51200.0KB) passed
lxcora01 30.0516GB (3.1511436E7KB) 50MB (51200.0KB) passed
Result: Available memory check passed
Check: Swap space
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 30.832GB (3.2329724E7KB) 16GB (1.6777216E7KB) passed
lxcora01 30.832GB (3.2329724E7KB) 16GB (1.6777216E7KB) passed
Result: Swap space check passed
Check: Free disk space for "lxcora02:/usr,lxcora02:/var,lxcora02:/etc,lxcora02:/sbin,lxcora02:/tmp"
Path Node Name Mount point Available Required Status
---------------- ------------ ------------ ------------ ------------ ------------
/usr lxcora02 / 849.4424GB 1.0635GB passed
/var lxcora02 / 849.4424GB 1.0635GB passed
/etc lxcora02 / 849.4424GB 1.0635GB passed
/sbin lxcora02 / 849.4424GB 1.0635GB passed
/tmp lxcora02 / 849.4424GB 1.0635GB passed
Result: Free disk space check passed for "lxcora02:/usr,lxcora02:/var,lxcora02:/etc,lxcora02:/sbin,lxcora02:/tmp"
Check: Free disk space for "lxcora01:/usr,lxcora01:/var,lxcora01:/etc,lxcora01:/sbin,lxcora01:/tmp"
Path Node Name Mount point Available Required Status
---------------- ------------ ------------ ------------ ------------ ------------
/usr lxcora01 / 849.4431GB 1.0635GB passed
/var lxcora01 / 849.4431GB 1.0635GB passed
/etc lxcora01 / 849.4431GB 1.0635GB passed
/sbin lxcora01 / 849.4431GB 1.0635GB passed
/tmp lxcora01 / 849.4431GB 1.0635GB passed
Result: Free disk space check passed for "lxcora01:/usr,lxcora01:/var,lxcora01:/etc,lxcora01:/sbin,lxcora01:/tmp"
Check: User existence for "grid"
Node Name Status Comment
------------ ------------------------ ------------------------
lxcora02 passed exists(1098)
lxcora01 passed exists(1098)
Checking for multiple users with UID value 1098
Result: Check for multiple users with UID value 1098 passed
Result: User existence check passed for "grid"
Check: Group existence for "oinstall"
Node Name Status Comment
------------ ------------------------ ------------------------
lxcora02 passed exists
lxcora01 passed exists
Result: Group existence check passed for "oinstall"
Check: Group existence for "asmadmin"
Node Name Status Comment
------------ ------------------------ ------------------------
lxcora02 passed exists
lxcora01 passed exists
Result: Group existence check passed for "asmadmin"
Check: Membership of user "grid" in group "oinstall" [as Primary]
Node Name User Exists Group Exists User in Group Primary Status
---------------- ------------ ------------ ------------ ------------ ------------
lxcora02 yes yes yes yes passed
lxcora01 yes yes yes yes passed
Result: Membership check for user "grid" in group "oinstall" [as Primary] passed
Check: Membership of user "grid" in group "asmadmin"
Node Name User Exists Group Exists User in Group Status
---------------- ------------ ------------ ------------ ----------------
lxcora02 yes yes yes passed
lxcora01 yes yes yes passed
Result: Membership check for user "grid" in group "asmadmin" passed
Check: Run level
Node Name run level Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 3 3,5 passed
lxcora01 3 3,5 passed
Result: Run level check passed
Check: Hard limits for "maximum open file descriptors"
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
lxcora02 hard 65536 65536 passed
lxcora01 hard 65536 65536 passed
Result: Hard limits check passed for "maximum open file descriptors"
Check: Soft limits for "maximum open file descriptors"
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
lxcora02 soft 65536 1024 passed
lxcora01 soft 65536 1024 passed
Result: Soft limits check passed for "maximum open file descriptors"
Check: Hard limits for "maximum user processes"
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
lxcora02 hard 123913 16384 passed
lxcora01 hard 123913 16384 passed
Result: Hard limits check passed for "maximum user processes"
Check: Soft limits for "maximum user processes"
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
lxcora02 soft 2047 2047 passed
lxcora01 soft 2047 2047 passed
Result: Soft limits check passed for "maximum user processes"
Check: System architecture
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 x86_64 x86_64 passed
lxcora01 x86_64 x86_64 passed
Result: System architecture check passed
Check: Kernel version
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 3.19.0-21-generic 2.6.39 passed
lxcora01 3.19.0-21-generic 2.6.39 passed
Result: Kernel version check passed
Check: Kernel parameter for "semmsl"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 250 250 250 passed
lxcora02 250 250 250 passed
Result: Kernel parameter check passed for "semmsl"
Check: Kernel parameter for "semmns"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 32000 32000 32000 passed
lxcora02 32000 32000 32000 passed
Result: Kernel parameter check passed for "semmns"
Check: Kernel parameter for "semopm"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 100 100 100 passed
lxcora02 100 100 100 passed
Result: Kernel parameter check passed for "semopm"
Check: Kernel parameter for "semmni"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 128 128 128 passed
lxcora02 128 128 128 passed
Result: Kernel parameter check passed for "semmni"
Check: Kernel parameter for "shmmax"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 68719476736 68719476736 16253511680 passed
lxcora02 68719476736 68719476736 16253511680 passed
Result: Kernel parameter check passed for "shmmax"
Check: Kernel parameter for "shmmni"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 4096 4096 4096 passed
lxcora02 4096 4096 4096 passed
Result: Kernel parameter check passed for "shmmni"
Check: Kernel parameter for "shmall"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 4294967296 4294967296 3174514 passed
lxcora02 4294967296 4294967296 3174514 passed
Result: Kernel parameter check passed for "shmall"
Check: Kernel parameter for "file-max"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 6815744 6815744 6815744 passed
lxcora02 6815744 6815744 6815744 passed
Result: Kernel parameter check passed for "file-max"
Check: Kernel parameter for "ip_local_port_range"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 between 9000 & 65500 between 9000 & 65500 between 9000 & 65535 passed
lxcora02 between 9000 & 65500 between 9000 & 65500 between 9000 & 65535 passed
Result: Kernel parameter check passed for "ip_local_port_range"
Check: Kernel parameter for "rmem_default"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 unknown 262144 262144 failed Current value incorrect.
lxcora02 unknown 262144 262144 failed Current value incorrect.
PRVF-7544 : Check cannot be performed for kernel parameter "rmem_default" on node "lxcora01"
PRVG-2044 : Command "/sbin/sysctl -a | grep rmem_default[[:space:]]*=" failed on node "lxcora01" and produced no output.
PRVF-7544 : Check cannot be performed for kernel parameter "rmem_default" on node "lxcora02"
PRVG-2044 : Command "/sbin/sysctl -a | grep rmem_default[[:space:]]*=" failed on node "lxcora02" and produced no output.
Result: Kernel parameter check failed for "rmem_default"
Check: Kernel parameter for "rmem_max"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 unknown 4194304 4194304 failed Current value incorrect.
lxcora02 unknown 4194304 4194304 failed Current value incorrect.
PRVF-7544 : Check cannot be performed for kernel parameter "rmem_max" on node "lxcora01"
PRVG-2044 : Command "/sbin/sysctl -a | grep rmem_max[[:space:]]*=" failed on node "lxcora01" and produced no output.
PRVF-7544 : Check cannot be performed for kernel parameter "rmem_max" on node "lxcora02"
PRVG-2044 : Command "/sbin/sysctl -a | grep rmem_max[[:space:]]*=" failed on node "lxcora02" and produced no output.
Result: Kernel parameter check failed for "rmem_max"
Check: Kernel parameter for "wmem_default"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 unknown 262144 262144 failed Current value incorrect.
lxcora02 unknown 262144 262144 failed Current value incorrect.
PRVF-7544 : Check cannot be performed for kernel parameter "wmem_default" on node "lxcora01"
PRVG-2044 : Command "/sbin/sysctl -a | grep wmem_default[[:space:]]*=" failed on node "lxcora01" and produced no output.
PRVF-7544 : Check cannot be performed for kernel parameter "wmem_default" on node "lxcora02"
PRVG-2044 : Command "/sbin/sysctl -a | grep wmem_default[[:space:]]*=" failed on node "lxcora02" and produced no output.
Result: Kernel parameter check failed for "wmem_default"
Check: Kernel parameter for "wmem_max"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 unknown 1048576 1048576 failed Current value incorrect.
lxcora02 unknown 1048576 1048576 failed Current value incorrect.
PRVF-7544 : Check cannot be performed for kernel parameter "wmem_max" on node "lxcora01"
PRVG-2044 : Command "/sbin/sysctl -a | grep wmem_max[[:space:]]*=" failed on node "lxcora01" and produced no output.
PRVF-7544 : Check cannot be performed for kernel parameter "wmem_max" on node "lxcora02"
PRVG-2044 : Command "/sbin/sysctl -a | grep wmem_max[[:space:]]*=" failed on node "lxcora02" and produced no output.
Result: Kernel parameter check failed for "wmem_max"
Check: Kernel parameter for "aio-max-nr"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 1048576 1048576 1048576 passed
lxcora02 1048576 1048576 1048576 passed
Result: Kernel parameter check passed for "aio-max-nr"
Check: Kernel parameter for "panic_on_oops"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 1 1 1 passed
lxcora02 1 1 1 passed
Result: Kernel parameter check passed for "panic_on_oops"
Check: Package existence for "binutils"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 binutils-2.20.51.0.2-5.36.el6 binutils-2.20.51.0.2 passed
lxcora01 binutils-2.20.51.0.2-5.36.el6 binutils-2.20.51.0.2 passed
Result: Package existence check passed for "binutils"
Check: Package existence for "compat-libcap1"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 compat-libcap1-1.10-1 compat-libcap1-1.10 passed
lxcora01 compat-libcap1-1.10-1 compat-libcap1-1.10 passed
Result: Package existence check passed for "compat-libcap1"
Check: Package existence for "compat-libstdc++-33(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 compat-libstdc++-33(x86_64)-3.2.3-69.el6 compat-libstdc++-33(x86_64)-3.2.3 passed
lxcora01 compat-libstdc++-33(x86_64)-3.2.3-69.el6 compat-libstdc++-33(x86_64)-3.2.3 passed
Result: Package existence check passed for "compat-libstdc++-33(x86_64)"
Check: Package existence for "libgcc(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 libgcc(x86_64)-4.4.7-4.el6 libgcc(x86_64)-4.4.4 passed
lxcora01 libgcc(x86_64)-4.4.7-4.el6 libgcc(x86_64)-4.4.4 passed
Result: Package existence check passed for "libgcc(x86_64)"
Check: Package existence for "libstdc++(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 libstdc++(x86_64)-4.4.7-4.el6 libstdc++(x86_64)-4.4.4 passed
lxcora01 libstdc++(x86_64)-4.4.7-4.el6 libstdc++(x86_64)-4.4.4 passed
Result: Package existence check passed for "libstdc++(x86_64)"
Check: Package existence for "libstdc++-devel(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 libstdc++-devel(x86_64)-4.4.7-4.el6 libstdc++-devel(x86_64)-4.4.4 passed
lxcora01 libstdc++-devel(x86_64)-4.4.7-4.el6 libstdc++-devel(x86_64)-4.4.4 passed
Result: Package existence check passed for "libstdc++-devel(x86_64)"
Check: Package existence for "sysstat"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 sysstat-9.0.4-22.el6 sysstat-9.0.4 passed
lxcora01 sysstat-9.0.4-22.el6 sysstat-9.0.4 passed
Result: Package existence check passed for "sysstat"
Check: Package existence for "gcc"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 gcc-4.4.7-4.el6 gcc-4.4.4 passed
lxcora01 gcc-4.4.7-4.el6 gcc-4.4.4 passed
Result: Package existence check passed for "gcc"
Check: Package existence for "gcc-c++"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 gcc-c++-4.4.7-4.el6 gcc-c++-4.4.4 passed
lxcora01 gcc-c++-4.4.7-4.el6 gcc-c++-4.4.4 passed
Result: Package existence check passed for "gcc-c++"
Check: Package existence for "ksh"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 ksh ksh passed
lxcora01 ksh ksh passed
Result: Package existence check passed for "ksh"
Check: Package existence for "make"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 make-3.81-20.el6 make-3.81 passed
lxcora01 make-3.81-20.el6 make-3.81 passed
Result: Package existence check passed for "make"
Check: Package existence for "glibc(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 glibc(x86_64)-2.12-1.132.el6 glibc(x86_64)-2.12 passed
lxcora01 glibc(x86_64)-2.12-1.132.el6 glibc(x86_64)-2.12 passed
Result: Package existence check passed for "glibc(x86_64)"
Check: Package existence for "glibc-devel(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 glibc-devel(x86_64)-2.12-1.132.el6 glibc-devel(x86_64)-2.12 passed
lxcora01 glibc-devel(x86_64)-2.12-1.132.el6 glibc-devel(x86_64)-2.12 passed
Result: Package existence check passed for "glibc-devel(x86_64)"
Check: Package existence for "libaio(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 libaio(x86_64)-0.3.107-10.el6 libaio(x86_64)-0.3.107 passed
lxcora01 libaio(x86_64)-0.3.107-10.el6 libaio(x86_64)-0.3.107 passed
Result: Package existence check passed for "libaio(x86_64)"
Check: Package existence for "libaio-devel(x86_64)"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 libaio-devel(x86_64)-0.3.107-10.el6 libaio-devel(x86_64)-0.3.107 passed
lxcora01 libaio-devel(x86_64)-0.3.107-10.el6 libaio-devel(x86_64)-0.3.107 passed
Result: Package existence check passed for "libaio-devel(x86_64)"
Check: Package existence for "nfs-utils"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 nfs-utils-1.2.3-39.el6 nfs-utils-1.2.3-15 passed
lxcora01 nfs-utils-1.2.3-39.el6 nfs-utils-1.2.3-15 passed
Result: Package existence check passed for "nfs-utils"
Checking availability of ports "6200,6100" required for component "Oracle Notification Service (ONS)"
Node Name Port Number Protocol Available Status
---------------- ------------ ------------ ------------ ----------------
lxcora02 6200 TCP yes successful
lxcora01 6200 TCP yes successful
lxcora02 6100 TCP yes successful
lxcora01 6100 TCP yes successful
Result: Port availability check passed for ports "6200,6100"
Checking availability of ports "42424" required for component "Oracle Cluster Synchronization Services (CSSD)"
Node Name Port Number Protocol Available Status
---------------- ------------ ------------ ------------ ----------------
lxcora02 42424 TCP yes successful
lxcora01 42424 TCP yes successful
Result: Port availability check passed for ports "42424"
Checking for multiple users with UID value 0
Result: Check for multiple users with UID value 0 passed
Check: Current group ID
Result: Current group ID check passed
Starting check for consistency of primary group of root user
Node Name Status
------------------------------------ ------------------------
lxcora02 passed
lxcora01 passed
Check for consistency of root user's primary group passed
Checking Devices for ASM...
Check: Package existence for "cvuqdisk"
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
lxcora02 cvuqdisk-1.0.9-1 cvuqdisk-1.0.9-1 passed
lxcora01 cvuqdisk-1.0.9-1 cvuqdisk-1.0.9-1 passed
Result: Package existence check passed for "cvuqdisk"
Checking for shared devices...
PRVG-5150 : could not determine if path /oradata/AsmSystem_01.img is a valid path on all nodes
PRVG-5150 : could not determine if path /oradata/AsmSystem_03.img is a valid path on all nodes
PRVG-5150 : could not determine if path /oradata/AsmSystem_02.img is a valid path on all nodes
Checking consistency of device owner across all nodes...
Consistency check of device owner for "/oradata/AsmSystem_03.img" PASSED
Consistency check of device owner for "/oradata/AsmSystem_02.img" PASSED
Consistency check of device owner for "/oradata/AsmSystem_01.img" PASSED
Checking consistency of device group across all nodes...
Consistency check of device group for "/oradata/AsmSystem_03.img" PASSED
Consistency check of device group for "/oradata/AsmSystem_02.img" PASSED
Consistency check of device group for "/oradata/AsmSystem_01.img" PASSED
Checking consistency of device permissions across all nodes...
Consistency check of device permissions for "/oradata/AsmSystem_03.img" PASSED
Consistency check of device permissions for "/oradata/AsmSystem_02.img" PASSED
Consistency check of device permissions for "/oradata/AsmSystem_01.img" PASSED
Checking consistency of device size across all nodes...
Consistency check of device size for "/oradata/AsmSystem_03.img" PASSED
Consistency check of device size for "/oradata/AsmSystem_02.img" PASSED
Consistency check of device size for "/oradata/AsmSystem_01.img" PASSED
UDev attributes check for ASM Disks started...
Result: UDev attributes check passed for ASM Disks
Checking that ASM devices are ASM Filter Driver capable.
WARNING:
PRVG-12805 : Device "/oradata/AsmSystem_01.img" cannot be managed by the ASM Filter Driver on node "lxcora02".
PRVG-12805 : Device "/oradata/AsmSystem_02.img" cannot be managed by the ASM Filter Driver on node "lxcora02".
PRVG-12805 : Device "/oradata/AsmSystem_03.img" cannot be managed by the ASM Filter Driver on node "lxcora02".
PRVG-12805 : Device "/oradata/AsmSystem_01.img" cannot be managed by the ASM Filter Driver on node "lxcora01".
PRVG-12805 : Device "/oradata/AsmSystem_02.img" cannot be managed by the ASM Filter Driver on node "lxcora01".
PRVG-12805 : Device "/oradata/AsmSystem_03.img" cannot be managed by the ASM Filter Driver on node "lxcora01".
Result: Devices check for ASM failed
Starting Clock synchronization checks using Network Time Protocol(NTP)...
Checking existence of NTP configuration file "/etc/ntp.conf" across nodes
Node Name File exists?
------------------------------------ ------------------------
lxcora02 yes
lxcora01 yes
The NTP configuration file "/etc/ntp.conf" is available on all nodes
NTP configuration file "/etc/ntp.conf" existence check passed
No NTP Daemons or Services were found to be running
PRVF-5507 : NTP daemon or service is not running on any node but NTP configuration file exists on the following node(s):
lxcora02,lxcora01
Result: Clock synchronization check using Network Time Protocol(NTP) failed
Checking Core file name pattern consistency...
Core file name pattern consistency check passed.
Checking to make sure user "grid" is not in "root" group
Node Name Status Comment
------------ ------------------------ ------------------------
lxcora02 passed does not exist
lxcora01 passed does not exist
Result: User "grid" is not part of "root" group. Check passed
Check default user file creation mask
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
lxcora02 0022 0022 passed
lxcora01 0022 0022 passed
Result: Default user file creation mask check passed
Checking integrity of file "/etc/resolv.conf" across nodes
Checking the file "/etc/resolv.conf" to make sure only one of 'domain' and 'search' entries is defined
"domain" and "search" entries do not coexist in any "/etc/resolv.conf" file
Checking if 'domain' entry in file "/etc/resolv.conf" is consistent across the nodes...
"domain" entry does not exist in any "/etc/resolv.conf" file
Checking if 'search' entry in file "/etc/resolv.conf" is consistent across the nodes...
Checking file "/etc/resolv.conf" to make sure that only one 'search' entry is defined
More than one "search" entry does not exist in any "/etc/resolv.conf" file
All nodes have same "search" order defined in file "/etc/resolv.conf"
Checking DNS response time for an unreachable node
Node Name Status
------------------------------------ ------------------------
lxcora01 passed
lxcora02 passed
The DNS response time for an unreachable node is within acceptable limit on all nodes
checking DNS response from all servers in "/etc/resolv.conf"
checking response for name "lxcora02" from each of the name servers specified in "/etc/resolv.conf"
Node Name Source Comment Status
------------ ------------------------ ------------------------ ----------
lxcora02 10.207.39.1 IPv4 passed
lxcora02 8.8.8.8 IPv4 passed
checking response for name "lxcora01" from each of the name servers specified in "/etc/resolv.conf"
Node Name Source Comment Status
------------ ------------------------ ------------------------ ----------
lxcora01 10.207.39.1 IPv4 passed
lxcora01 8.8.8.8 IPv4 passed
Check for integrity of file "/etc/resolv.conf" failed
Check: Time zone consistency
Result: Time zone consistency check passed
Checking integrity of name service switch configuration file "/etc/nsswitch.conf" ...
Checking if "hosts" entry in file "/etc/nsswitch.conf" is consistent across nodes...
Checking file "/etc/nsswitch.conf" to make sure that only one "hosts" entry is defined
More than one "hosts" entry does not exist in any "/etc/nsswitch.conf" file
All nodes have same "hosts" entry defined in file "/etc/nsswitch.conf"
Check for integrity of name service switch configuration file "/etc/nsswitch.conf" passed
Checking daemon "avahi-daemon" is not configured and running
Check: Daemon "avahi-daemon" not configured
Node Name Configured Status
------------ ------------------------ ------------------------
lxcora02 no passed
lxcora01 no passed
Daemon not configured check passed for process "avahi-daemon"
Check: Daemon "avahi-daemon" not running
Node Name Running? Status
------------ ------------------------ ------------------------
lxcora02 no passed
lxcora01 no passed
Daemon not running check passed for process "avahi-daemon"
Starting check for /dev/shm mounted as temporary file system ...
Check for /dev/shm mounted as temporary file system passed
Starting check for /boot mount ...
ERROR:
PRVE-10073 : Required /boot data is not available on node "lxcora02"
PRVE-10073 : Required /boot data is not available on node "lxcora02"
PRVE-10073 : Required /boot data is not available on node "lxcora01"
PRVE-10073 : Required /boot data is not available on node "lxcora01"
Check for /boot mount failed
Starting check for zeroconf check ...
Check for zeroconf check passed
Pre-check for cluster services setup was unsuccessful on all the nodes.
[grid@lxcora01 grid]$
Explanation of Cluster Verification Exceptions
The /boot Mount Exception
The /boot mount error arises because /boot for an LXC Oracle Enterprise Linux 6.5 container is very different than what would be found in an actual bare metal OEL 6.5 server. This exception can be safely ignored.
The NTP Exception
There are issues with running NTP in a container. For one thing, NTP cannot be started in the normal way using "chkconfig" for unknown reasons. One point to understand is that NTP runs on the base Ubuntu OS. However, during install of Oracle Clusterware, the installer will be looking for NTP so it is useful to have NTP running on the LXC containers.
The workaround is to start NTP in /etc/rc.local as was previously described in an earlier section. Therefore, to validate this check, manually check to ensure that NTP is up and running as shown below. If NTP passes the below manual checks, it is safe to proceed with the Oracle Clusterware install.
gstanden@u1504:~$ ssh root@lxcora01
root@lxcora01's password:
Last login: Sun Jun 28 12:55:10 2015
[root@lxcora01 ~]# ps -ef | grep ntp
root 360 1 0 12:55 ? 00:00:00 ntpd <-- Verify ntpd is running on both nodes
root 3429 3417 0 13:39 pts/4 00:00:00 grep ntp
[root@lxcora01 ~]# ntpq -p <-- Verify ntpq output on both nodes
remote refid st t when poll reach delay offset jitter
==============================================================================
*pool-test.ntp.o 216.218.254.202 2 u 123 128 377 83.394 0.016 37.793
-pawn.slash31.co 128.227.205.3 2 u 58 128 277 39.094 -4.702 194.382
+142.54.181.202 64.113.32.5 2 u 90 128 317 47.164 1.978 68.609
+ntp.mocana.com 204.26.59.157 3 u 225 128 376 84.518 -1.296 273.107
[root@lxcora01 ~]# su - grid
[grid@lxcora01 ~]$ ./no-passwd.sh <-- Verify that time is synchronized on nodes
Sun Jun 28 13:39:42 EDT 2015
Sun Jun 28 13:39:42 EDT 2015
Sun Jun 28 13:39:42 EDT 2015
Sun Jun 28 13:39:42 EDT 2015
Sun Jun 28 13:39:43 EDT 2015
Sun Jun 28 13:39:43 EDT 2015
Sun Jun 28 13:39:43 EDT 2015
Sun Jun 28 13:39:43 EDT 2015
[grid@lxcora01 ~]$ exit
logout
[root@lxcora01 ~]# ssh lxcora02
root@lxcora02's password:
Last login: Sun Jun 28 12:57:00 2015
[root@lxcora02 ~]# ps -ef | grep ntp
root 360 1 0 12:56 ? 00:00:00 ntpd
root 3086 3074 0 13:39 pts/4 00:00:00 grep ntp
[root@lxcora02 ~]# ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
-108.61.194.85.v 200.23.51.102 2 u 58 64 337 89.653 6.003 195.037
*pawn.slash31.co 128.227.205.3 2 u 159 64 324 39.666 -4.087 1.255
+clock.trit.net 43.77.130.254 2 u 41 64 367 75.905 -4.368 136.249
+kahuna.ruselabs 199.102.46.70 2 u 10 64 207 20.295 2.602 0.701
[root@lxcora02 ~]# su - grid
[grid@lxcora02 ~]$ cat no-passwd.sh <-- no-passwd.sh script checks both nodes
ssh lxcora02 date
ssh lxcora01 date
ssh lxcora01.vmem.org date
ssh lxcora02.vmem.org date
ssh 10.207.39.70 date
ssh 10.207.39.71 date
ssh localhost date
ssh 127.0.0.1 date
[grid@lxcora02 ~]$ ./no-passwd.sh
Sun Jun 28 13:40:23 EDT 2015
Sun Jun 28 13:40:23 EDT 2015
Sun Jun 28 13:40:23 EDT 2015
Sun Jun 28 13:40:23 EDT 2015
Sun Jun 28 13:40:23 EDT 2015
Sun Jun 28 13:40:24 EDT 2015
Sun Jun 28 13:40:24 EDT 2015
Sun Jun 28 13:40:24 EDT 2015
[grid@lxcora02 ~]$
The ASM Filter Driver Warning Exception
The ASM Filter Driver is an optional storage feature which does not have to be used and is only a warning. It can be safely ignored if not electing to use the ASM Filter Driver.
Checking that ASM devices are ASM Filter Driver capable.
WARNING:
PRVG-12805 : Device "/oradata/AsmSystem_01.img" cannot be managed by the ASM Filter Driver on node "lxcora02".
PRVG-12805 : Device "/oradata/AsmSystem_02.img" cannot be managed by the ASM Filter Driver on node "lxcora02".
PRVG-12805 : Device "/oradata/AsmSystem_03.img" cannot be managed by the ASM Filter Driver on node "lxcora02".
PRVG-12805 : Device "/oradata/AsmSystem_01.img" cannot be managed by the ASM Filter Driver on node "lxcora01".
PRVG-12805 : Device "/oradata/AsmSystem_02.img" cannot be managed by the ASM Filter Driver on node "lxcora01".
PRVG-12805 : Device "/oradata/AsmSystem_03.img" cannot be managed by the ASM Filter Driver on node "lxcora01".
Result: Devices check for ASM failed
The ASM Shared Devices Exception
This is a known-issue with CVU sometimes not being able to execute this check correctly, and can safely be ignored.
Checking for shared devices...
PRVG-5150 : could not determine if path /oradata/AsmSystem_01.img is a valid path on all nodes
PRVG-5150 : could not determine if path /oradata/AsmSystem_03.img is a valid path on all nodes
PRVG-5150 : could not determine if path /oradata/AsmSystem_02.img is a valid path on all nodes
The rmem and wmem Kernel Parameter Exceptions
The rmem_default, rmem_max, wmem_default, and wmem_max are not supported inside the LXC containers. However, note that LXC containers all share the same kernel as the base OS, in this case, Ubuntu, so if these are set to correct values in the Ubuntu Base OS kernel, then they will in fact be correct, even though the "sysctl -a | grep wmem" checks will fail for CVU, as shown below.
Check: Kernel parameter for "rmem_default"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 unknown 262144 262144 failed Current value incorrect.
lxcora02 unknown 262144 262144 failed Current value incorrect.
PRVF-7544 : Check cannot be performed for kernel parameter "rmem_default" on node "lxcora01"
PRVG-2044 : Command "/sbin/sysctl -a | grep rmem_default[[:space:]]*=" failed on node "lxcora01" and produced no output.
PRVF-7544 : Check cannot be performed for kernel parameter "rmem_default" on node "lxcora02"
PRVG-2044 : Command "/sbin/sysctl -a | grep rmem_default[[:space:]]*=" failed on node "lxcora02" and produced no output.
Result: Kernel parameter check failed for "rmem_default"
Check: Kernel parameter for "rmem_max"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 unknown 4194304 4194304 failed Current value incorrect.
lxcora02 unknown 4194304 4194304 failed Current value incorrect.
PRVF-7544 : Check cannot be performed for kernel parameter "rmem_max" on node "lxcora01"
PRVG-2044 : Command "/sbin/sysctl -a | grep rmem_max[[:space:]]*=" failed on node "lxcora01" and produced no output.
PRVF-7544 : Check cannot be performed for kernel parameter "rmem_max" on node "lxcora02"
PRVG-2044 : Command "/sbin/sysctl -a | grep rmem_max[[:space:]]*=" failed on node "lxcora02" and produced no output.
Result: Kernel parameter check failed for "rmem_max"
Check: Kernel parameter for "wmem_default"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 unknown 262144 262144 failed Current value incorrect.
lxcora02 unknown 262144 262144 failed Current value incorrect.
PRVF-7544 : Check cannot be performed for kernel parameter "wmem_default" on node "lxcora01"
PRVG-2044 : Command "/sbin/sysctl -a | grep wmem_default[[:space:]]*=" failed on node "lxcora01" and produced no output.
PRVF-7544 : Check cannot be performed for kernel parameter "wmem_default" on node "lxcora02"
PRVG-2044 : Command "/sbin/sysctl -a | grep wmem_default[[:space:]]*=" failed on node "lxcora02" and produced no output.
Result: Kernel parameter check failed for "wmem_default"
Check: Kernel parameter for "wmem_max"
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
lxcora01 unknown 1048576 1048576 failed Current value incorrect.
lxcora02 unknown 1048576 1048576 failed Current value incorrect.
PRVF-7544 : Check cannot be performed for kernel parameter "wmem_max" on node "lxcora01"
PRVG-2044 : Command "/sbin/sysctl -a | grep wmem_max[[:space:]]*=" failed on node "lxcora01" and produced no output.
PRVF-7544 : Check cannot be performed for kernel parameter "wmem_max" on node "lxcora02"
PRVG-2044 : Command "/sbin/sysctl -a | grep wmem_max[[:space:]]*=" failed on node "lxcora02" and produced no output.
Result: Kernel parameter check failed for "wmem_max"
The values can be checked manually at the Ubuntu Host OS level as shown below. Also, note in the above output that the "Configured" value should match the "Required" value as listed in the CVU output (shown in bold above). As long as "Configured" value matches the "Required" value for these kernel parameter, and as long as they are set in the Ubuntu Host OS as shown below, this check is passed.
gstanden@u1504:~$ cat /etc/sysctl.conf | egrep 'wmem|rmem'
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
gstanden@u1504:~$ sudo sysctl -a | egrep 'wmem_|rmem_' | egrep 'default|max'
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
gstanden@u1504:~$
Create Additional LXC Containers for n-Node Oracle RAC
One of the great things about building Oracle RAC in LXC Containers is that it's feasible to run n-Node RAC because resource-intensive fully-virtualized machines do not need to be created (i.e. only containers are being used) so additional LXC containers can be cloned up to the number of nodes desired for this RAC deployment on Ubuntu 15.04 or whatever LXC base OS is being used. In this example, six RAC LXC nodes are created for a six-node Oracle RAC deployment on LXC containers. It's important to use lxcora02 as the source for the additional node clones, because lxcora01 has the install media under /home/grid, which does not need to be cloned as it only needs to exist on the first node, lxcora01, so as shown below, the additional nodes are quickly cloned.
gstanden@u1504:~$ sudo lxc-clone -o lxcora02 -n lxcora03
Created container lxcora03 as copy of lxcora02
gstanden@u1504:~$ sudo lxc-clone -o lxcora02 -n lxcora04
Created container lxcora04 as copy of lxcora02
gstanden@u1504:~$ sudo lxc-clone -o lxcora02 -n lxcora05
Created container lxcora05 as copy of lxcora02
gstanden@u1504:~$ sudo lxc-clone -o lxcora02 -n lxcora06
Created container lxcora06 as copy of lxcora02
gstanden@u1504:~$
Each of the newly created nodes will need all of the following completed before Oracle Clusterware can be installed:
Create "/etc/network/if-up.d" and "/etc/network/if-down.d" files for OpenvSwitch (as shown previsouly in section )
Establish no-password SSH for all nodes (every node must be able to ssh to every other node with no password required)
Edit "/var/lib/lxc/lxcora0X/config" for OpenvSwitch config stanzas (as shown previously in section )
Re-run "/home/grid/no-passwd.sh" script to ensure that no password is required
Re-run the CVU utility to ensure that all cluster verification checks are passed by the added LXC nodes
Create "if-up.d" and "if-down.d" Files for Cloned Nodes
I created a script that can be used to create the needed files for the additional nodes. The script is not entirely generic so it would need some modification if the names of the LXC containers are not of the form "lxcora0X" so modify it as needed for other container names.
SCST Discovery Issue
If the following issue is encountered when attempting to connect to SCST and discover LUNS as shown below.
[root@oracle651 ~]# iscsiadm --mode discovery --type sendtargets --portal 10.207.40.76
Starting iscsid: [ OK ]
iscsiadm: Login response timeout. Waited 30 seconds and did not get reponse PDU.
iscsiadm: discovery login to 10.207.40.76 failed, giving up 2
iscsiadm: Could not perform SendTargets discovery: encountered non-retryable iSCSI login failure
[root@oracle651 ~]#
Run the following command as shown below.
[root@oracle651 ~]# scstadmin -set_drv_attr iscsi -attributes enabled=1
Collecting current configuration: done.
Performing this action may result in lost or corrupt data, are you sure you wish to continue (y/[n]) ? y
-> Making requested changes.
-> Enabling driver 'iscsi': done.
-> Done, 0 change(s) made.
All done.
[root@oracle651 ~]
Retry discovery as shown below first from the SCST SAN host itself.
[root@oracle651 ~]# iscsiadm -m discovery -t sendtargets -p 10.207.40.76
10.207.40.76:3260,1 iqn.2015-07.org.vmem:oracle651.san.asm.luns
10.207.39.76:3260,1 iqn.2015-07.org.vmem:oracle651.san.asm.luns
10.207.41.76:3260,1 iqn.2015-07.org.vmem:oracle651.san.asm.luns
[root@oracle651 ~]#
Retry discovery as shown below from the Ubuntu SAN client.
gstanden@u1504:~$ sudo iscsiadm -m discovery -t sendtargets -p 10.207.40.76
[sudo] password for gstanden:
10.207.40.76:3260,1 iqn.2015-07.org.vmem:oracle651.san.asm.luns
10.207.39.76:3260,1 iqn.2015-07.org.vmem:oracle651.san.asm.luns
10.207.41.76:3260,1 iqn.2015-07.org.vmem:oracle651.san.asm.luns
gstanden@u1504:~$
Note that the initiators were previously added to the SCST target group as shown below.
[root@oracle651 ~]# cat /etc/scst.conf
# Automatically generated by SCST Configurator v3.0.2.
HANDLER vdisk_fileio {
DEVICE systemdg_01 {
filename /storage_0512/systemdg_01.img
}
DEVICE systemdg_02 {
filename /storage_0512/systemdg_02.img
}
DEVICE systemdg_03 {
filename /storage_0512/systemdg_03.img
}
}
TARGET_DRIVER iscsi {
enabled 0
TARGET iqn.2015-07.org.vmem:oracle651.san.asm.luns {
enabled 1
rel_tgt_id 1
GROUP ubuntu1504 {
LUN 0 systemdg_01
LUN 1 systemdg_02
LUN 2 systemdg_03
INITIATOR iqn.1988-12.com.oracle:b59b40c12ce
INITIATOR iqn.1993-08.org.debian:01:359756c6aac
}
}
}
[root@oracle651 ~]#
Run orainstRoot.sh on All Nodes
Run the orainstRoot.sh as shown below.
[root@lxcora01 oraInventory]# ./orainstRoot.sh
Changing permissions of /u00/app/12.1.0/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u00/app/12.1.0/oraInventory to oinstall.
The execution of the script is complete.
[root@lxcora01 oraInventory]# ssh lxcora02
root@lxcora02's password:
Last login: Sun Jun 28 22:30:35 2015 from 10.207.39.70
[root@lxcora02 ~]# cd /u00/app/12.1.0/oraInventory
[root@lxcora02 oraInventory]# ./orainstRoot.sh
Changing permissions of /u00/app/12.1.0/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u00/app/12.1.0/oraInventory to oinstall.
The execution of the script is complete.
[root@lxcora02 oraInventory]# ssh lxcora03
root@lxcora03's password:
Last login: Sun Jun 28 22:31:11 2015 from 10.207.39.71
[root@lxcora03 ~]# cd /u00/app/12.1.0/oraInventory
[root@lxcora03 oraInventory]# ./orainstRoot.sh
Changing permissions of /u00/app/12.1.0/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u00/app/12.1.0/oraInventory to oinstall.
The execution of the script is complete.
[root@lxcora03 oraInventory]# ssh lxcora04
The authenticity of host 'lxcora04 (10.207.39.73)' can't be established.
RSA key fingerprint is 26:48:0f:ee:41:65:14:38:56:06:b3:14:a0:91:fc:85.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'lxcora04,10.207.39.73' (RSA) to the list of known hosts.
root@lxcora04's password:
Last login: Sun Jun 28 21:03:41 2015
[root@lxcora04 ~]# cd /u00/app/12.1.0/oraInventory
[root@lxcora04 oraInventory]# ./orainstRoot.sh
Changing permissions of /u00/app/12.1.0/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u00/app/12.1.0/oraInventory to oinstall.
The execution of the script is complete.
[root@lxcora04 oraInventory]# ssh lxcora05
The authenticity of host 'lxcora05 (10.207.39.74)' can't be established.
RSA key fingerprint is 26:48:0f:ee:41:65:14:38:56:06:b3:14:a0:91:fc:85.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'lxcora05,10.207.39.74' (RSA) to the list of known hosts.
root@lxcora05's password:
Last login: Sun Jun 28 21:06:02 2015
[root@lxcora05 ~]# cd /u00/app/12.1.0/oraInventory
[root@lxcora05 oraInventory]# ./orainstRoot.sh
Changing permissions of /u00/app/12.1.0/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u00/app/12.1.0/oraInventory to oinstall.
The execution of the script is complete.
[root@lxcora05 oraInventory]# ssh lxcora06
The authenticity of host 'lxcora06 (10.207.39.75)' can't be established.
RSA key fingerprint is 26:48:0f:ee:41:65:14:38:56:06:b3:14:a0:91:fc:85.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'lxcora06,10.207.39.75' (RSA) to the list of known hosts.
root@lxcora06's password:
Last login: Sun Jun 28 21:06:35 2015
[root@lxcora06 ~]# cd /u00/app/12.1.0/oraInventory
[root@lxcora06 oraInventory]# ./orainstRoot.sh
Changing permissions of /u00/app/12.1.0/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u00/app/12.1.0/oraInventory to oinstall.
The execution of the script is complete.
[root@lxcora06 oraInventory]#
Create the /etc/network/if-up.d/openvswitch-net file
gstanden@u1504:~$ cat /etc/network/if-up.d/openvswitch-net
#!/bin/sh -e
/home/gstanden/OpenvSwitch/crt_ovs_sw1.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw1.log
/home/gstanden/OpenvSwitch/crt_ovs_sx1.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sx1.log
/home/gstanden/OpenvSwitch/crt_ovs_sw2.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw2.log
/home/gstanden/OpenvSwitch/crt_ovs_sw3.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw3.log
/home/gstanden/OpenvSwitch/crt_ovs_sw4.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw4.log
/home/gstanden/OpenvSwitch/crt_ovs_sw5.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw5.log
/home/gstanden/OpenvSwitch/crt_ovs_sw6.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw6.log
/home/gstanden/OpenvSwitch/crt_ovs_sw7.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw7.log
/home/gstanden/OpenvSwitch/crt_ovs_sw8.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw8.log
/home/gstanden/OpenvSwitch/crt_ovs_sw9.sh 2>&1 > /home/gstanden/OpenvSwitch/crt_ovs_sw9.log
exit 0
gstanden@u1504:~$