Tuesday, January 28, 2014

OpenStack: Overview of Service Components

OpenStack is made up of several projects that together build up the OpenStack cloud. Three core components are Compute, Storage, and Network.

OpenStack Training Guides provide a nice conceptual architecture of OpenStack service components, created by Solinea.

OpenStack Conceptual Architecture

Compute (Nova)

Nova services provide computing resources through virtual machine (VM) instances and works with several virtualization technologies, such as KVM, QEMU, VMWare ESX, Xen, Hyper-V, and LXC. A complete list of supported Hypervisors is listed at OpenStack Wiki.

Nova is also called Cloud Controller as it provides framework for provisioning and managing VMs. Nova also provides ephemeral storage.

With Nova services, IT departments can offer private cloud services to internal departments that can scale computing resources on-demand as workload varies. It is analogous to Amazon EC2.
Nova is made up of several service components:
  • nova-compute: Runs the VM instances.
  • nova-scheduler: Decides which host will run the requested instance.
  • nova-api: Service interface to Nova, such as a call to start up a Nova instance.
  • nova-network: Network services
  • nova-objectstore: File storage services
  • nova-common: The underlying common libraries
  • nova-cert: Certificate management service used to authenticate to Nova

Object Storage (Swift)

By default, all storage disappears when VM instance is terminated. Swift services provide cost-effective scale-out redundant persistent storage to VM instances and responsible for ensuring data replication and integrity. The object storage is suitable for static data and stored as Objects. The objects are stored and replicated on disks spread across storage cluster nodes. The stored data persists until deleted by users.

Swift is analogous to Amazon S3. Swift provides RESTful API for integration with other application, storage and services.

Swift is made of several service components:
  • swift-proxy: Accepts, authorizes, authenticates incoming requests.
  • swift-account: Manages database of accounts.
  • swift-container: Contains mapping of containers.
  • swift-object: Contains mapping of objects.

Block Storage (Cinder)

Cinder provides high performance persistent block storage for use with VM instances. Cinder volumes are appropriate for database, file system and raw block storage. Cinder is supported by several storage platforms including Ceph, Nexenta, SolidFire, Zadara, CloudByte, Coraid and Scality to name a few startups in this area. A complete lists of storage devices supported by Cinder drivers is available at Cinder Support Matrix.

Cinder is analogous to Amazon EBS. It also provides snapshot management with capability to either restore snapshot or use snapshot to create a new block storage volume.
Following services are available with Cinder:
  • cinder-api: Authenticates and routes requests to block storage.
  • cinder-scheduler: Scheduling and routing requests to volume service.
  • cinder-volume: Managing back-end block storage devices.
  • cinder-backup: Backup Cinder volume to Swift.

Network (Neutron)

Neutron (formerly Quantum) provides virtual network service for connectivity and addressing used by other services. OpenStack networking can have multiple private networks with overlapping IP addressing schemes and relies on Keystone for authentication and authorization for API access. A standard implementation includes management network, data network, external network, and API network. Though started with basic Linux VLANs and IP tables, it now includes plugins which are pluggable back-end implementation of OpenStack Networking API: Open vSwitch, Linux Bridge, NEC OpenFlow, etc.

Following services are available with Networking:
  • quantum-server: Passing user requests to the configured Plugin for processing.
  • plugin agent (quantum-*-plugin-agent): Perform local vSwitch configuration on hypervisor.
  • dhcp agent (quantum-dhcp-agent): Provides DHCP services to internal networks.
  • l3 agent (quantum-l3-agent): Provides L3/NAT forwarding for access to external network.

Image (Glance)

Glance provides catalog and repository for disk, server and snapshot images. Glance API service is used for discovery, registration and delivery of images. Users can choose from available images, create their own from existing servers and start new compute instances from base templates. Glance supports multi-format image registry including raw, AMI (machine), QCOW2 (QEMU/KVM), VMDK (VMWare), VHD (Hyper-V), VDI (VirtualBox), and OVF.

Glance is made of several service components:
  • glance-api: Accept incoming API requests and communicate with other components.
  • glance-registry: Stores and retrieves metadata about images.
  • image store: Stores actual image blobs, can be local filesystem or Swift.

Identity (Keystone)

Keystone provides authentication authorization for managing users, accounts and role information for all OpenStack services. It also provides catalog of available services. Keystone provides an authentication token that is passed between services after validation for users to be able to use OpenStack resources. Keystone can also integrate with existing backend director services like LDAP.

Dashboard (Horizon)

Horizon provides a web-based graphical interface to access, provision and automate resources for administrators and users. It is implemented as Python/WSGI/Django web application running on Apache Web Server. As an alternate, custom command line scripts can be developed using OpenStack API.

In the next blog post, I plan to perform a quick install of OpenStack.

Thursday, January 23, 2014

OpenStack: The Building Block for Private Cloud

I am back after an extended break from blogging about data storage topics. As I am no longer working at Quantum, I am free to blog about the recent developments in data storage without any concerns.

Last year, I started using cloud servers on Digital Ocean for PeerCube and worked with Amazon Web Services during Coursera's Introduction to Data Science course. Since then, I became very interested in private clouds and methods to establish and manage them.

With the rise of server virtualization, users and applications ability to spin up and spin down pre-built images as needed, and the success of Amazon Web Services (AWS) public cloud, there are fewer reasons for IT administrators to actively manage pooling and allocation of IT infrastructure resources. While there are several commercial (VMware vCloud) and open source platforms (Apache CloudStack, Eucalyptus, OpenStack) that let you build Infrastructure as a Service (IaaS) private cloud, I am particularly excited with the developments happening in OpenStack community.

I was sold on OpenStack as soon as I was able to install a DevStack environment on an old laptop and spin up images within couple of hours. Since then, I have been exploring OpenStack documentation and source code to understand this platform better with private cloud as a specific application in mind.

OpenStack Components

The modularized architecture of OpenStack includes following service components:
  1. Compute (Nova)
  2. Object Store (Swift)
  3. Block Storage (Cinder)
  4. Neutron, formerly Quantum (Network)
  5. Image (Glance)
  6. Identity (Keystone)
  7. Dashboard (Horizon)
In the next blog post, I will elaborate on each services components. Being from data storage industry, I am very interested in learning about the storage service components of OpenStack in-depth.

As I realized that I learn best by writing and sharing, future blog posts will be my journey toward understanding OpenStack and how to use it to establish and manage private clouds.

Book Resources