This guide provides technical details for deploying OpenStack with Blockbridge iSCSI storage using the Blockbridge Cinder Volume driver.

Most readers will want to start with the Quickstart section. The rest of the guide has detailed information about features, driver configuration options and troubleshooting.

Supported Versions

This guide covers the 2.0.0 release of the Blockbridge volume driver.

The 2.0.0 release has been qualified with the following OpenStack releases:

• Victoria
• Ussuri

Additionally, this release is supported with the following Blockbridge releases:

• 5.1.0
• 4.4.14

QUICKSTART

This is a quick reference for installing and configuring the Blockbridge Cinder driver with project isolation disabled. We recommend working through the installation in this order:

1. Install the driver.
2. Configure your Blockbridge installation to permit the driver to connect.
3. Configure the Blockbridge volume type in OpenStack.

Some of these topics have more information available by selecting the information ⓘ links next to items where they appear.

Driver Installation

Install the Blockbridge driver on each OpenStack node that will access Blockbridge volumes. ⓘ

Add the YUM repository

sudo yum install -y yum-utils
sudo yum-config-manager \
  --add-repo=https://get.blockbridge.com/tools/5.2/centos/blockbridge-tools.repo

Install the Blockbridge driver package

sudo yum install -y python3-blockbridge_cinder

When prompted, verify the Blockbridge signing key fingerprint

Retrieving key from https://get.blockbridge.com/RPM-GPG-KEY-Blockbridge
Importing GPG key 0x7ECF5373:
 Userid     : "Blockbridge (Official Signing Key) <security@blockbridge.com>"
 Fingerprint: 9c1d e2ae 5970 cfd4 adc5 e0ba ddde 845d 7ecf 5373
 From       : https://get.blockbridge.com/RPM-GPG-KEY-Blockbridge
Is this ok [y/N]: y

Blockbridge Configuration

The following steps use the bb command-line utility.

1. Authenticate as the system user: ⓘ

    $ bb -kH localhost auth login
   

2. Create the openstack tenant account: ⓘ

    $ bb -kH localhost account create --name openstack
   

3. Log in as the new openstack tenant, using the “substitute user” switch: ⓘ

    $ bb -kH localhost auth login --su openstack
   

4. Create a persistent authorization token for the cinder volume driver API access: ⓘ

    $ bb -kH localhost authorization create --notes 'cinder volume driver api access'
   

Remember to record the access token!

OpenStack Configuration

1. On your OpenStack Cinder node, configure a new backend by adding a named configuration group to the /etc/cinder/cinder.conf file. ⓘ

    [bb-general]
    volume_backend_name = bb-general
    blockbridge_pools = pool:
    blockbridge_api_host = blockbridge-storage
    blockbridge_auth_token = 1/j2JUTZQdum2HnO76HbF9adhol0ucgXDataE6tXcV7U8PeQlMlB2wLA
    blockbridge_project_isolation = disabled
    blockbridge_ssl_verify_peer = False
   

2. Add bb-general to the enabled_backends list in the [DEFAULT] group: ⓘ

    [DEFAULT]
    enabled_backends=lvm,bb-general
   

3. Restart the cinder-volume service. ⓘ Keep in mind that the cinder service name varies depending on what OpenStack distribution you’re deploying.

    systemctl restart openstack-cinder-volume
   

4. Create the bb-general volume type: ⓘ

    openstack --os-username admin --os-tenant-name admin volume type create bb-general
   

5. Map the volume type to its corresponding volume_backend_name: ⓘ

    openstack --os-username admin --os-tenant-name admin volume type set blockbridge \
      --property volume_backend_name=bb-general
   

DEPLOYMENT & MANAGEMENT

This section works through details about how best to deploy Blockbridge in your OpenStack environment.

Project Isolation

When you deploy Blockbridge as a volume backend for OpenStack, you can choose how to best integrate it with your OpenStack installation.

If you have a single OpenStack cloud, you can integrate with it using Blockbridge’s Project Isolation support. In this mode, Blockbridge automatically creates tenant accounts on demand to service your OpenStack users. Using Keystone, the OpenStack identity is mapped to a backend Blockbridge account and user, automatically partitioning each project’s storage resources.

With project isolation disabled, you create a Blockbridge account for each of your OpenStack clouds and define an OpenStack volume backend for that account in the Cinder configuration for that OpenStack cloud. The same account may be used by multiple Cinder backends.

With either deployment modes, tenants can log in to Blockbridge to view auditing and statistics related to their volumes. When Project Isolation enabled, tenants use their OpenStack Keystone credentials to log in. With Project Isolation disabled, they’ll use credentials assigned by the Blockbridge storage administrator.

Cinder Driver

The Blockbridge Cinder driver consists of a single python source file. We’re currently in the process of including the driver in the upstream OpenStack cinder distribution; until then, RPM packages are available for CentOS 7 and 8.

The driver must be installed on all OpenStack nodes running the cinder-volume service.

Ensure the yum-config-manager utility is available:

yum install -y yum-utils

Next, add the Blockbridge tools YUM repository:

sudo yum-config-manager --add-repo=https://get.blockbridge.com/tools/6.0/centos/blockbridge-tools.repo

Finally, install the python3-blockbridge_cinder package:

sudo yum install python3-blockbridge_cinder

When prompted, verify and accept the Blockbridge software signing key:

Retrieving key from https://get.blockbridge.com/RPM-GPG-KEY-Blockbridge
Importing GPG key 0x7ECF5373:
 Userid     : "Blockbridge (Official Signing Key) <security@blockbridge.com>"
 Fingerprint: 9c1d e2ae 5970 cfd4 adc5 e0ba ddde 845d 7ecf 5373
 From       : https://get.blockbridge.com/RPM-GPG-KEY-Blockbridge
Is this ok [y/N]: y

For assistance with other OpenStack deployments, contact support@blockbridge.com.

Authentication

The Blockbridge cinder driver requires a Blockbridge account. With Project Isolation enabled, the account must be an administrative user that is granted the “switch user” permission. The driver will switch from the admin authentication context to a tenant’s context on demand. With Project Isolation disabled, the tenant’s own Blockbridge account is used, and therefore does not require administrative privileges.

You can use the Blockbridge CLI to create authorization tokens suitable for cut-and-pasting into cinder.conf. These tokens can be individually revoked if the need arises. For this reason, we recommend allocating one for each account on each OpenStack cluster.

Blockbridge Login

Log in as the administrative system user to a Blockbridge management cluster using the command-line tool. Here, the management address is blockbridge-storage.

$ bb -kH blockbridge-storage auth login
Authenticating to https://blockbridge-storage/api

Enter user or access token: system
Password for system:
Authenticated; token expires in 3599 seconds.

== Authenticated as user system.

Account Creation

Create a dedicated tenant account (e.g. openstack) as follows:

$ bb -kH blockbridge-storage account create --name openstack
== Created account: openstack (ACT0762194C407FBCF4)

== Account: openstack (ACT0762194C407FBCF4)
name                  openstack
label                 openstack
serial                ACT0762194C407FBCF4
created               2021-03-10 17:11:35 -0500
disabled              no

Administrative Token

For Keystone integrated deployments, create an unrestricted token for the system user as follows:

$ bb -H blockbridge-storage authorization create --notes "OpenStack Admin Token" --restrict none
Authenticating to https://blockbridge-storage/api

Enter user or access token: system
Password for system:
Authenticated; token expires in 3599 seconds.

== Authorization: ATH4762894C40626410
notes                 OpenStack Admin Token
serial                ATH4762894C40626410
account               system (ACT0762594C40626440)
user                  system (USR1B62094C40626440)
enabled               yes
created at            2015-10-24 22:08:48 +0000
access type           online
token suffix          xaKUy3gw
restrict              none

== Access Token
access token          1/elvMWilMvcLAajl...3ms3U1u2KzfaMw6W8xaKUy3gw

*** Remember to record your access token!

Make note of the displayed access token; if you lose it, you’ll have to create another persistent authorization.

Tenant Token

For deployments with Keystone integration disabled, authenticate as the tenant and create a token with default options:

bb -kH localhost auth login --su openstack
... [output trimmed]
bb -kH blockbridge-storage authorization create --notes 'Cinder volume driver API access for Cloud-BOS-1'
... [output trimmed]

Cinder Backend

On every OpenStack Cinder node, configure a new backend by adding a named configuration group to the /etc/cinder/cinder.conf file. Set the blockbridge_api_host to the DNS name or IP address of the Blockbridge management node. If you have a clustered management installation, be sure to use a cluster-managed VIP, otherwise cinder won’t be able to communicate with the backend after service failover. For blockbridge_auth_token, use the access token generated in the previous step.

[bb-general]
volume_backend_name = bb-general
blockbridge_pools = pool: +ssd
blockbridge_api_host = blockbridge-storage
blockbridge_auth_token = 1/j2JUTZQdum2HnO76HbF9adhol0ucgXDataE6tXcV7U8PeQlMlB2wLA
blockbridge_project_isolation = disabled
blockbridge_ssl_verify_peer = False

You’ll also need to add bb-general to the enabled_backends list in the [DEFAULT] group:

[DEFAULT]
enabled_backends=lvm,bb-general

Though we’ve used bb-general for our configuration group name and the volume_backend_name, they don’t have to match. The enabled_backends parameter is a list of configuration group names (as specified in square brackets in cinder.conf), not a list of volume_backend_name values. More information can be found in the Multiple Volume Types section below.

After editing the cinder.conf configuration file, restart the cinder-volume service.

systemctl restart openstack-cinder-volume

Volume Type

The final piece of configuration is to expose a volume type (e.g. blockbridge) to users so that they may create volumes with the bb-general backend. (The volume_backend_name parameter is Cinder’s internal volume type identifier.) The process to map this internal parameter to a public volume type has two steps:

First, create the volume type (here, blockbridge):

openstack --os-username admin --os-tenant-name admin volume type create blockbridge

And finally, map the volume type to its corresponding volume_backend_name:

openstack --os-username admin --os-tenant-name admin volume type set blockbridge \
  --property volume_backend_name=bb-general

Resource Queries

Each Blockbridge volume backend can include attributes that influence the type of storage that can be provisioned for the backend. These are specified in the blockbridge_pools parameter in the backend definition. For example, a query could specify +ssd +10.0.0.0 +6nines -production iops.reserve=1000 capacity.reserve=30%. This query is satisfied by selecting SSD resources, accessible on the 10.0.0.0 network, with high resiliency, for non-production workloads, with guaranteed IOPS of 1000 and a storage reservation for 30% of the volume capacity specified at create time.

To make this style of attribute-based provisioning work, tag Blockbridge datastores with the appropriate information. In the example above, the query would only provision storage from datastores configured with the ssd, 10.0.0.0, and 6nines tags. It would skip datastores tagged production. The iops parameter requires a datastore that supports. IOPS scheduling.

By default, each backend has an implicit +openstack query. Within Blockbridge, datastores that are tagged openstack will be selected to fulfill OpenStack provisioning requests. If you prefer a more specific query, define a custom pool configuration.

blockbridge_pools = Production: +production +qos iops.reserve=5000

Configuration options

Description of Blockbridge EPS volume driver configuration options

Example cinder.conf

[Default]
enabled_backends = bb_devel,bb_prod

[bb_prod]
volume_driver = cinder.volume.drivers.blockbridge.BlockbridgeISCSIDriver
volume_backend_name = blockbridge_prod
blockbridge_api_host = [ip or dns of management cluster]
blockbridge_auth_token = 1/elvMWilMvcLAajl...3ms3U1u2KzfaMw6W8xaKUy3gw
blockbridge_pools = Production: +production +qos iops.reserve=5000
blockbridge_project_isolation = idp

[bb_devel]
volume_driver = cinder.volume.drivers.blockbridge.BlockbridgeISCSIDriver
volume_backend_name = blockbridge_devel
blockbridge_api_host = [ip or dns of management cluster]
blockbridge_auth_token = 1/elvMWilMvcLAajl...3ms3U1u2KzfaMw6W8xaKUy3gw
blockbridge_pools = Development: +development
blockbridge_project_isolation = disabled

REFERENCE

Supported Operations

The Blockbridge cinder driver supports the following operations:

Supported protocols

Blockbridge uses iSCSI for data access. Our Cinder driver communicates directly with our transactional API to build tenant-isolated SAN fabrics on demand.

Version History