Accessing IBM Object Store from Python

IBM Object Store

SWIFT Object StoreIBM offers a S3 compatible Object Store as a file storage. Beside S3 the storage can also be accessed via the SWIFT protocol by selecting a different deploy model. As the cost for this storage is extremely low compared to Database storage it is perfect for storing sensor data or other kind of data for machine learning.

I use the storage for example to host my training data or trained model for Tensorflow. Access and payment for the Object Store is managed via IBM Cloud aka Bluemix. And as this offering is included in the Lite offering the first 25GB are for free. πŸ™‚

As there is a problem getting the S3 credentials right now I use the SWIFT access model. Please make sure when you request the Object Store service to access the SWIFT version to select the right access model.

Python libs

As the SWIFT protocol is part of openstack, the python access client can be found atΒ https://docs.openstack.org/python-swiftclient. Depending on the security access model you also need the openstack Identity API (Keystone). Both libs are on github (swiftclient and keystone) and also available via pip.

Access storage

Inside the IBM Cloud web interface you can create or read existing credentials. If your program runs on IBM Cloud (Cloudfoundry or Kubernetes) the credentials are also available via the VCAP environment variable. In both cases they look like mine here:

Important informations are the projectId, region, userId and password. The access with keystone the swift python client looks like this:

Important is the version information, also as part of the authurl.

Accessing data

Objects can be read and written, containers (aka buckets) can we read and modified as described in the documentation. For example:

 

 

2 Replies to “Accessing IBM Object Store from Python”

  1. Hi ANSI,

    I am trying to get the credentials as you mentioned on your blog. Having a hard time finding these on the IBM Cloud Interface.

    I would appreciate your help.

    ——-

    Inside the IBM Cloud web interface, you can create or read existing credentials.

    {
    “auth_url”: “https://identity.open.softlayer.com”,
    “project”: “object_storage_xxxxxxxx_xxxx_xxxx_b35a_6d007e3f9118”,
    “projectId”: “512xxxxxxxxxxxxxxxxxxxxxe00fe4e1”,
    “region”: “dallas”,
    “userId”: “e8c19efxxxxxxxxxxxxxxxxxxx91d53e”,
    “username”: “admin_1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxa66”,
    “password”: “fTxxxxxxxxxxw8}l”,
    “domainId”: “15xxxxxxxxxxxxxxxxxxxxxxxxxxxx2a”,
    “domainName”: “77xxx3”,
    “role”: “admin”
    }

Leave a Reply

Your email address will not be published. Required fields are marked *