Add image registration examples
authorStavros Sachtouris <saxtouri@admin.grnet.gr>
Wed, 19 Jun 2013 11:58:16 +0000 (14:58 +0300)
committerStavros Sachtouris <saxtouri@admin.grnet.gr>
Wed, 19 Jun 2013 14:59:32 +0000 (17:59 +0300)
(cherry picked from commit 0b158e6b7ca54bb2cb95aefcb49a55420a75145e)

docs/collection_of_examples/examples.rst
docs/collection_of_examples/imageregister.rst [new file with mode: 0644]
docs/collection_of_examples/listing.rst
docs/examples.rst
kamaki/cli/commands/image.py

index a66d8f1..63d916c 100644 (file)
@@ -1,16 +1,4 @@
 
-Image register
---------------
-
-Upload and register image
-^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Register with metadata template
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Upload and register multiple images
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
 virtual machines
 ----------------
 
diff --git a/docs/collection_of_examples/imageregister.rst b/docs/collection_of_examples/imageregister.rst
new file mode 100644 (file)
index 0000000..2a52325
--- /dev/null
@@ -0,0 +1,494 @@
+Image registration
+==================
+
+In Synnefo, an image is loaded as a file to the storage service (Pithos+), and
+then is registered to the image service (Plankton). The image location at the
+storage server is unique through out a deployment and also necessary for the
+image to exist.
+
+The image location format::
+
+    pithos://<user id>/<container>/<object path>
+
+    e.g.:
+
+    pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump
+
+
+Register an image
+-----------------
+
+Let the image file `debian_base3.diskdump` be a debian image located at the
+current directory.
+
+Upload the image to container `pithos`
+
+.. code-block:: console
+
+    [kamaki]: file upload debian_base3.diskdump pithos
+    Uploading /home/someuser/debian_base3.diskdump --> pithos:debian_base3.diskdump
+    Done
+    [kamaki]:
+
+Register the image object with the name 'Debian Base Alpha'
+
+.. code-block:: console
+
+    [kamaki]: image register 'Debian Base Alpha' pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump
+    checksum:         3cb03556ec971f...e8dd6190443b560cb7
+    container-format: bare
+    created-at:       2013-06-19 08:00:22
+    deleted-at:       
+    disk-format:      diskdump
+    id:               7h1rd-1m4g3-1d
+    is-public:        False
+    location:         pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump
+    name:             Debian Base Alpha
+    owner:            s0m3-u53r-1d
+    properties:      
+    size:             903471104
+    status:           available
+    updated-at:       2013-06-19 08:01:00
+    Metadata file uploaded as pithos:debian_base3.diskdump.meta (version 1352)
+    [kamaki]:
+
+.. note:: The `image register` command automatically create a meta file and
+    uploads it to the same location as the image. The meta file can be
+    downloaded and reused for more image registrations.
+
+Read the metafile
+
+.. code-block:: console
+
+    [kamaki]: file cat pithos:debian_base3.diskdump
+    {
+      "status": "available", 
+      "name": "Debian Base Gama", 
+      "checksum": "3cb03556ec971f...e8dd6190443b560cb7", 
+      "id": "7h1rd-1m4g3-1d2", 
+      "updated-at": "2013-06-19 08:01:00", 
+      "created-at": "2013-06-19 08:00:22", 
+      "properties": {}, 
+      "location": "pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump", 
+      "is-public": "False", 
+      "owner": "s0m3-u53r-1d", 
+      "disk-format": "diskdump", 
+      "size": "903471104", 
+      "deleted-at": "", 
+      "container-format": "bare"
+    }
+    [kamaki]:
+
+Images registered by me
+-----------------------
+
+List all images, then list only images owned by the user with id s0m3-u53r-1d
+
+.. code-block:: console
+
+    [kamaki]: image list
+    f1r57-1m4g3-1d Debian Base Alpha
+        container_format: bare
+        disk_format:      diskdump
+        size:             474066944
+        status:           available
+    53c0nd-1m4g3-1d Beta Debian Base
+        container_format: bare
+        disk_format:      diskdump
+        size:             474066944
+        status:           available
+    7h1rd-1m4g3-1d Debian Base Gama
+        container_format: bare
+        disk_format:      diskdump
+        size:             474066944
+        status:           available
+    [kamaki]: image list --owner=s0m3-u53r-1d
+    7h1rd-1m4g3-1d Debian Base Gama
+        container_format: bare
+        disk_format:      diskdump
+        size:             474066944
+        status:           available
+    [kamaki]:
+
+.. note:: To get the current user id, use `user authenticate` in kamaki
+
+Unregister an image
+-------------------
+
+An image can be unregistered by its image id, but only if the current user is
+also the image owner. In this example, there is only one image owned by current
+user.
+
+Unregister image owned by current user 
+
+.. code-block:: console
+
+    [kamaki]: image unregister 7h1rd-1m4g3-1d
+    [kamaki]:
+
+Check if the image is deleted
+
+.. code-block:: console
+
+    [kamaki]: image list --owner=s0m3-u53r-1d
+    [kamaki]:
+
+Attempt to unregister an image of another user
+
+.. code-block:: console
+
+    [kamaki]: image unregister f1r57-1m4g3-1d
+    (403) FORBIDDEN forbidden ()
+    [kamaki]:
+
+Register with properties
+------------------------
+
+The image will be registered again, but with some custom properties::
+
+    OS: Linux
+    user: someuser
+
+These properties can be added freely by the user, and they have no significance
+for the image server, but they could be used to help using the image more
+efficiently.
+
+Attempt to register with properties
+
+.. code-block:: console
+
+    [kamaki]: image register 'Debian Base Gama' pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump -p OS=Linux -p user=someuser
+    Metadata file pithos:debian_base3.diskdump.meta already exists
+    [kamaki]:
+
+It's true that the metafile is already there, but we can override it (**-f**)
+
+.. code-block:: console
+
+    [kamaki]: image register -f 'Debian Base Gama' pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump -p OS=Linux -p user=someuser
+    [kamaki]:
+
+Register with a meta file
+-------------------------
+
+Download the meta file of the image (it was uploaded recently)
+
+.. code-block:: console
+
+    [kamaki]: file download pithos:debian_base3.diskdump.meta
+    Downloading pithos:debian_base3.diskdump.meta --> /home/someuser/debian_base3.diskdump.meta
+    Done
+    [kamaki]:
+
+The metadata file can be edited. Let's edit the file, by adding properties::
+
+    OS: Linux
+    user: root
+
+The resulting file will look like this:
+
+.. code-block:: javascript
+
+    {
+      "status": "available", 
+      "name": "Debian Base Gama", 
+      "checksum": "3cb03556ec971f...e8dd6190443b560cb7", 
+      "id": "7h1rd-1m4g3-1d2", 
+      "updated-at": "2013-06-19 08:01:00", 
+      "created-at": "2013-06-19 08:00:22", 
+      "properties": {
+        "OS": "Linux",
+        "USER": "root"
+      }, 
+      "location": "pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump", 
+      "is-public": "False", 
+      "owner": "s0m3-u53r-1d", 
+      "disk-format": "diskdump", 
+      "size": "903471104", 
+      "deleted-at": "", 
+      "container-format": "bare"
+    }
+
+.. warning:: make sure the file is in a valid json format, otherwise image
+    register will fail
+
+In the following registration, a different name will be used for the image.
+
+Register the image (don't forget the -f parameter, to override the metafile).
+
+.. code-block:: console
+
+    [kamaki]: image register -f 'Debian Base Delta' pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump --metafile=debian_base3.diskdump.meta
+    checksum:         3cb03556ec971f...e8dd6190443b560cb7
+    container-format: bare
+    created-at:       2013-06-19 08:00:22
+    deleted-at:       
+    disk-format:      diskdump
+    id:               7h1rd-1m4g3-1d
+    is-public:        False
+    location:         pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump
+    name:             Debian Base Delta
+    owner:            s0m3-u53r-1d
+    properties:      
+            OS:     Linux
+            USER:   root
+    size:             903471104
+    status:           available
+    updated-at:       2013-06-19 08:01:00
+    Metadata file uploaded as pithos:debian_base3.diskdump.meta (version 1359)
+    [kamaki]:
+
+Reregistration: priorities and overrides
+----------------------------------------
+
+Let's review the examples presented above::
+
+    - Register an image with name `Debian Base Gama`
+    - Unregister the image
+    - Register a new image of the uploaded image object, with custom properties
+    - Reregister the image with a meta file and modified properties and name
+
+**The image id is related to the image object**
+
+Although the image was unregistered and reregistered, the image id, that is
+produced automatically at the server side, was the same. This is due to the
+fact that image ids are 1 to 1 related to image objects uploaded to Pithos+
+
+**An explicit name overrides the metafile**
+
+Each image needs a name and this is given as the first argument of the
+`register` command. This name overrides the name in the metafile.
+
+**Reregistration is not update, but an override**
+
+The property `user: root` won over `user: someuser`, because it was set last.
+Actually, all properties were replaced by the new ones, when the image was
+reregistered, and the same holds with all customizable attributes of the image.
+
+Command line wins the metafile
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Let's compine the metafile with a command line attribute `user: admin`
+
+.. code-block:: console
+
+    [kamaki]: image register -f 'Debian Base Delta' pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump --metafile=debian_base3.diskdump.meta
+    checksum:         3cb03556ec971f...e8dd6190443b560cb7
+    container-format: bare
+    created-at:       2013-06-19 08:00:22
+    deleted-at:       
+    disk-format:      diskdump
+    id:               7h1rd-1m4g3-1d
+    is-public:        False
+    location:         pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump
+    name:             Debian Base Delta
+    owner:            s0m3-u53r-1d
+    properties:      
+            OS:     Linux
+            USER:   root
+    size:             903471104
+    status:           available
+    updated-at:       2013-06-19 08:01:00
+    Metadata file uploaded as pithos:debian_base3.diskdump.meta (version 1377)
+    [kamaki]:
+
+Although the property `OS` was read from the metafile, the property `USER` was
+set by the command line property to `admin`.
+
+.. note:: This feature allows the use of a metafile as a template for uploading
+    multiple images with many common attributes but slight modifications per
+    image
+
+Multiple metafile versions
+--------------------------
+
+.. warning:: Make sure your container is set to auto, otherwise, there will be
+    no object versions
+
+    .. code-block:: console
+
+        [kamaki]: file versioning get pithos
+        x-container-policy-versioning: auto
+        [kamaki]:
+
+    To set versioning to auto
+
+    .. code-block:: console
+
+        [kamaki]: file versioning set auto pithos
+        [kamaki]:
+
+In the above examples, the image was registered many times by overriding the
+metafile. It is possible to avoid writing a metafile, as well as accessing
+older versions of the file.
+
+Register the image without uploading a metafile
+
+.. code-block:: console
+
+    [kamaki]: image register 'Debian Base Delta' pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump --metafile=debian_base3.diskdump.meta --no-metafile-upload
+    checksum:         3cb03556ec971f...e8dd6190443b560cb7
+    container-format: bare
+    created-at:       2013-06-19 08:00:22
+    deleted-at:       
+    disk-format:      diskdump
+    id:               7h1rd-1m4g3-1d
+    is-public:        False
+    location:         pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump
+    name:             Debian Base Delta
+    owner:            s0m3-u53r-1d
+    properties:      
+            OS:     Linux
+            USER:   root
+    size:             903471104
+    status:           available
+    updated-at:       2013-06-19 08:01:00
+    [kamaki]:
+
+Uploaded metafiles are kept in versions, thanks to Pithos+ versioning support
+
+.. code-block:: console
+
+    [kamaki]: file versions pithos:debian_base3.diskdump.meta
+    1352
+     created: 19-06-2013 11:00:22
+    1359
+     created: 19-06-2013 11:01:00
+    1377
+     created: 19-06-2013 11:34:37
+    [kamaki]:
+
+Consult the first version of the metafile
+
+.. code-block:: console
+
+    [kamaki]: file cat --object-version=1352 pithos:debian_base3.diskdump.meta
+    {
+      "status": "available", 
+      "name": "Debian Base Gama", 
+      "checksum": "3cb03556ec971f...e8dd6190443b560cb7", 
+      "id": "7h1rd-1m4g3-1d2", 
+      "updated-at": "2013-06-19 08:01:00", 
+      "created-at": "2013-06-19 08:00:22", 
+      "properties": {}, 
+      "location": "pithos://s0m3-u53r-1d/pithos/debian_base3.diskdump", 
+      "is-public": "False", 
+      "owner": "s0m3-u53r-1d", 
+      "disk-format": "diskdump", 
+      "size": "903471104", 
+      "deleted-at": "", 
+      "container-format": "bare"
+    }
+    [kamaki]:
+
+Download the second version
+
+.. code-block:: console
+
+    [kamaki]: file download --object-version=1359 pithos:debian_base3.diskdump.meta debian_base3.diskdump.meta.v1359
+    Downloading pithos:debian_base3.diskdump.meta --> /home/someuser/debian_base3.diskdump.meta.v1359
+    Done
+    [kamaki]:
+
+Batch image upload
+------------------
+
+Let a directory at /home/someuser/images with a variety of images needed to be
+uploaded and registered.
+
+Batch-upload the images
+
+.. code-block:: console
+
+    [kamaki]: file upload -R images pithos
+    mkdir pithos:images
+    Uploading /home/someuser/images/debian.diskdump --> pithos:images/debian.diskdump
+    Uploading /home/someuser/images/win8.diskdump --> pithos:images/win8.diskdump
+    ...
+    Done
+    [kamaki]:
+
+Make sure the images are uploaded to pithos:images/ remote directory object
+
+.. code-block:: console
+
+    [kamaki]: file list pithos:images/
+    D       images/
+    983MB   images/debian.diskdump
+    2.2GB   images/win8.diskdump
+    ...
+    [kamaki]:
+
+Use the host shell capabilities to streamline the registration, so exit kamaki
+
+.. code-block:: console
+
+    [kamaki]: /exit
+
+The following is a bash script that attempts to register the already uploaded
+images:
+
+.. code-block:: bash
+
+    #!/bin/bash
+
+    userid=... # e.g. s0m3-u53r-1d
+    container=... # e.g. pithos
+
+    for path in images/*.diskdump; do
+        location=pithos://$userid/$container/${path}
+        kamaki image register $path $location
+    done
+
+Let's use the script (enriched with a separator message) to batch-register the
+images (all images will be named after their relative paths).
+
+Also, let the registered images be public (accessible to all users for creating
+VMs) by adding the **--public** flag argument when calling `image register`.
+
+.. code-block:: console
+
+    $ for path in images/*.diskdump; do
+        location=pithos://s0m3-u53r-1d/pithos/${path}
+        echo "- - - Register ${path} - - -"
+        kamaki image register $path $location --public
+    done
+    - - - Register images/debian.diskdump ---
+    checksum:         3cb03556ec971f...e8dd6190443b560cb7
+    container-format: bare
+    created-at:       2013-06-19 08:00:22
+    deleted-at:       
+    disk-format:      diskdump
+    id:               d3b14n-1m4g3-1d
+    is-public:        False
+    location:         pithos://s0m3-u53r-1d/pithos/images/debian.diskdump
+    name:             images/debian.diskdump
+    owner:            s0m3-u53r-1d
+    properties:
+    size:             903471104
+    status:           available
+    updated-at:       2013-06-19 08:01:00
+    Metadata file uploaded as pithos:images/debian.diskdump.meta (version 4201)
+    - - - Register images/win8.diskdump ---
+    checksum:         4cb03556ec971f...e8dd6190443b560cb6
+    container-format: bare
+    created-at:       2013-06-19 08:00:22
+    deleted-at:       
+    disk-format:      diskdump
+    id:               w1nd0w5-1m4g3-1d
+    is-public:        False
+    location:         pithos://s0m3-u53r-1d/pithos/images/win8.diskdump
+    name:             images/win8.diskdump
+    owner:            s0m3-u53r-1d
+    properties:
+    size:             2103471104
+    status:           available
+    updated-at:       2013-06-19 08:01:00
+    Metadata file uploaded as pithos:images/debian.diskdump.meta (version 4301)
+    ...
+    $
+
+.. note:: All images can be re-registered, either individually or with a batch
+    process.
+
+
index 770e755..4df149c 100644 (file)
@@ -509,14 +509,14 @@ Detailed listing
 
     [image]: list -l
     f1r57-1m4g3-1d Debian Base Alpha
-        checksum:         9344d77620cde1dd77da556e5f72ed4de8533931f86c857b70badda34b26d782
+        checksum:         9344d77620cde1dd77da...7b70badda34b26d782
         container_format: bare
         created_at:       2013-06-03 16:44:16
         deleted_at:       
         disk_format:      diskdump
         is_public:        True
-        location:         pithos://s0m3-u53r-1d/pithos/debian_base1.diskdump
-        owner:            s0m3-u53r-1d
+        location:         pithos://s0m3-5up3r-u53r-1d/pithos/debian_base1.diskdump
+        owner:            s0m3-5up3r-u53r-1d
         properties:      
                     description:    Debian 6.0.6 (Squeeze) Base System
                     gui:            No GUI
@@ -530,14 +530,14 @@ Detailed listing
         status:           available
         updated_at:       2013-06-03 16:44:16
     53c0nd-1m4g3-1d Beta Debian Base
-        checksum:         9344d77620cde1dd77da556e5f72ed4de8533931f86c857b70badda34b26d782
+        checksum:         9344d77620cde1dd77da...7b70badda34b26d782
         container_format: bare
         created_at:       2013-06-03 16:44:16
         deleted_at:       
         disk_format:      diskdump
         is_public:        True
-        location:         pithos://s0m3-u53r-1d/pithos/debian_base2.diskdump
-        owner:            s0m3-u53r-1d
+        location:         pithos://s0m3-5up3r-u53r-1d/pithos/debian_base2.diskdump
+        owner:            s0m3-5up3r-u53r-1d
         properties:      
                     description:    Debian 6.0.6 (Squeeze) Base System
                     gui:            No GUI
@@ -573,9 +573,9 @@ Detailed listing
           users:          root
         progress:  100
         status:    ACTIVE
-        tenant_id: s0m3-u53r-1d
+        tenant_id: s0m3-5up3r-u53r-1d
         updated:   2013-06-03T16:21:53+00:00
-        user_id:   s0m3-u53r-1d
+        user_id:   s0m3-5up3r-u53r-1d
     53c0nd-1m4g3-1d Beta Debian Base
         created:   2013-06-03T16:21:53+00:00
         links:    
@@ -598,9 +598,9 @@ Detailed listing
           users:          root
         progress:  100
         status:    ACTIVE
-        tenant_id: s0m3-u53r-1d
+        tenant_id: s0m3-5up3r-u53r-1d
         updated:   2013-06-03T16:21:53+00:00
-        user_id:   s0m3-u53r-1d
+        user_id:   s0m3-5up3r-u53r-1d
     [image]:
 
 Filter listing by prefix, suffix or words in image names
index d7f74d3..930f29b 100644 (file)
@@ -9,3 +9,4 @@ Examples
     collection_of_examples/listing
     collection_of_examples/updownload
     collection_of_examples/sharing
+    collection_of_examples/imageregister
index 707459e..6f6cfc6 100644 (file)
@@ -438,7 +438,7 @@ class image_register(_init_image, _optional_json):
                 print('Metadata file uploaded as %s:%s (version %s)' % (
                     container, meta_path, meta_headers['x-object-version']))
 
-    def main(self, name, location=None):
+    def main(self, name, location):
         super(self.__class__, self)._run()
         self._run(name, location)