Statistics
| Branch: | Tag: | Revision:

root / docs / collection_of_examples / updownload.rst @ ef04bdeb

History | View | Annotate | Download (3.9 kB)

1 efae4212 Stavros Sachtouris
Upload and Downloads
2 efae4212 Stavros Sachtouris
====================
3 efae4212 Stavros Sachtouris
4 efae4212 Stavros Sachtouris
The operations of uploading files to pithos as objects, and downloading
5 efae4212 Stavros Sachtouris
objects from pithos as a file are presented in this section.
6 efae4212 Stavros Sachtouris
7 efae4212 Stavros Sachtouris
Enter file context
8 efae4212 Stavros Sachtouris
9 efae4212 Stavros Sachtouris
.. code-block:: console
10 efae4212 Stavros Sachtouris
11 efae4212 Stavros Sachtouris
    $ kamaki
12 efae4212 Stavros Sachtouris
    [kamaki]: file
13 efae4212 Stavros Sachtouris
    [file]:
14 efae4212 Stavros Sachtouris
15 efae4212 Stavros Sachtouris
Upload a file or a directory
16 efae4212 Stavros Sachtouris
----------------------------
17 efae4212 Stavros Sachtouris
18 efae4212 Stavros Sachtouris
First, check the files at the current directory
19 efae4212 Stavros Sachtouris
20 efae4212 Stavros Sachtouris
.. code-block:: console
21 efae4212 Stavros Sachtouris
22 efae4212 Stavros Sachtouris
    [file]: ! ls -F
23 efae4212 Stavros Sachtouris
    file2upload.txt
24 efae4212 Stavros Sachtouris
    dir2upload/
25 efae4212 Stavros Sachtouris
    [file]: list
26 efae4212 Stavros Sachtouris
    pithos (36MB, 5 objects)
27 efae4212 Stavros Sachtouris
    trash  (0B, 0 objects)
28 efae4212 Stavros Sachtouris
29 efae4212 Stavros Sachtouris
.. note:: the `!` symbol is used to run host shell commands
30 efae4212 Stavros Sachtouris
31 efae4212 Stavros Sachtouris
Upload `file2upload.txt` to remote container `pithos`
32 efae4212 Stavros Sachtouris
33 efae4212 Stavros Sachtouris
.. code-block:: console
34 efae4212 Stavros Sachtouris
35 efae4212 Stavros Sachtouris
    [file]: upload file2upload.txt pithos
36 efae4212 Stavros Sachtouris
    Uploading /home/someuser/file2upload.txt --> pithos:file2upload.txt
37 efae4212 Stavros Sachtouris
    Done
38 efae4212 Stavros Sachtouris
39 efae4212 Stavros Sachtouris
Check remote container `pithos` to confirm
40 efae4212 Stavros Sachtouris
41 efae4212 Stavros Sachtouris
.. code-block:: console
42 efae4212 Stavros Sachtouris
43 efae4212 Stavros Sachtouris
    [file]: list pithos
44 efae4212 Stavros Sachtouris
    2 KB file2upload.txt
45 efae4212 Stavros Sachtouris
    2KB  info.txt
46 efae4212 Stavros Sachtouris
    D    video/
47 efae4212 Stavros Sachtouris
    11MB video/tk1.mpg
48 efae4212 Stavros Sachtouris
    12MB video/tk2.mpg
49 efae4212 Stavros Sachtouris
    13MB video/tk3.mpg
50 efae4212 Stavros Sachtouris
    [file]:
51 efae4212 Stavros Sachtouris
52 efae4212 Stavros Sachtouris
Attempt to upload a whole directory, fail and retry with correct arguments
53 efae4212 Stavros Sachtouris
54 efae4212 Stavros Sachtouris
.. code-block:: console
55 efae4212 Stavros Sachtouris
56 efae4212 Stavros Sachtouris
    [file]: upload dir2upload pithos
57 efae4212 Stavros Sachtouris
    /home/someuser/dir2upload is a directory
58 efae4212 Stavros Sachtouris
    |  Use -R to upload directory contents
59 efae4212 Stavros Sachtouris
    [file]: upload -R dirupload pithos
60 efae4212 Stavros Sachtouris
    mkdir pithos:dir2upload
61 efae4212 Stavros Sachtouris
    Uploading /home/someuser/dir2upload/large.mov --> pithos:dir2upload/large.mov
62 efae4212 Stavros Sachtouris
    Uploading /home/someuser/dir2upload/small.mov --> pithos:dir2upload/small.mov
63 efae4212 Stavros Sachtouris
    Done
64 efae4212 Stavros Sachtouris
    [file]: list pithos
65 efae4212 Stavros Sachtouris
    D    dir2upload/
66 efae4212 Stavros Sachtouris
    1GB  dir2upload/large.mov
67 efae4212 Stavros Sachtouris
    1MB  dir2upload/small.mov
68 efae4212 Stavros Sachtouris
    2 KB file2upload.txt
69 efae4212 Stavros Sachtouris
    2KB  info.txt
70 efae4212 Stavros Sachtouris
    D    video/
71 efae4212 Stavros Sachtouris
    11MB video/tk1.mpg
72 efae4212 Stavros Sachtouris
    12MB video/tk2.mpg
73 efae4212 Stavros Sachtouris
    13MB video/tk3.mpg
74 efae4212 Stavros Sachtouris
    [file]:
75 efae4212 Stavros Sachtouris
76 efae4212 Stavros Sachtouris
.. note:: Try to reupload the files (use the -f option to override) and notice
77 efae4212 Stavros Sachtouris
    how much faster is the uploading now. Pithos can determine what parts of
78 efae4212 Stavros Sachtouris
    the file are already uploaded so that kamaki won't attempt to upload them
79 efae4212 Stavros Sachtouris
    again.
80 efae4212 Stavros Sachtouris
81 efae4212 Stavros Sachtouris
Download an object or a directory
82 efae4212 Stavros Sachtouris
---------------------------------
83 efae4212 Stavros Sachtouris
84 efae4212 Stavros Sachtouris
Download object `info.txt` as a local file of the same name
85 efae4212 Stavros Sachtouris
86 efae4212 Stavros Sachtouris
.. code-block:: console
87 efae4212 Stavros Sachtouris
88 efae4212 Stavros Sachtouris
    [file]: download pithos:info.txt
89 efae4212 Stavros Sachtouris
    Downloading pithos:info.txt --> /home/someuser/info.txt
90 efae4212 Stavros Sachtouris
    Donw
91 efae4212 Stavros Sachtouris
    [file]:
92 efae4212 Stavros Sachtouris
93 efae4212 Stavros Sachtouris
Download directory `video` as a local directory with its contents.
94 efae4212 Stavros Sachtouris
We will suppose that a power failure causes the operation to stop unexpectingly
95 efae4212 Stavros Sachtouris
before it's completed.
96 efae4212 Stavros Sachtouris
97 efae4212 Stavros Sachtouris
.. code-block:: console
98 efae4212 Stavros Sachtouris
99 efae4212 Stavros Sachtouris
    [file]: download -R pithos:video
100 efae4212 Stavros Sachtouris
    mkdir video
101 efae4212 Stavros Sachtouris
    Downloading pithos:video/tk1.mpg --> /home/someuser/video/tk1.mpg
102 efae4212 Stavros Sachtouris
    Done
103 efae4212 Stavros Sachtouris
    Downloading pithos:video/tk2.mpg --> /home/someuser/video/tk2.mpg
104 efae4212 Stavros Sachtouris
    <POWER FAILURE>
105 efae4212 Stavros Sachtouris
106 efae4212 Stavros Sachtouris
After we recover the terminal and load kamaki in `file` context, we find out
107 efae4212 Stavros Sachtouris
that `tk1.mpg` had been downloaded while `tk2.mpg` download is incomplete.
108 efae4212 Stavros Sachtouris
109 efae4212 Stavros Sachtouris
.. code-block:: console
110 efae4212 Stavros Sachtouris
111 efae4212 Stavros Sachtouris
    $ ls -F video
112 efae4212 Stavros Sachtouris
    tk1.mpg 11MB
113 efae4212 Stavros Sachtouris
    tk2.mpg 4MB
114 efae4212 Stavros Sachtouris
    $
115 efae4212 Stavros Sachtouris
116 efae4212 Stavros Sachtouris
Let's resume the download uperations (use -r)
117 efae4212 Stavros Sachtouris
118 efae4212 Stavros Sachtouris
.. code-block:: console
119 efae4212 Stavros Sachtouris
120 efae4212 Stavros Sachtouris
    [file]: download -R pithos:video
121 efae4212 Stavros Sachtouris
    Directory video already exists
122 efae4212 Stavros Sachtouris
    | Use -r to resume
123 efae4212 Stavros Sachtouris
    [file]: download -R -r pithos:video
124 efae4212 Stavros Sachtouris
    Resuming pithos:video/tk2.mpg --> /home/someuser/video/tk2.mpg
125 efae4212 Stavros Sachtouris
    Downloading pithos:video/tk3.mpg --> /home/someuser/video/tk3.mpg
126 efae4212 Stavros Sachtouris
    Done
127 efae4212 Stavros Sachtouris
    [file]:
128 efae4212 Stavros Sachtouris
129 efae4212 Stavros Sachtouris
Upload all
130 efae4212 Stavros Sachtouris
----------
131 efae4212 Stavros Sachtouris
132 efae4212 Stavros Sachtouris
.. code-block:: console
133 efae4212 Stavros Sachtouris
134 efae4212 Stavros Sachtouris
    [file]: upload -R -f . pithos
135 efae4212 Stavros Sachtouris
    Done
136 efae4212 Stavros Sachtouris
    [file]:
137 efae4212 Stavros Sachtouris
138 efae4212 Stavros Sachtouris
.. note:: Kamaki calculated that all information is already uploaded, there was
139 efae4212 Stavros Sachtouris
    nothing to be done. If there is was a new or modified file, kamaki would
140 efae4212 Stavros Sachtouris
    detect and upload it.
141 efae4212 Stavros Sachtouris
142 efae4212 Stavros Sachtouris
Download all
143 efae4212 Stavros Sachtouris
------------
144 efae4212 Stavros Sachtouris
145 efae4212 Stavros Sachtouris
.. code-block:: console
146 efae4212 Stavros Sachtouris
147 efae4212 Stavros Sachtouris
    [file]: download -R -r pithos
148 efae4212 Stavros Sachtouris
    Done
149 efae4212 Stavros Sachtouris
    [file]:
150 efae4212 Stavros Sachtouris
151 efae4212 Stavros Sachtouris
.. note:: Kamaki determined that all files have already been calculated. This
152 efae4212 Stavros Sachtouris
    operation examines all local files against the remote objects and
153 efae4212 Stavros Sachtouris
    downloads only missing data.
154 4dd5ea2c Stavros Sachtouris
155 4dd5ea2c Stavros Sachtouris
Exit Context
156 4dd5ea2c Stavros Sachtouris
------------
157 4dd5ea2c Stavros Sachtouris
158 4dd5ea2c Stavros Sachtouris
.. code-block:: console
159 4dd5ea2c Stavros Sachtouris
160 4dd5ea2c Stavros Sachtouris
    [file]: exit
161 4dd5ea2c Stavros Sachtouris
    [kamaki]: