root / docs / collection_of_examples / updownload.rst @ ef04bdeb
History | View | Annotate | Download (3.9 kB)
1 | efae4212 | Stavros Sachtouris | Upload and Downloads |
---|---|---|---|
2 | efae4212 | Stavros Sachtouris | ==================== |
3 | efae4212 | Stavros Sachtouris | |
4 | efae4212 | Stavros Sachtouris | The operations of uploading files to pithos as objects, and downloading |
5 | efae4212 | Stavros Sachtouris | objects from pithos as a file are presented in this section. |
6 | efae4212 | Stavros Sachtouris | |
7 | efae4212 | Stavros Sachtouris | Enter file context |
8 | efae4212 | Stavros Sachtouris | |
9 | efae4212 | Stavros Sachtouris | .. code-block:: console |
10 | efae4212 | Stavros Sachtouris | |
11 | efae4212 | Stavros Sachtouris | $ kamaki |
12 | efae4212 | Stavros Sachtouris | [kamaki]: file |
13 | efae4212 | Stavros Sachtouris | [file]: |
14 | efae4212 | Stavros Sachtouris | |
15 | efae4212 | Stavros Sachtouris | Upload a file or a directory |
16 | efae4212 | Stavros Sachtouris | ---------------------------- |
17 | efae4212 | Stavros Sachtouris | |
18 | efae4212 | Stavros Sachtouris | First, check the files at the current directory |
19 | efae4212 | Stavros Sachtouris | |
20 | efae4212 | Stavros Sachtouris | .. code-block:: console |
21 | efae4212 | Stavros Sachtouris | |
22 | efae4212 | Stavros Sachtouris | [file]: ! ls -F |
23 | efae4212 | Stavros Sachtouris | file2upload.txt |
24 | efae4212 | Stavros Sachtouris | dir2upload/ |
25 | efae4212 | Stavros Sachtouris | [file]: list |
26 | efae4212 | Stavros Sachtouris | pithos (36MB, 5 objects) |
27 | efae4212 | Stavros Sachtouris | trash (0B, 0 objects) |
28 | efae4212 | Stavros Sachtouris | |
29 | efae4212 | Stavros Sachtouris | .. note:: the `!` symbol is used to run host shell commands |
30 | efae4212 | Stavros Sachtouris | |
31 | efae4212 | Stavros Sachtouris | Upload `file2upload.txt` to remote container `pithos` |
32 | efae4212 | Stavros Sachtouris | |
33 | efae4212 | Stavros Sachtouris | .. code-block:: console |
34 | efae4212 | Stavros Sachtouris | |
35 | efae4212 | Stavros Sachtouris | [file]: upload file2upload.txt pithos |
36 | efae4212 | Stavros Sachtouris | Uploading /home/someuser/file2upload.txt --> pithos:file2upload.txt |
37 | efae4212 | Stavros Sachtouris | Done |
38 | efae4212 | Stavros Sachtouris | |
39 | efae4212 | Stavros Sachtouris | Check remote container `pithos` to confirm |
40 | efae4212 | Stavros Sachtouris | |
41 | efae4212 | Stavros Sachtouris | .. code-block:: console |
42 | efae4212 | Stavros Sachtouris | |
43 | efae4212 | Stavros Sachtouris | [file]: list pithos |
44 | efae4212 | Stavros Sachtouris | 2 KB file2upload.txt |
45 | efae4212 | Stavros Sachtouris | 2KB info.txt |
46 | efae4212 | Stavros Sachtouris | D video/ |
47 | efae4212 | Stavros Sachtouris | 11MB video/tk1.mpg |
48 | efae4212 | Stavros Sachtouris | 12MB video/tk2.mpg |
49 | efae4212 | Stavros Sachtouris | 13MB video/tk3.mpg |
50 | efae4212 | Stavros Sachtouris | [file]: |
51 | efae4212 | Stavros Sachtouris | |
52 | efae4212 | Stavros Sachtouris | Attempt to upload a whole directory, fail and retry with correct arguments |
53 | efae4212 | Stavros Sachtouris | |
54 | efae4212 | Stavros Sachtouris | .. code-block:: console |
55 | efae4212 | Stavros Sachtouris | |
56 | efae4212 | Stavros Sachtouris | [file]: upload dir2upload pithos |
57 | efae4212 | Stavros Sachtouris | /home/someuser/dir2upload is a directory |
58 | efae4212 | Stavros Sachtouris | | Use -R to upload directory contents |
59 | efae4212 | Stavros Sachtouris | [file]: upload -R dirupload pithos |
60 | efae4212 | Stavros Sachtouris | mkdir pithos:dir2upload |
61 | efae4212 | Stavros Sachtouris | Uploading /home/someuser/dir2upload/large.mov --> pithos:dir2upload/large.mov |
62 | efae4212 | Stavros Sachtouris | Uploading /home/someuser/dir2upload/small.mov --> pithos:dir2upload/small.mov |
63 | efae4212 | Stavros Sachtouris | Done |
64 | efae4212 | Stavros Sachtouris | [file]: list pithos |
65 | efae4212 | Stavros Sachtouris | D dir2upload/ |
66 | efae4212 | Stavros Sachtouris | 1GB dir2upload/large.mov |
67 | efae4212 | Stavros Sachtouris | 1MB dir2upload/small.mov |
68 | efae4212 | Stavros Sachtouris | 2 KB file2upload.txt |
69 | efae4212 | Stavros Sachtouris | 2KB info.txt |
70 | efae4212 | Stavros Sachtouris | D video/ |
71 | efae4212 | Stavros Sachtouris | 11MB video/tk1.mpg |
72 | efae4212 | Stavros Sachtouris | 12MB video/tk2.mpg |
73 | efae4212 | Stavros Sachtouris | 13MB video/tk3.mpg |
74 | efae4212 | Stavros Sachtouris | [file]: |
75 | efae4212 | Stavros Sachtouris | |
76 | efae4212 | Stavros Sachtouris | .. note:: Try to reupload the files (use the -f option to override) and notice |
77 | efae4212 | Stavros Sachtouris | how much faster is the uploading now. Pithos can determine what parts of |
78 | efae4212 | Stavros Sachtouris | the file are already uploaded so that kamaki won't attempt to upload them |
79 | efae4212 | Stavros Sachtouris | again. |
80 | efae4212 | Stavros Sachtouris | |
81 | efae4212 | Stavros Sachtouris | Download an object or a directory |
82 | efae4212 | Stavros Sachtouris | --------------------------------- |
83 | efae4212 | Stavros Sachtouris | |
84 | efae4212 | Stavros Sachtouris | Download object `info.txt` as a local file of the same name |
85 | efae4212 | Stavros Sachtouris | |
86 | efae4212 | Stavros Sachtouris | .. code-block:: console |
87 | efae4212 | Stavros Sachtouris | |
88 | efae4212 | Stavros Sachtouris | [file]: download pithos:info.txt |
89 | efae4212 | Stavros Sachtouris | Downloading pithos:info.txt --> /home/someuser/info.txt |
90 | efae4212 | Stavros Sachtouris | Donw |
91 | efae4212 | Stavros Sachtouris | [file]: |
92 | efae4212 | Stavros Sachtouris | |
93 | efae4212 | Stavros Sachtouris | Download directory `video` as a local directory with its contents. |
94 | efae4212 | Stavros Sachtouris | We will suppose that a power failure causes the operation to stop unexpectingly |
95 | efae4212 | Stavros Sachtouris | before it's completed. |
96 | efae4212 | Stavros Sachtouris | |
97 | efae4212 | Stavros Sachtouris | .. code-block:: console |
98 | efae4212 | Stavros Sachtouris | |
99 | efae4212 | Stavros Sachtouris | [file]: download -R pithos:video |
100 | efae4212 | Stavros Sachtouris | mkdir video |
101 | efae4212 | Stavros Sachtouris | Downloading pithos:video/tk1.mpg --> /home/someuser/video/tk1.mpg |
102 | efae4212 | Stavros Sachtouris | Done |
103 | efae4212 | Stavros Sachtouris | Downloading pithos:video/tk2.mpg --> /home/someuser/video/tk2.mpg |
104 | efae4212 | Stavros Sachtouris | <POWER FAILURE> |
105 | efae4212 | Stavros Sachtouris | |
106 | efae4212 | Stavros Sachtouris | After we recover the terminal and load kamaki in `file` context, we find out |
107 | efae4212 | Stavros Sachtouris | that `tk1.mpg` had been downloaded while `tk2.mpg` download is incomplete. |
108 | efae4212 | Stavros Sachtouris | |
109 | efae4212 | Stavros Sachtouris | .. code-block:: console |
110 | efae4212 | Stavros Sachtouris | |
111 | efae4212 | Stavros Sachtouris | $ ls -F video |
112 | efae4212 | Stavros Sachtouris | tk1.mpg 11MB |
113 | efae4212 | Stavros Sachtouris | tk2.mpg 4MB |
114 | efae4212 | Stavros Sachtouris | $ |
115 | efae4212 | Stavros Sachtouris | |
116 | efae4212 | Stavros Sachtouris | Let's resume the download uperations (use -r) |
117 | efae4212 | Stavros Sachtouris | |
118 | efae4212 | Stavros Sachtouris | .. code-block:: console |
119 | efae4212 | Stavros Sachtouris | |
120 | efae4212 | Stavros Sachtouris | [file]: download -R pithos:video |
121 | efae4212 | Stavros Sachtouris | Directory video already exists |
122 | efae4212 | Stavros Sachtouris | | Use -r to resume |
123 | efae4212 | Stavros Sachtouris | [file]: download -R -r pithos:video |
124 | efae4212 | Stavros Sachtouris | Resuming pithos:video/tk2.mpg --> /home/someuser/video/tk2.mpg |
125 | efae4212 | Stavros Sachtouris | Downloading pithos:video/tk3.mpg --> /home/someuser/video/tk3.mpg |
126 | efae4212 | Stavros Sachtouris | Done |
127 | efae4212 | Stavros Sachtouris | [file]: |
128 | efae4212 | Stavros Sachtouris | |
129 | efae4212 | Stavros Sachtouris | Upload all |
130 | efae4212 | Stavros Sachtouris | ---------- |
131 | efae4212 | Stavros Sachtouris | |
132 | efae4212 | Stavros Sachtouris | .. code-block:: console |
133 | efae4212 | Stavros Sachtouris | |
134 | efae4212 | Stavros Sachtouris | [file]: upload -R -f . pithos |
135 | efae4212 | Stavros Sachtouris | Done |
136 | efae4212 | Stavros Sachtouris | [file]: |
137 | efae4212 | Stavros Sachtouris | |
138 | efae4212 | Stavros Sachtouris | .. note:: Kamaki calculated that all information is already uploaded, there was |
139 | efae4212 | Stavros Sachtouris | nothing to be done. If there is was a new or modified file, kamaki would |
140 | efae4212 | Stavros Sachtouris | detect and upload it. |
141 | efae4212 | Stavros Sachtouris | |
142 | efae4212 | Stavros Sachtouris | Download all |
143 | efae4212 | Stavros Sachtouris | ------------ |
144 | efae4212 | Stavros Sachtouris | |
145 | efae4212 | Stavros Sachtouris | .. code-block:: console |
146 | efae4212 | Stavros Sachtouris | |
147 | efae4212 | Stavros Sachtouris | [file]: download -R -r pithos |
148 | efae4212 | Stavros Sachtouris | Done |
149 | efae4212 | Stavros Sachtouris | [file]: |
150 | efae4212 | Stavros Sachtouris | |
151 | efae4212 | Stavros Sachtouris | .. note:: Kamaki determined that all files have already been calculated. This |
152 | efae4212 | Stavros Sachtouris | operation examines all local files against the remote objects and |
153 | efae4212 | Stavros Sachtouris | downloads only missing data. |
154 | 4dd5ea2c | Stavros Sachtouris | |
155 | 4dd5ea2c | Stavros Sachtouris | Exit Context |
156 | 4dd5ea2c | Stavros Sachtouris | ------------ |
157 | 4dd5ea2c | Stavros Sachtouris | |
158 | 4dd5ea2c | Stavros Sachtouris | .. code-block:: console |
159 | 4dd5ea2c | Stavros Sachtouris | |
160 | 4dd5ea2c | Stavros Sachtouris | [file]: exit |
161 | 4dd5ea2c | Stavros Sachtouris | [kamaki]: |